Line-of-sight angular rate extraction method based on depth target tracking

Abstract

The invention relates to a sight angle rate extraction method based on depth target tracking, and relates to the field of navigation control and computer vision. Comprising the steps of deploying a variable-focus optical camera on an unmanned aerial vehicle, and setting a normal of the center of a mirror surface of the camera to be inclined at a certain angle with a horizontal plane for acquiring complete information of a target; calibrating the target in the camera based on an optical parameter calibration method, and obtaining a field angle of the target relative to the center of the camera; deploying a target tracking algorithm on the unmanned aerial vehicle, and identifying a target position based on the target tracking algorithm; obtaining a horizontal deflection angle and a pitching deflection angle of the target relative to the center of the camera by combining a target position identified by a target tracking algorithm based on the field angle; and estimating the line-of-sight angular rate through Kalman filtering based on the obtained horizontal deflection angle, the obtained pitching deflection angle and the flight parameters of the unmanned aerial vehicle. Through combination of parameter calibration of the optical camera and a target tracking algorithm based on deep learning, the horizontal deflection angle and the pitching deflection angle of the target can be accurately obtained.

Description

Technical Field

[0001] The present invention relates to the fields of navigation control and computer vision, and in particular to a line of sight angular rate extraction method based on deep target tracking. Background Art

[0002] With the development of drone technology, its application in military, security, traffic monitoring and other fields is becoming more and more extensive. As an important part of drone flight, target tracking technology can capture and track moving targets in real time. However, when completing target tracking tasks, most single drones currently rely on real-time imaging images returned by cameras and remote operations of personnel to achieve target positioning and tracking, which not only increases labor costs, but also has very low efficiency. Therefore, it is particularly important to develop an efficient and reliable single drone autonomous target tracking technology.

[0003] Traditional target tracking methods mainly rely on image processing technology and physical methods based on flight control, such as feature point-based tracking algorithms and optical flow methods, but these methods are often limited in performance in dynamic environments. For example, when the target is blocked by other objects or when lighting conditions change, tracking accuracy and stability will drop significantly. In addition, the structure based on gyroscopes and complex optical systems is also difficult to implement. On the other hand, there are also a small number of solutions that apply deep learning methods to drones, but due to the limitations of drone payloads and optical systems, tracking tasks usually require multiple drones to complete, and the cost and difficulty of implementation are also high. Summary of the invention

[0004] The technical problems to be solved by the present invention are:

[0005] In order to avoid the shortcomings of the prior art, the present invention provides a line of sight angular rate extraction method based on deep target tracking, which uses a method combining a deep convolutional neural network and an optical system structure design to estimate the line of sight angular rate through Kalman filtering to achieve accurate tracking of the target. At the same time, combined with sensor data and environmental information, the robustness and real-time performance of target tracking are improved.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A method for extracting sight angle rate based on deep target tracking, characterized by comprising:

[0008] A variable-focus optical camera is deployed on the drone, and the normal of the camera mirror center is set to be tilted at a certain angle to the horizontal plane to obtain complete information of the target;

[0009] Calibrate the target in the camera based on the optical parameter calibration method to obtain the field of view angle of the target relative to the center of the camera;

[0010] Deploy a target tracking algorithm on the UAV and identify the target location based on the target tracking algorithm;

[0011] Based on the field of view angle and the target position identified by the target tracking algorithm, the horizontal and pitch angles of the target relative to the center of the camera are obtained;

[0012] Based on the acquired horizontal deflection angle, pitch deflection angle and flight parameters of the UAV, the line of sight angular rate is estimated through Kalman filtering.

[0013] A further technical solution of the present invention is as follows: the step of calibrating the target in the camera based on the optical parameter calibration method and obtaining the field of view angle of the target relative to the center of the camera includes:

[0014] Fix the target and the camera at a predetermined position, and measure the horizontal deflection angle and the pitch deflection angle of the target relative to the center of the camera; capture the target image through the camera, record the position of the target center in the imaging result, calculate the offset of the target position relative to the center position of the camera, and obtain the offset and the offset parameters composed of the horizontal deflection angle and the pitch deflection angle;

[0015] Keep the position of the camera unchanged, change the position of the target on the straight line of the angle between the two, so that the horizontal and pitch angles between the target and the camera remain unchanged, record the position of the target in the camera, and compare it with the initial position, and continuously correct the calibrated offset parameters; when the target and the camera move under the condition of unchanged angle, and the imaging results are completely consistent, the camera angle calibration of that point is completed;

[0016] Calibrate the deflection angle parameters of the center point of the edge of the imaging result. The horizontal and pitch deflection angles of the midpoints of the left and lower edges of the image are recorded as (α 0 ,0) and (0,β 0 ), then the horizontal field of view and the pitch field of view of the current focal length of the camera are 2α respectively 0 and 2β 0 .

[0017] A further technical solution of the present invention: the target tracking algorithm comprises:

[0018] In the same video sequence, the first frame is selected as a template image, and another frame image separated by a certain time is selected as a search image. The same convolutional neural network is used to convolve the template image and the search image to extract features to obtain a first feature map and a second feature map.

[0019] A response map is obtained by performing cross-correlation calculation on the first feature map and the second feature map. The area with the largest response value in the response map is the location of the target in the search area.

[0020] A further technical solution of the present invention is as follows: the cross-correlation calculation of the first characteristic map and the second characteristic map is performed to obtain a response map, specifically:

[0021] Use the first feature map as the convolution kernel and perform convolution on the second feature map:

[0022]

[0023] Among them, f(z,x) is the result of convolution, and γ and b are the scaling and enhancement of this process.

[0024] A further technical solution of the present invention is as follows: the step of obtaining the horizontal deflection angle and the pitch deflection angle of the target relative to the center of the camera by combining the field of view angle with the target position identified by the target tracking algorithm comprises:

[0025]

[0026] Among them, α 0 and β 0 are half of the calibrated horizontal field of view and pitch field of view, respectively. (x, y) is the target center position, (x0, y0) is the camera center, α is the horizontal deflection angle, and β is the pitch deflection angle.

[0027] A further technical solution of the present invention is as follows: the step of estimating the line of sight angular rate by Kalman filtering based on the acquired horizontal deflection angle and pitch deflection angle and the flight parameters of the UAV comprises:

[0028] The sight horizontal angle, the sight horizontal angle rate, the sight pitch angle, and the sight pitch angle rate are used as state quantities; the sight horizontal angle is the calculated horizontal deflection angle, and the sight pitch angle is the calculated pitch deflection angle;

[0029] Establish state equation based on state quantity;

[0030] A Kalman filter line of sight angular rate estimation model is established based on the state equation;

[0031] The line of sight horizontal angular rate and line of sight pitch angular rate are estimated based on the Kalman filter line of sight angular rate estimation model.

[0032] A further technical solution of the present invention: the Kalman filter line of sight angular rate estimation model is expressed as follows:

[0033]

[0034] Among them, X k is the state quantity at time k, W is the system noise matrix, F k is the state matrix of the system, V is the measurement noise matrix, Z k is the estimated value of pitch angle and horizontal angle, Hk is the observation matrix of the system.

[0035] A computer system, characterized in that it comprises: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0036] A computer-readable storage medium, characterized in that it stores computer-executable instructions, which are used to implement the above method when executed.

[0037] A computer program product, characterized by comprising computer executable instructions, wherein the instructions are used to implement the above method when executed.

[0038] The beneficial effects of the present invention are:

[0039] The present invention provides a line of sight angular rate extraction method based on deep target tracking, which can obtain the horizontal deflection angle and pitch deflection angle of the target more accurately by combining the parameter calibration of the optical camera and the target tracking algorithm based on deep learning. Compared with the existing methods, it does not require complex optical structures and other auxiliary equipment. It only needs to strapdown the optical camera with a single UAV, which greatly reduces the difficulty and cost of implementation; and the detection method based on deep learning can reduce the problems of false detection and missed detection in the case of complex background and target occlusion, effectively solving the shortcomings of traditional methods, and at the same time has good speed and accuracy; finally, the line of sight angular rate with small error is estimated through Kalman filtering to achieve tracking control of the UAV. The advantages are as follows:

[0040] 1. Most UAVs that rely on optical systems to achieve target tracking require the design of a seeker structure, which increases the load and manufacturing cost of the UAV. The present invention uses a strapdown camera to replace the seeker, which simplifies the structure and reduces the difficulty and cost of manufacturing.

[0041] 2. The traditional target tracking algorithm based on feature points is weak in accuracy and anti-interference ability, especially when the target undergoes large changes in appearance or is obscured. The present invention uses a deep learning method to achieve target tracking, improves the tracking accuracy, and can ensure the reliability of tracking under most interferences.

[0042] 3. When a drone performs tracking and other tasks, it usually requires several or even more drones to form a drone group to achieve stable cruising and tracking tasks. The present invention combines the optical system and the deep learning algorithm, so that the tracking task can be completed by deploying the tracking algorithm on a single drone, thereby improving the diversity of the functions of a single drone.

[0043] 4. Most common UAV tracking and navigation algorithms estimate the line of sight angular rate through gyroscopes and radars, which is a waste of resources for tasks with relatively low flight speed and real-time requirements. The present invention realizes line of sight angular rate estimation through the target tracking algorithm deployed on the UAV and the UAV's own parameters, which reduces mechanical and power losses while also being able to achieve target tracking tasks more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0045] Figure 1 It is a schematic flow chart of the implementation steps of the present invention.

[0046] Figure 2 Schematic diagram of horizontal deflection angle and pitch deflection angle.

[0047] Figure 3 Schematic diagram of the UAV roll angle. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0050] like Figure 1 As shown, the present invention provides a method for acquiring sight angle rate based on deep target tracking, comprising the following steps:

[0051] Step 1: Deploy a variable-focus optical camera on the drone, and make the normal line passing through the center of the camera mirror tilt at a certain angle to the horizontal plane to obtain complete information of the target;

[0052] Step 2: Calibrate the horizontal deflection angle and pitch deflection angle corresponding to any point of the target in the camera through field tests, and repeat the test multiple times to ensure the accuracy of the calibration;

[0053] Step 3: Deploy the target tracking algorithm on the drone and train the algorithm using a large amount of data sets in the early stage so that it has good tracking accuracy for common targets.

[0054] Step 4: By calibrating several sets of accurate offset parameters and optical camera parameters, combined with the target position identified by the target tracking algorithm, the horizontal deflection angle and pitch deflection angle of the target at any position in the camera relative to the current camera center can be obtained;

[0055] Step 5: According to the two deflection angle parameters and the flight parameters of the UAV, the line of sight angular rate is estimated through Kalman filtering, and the speed and flight distance of the UAV in two directions are adjusted so that the target always remains in the center of the optical camera, thereby achieving target tracking.

[0056] The above steps are as follows:

[0057] Step 1: Deploy a variable-focus optical camera on the drone, and make the normal line passing through the center of the camera mirror tilt at a certain angle to the horizontal plane to obtain complete information of the target;

[0058] The present invention deploys a variable-focus high-resolution camera on the drone. The camera has the following characteristics:

[0059] High resolution: Able to capture subtle features of the target. Clearer imaging results help the tracking algorithm to better extract target features and improve the accuracy of target recognition.

[0060] Variable focal length: allows the drone to focus on targets at different distances to adapt to different tracking scenarios and distance requirements.

[0061] Camera tilt angle setting

[0062] In order to obtain complete information of the target, the camera is designed so that the normal line passing through the center of the camera mirror is tilted at a certain angle to the horizontal plane. This strapdown design enables the camera to capture the target from a high altitude, thereby providing more comprehensive perspective information, and has a simple structure and is easy to implement.

[0063] Tilt angle: Based on the experimental results of the previous training, the tilt angle of the camera relative to the direction of flight of the UAV is precisely set to the highest universality to optimize the acquisition of target information.

[0064] Step 2: Optical system design and camera parameter calibration: Strapdown the optical camera and the drone at a certain angle, and complete the offset parameter calibration on this basis.

[0065] In a specific embodiment, the angle between the camera and the drone is determined by the focal length of the camera, the flight altitude of the drone and the target size. The angle value is selected by verifying that the target has clear and complete features in the imaging result, and the angle is applicable to most target types.

[0066] Optical parameter calibration method

[0067] Calibration process: In a known test environment, fix the target and camera at a predetermined position, and measure the horizontal deflection angle α and pitch deflection angle β of the target relative to the center of the camera; capture the target image through the camera, record the position T(x,y) of the target center in the imaging result, and calculate the offset ΔT(Δx,Δy) of the target position T relative to the camera center position O(x0,y0); ΔT is calculated as follows:

[0068]

[0069] Wherein, Δx and Δy can be negative numbers, representing the position coordinates of the target in the plane direct coordinate system with the camera center O as the origin; the four parameters α, β, Δx, and Δy are recorded as a group, and the offset parameters B(Δx, α, Δy, β) of the target relative to the camera center are obtained.

[0070] Data collection: Keep the position of the camera unchanged, change the position of the target on the straight line of the angle between the two, so that the horizontal and pitch angles between the target and the camera are always kept at α and β, record the position of the target in the camera, and compare it with the initial position T, and continuously correct the calibrated offset parameters. When the target and the camera move under the condition of unchanged angle, and the results of several imaging are completely consistent, the camera angle calibration of that point is completed.

[0071] Calibration accuracy: According to the above method, the deflection parameters of the center point of the edge of the imaging result can be calibrated. The horizontal and pitch deflection angles of the midpoints of the left and lower edges of the image are recorded as (α 0 ,0) and (0,β 0 ), then the horizontal field of view and the pitch field of view of the current focal length of the camera are 2α respectively 0 and 2β 0 Assuming the image (video) resolution is 2a×2b, the offset parameters of the left edge center point and the bottom edge center point are B and w (-a,α 0 ,0,0) and B h (0,0,b,β 0). Based on the camera optical parameters and the calibrated offset parameters, the offset parameters of any point in the image relative to the center point can be calculated. Finally, the above calibration steps are repeated, and the accuracy of the offset parameters in the entire image is verified by comparing the calibration and calculation results.

[0072] Step 2: Target tracking algorithm training and deployment: Use a large number of standard data sets to train the target tracking algorithm so that it has relatively good tracking accuracy for common targets and is deployed on drones.

[0073] In a specific embodiment, according to the different application scenarios and tasks of the drone, firstly, the types of data sets are as rich as possible, including pedestrians, bicycles, motorcycles, cars and large vehicles, etc.; secondly, the data has better quality after preprocessing, the original data of target tracking is the video signal, the features of the target in the same video do not change significantly and the annotation format is unified; finally, there is some background interference in the video signal and the comparison with other instances can enhance the feature extraction ability and robustness of the network.

[0074] The traditional filtering algorithm is replaced by deep learning, and the detection-based target tracking task is realized by introducing a twin neural network. The first frame of the video sequence is used as the template x, and the other frame is used as the search image z. The same convolutional neural network is used for feature extraction. The convolution process is recorded as Then the feature maps obtained by convolution of the template frame image and the search frame image are and

[0075] Will As the convolution kernel, Convolution is performed on the two to complete the cross-correlation operation. This process is also the target tracking process. * is used to represent the convolution operation, and this process can be recorded as:

[0076]

[0077] In the formula, f(z,x) is the result of the convolution, recorded as the response map, and the area with the largest response value in the map is the location of the target in the search area. γ and b are the scaling and enhancement of this process. Since this is a common change for all response values ​​in the response map, it does not change the area where the maximum value is located, and therefore does not affect the positioning result.

[0078] It should be noted that the image size of the search frame is different from that of the template frame. Its side length is twice that of the template frame. The size of the template frame image is set to A×A, where A is defined as:

[0079] A 2 =(w+p)×(h+p);

[0080] Where w and h represent the size of the target bounding box, and p is defined as:

[0081]

[0082] The size of the template frame can be expressed as:

[0083]

[0084] In another specific embodiment, the located target area is recorded as (A x ,A y ,A w ,A h ), and the real bounding box of the target is recorded as (T x ,T y ,T w ,T h ), then the regression target can be expressed as follows:

[0085]

[0086] The training process of a neural network is usually represented by a loss function, the value of which represents the network's ability to regress the target, and the two are inversely proportional. In another specific embodiment, the loss function is It means that based on the above regression target, the tracking loss Loss can be expressed as:

[0087]

[0088] in The expression is:

[0089]

[0090] Among them, σ is a hyperparameter that controls the range of the smoothing area. Different values ​​determine where to transition from square loss to absolute value loss. The gradient of the loss function is continuously calculated through back propagation, so that its value tends to the global minimum, and finally the parameters of the tracking network are obtained.

[0091] Step 3: Calculate the horizontal and pitch angles: By combining the output of the target tracking algorithm with the offset parameters calibrated by the optical camera, the offset angle of the target in the current frame relative to the center of the camera can be obtained.

[0092] In a specific embodiment, the offset angle is defined as follows: the line connecting the target center point and the camera center is taken as the target axis, a spatial rectangular coordinate system is established with the camera center as the origin, the direction facing the camera center is the optical axis, the direction perpendicular to the ground is the pitch axis, and the direction parallel to the ground and perpendicular to the two is the cruise axis. The horizontal deflection angle and the pitch deflection angle are the angles between the projection of the target axis on the two planes and the optical axis, such as Figure 2 shown.

[0093] In the figure, the angle formed by the target axis mapped on the cruise axis-optical axis plane and the optical axis is the horizontal deflection angle, represented by α; the angle formed by the target axis mapped on the pitch axis-optical axis plane and the optical axis is the pitch deflection angle, represented by β.

[0094] In a specific embodiment, the horizontal and pitch angles of the target at any position in the image relative to the center of the camera can be calculated by using several offset parameter values ​​calibrated by the optical camera and the target position coordinates obtained by the target tracking algorithm.

[0095] Under ideal conditions without considering other influencing factors such as lens distortion, the horizontal deflection angle α and the pitch deflection angle β of the target relative to the center of the camera can be expressed by the target imaging position and the focal length of the lens. The center points of the left edge and the bottom edge of the imaging result have been clearly calibrated as B w (-a,α 0 ,0,0) and B h (0,0,b,β 0 ), so the horizontal deflection angle α and pitch deflection angle β of the target center position (x, y) relative to the camera center (x0, y0) can be expressed as:

[0096]

[0097] In the formula, α 0 and β 0 They are half of the calibrated horizontal field of view and pitch field of view, respectively. Therefore, the absolute values ​​of the horizontal deflection angle α and the pitch deflection angle β should not exceed half of the calibrated maximum field of view of the camera.

[0098] It should be noted that the horizontal field of view and the pitch field of view of the current focal length of the camera are 2α respectively. 0 and 2β 0 , the absolute values ​​of the horizontal deflection angle α and the pitch deflection angle β do not exceed half of the calibrated maximum field of view of the camera. Moreover, this method is an approximate estimation result under ideal conditions. If the deviation is large in actual application, other influencing factors such as optical distortion should be considered, or each pixel should be calibrated to achieve a more accurate result.

[0099] Step 4: Estimation of sight angular rate: Estimate the sight angular rate based on the horizontal deflection angle α and pitch deflection angle β of the target relative to the camera center and the flight parameters of the UAV.

[0100] Ideally, the drone body will remain parallel to the ground during flight, so the camera will not be affected by shaking. However, in actual flight, due to the influence of resistance, weather and other external environments, the drone will shake slightly in the optical axis, pitch axis and cruise axis, causing the optical lens to have an angle with each axis in the ideal state, thereby affecting the calculation of the target horizontal and pitch angles. Therefore, it is also necessary to consider the three shaking angles of the optical lens, namely azimuth, pitch angle and roll angle, recorded as ψ, θ and γ. Usually, the values ​​of these three angles can be given by the drone system through measurement and calculation of related components and airborne navigation algorithms.

[0101] In a specific embodiment, the definitions of the azimuth and pitch angles are consistent with the horizontal deflection and pitch deflection of the target, and the roll angle is defined as follows: Figure 3 As shown in the figure, the red color is the target bounding box in the ideal state, the blue color is the actual target bounding box after jittering, and the angle between the two is the rolling angle γ.

[0102] Step 3 has obtained the horizontal deflection angle α and the pitch deflection angle β of the target relative to the center of the camera, but this includes lens jitter and other measurement noise. In order to further accurately estimate the line of sight angular velocity, Kalman filtering is used here to estimate. Suppose the required line of sight horizontal angle is q α (i.e. horizontal deflection angle α), the horizontal angular rate of sight is The line of sight elevation angle is q β (i.e., pitch angle β), the line of sight pitch angle rate is And make

[0103] The state equation is established as follows:

[0104]

[0105] In the formula, r represents the distance from the target to the camera, which can be measured and given by the drone.

[0106] The following Kalman filter line of sight angular rate estimation model is established based on the state equation:

[0107]

[0108] Where, X k is the state quantity at time k, W is the system noise matrix, F k is the state matrix of the system, given by the state equation. k is the estimated value of pitch angle and horizontal angle, H k is the observation matrix of the system, whose value can be given by the state equation and the matrix L(γ,θ,ψ). V is the measurement noise matrix, and the estimated value and It is given by the observation equation:

[0109]

[0110] Where, L ij Represents the elements in the matrix L(γ,θ,ψ) composed of the flight azimuth, pitch angle and roll angle ψ,θ,γ of the drone, and its expression is as follows:

[0111]

[0112] Finally, after introducing the initial value of the state quantity, system noise, and observation noise, the pitch and horizontal line of sight angular rates with smaller errors can be obtained according to the Kalman filter algorithm.

[0113] Directly dividing the line of sight angular rate by the deflection angle value by the time interval will result in a large error. However, after Kalman filter estimation, a value with a smaller error can be obtained, achieving more precise drone control.

[0114] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A method for extracting sight angle rate based on deep target tracking, characterized in that: include: A variable-focus optical camera is deployed on the drone, and the normal of the camera mirror center is set to be tilted at a certain angle to the horizontal plane to obtain complete information of the target; Calibrate the target in the camera based on the optical parameter calibration method to obtain the field of view angle of the target relative to the center of the camera; Deploy a target tracking algorithm on the UAV and identify the target location based on the target tracking algorithm; Based on the field of view angle and the target position identified by the target tracking algorithm, the horizontal and pitch angles of the target relative to the center of the camera are obtained; Based on the acquired horizontal deflection angle, pitch deflection angle and flight parameters of the UAV, the line of sight angular rate is estimated through Kalman filtering.

2. According to claim 1, a method for extracting line of sight angular velocity based on deep target tracking is characterized in that: The step of calibrating the target in the camera based on the optical parameter calibration method and obtaining the field of view angle of the target relative to the center of the camera includes: Fix the target and the camera at a predetermined position, and measure the horizontal deflection angle and the pitch deflection angle of the target relative to the center of the camera; capture the target image through the camera, record the position of the target center in the imaging result, calculate the offset of the target position relative to the center position of the camera, and obtain the offset and the offset parameters composed of the horizontal deflection angle and the pitch deflection angle; Keep the position of the camera unchanged, change the position of the target on the straight line of the angle between the two, so that the horizontal and pitch angles between the target and the camera remain unchanged, record the position of the target in the camera, and compare it with the initial position, and continuously correct the calibrated offset parameters; when the target and the camera move under the condition of unchanged angle, and the imaging results are completely consistent, the camera angle calibration of that point is completed; Calibrate the deflection angle parameters of the center point of the edge of the imaging result. The horizontal and pitch deflection angles of the midpoints of the left and lower edges of the image are recorded as (α 0 ,0) and (0,β 0 ), then the horizontal field of view and the pitch field of view of the current focal length of the camera are 2α respectively 0 and 2β 0 .

3. The method for extracting sight angle rate based on deep target tracking according to claim 1, characterized in that: The target tracking algorithm comprises: In the same video sequence, the first frame is selected as a template image, and another frame image separated by a certain time is selected as a search image. The same convolutional neural network is used to convolve the template image and the search image to extract features to obtain a first feature map and a second feature map. A response map is obtained by performing cross-correlation calculation on the first feature map and the second feature map. The area with the largest response value in the response map is the location of the target in the search area.

4. According to claim 3, a method for extracting line of sight angular velocity based on deep target tracking is characterized in that: The cross-correlation calculation of the first feature map and the second feature map is performed to obtain the response map, specifically: Use the first feature map as the convolution kernel and perform convolution on the second feature map: Among them, f(z,x) is the result of convolution, and γ and b are the scaling and enhancement of this process.

5. The method for extracting sight angle rate based on deep target tracking according to claim 1, characterized in that: The step of obtaining the horizontal deflection angle and the pitch deflection angle of the target relative to the center of the camera by combining the field of view angle with the target position identified by the target tracking algorithm includes: Among them, α 0 and β 0 are half of the calibrated horizontal field of view and pitch field of view, respectively. (x, y) is the target center position, (x0, y0) is the camera center, α is the horizontal deflection angle, and β is the pitch deflection angle.

6. According to claim 5, a method for extracting sight angle rate based on deep target tracking is characterized in that: The step of estimating the line of sight angular rate through Kalman filtering based on the acquired horizontal deflection angle and pitch deflection angle and the flight parameters of the UAV includes: The sight horizontal angle, the sight horizontal angle rate, the sight pitch angle, and the sight pitch angle rate are used as state quantities; the sight horizontal angle is the calculated horizontal deflection angle, and the sight pitch angle is the calculated pitch deflection angle; Establish state equation based on state quantity; A Kalman filter line of sight angular rate estimation model is established based on the state equation; The line of sight horizontal angular rate and line of sight pitch angular rate are estimated based on the Kalman filter line of sight angular rate estimation model.

7. The method for extracting sight angle rate based on deep target tracking according to claim 1, characterized in that: The Kalman filter line of sight angular rate estimation model is expressed as follows: Among them, X k is the state quantity at time k, W is the system noise matrix, F k is the state matrix of the system, V is the measurement noise matrix, Z k is the estimated value of pitch angle and horizontal angle, H k is the observation matrix of the system.

8. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

9. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

10. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when being executed.

Images