A method for target stable tracking combined with ground speed compensation

Abstract

The present invention relates to the technical field of UAV target tracking, and particularly to a method for stable target tracking combined with ground speed compensation. The method includes measuring the flight speed and direction of the UAV to determine the ground speed; using an angle measurement system to obtain the angle parameters on the flight path of the UAV; calculating the ground speed compensation value according to the ground speed and the angle parameters; the optoelectronic turret captures visible light images and infrared images and performs real-time registration and fusion; according to the registered and fused images, the local contrast and information entropy are extracted, and the fusion ratio of the visible light image and the infrared image is dynamically adjusted through a fusion tracking method to generate a fusion tracking image; the fusion tracking image is input into a correlation tracking algorithm for target tracking; the target search area is offset according to the ground speed compensation parameters; the servo system inputs the ground speed compensation parameters into the speed closed-loop control loop to stably track the target in combination with the miss distance. The advantages are as follows: real-time registration improves the image registration efficiency; the target tracking performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of UAV target tracking, and particularly to a method for stable target tracking combined with ground speed compensation. Background Art

[0002] In the technical field of UAV airborne optoelectronic turrets, target tracking technology is a core component, which directly affects the execution efficiency and accuracy of tasks such as reconnaissance, surveillance, and target recognition. With the development of UAV technology, higher requirements are put forward for target tracking technology, especially the stability and accuracy in a dynamic flight environment. In a UAV airborne optoelectronic turret system, achieving stable target tracking is crucial for improving the efficiency and effectiveness of UAV mission execution.

[0003] In the prior art, a UAV airborne optoelectronic turret system usually includes a visible light camera and an infrared camera, which can provide visible light images and infrared images of the target. However, due to the influence of flight speed and attitude changes during high-speed flight or maneuvering flight of the UAV, the computing power of the airborne embedded computer is limited, and there are certain limitations in target tracking. Especially in a dynamically changing environment, the UAV needs to adjust the pointing of its optoelectronic turret in real time to maintain stable tracking of the target, which is very challenging technically.

[0004] First, when the UAV is flying at high speed or performing maneuvering flight, due to flight speed and attitude changes, it is difficult for the target tracking system to accurately predict the moving trajectory of the target. Second, existing systems often lack an effective ground speed compensation mechanism and cannot accurately compensate for the movement of the UAV relative to the ground, thus affecting tracking stability. In addition, the image registration algorithms of the prior art usually cannot achieve real-time processing, which limits the real-time performance and reliability of target tracking.

[0005] In practical applications, a UAV airborne optoelectronic turret system needs to perform tasks under various environmental conditions, including different flight altitudes, speeds, and complex meteorological conditions. These factors all pose challenges to the performance of the target tracking system. For example, when flying at high speed, the UAV needs to quickly adjust the pointing of the optoelectronic turret to maintain stable tracking of the target; under complex meteorological conditions such as fog, rain, or at night, the UAV needs to use multi-modal information such as infrared images for target tracking.

[0006] To improve the accuracy and stability of target tracking, researchers have proposed various methods, including model-based prediction methods, image processing techniques, and machine learning algorithms, etc. However, these methods still have deficiencies in terms of real-time performance, accuracy, and robustness, especially in a UAV dynamic flight environment.

[0007] In addition, the image registration algorithms in the prior art usually require complex ground calibration processes, including precise measurements of the offset of the optical axis center of the camera, edge distortion, and pixel mapping relationships. These calibration processes are not only time-consuming and laborious but also difficult to meet the application requirements of the variable focal length camera in the airborne optoelectronic turret system of unmanned aerial vehicles (UAVs).

[0008] In summary, the existing airborne optoelectronic turret systems of UAVs face problems such as poor adaptability to dynamic flight environments, insufficient ground speed compensation, and poor real-time performance of image registration in target tracking. These problems limit the efficiency and effectiveness of UAVs in performing tasks in complex environments. Therefore, it is necessary to develop a new target stable tracking method combined with ground speed compensation to improve the target tracking performance of UAVs in dynamic flight environments. Summary of the Invention

[0009] The present invention provides a method for target stable tracking combined with ground speed compensation to solve the above problems.

[0010] The object of the present invention is to provide a method for target stable tracking combined with ground speed compensation, which specifically includes the following steps:

[0011] S1. Measure the flight speed and flight direction of the UAV through the onboard IMU of the UAV to determine the ground speed; obtain the angular parameters on the flight path of the UAV using the angle measuring system of the airborne optoelectronic turret of the UAV; calculate the ground speed compensation value based on the ground speed and angular parameters.

[0012] S2. Install both the visible light camera and the infrared camera on the airborne optoelectronic turret of the UAV, capture visible light images and infrared images using the airborne optoelectronic turret of the UAV, and perform real-time registration and fusion through the image registration algorithm.

[0013] S3. Extract the local contrast and information entropy from the registered and fused images, dynamically adjust the fusion ratio of the visible light image and the infrared image through the fusion tracking method to generate a fusion tracking image; input the fusion tracking image into the correlation tracking algorithm for target tracking; offset the target search area according to the ground speed compensation parameter to ensure that the target is searched with the highest probability.

[0014] S4. The servo system inputs the ground speed compensation parameter into the speed closed-loop control loop and combines the miss distance of the correlation tracking to stably track the target.

[0015] Preferably, the angular parameters in step S1 include the azimuth angle α and the pitch angle β;

[0016] The specific calculation method of the ground speed compensation value specifically includes:

[0017] S101. Define the eastward speed of the UAV by the azimuth angle α and the northward speed ; Convert the eastward speed and northward speed of the drone into speed components relative to the flight path of the drone : :

[0018] ;

[0019] where α represents the azimuth angle on the flight path of the drone, represents the eastward speed, represents the northward speed;

[0020] S102. Adjust the speed components according to the pitch angle β on the flight path of the drone , and obtain the adjusted horizontal speed components. The calculation formula is as follows:

[0021] ;

[0022] where is the horizontal speed component calculated according to the eastward speed and northward speed , azimuth angle α and pitch angle β of the drone; represents the speed component; β represents the pitch angle on the flight path of the drone;

[0023] S103. Calculate the ground speed compensation value; the calculation formula is as follows:

[0024] ;

[0025] where represents the ground speed compensation value; is the proportional constant used to adjust the compensation intensity; represents the horizontal speed component.

[0026] Preferably, step S2 specifically includes the following sub-steps:

[0027] S201. Ground calibration: Measure and calibrate the visible light Y-direction field of view angle at each focal length of the visible light camera and the infrared Y-direction field of view angle at each focal length of the infrared camera on the ground, record the relationship between the field of view angle and the focal length as a file, and store it on the camera control board;

[0028] S202. Coarse matching of the field of view angle: The camera control board controls the movement of the visible light camera or the infrared camera through the PID algorithm, so that the field of view angle of a single pixel of the visible light image is equal to the field of view angle of a single pixel of the infrared image;

[0029] S203. Image descriptor extraction and registration: Extract the heterogeneous image descriptors of the visible light image and the infrared image, match the descriptors, and store the projection relationship of each pixel coordinate of the visible light image and the infrared image as matrix A; call matrix A to complete the registration.

[0030] Preferably, the heterogeneous image descriptors of the visible light image and the infrared image in step S203 include phase consistency descriptors and gradient descriptors; the method of weighted correlation distance is used for matching, and the specific method is as follows:

[0031] S2031. Calculate the phase consistency descriptor of the heterogeneous image, which is used to measure the phase of specific frequency components in the heterogeneous image; for each pixel, the phase can be calculated by the following formula:

[0032] ;

[0033] where is the pixel coordinate, w is different scales, is the phase at scale w, and M represents the number of sizes;

[0034] S2032. Calculate the gradient descriptor of the heterogeneous image;

[0035] The gradient direction descriptor of the heterogeneous image is as follows:

[0036] ; ;

[0037] where and are the gradients in the x - direction and y - direction at scale m respectively, and 𝐼 is the brightness of the image; thus, the gradient descriptor of the heterogeneous image is expressed as:

[0038]

[0039] ;

[0040] In the formula, represents the direction of the gradient, that is, it represents the direction of the brightness change of the image at the point (x,y);

[0041] S2033. Set the statistical scale as m, and the gradient histogram of 10×10 pixels around the pixel (x,y) is as follows:

[0042] ;

[0043] S2034. Based on the target size of the image, take M / 2 scales upward and downward respectively, for a total of M scales; the phase consistency descriptor at each scale , Gradient Descriptor , Histogram of Oriented Gradients constitute the descriptor vectors at M scales;

[0044] S2035. Matching is performed by using the weighted correlation distance method; the expression is as follows:

[0045] ;

[0046] In the formula, WCD represents the weighted correlation distance, that is, the weighted sum of the differences between the descriptor vectors of the visible light image and the infrared image from scale 1 to scale M; m is the scale of the descriptor; M represents the number of scales; and respectively represent the multi-scale descriptor vectors at the m-th scale in the visible light image and the infrared image; is the weight of the m-th scale, expressed as: ; is the descriptor vector of the m-th scale and of the variance;

[0047] When the WCD value is greater than 0.5, it is considered that and the matching is successful; the projection relationship of each pixel coordinate in the visible light image and the infrared image is stored as matrix A.

[0048] Preferably, the method for adjusting the field of view angle of a single pixel of the visible light image to be equal to the field of view angle of a single pixel of the infrared image in step S202 is as follows:

[0049] When the drone performs a mission, if the visible lens is the main one and the infrared is fused and tracked, the following steps are executed: taking the visible lens as the main lens, when the ground station sends the wide field of view and narrow field of view commands, only the visible lens responds; after the ground station control ends, the visible lens is in a stationary state. At this time, the focal length value g of the visible light is read CCD ; According to the focal length value g of the visible light CCD , the visible pixel size and the infrared pixel size, calculate the infrared focal length under the condition of the same field of view for a single pixel. The camera control board controls the repeated movement of the infrared camera through the PID algorithm until the visible light focal length is 1 / 6 of the infrared focal length;

[0050] When the drone performs a mission, if the infrared lens is the main one and the visible lens is fused and tracked, the following steps are executed: taking the infrared lens as the main lens, when the ground station sends the wide field of view and narrow field of view commands, only the infrared lens responds; after the ground station control ends, the infrared lens is in a stationary state. At this time, the focal length value g of the infrared lens is read IR ; According to the focal length value g of the infrared lens IR, according to the visible pixel size and the infrared pixel size, calculate the visible light focal length under the condition of the same field of view for a single pixel. The camera control board controls the repeated movement of the visible light camera through the PID algorithm until the visible light focal length is 1 / 6 of the infrared focal length.

[0051] Preferably, the fusion tracking method in step S3 specifically includes the following sub-steps:

[0052] S301. Calculate the local contrast of the visible light image and the infrared image; normalize the local contrast of the visible light image and the infrared image to obtain the weights of the local contrast of the visible light image and the infrared image;

[0053] S302. Calculate the information entropy of the visible light image and the infrared image, normalize the information entropy of the visible light image and the infrared image to obtain the weights of the information entropy of the visible light image and the infrared image;

[0054] S303. Convert the RGB format file of the visible light image to YUV, and make the Y-channel brightness value of the visible light image Y ccd be superimposed with the Y-channel brightness value of the infrared image Y ir, and perform fusion tracking pixel by pixel to generate a fusion tracking image Y fused , and the expression is as follows:

[0055] ;

[0056] In the formula, Y fused represents the fusion tracking image, Y ccd represents the Y-channel brightness value of the visible light image, Y ir is the Y-channel brightness value of the infrared image; represents the weight of the visible light image based on the local contrast, represents the weight of the infrared image based on the local contrast; represents the weight of the visible light image based on the information entropy, represents the weight of the infrared image based on the information entropy.

[0057] Preferably, step S301 specifically includes the following sub-steps:

[0058] S3011. Determine the local area centered on the feature, apply a mean filter to the local area to obtain the local average brightness, calculate the standard deviation of the deviation between the pixel values in the local area and the local average brightness, and take the maximum value of the standard deviation as the local contrast;

[0059] The local standard deviation calculation formulas for the visible light image and the infrared image are as follows:

[0060] ; ;

[0061] Wherein, is the pixel value of the visible image at the coordinate ( x , y ); is the pixel value of the infrared image at the coordinate ( x , y ); is the average pixel value of the local region Ω f in the visible light image or the infrared image, |Ω f | represents the total number of pixels in the local region Ω f in the visible light image or the infrared image;

[0062] The local contrast expression of the visible light image or the infrared image is as follows:

[0063] CCDLocalContrast = max( );

[0064] IRLocalContrast = max( );

[0065] S3012. Normalize the local contrast of the visible light image and the infrared image so that the sum of CCDLocalContrast and IRLocalContrast is 1; obtain the weights of the local contrast of the visible light image and the infrared image:

[0066] ;

[0067] Wherein, represents the weight of the visible light image based on the local contrast, represents the weight of the infrared image based on the local contrast;

[0068] The step S302 specifically includes the following sub-steps:

[0069] S3021. Use a histogram to represent the number of times the gray value i appears in the visible light image or the infrared image; for the gray value of each pixel point in the gray image or , perform the following operations:

[0070] ;

[0071] Wherein, and represent the visible histogram and the infrared histogram; i represents the gray value; represents the indicator function; when ; When ; represents the gray value of the pixel at coordinates (x, y) in the visible image, represents the gray value of the pixel at coordinates (x, y) in the infrared image;

[0072] S3022. Perform normalization processing to convert the histogram into a probability distribution by dividing the frequency of each gray value by the total number of pixels in the image:

[0073] ;

[0074] ;

[0075] In the formula, and are the probability distributions of each gray value of the visible light image and the infrared image respectively, and are the width and height of the visible light image respectively, and are the width and height of the infrared image respectively;

[0076] S3023. Use the Shannon information entropy formula to multiply the probability of each gray value by its logarithm to the base 2, and accumulate for all 256 possible gray values to calculate the visible light image information entropy H ccd and the infrared image information entropy H ir;

[0077] ;

[0078] ;

[0079] S3024. Normalize the information entropy of the visible light image and the infrared image so that H ccd and H ir sum to 1 to obtain the weights of the visible light image and the infrared image information entropy:

[0080] ;

[0081] In the formula, represents the weight of the visible light image based on information entropy, represents the weight of the infrared image based on information entropy.

[0082] Preferably, the correlation tracking algorithm in step S3 specifically includes the following steps:

[0083] S304. Target region initialization: in the fused tracking image, the initial region of the target is determined by the target detection algorithm, and the initial region is used as the starting point of the tracking algorithm;

[0084] S305. Template matching and correlation calculation: Use the initial area of ​​the target as a template and search for the target by calculating the correlation between the template and each possible position in the image; the correlation can be calculated by the following formula:

[0085] ;

[0086] in, R ( x , y ) is at the fused image position ( x , y ), T ( i , j ) is the pixel value of the fused template image, I ( x + i , y + j ) is the pixel value of the target fused image, μT and μI are the means of the template and target regions, respectively;

[0087] S306. Peak detection and target positioning: In the correlation graph, the new position of the target is determined by finding the local maximum; if the peak value is higher than the preset threshold, the target is considered to be successfully tracked at this position, and the target model is updated according to the tracking result.

[0088] Preferably, the method for performing target search area offset according to ground speed compensation parameters in step S3 specifically comprises the following steps:

[0089] S307. Compensate value based on ground speed C Calculate the ground speed of the drone in Δt time C Distance moved Δ P offset, the expression is:

[0090] S308. Determine the initial search area center as the target position currently observed by the drone P current ; Set the search area according to Δ P offset Move, the new search area center is:

[0091] ;

[0092] S309. Execute the relevant tracking algorithm within the new search area to search for the target with the highest probability and calculate the target miss distance Δ P error 。

[0093] Preferably, step S4 specifically includes the following sub-steps:

[0094] S401. The servo system includes two closed control loops: a speed loop and a tracking loop; the speed loop receives the ground speed compensation value C , forming a speed loop circuit with the ground speed compensation value;

[0095] ;

[0096] In the formula: Δ P speed_error represents the angular rate error of the speed loop, that is, the difference between the desired angular rate and the actual angular rate; C is the ground speed compensation value; Kp_ speed is the proportional gain of the speed loop; Ki_ speed is the integral gain of the speed loop; Δ P speed_correction is the output of the speed loop control quantity, used to complete the speed loop closed-loop;

[0097] S402. The error input of the tracking loop of the servo system is the miss distance Δ P error ;According to the miss distance Δ P error Adjust the pointing of the optoelectronic turret to reduce the deviation and achieve stable tracking:

[0098] ;

[0099] In the formula, Δ P correction is the output of the tracking loop control quantity, used to complete the tracking loop closed-loop to achieve precise tracking of the target, Δ P error represents the miss distance, Kp 、 Ki and Kd are respectively the proportional, integral, and differential gains of the tracking loop of the servo control system.

[0100] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0101] Introducing a ground speed compensation mechanism significantly improves the tracking stability and accuracy of the UAV for the target in a dynamic flight environment.

[0102] Based on the image registration algorithm of multi-scale descriptors and optimized search strategies, real-time registration and fusion of images are achieved, improving the real-time performance of target tracking.

[0103] The fusion tracking method combines local contrast and information entropy, dynamically adjusts the image fusion ratio, maximizes the information content of the fused image, and enhances the accuracy and robustness of target tracking.

[0104] It is applicable to image registration under different resolutions and variable focal length conditions, reduces the workload of registration calibration, and improves the efficiency of image registration.

[0105] In summary, the real-time registration and dynamic fusion of visible and infrared images in the present invention result in a fused image with rich features, which can effectively extract target features. The target search area is offset based on the ground speed compensation parameter, effectively improving the reliability of the target tracking algorithm. The servo system inputs the ground speed compensation parameter into the speed closed-loop control loop and combines the miss distance to stably track the target, effectively improving the tracking performance of the UAV for ground targets in a dynamic flight environment. It is applicable to the registration of visible and infrared images with different resolutions and when the visible and infrared cameras are variable focal length lenses, especially applicable to fields such as image registration between a variable focal length visible light camera and a variable focal length infrared camera on an airborne optoelectronic turret of a UAV. It has broad application prospects and practical application value, and significantly reduces the workload of registration calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1 It is a flowchart of a method for stable target tracking combined with ground speed compensation provided according to an embodiment of the present invention.

[0107] Figure 2 It is a visible light image taken by a UAV according to an embodiment of the present invention.

[0108] Figure 3 It is an infrared image taken by a UAV according to an embodiment of the present invention.

[0109] Figure 4 It is a fused tracking image generated after the fusion of a visible light image and an infrared image on a UAV according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0110] In the following, embodiments of the present invention will be described with reference to the drawings. In the following description, the same modules are denoted by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, their detailed descriptions will not be repeated.

[0111] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0112] See Figure 1, the present invention provides a method for target stable tracking combined with ground speed compensation, which specifically includes the following steps:

[0113] S1. Measure the flight speed and flight direction of the UAV through the on-board IMU of the UAV to determine the ground speed; obtain the angular parameters on the flight path of the UAV using the angle measurement system of the on-board optoelectronic turret of the UAV, and the angular parameters include the azimuth angle α and the pitch angle β; calculate the ground speed compensation value according to the ground speed and the angular parameters.

[0114] Ground speed compensation refers to compensating for the movement of the UAV relative to the ground, that is, converting the celestial, eastward, and northward speeds into projections on the ground plane; the calculation method of the ground speed compensation value specifically includes:

[0115] The UAV has a celestial speed 、eastward speed and northward speed , and α is the angle between the azimuth angle of the optoelectronic turret and the flight path of the UAV, β is the pitch angle of the optoelectronic turret, and the ground speed compensation value is calculated according to the above speed components and angles.

[0116] S101. Convert the eastward speed and northward speed into speed components relative to the flight path of the UAV (defined by the azimuth angle α):

[0117] ;

[0118] In the formula, α represents the azimuth angle on the flight path of the UAV, represents the eastward speed, represents the northward speed;

[0119] S102. Adjust the speed component according to the pitch angle β on the flight path of the UAV to consider the tilt of the UAV relative to the horizontal plane, and obtain the adjusted horizontal speed component. The calculation formula is as follows:

[0120] ;

[0121] In the formula, is the horizontal speed component calculated according to the eastward speed and northward speed of the UAV, the azimuth angle α and the pitch angle β of the optoelectronic turret; represents the speed component; β represents the pitch angle on the flight path of the UAV;

[0122] S103. Calculate the ground speed compensation value; the calculation formula is as follows:

[0123] ;

[0124] In the formula, represents the ground speed compensation value; is a proportionality constant used to adjust the compensation intensity; represents the horizontal speed component.

[0125] Step S1 further includes: inputting the ground speed compensation value into a relevant tracking algorithm to predict the dynamic position change of the target.

[0126] S2. Install both the visible light camera and the infrared camera on the airborne optoelectronic turret of the UAV, and use the airborne optoelectronic turret of the UAV to capture visible light images and infrared images (see Figure 2 and Figure 3 ), and perform real-time registration and fusion through an image registration algorithm; specifically, it includes the following sub-steps:

[0127] S201. Ground calibration: Measure and calibrate the visible light Y-direction field of view angle of the visible light camera at each focal length and the infrared Y-direction field of view angle of the infrared camera at each focal length on the ground, record the relationship between the field of view angle and the focal length as a file, and store it on the camera control board;

[0128] The measurement and calibration method specifically includes: using a folding visible-infrared dual-light co-visual field light pipe (the light pipe contains a crosshair, and the stepping length is 0.1 m) to gradually measure the visible light Y-direction field of view angle of the visible light camera and the infrared Y-direction field of view angle of the infrared camera;

[0129] Specifically, the inside of the optoelectronic turret is equipped with a measured azimuth and elevation encoder; assume that the focal length of the visible light camera lens at this time is CCD_F, the elevation angle of the optoelectronic turret is locked at 0, align the upper edge of the visible light camera with the crosshair of the light pipe, and read the azimuth angle CCDup of the optoelectronic turret at this time; rotate the optoelectronic turret so that the lower edge of the visible light camera aligns with the crosshair of the light pipe, and read the azimuth angle CCDdown of the optoelectronic turret at this time. Then, at the visible light camera focal length CCD_F, the visible light Y-direction field of view angle is CCD_Y = CCDup - CCDdown; record the corresponding relationship between the visible light lens focal length and the field of view angle into the array c; similarly, measure the infrared camera's Y-direction field of view angle, IR_Y = IRup - IRdown; record the corresponding relationship between the infrared lens focal length and the field of view angle into the array d, and store it in the airborne optoelectronic turret recorder of the UAV.

[0130] S202. Coarse matching of the field of view angle: The camera control board controls the movement of the visible light camera or the infrared camera through the PID algorithm to make the field of view angle of a single pixel of the visible light image equal to the field of view angle of a single pixel of the infrared image; the specific operation is as follows:

[0131] If the field of view angle of each pixel of the visible light image and the infrared image is to be the same, the derivation of the focal length relationship between the visible light lens and the infrared lens is as follows:

[0132] Visible pixel size: ; Infrared pixel size: ; The focal length of the visible light lens is fvi ; The focal length of the infrared lens is fir ;

[0133] The angular resolution θ can be expressed by the following formula: θ = pixel size / focal length;

[0134] The angular resolution of the visible light image is θ vis = dvis / fvis ; The angular resolution of the infrared image is ;

[0135] To make θ vis equal to θ ir , the following equation can be established: ; ; That is, in the case of a visible pixel size of 2.5 μm and an infrared pixel size of 15 μm, that is, when the visible focal length is 1 / 6 of the infrared focal length, the field of view angle of a single pixel of the visible light image is equal to the field of view angle of a single pixel of the infrared image.

[0136] The visible light camera and the infrared camera adopt an external trigger method to ensure that the visible light and the infrared are exposed at the same time.

[0137] Adjust the field of view angle of a single pixel of the visible light image to be equal to the field of view angle of a single pixel of the infrared image. The method of adjusting the visible focal length to be 1 / 6 of the infrared focal length is as follows:

[0138] When the UAV performs a mission, if the visible light lens is the main one and fuses and tracks the infrared, the following steps are executed: Taking the visible light lens as the main lens, when the ground station sends the large field of view and small field of view commands, only the visible light lens responds; after the ground station control ends, the visible light lens is in a stationary state. At this time, read out the focal length value g of the visible light CCD ; According to the focal length value g of the visible light CCD , the visible pixel size and the infrared pixel size, calculate the infrared focal length under the condition of the same field of view of a single pixel. The camera control board controls the repeated movement of the infrared camera through the PID algorithm until the visible focal length is 1 / 6 of the infrared focal length;

[0139] When the UAV performs a mission, if the infrared lens is the main one and fuses and tracks the visible light lens, the following steps are executed: Taking the infrared lens as the main lens, when the ground station sends the large field of view and small field of view commands, only the infrared lens responds; after the ground station control ends, the infrared lens is in a stationary state. At this time, read out the focal length value g of the infrared lens IR ; According to the focal length value g of the infrared lens IR, calculate the visible light focal length under the condition of the same field of view for a single pixel according to the visible pixel size and the infrared pixel size. The camera control board controls the repeated movement of the visible light camera through the PID algorithm until the visible light focal length is 1 / 6 of the infrared focal length.

[0140] S203. Image descriptor extraction and registration: Extract the heterogeneous image descriptors of the visible light image and the infrared image, match the descriptors, and store the projection relationship of each pixel coordinate of the visible light image and the infrared image as matrix A; call matrix A to complete the registration.

[0141] The heterogeneous image descriptors of the visible light image and the infrared image include the phase consistency descriptor and the gradient descriptor; the method of matching the descriptors is to combine the phase consistency descriptor and the gradient descriptor, extract the common feature descriptors of the visible light image and the infrared image, and use the method of weighted correlation distance (WCD) for matching; the specific method is as follows:

[0142] S2031. Calculate the phase consistency descriptor of the heterogeneous image to measure the phase of specific frequency components in the heterogeneous image; for each pixel, the phase can be calculated by the following formula:

[0143] ;

[0144] where is the pixel coordinate, w is different scales, is the phase at scale w, and M represents the number of sizes (usually taking the value of 10);

[0145] S2032. Calculate the gradient descriptor of the heterogeneous image;

[0146] The gradient direction descriptor of the heterogeneous image is as follows:

[0147] ; ;

[0148] where and are the gradients in the x - direction and y - direction at scale m respectively, and 𝐼 is the brightness of the image; thus, the gradient descriptor of the heterogeneous image is expressed as:

[0149]

[0150] ;

[0151] In the formula, represents the direction of the gradient, that is, it represents the direction of the brightness change of the image at point (x,y);

[0152] S2033. Set the statistical scale to m. The gradient histogram of 10×10 pixels surrounding the (x, y) pixel is represented as follows:

[0153] ;

[0154] S2034. Based on the image target size, take M / 2 scales upward and M / 2 scales downward, for a total of M scales. The phase consistency descriptor at the m-th scale , the gradient descriptor (gradient magnitude) , and the gradient histogram constitute the descriptor vector at the m-th scale :

[0155] ;

[0156] Both the visible light image and the infrared image are operated according to the above method, and a rich multi-scale descriptor vector is generated for each feature point in the visible light image and the infrared image and . This descriptor effectively describes the local structural information of the visible image and the infrared image and provides a solid foundation for feature matching.

[0157] S2035. Use the method of weighted correlation distance (WCD) for matching; the expression is as follows:

[0158] ;

[0159] In the formula, WCD represents the weighted correlation distance, that is, the weighted sum of the differences between the descriptor vectors of the visible light image and the infrared image from scale 1 to scale M; m is the scale of the descriptor (the value range of m is from 1 to M); M represents the number of sizes (usually taken as 10); and respectively represent the multi-scale descriptor vectors at the m-th scale in the visible light image and the infrared image; is the weight of the m-th scale, expressed as: ; is the variance of the descriptor vector of the m-th scale and ;

[0160] When the WCD value is greater than 0.5, it is considered that and are successfully matched; the projection relationship of each pixel coordinate in the visible light image and the infrared image is stored as matrix A.

[0161] Principle: After step S202, the visible and infrared fields of view are basically consistent in the Y direction in theory. However, due to the field of view angle calibration error in step S1 and the focus control error in step S202, there are still slight differences in the visible and infrared fields of view, so registration is required. The calculation of the common feature descriptors of visible light images and infrared images is a key step in the matching of visible light images and infrared images. It provides a unique vector representation for each feature point of the visible light image and the infrared image.

[0162] The weighted correlation distance (WCD) method not only considers the Euclidean distance between feature vectors, but also the correlation between feature dimensions. This method is particularly suitable for situations where there are intrinsic connections between feature dimensions.

[0163] Because visible infrared uses external triggering to ensure that visible infrared is exposed at the same time, after exposure, a new frame of visible image and infrared image arrive at the onboard computer. Only by calling matrix A can the real-time registration of visible infrared be completed, so that the real-time performance of the registration is effectively guaranteed.

[0164] S3. Extract local contrast and information entropy based on the registered fused image, and dynamically adjust the fusion ratio of the visible light image and the infrared image through the fusion tracking method to generate a fused tracking image ( Figure 4 ); Input the fused tracking image into the relevant tracking algorithm to track the target; offset the target search area according to the ground speed compensation parameters to ensure that the target is searched with the maximum probability;

[0165] The fusion tracking method specifically includes:

[0166] S301. Calculate the local contrast of the visible light image and the infrared image; normalize the local contrast of the visible light image and the infrared image to obtain the weight of the local contrast of the visible light image and the infrared image; specifically include the following sub-steps:

[0167] S3011. Determine a local area centered on the feature, apply a mean filter to the local area to obtain the local average brightness, calculate the standard deviation of the deviation between the pixel values ​​in the local area and the local average brightness, and take the maximum value of the standard deviation as the local contrast.

[0168] Local standard deviation It is a local area The measure of the internal pixel value fluctuation, the local standard deviation calculation formulas for visible light images and infrared images are as follows:

[0169] ; ;

[0170] In the formula, is the pixel value of the visible image at the coordinate ( x , y ); is the pixel value of the infrared image at the coordinate ( x , y ); is the average pixel value of the local region Ω f in the visible light image or the infrared image, |Ω f | represents the total number of pixels in the local region Ω f in the visible light image or the infrared image.

[0171] The local standard deviation is calculated by computing the sum of the squared deviations of the pixel values within the local region Ω f from its average value μf , and then taking the square root. This value reflects the degree of dispersion of the pixel intensities within the feature region and is an indicator of the local texture complexity.

[0172] The local contrast of the visible light image or the infrared image is defined as the maximum value of its local standard deviation:

[0173] CCDLocalContrast = max( );

[0174] IRLocalContrast = max( );

[0175] S3012. Normalize the local contrasts of the visible light image and the infrared image so that the sum of CCDLocalContrast and IRLocalContrast is 1; obtain the weights of the local contrasts of the visible light image and the infrared image:

[0176] ;

[0177] In the formula, represents the weight of the visible light image based on the local contrast, represents the weight of the infrared image based on the local contrast.

[0178] S302. Calculate the information entropy of the visible light image and the information entropy of the infrared image, normalize the information entropies of the visible light image and the infrared image, and obtain the weights of the information entropies of the visible light image and the infrared image; specifically, it includes the following sub-steps:

[0179] S3021. Use a histogram to represent the number of occurrences of the gray value i in the visible light image or the infrared image; for the gray value or of each pixel point in the gray image, perform the following operations:

[0180] ;

[0181] In the formula, and represent the visible histogram and the infrared histogram; i represents the gray value; represents the indicator function; when ; when ; represents the gray value of the pixel at coordinates (x, y) in the visible image, represents the gray value of the pixel at coordinates (x, y) in the infrared image;

[0182] S3022. Perform normalization processing to convert the histogram into a probability distribution by dividing the frequency of each gray value by the total number of pixels in the image:

[0183] ;

[0184] ;

[0185] In the formula, and are the probability distributions of each gray value of the visible light image and the infrared image respectively, and are the width and height of the visible light image respectively, and are the width and height of the infrared image respectively.

[0186] S3023. Use the Shannon information entropy formula, multiply the probability of each gray value by its logarithm to the base 2, and accumulate over all 256 possible gray values to calculate the visible light image information entropy H ccd and the infrared image information entropy H ir;

[0187] ;

[0188] ;

[0189] The range of the information entropy is from 0 to the maximum value of 8 bits, and this range reflects the amount of information in the image from completely disordered (the gray value of each pixel is random and has equal probability) to completely ordered (all pixels have the same gray value); when all pixels have the same gray value (i.e., the image is completely uniform), H ccd = 0, H ir = 0; when the probability of each gray value appearing is equal, the information entropy H ccd = 8, H ir = 8, reaching the maximum value.

[0190] S3024. Normalize the information entropy of the visible light image and the infrared image so that H ccd and H the sum of ir is 1 to obtain the weights of the information entropy of the visible light image and the infrared image:

[0191] ;

[0192] In the formula, represents the weight of the visible light image based on the information entropy, represents the weight of the infrared image based on the information entropy.

[0193] S303. Convert the RGB format file of the visible light image to YUV, and make the Y-channel brightness value of the visible light image Y ccd and the Y-channel brightness value of the infrared image Y ir are superimposed, and fusion tracking is performed pixel by pixel to generate a fusion tracking image Y fused , and the expression is as follows:

[0194] ;

[0195] In the formula, Y fused represents the fusion tracking image, Y ccd represents the Y-channel brightness value of the visible light image, Y ir is the Y-channel brightness value of the infrared image; represents the weight of the visible light image based on the local contrast, represents the weight of the infrared image based on the local contrast; represents the weight of the visible light image based on the information entropy, represents the weight of the infrared image based on the information entropy.

[0196] In step S3, the correlation tracking algorithm specifically includes the following steps:

[0197] S304. Target area initialization: In the fusion tracking image, determine the initial area of the target through the target detection algorithm, and use this initial area as the starting point of the tracking algorithm;

[0198] S305. Template matching and correlation calculation: Use the initial area of the target as the template, and search for the target by calculating the correlation between the template and each possible position in the image; the correlation can be calculated by the following formula:

[0199] ;

[0200] Among them, R ( x ,y ) is the correlation score at the fused image position ( x , y ), T ( i , j ) are the pixel values of the fused template image, I ( x + i , y + j ) are the pixel values of the target fused image, μT and μI are the means of the template and target regions respectively.

[0201] Due to the limited computing power of the embedded airborne computer, it is impossible to perform a search of the entire frame for all regions. The traditional method is to perform a search of a limited region near the position where the target appeared in the previous frame.

[0202] S306. Peak detection and target localization: In the correlation map, determine the new position of the target by finding the local maximum; if the peak is higher than the preset threshold, it is considered that the target has been successfully tracked at this position, and update the target model according to the tracking result, including the shape, size, and appearance features of the target.

[0203] In step S3, the method of offsetting the target search area according to the ground speed compensation parameters specifically includes the following steps:

[0204] S307. Calculate the distance Δ C offset that the UAV moves at the ground speed C within the time Δt according to the ground speed compensation value, and the expression is: P ; ;

[0205] S308. Determine the center of the initial search area as the target position currently observed by the UAV P current ; Move the search area according to Δ P offset , and the center of the new search area is:

[0206] ;

[0207] S309. Execute the relevant tracking algorithm within the new search area to search for the target with the highest probability and calculate the target miss distance Δ P error .

[0208] S4. The servo system inputs the ground speed compensation parameters into the speed closed-loop control loop and combines the miss distance of the relevant tracking to stably track the target; specifically includes the following sub-steps:

[0209] S401. The servo system includes two closed control loops: a speed loop and a tracking loop; the speed loop receives the ground speed compensation value C , forming a speed loop circuit with the ground speed compensation value;

[0210] ;

[0211] Where: Δ P speed_error represents the angular rate error of the speed loop, that is, the difference between the desired angular rate and the actual angular rate. This difference is measured by the angular rate gyroscope built into the optoelectronic turret. The angular rate gyroscope can only measure the change in the angular velocity of the optoelectronic turret and cannot measure the linear velocity change caused by the flight of the UAV; C is the ground speed compensation value, that is, the linear velocity change caused by the flight of the UAV; Δ P speed_error + C represents the angular velocity of the optoelectronic turret + the linear velocity caused by the flight of the UAV, which can comprehensively reflect the speed loop error of the optoelectronic turret during actual flight; by adding the ground speed compensation value C to the error term Δ P speed_error, it can ensure that the speed loop takes into account the linear velocity caused by the flight of the UAV when calculating the correction amount, thereby improving the accuracy of the speed loop response; Kp_ speed is the proportional gain of the speed loop; Ki_ speed is the integral gain of the speed loop; Δ P speed_correction is the output of the speed loop control quantity, which is used to complete the speed loop closed-loop.

[0212] S402. The error input of the tracking loop of the servo system is the miss distance Δ P error ; According to the miss distance Δ P error adjust the pointing of the optoelectronic turret to reduce the deviation and achieve stable tracking:

[0213] ;

[0214] Where, Δ P correction is the output of the tracking loop control quantity, which is used to complete the tracking loop closed-loop to achieve precise tracking of the target, Δ P error represents the miss distance, Kp , Ki and Kd are respectively the proportional, integral and differential gains of the tracking loop of the servo control system.

[0215] The core of the present invention lies in introducing a ground speed compensation mechanism. By calculating the ground speed of the unmanned aerial vehicle (UAV) in real time and using it as an input parameter for the tracking algorithm, the dynamic changes of the target can be predicted to achieve stable tracking of the target. First, the flight speed and direction of the UAV are obtained through its navigation system, and then the ground speed vector is calculated based on the azimuth and elevation angles of the airborne optoelectronic turret. This ground speed vector is then used to adjust the relevant tracking algorithm to ensure that during the flight of the UAV, the target tracking system can accurately predict the changes in the target position.

[0216] The present invention also includes an image registration algorithm based on multi-scale descriptors and an optimized search strategy. This algorithm can run in real time on the airborne computer to further improve the accuracy of target tracking. In addition, a fusion tracking method is provided. This method combines local contrast and information entropy to dynamically adjust the fusion ratio of visible light images and infrared images to maximize the information content of the fused images.

[0217] The advantages of the present invention are that it is not only applicable to image registration under different resolutions and variable focal lengths, but also significantly reduces the workload of registration calibration and improves the efficiency of image registration. Through the fusion tracking technology, the accuracy and robustness of target tracking are enhanced, meeting the requirements of real-time image processing of UAVs, and having broad application prospects and practical application values.

[0218] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the disclosure of the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved. No limitations are imposed herein.

[0219] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for stable target tracking combined with ground speed compensation, characterized in that: The specific steps include: S1. The flight speed and flight direction of the UAV are measured by the UAV-mounted IMU to determine the ground speed; the angle parameters on the flight path of the UAV are obtained by using the angle measurement system of the UAV-mounted optoelectronic turret; the ground speed compensation value is calculated according to the ground speed and angle parameters; the angle parameters include the azimuth angle α and the pitch angle β; the calculation method of the ground speed compensation value specifically includes: S101. Define the eastward speed of the drone by the azimuth angle α and northbound speed ; Set the drone's eastward speed and northbound speed Converted to velocity component relative to the UAV flight path : ； In the formula, α represents the azimuth angle of the UAV flight path, represents the eastward speed, represents the north speed; S102. Adjust the velocity component according to the pitch angle β on the UAV flight path , and the adjusted horizontal velocity component is obtained. The calculation formula is as follows: ； In the formula, Based on the eastward speed of the drone and northbound speed , the horizontal velocity component calculated from the azimuth angle α and the pitch angle β; represents the velocity component; β represents the pitch angle on the flight path of the UAV; S103. Calculate the ground speed compensation value; the calculation formula is as follows: ； In the formula, Indicates the ground speed compensation value; is a proportionality constant used to adjust the compensation intensity; represents the horizontal velocity component; S2. Install both the visible light camera and the infrared camera on the optoelectronic turret on the drone, use the optoelectronic turret on the drone to capture visible light images and infrared images, and perform real-time registration and fusion through image registration algorithm; S3. Extract local contrast and information entropy based on the registered fused image, and dynamically adjust the fusion ratio of the visible light image and the infrared image through the fusion tracking method to generate a fused tracking image; input the fused tracking image into the relevant tracking algorithm to track the target; offset the target search area based on the parameters of the ground speed compensation to ensure that the target is searched with the maximum probability; S4. The servo system inputs the ground speed compensation parameter into the speed closed loop control loop, and combines the miss distance of the relevant tracking to stably track the target; the servo system contains two levels of closed control loops: speed loop and tracking loop; the speed loop receives the ground speed compensation value C , forming a speed loop including ground speed compensation value; ； Where: Δ P speed_error represents the angular rate error of the speed loop, that is, the difference between the expected angular rate and the actual angular rate; C is the ground speed compensation value; Kp_ speed is the proportional gain of the speed loop; Ki_ speed is the integral gain of the speed loop; Δ P speed_correction is the speed loop control output, which is used to complete the speed loop closed loop.

2. The method for stable target tracking combined with ground speed compensation according to claim 1, characterized in that: The step S2 specifically includes the following sub-steps: S201. Ground calibration: calibrate the visible light Y-direction field angle of the visible light camera at each focal length and the infrared Y-direction field angle of the infrared camera at each focal length on the ground, record the relationship between the field angle and the focal length as a file, and store it on the camera control board; S202. Coarse matching of field of view angle: The camera control board controls the movement of the visible light camera or the infrared camera through the PID algorithm so that the field of view angle of a single pixel of the visible light image is equal to the field of view angle of a single pixel of the infrared image; S203. Image descriptor extraction and registration: extract heterogeneous image descriptors of the visible light image and the infrared image, match the descriptors, and store the projection relationship of each pixel coordinate of the visible light image and the infrared image as a matrix A; Call matrix A to complete the registration.

3. The method for stable target tracking combined with ground speed compensation according to claim 2, characterized in that: In step S203, the heterogeneous image descriptors of the visible light image and the infrared image include a phase consistency descriptor and a gradient descriptor; the weighted correlation distance method is used for matching, and the specific method is as follows: S2031. Calculate the phase consistency descriptor of the heterogeneous image, which is used to measure the phase of a specific frequency component in the heterogeneous image; for each pixel, the phase can be calculated by the following formula: ； in, are pixel coordinates, w are different scales, is the phase at scale w, M represents the number of sizes; S2032. Calculate the gradient descriptor of the heterogeneous image; The gradient direction descriptor of the heterogeneous image is as follows: ； ； in, and are the gradients in the x and y directions when the scale is m, respectively, and 𝐼 is the brightness of the image; thus, the gradient descriptor of the heterogeneous image is expressed as: ； In the formula, Indicates the direction of the gradient, that is, the direction of the brightness change of the image at the point (x, y); S2033. Set the statistical scale to m, and take the pixel (x, y) as the center, and the gradient histogram of the surrounding 10×10 pixels is expressed as follows: ； S2034. Taking the image target size as the benchmark, take M / 2 scales upward and downward respectively, for a total of M scales; the phase consistency descriptor at each scale , gradient descriptor , gradient histogram It forms a descriptor vector at M scales; S2035. Use the weighted correlation distance method for matching; the expression is as follows: ； Where WCD represents the weighted correlation distance, which is the weighted sum of the differences between the descriptor vectors of the visible light image and the infrared image from scale 1 to scale M; m is the scale of the descriptor; M represents the number of sizes; and Represent the multi-scale descriptor vector at the mth scale in the visible light image and infrared image respectively; is the weight of the mth scale, expressed as: ; is the descriptor vector of the mth scale and The variance of When the WCD value is greater than 0.5, it is considered and The match is successful; the projection relationship between the coordinates of each pixel in the visible light image and the infrared image is stored as matrix A.

4. The method for stable target tracking combined with ground speed compensation according to claim 2, characterized in that: The method for adjusting the field of view angle of a single pixel of the visible light image to be equal to the field of view angle of a single pixel of the infrared image in step S202 is as follows: When the UAV performs a mission, if the visible lens is used as the main lens and the infrared is integrated for tracking, the following steps are performed: when the visible lens is used as the main lens and the ground station sends a large field of view or small field of view command, only the visible lens responds; after the ground station control ends, the visible lens is in a stationary state. At this time, the focal length value g of the visible light is read out. CCD ; According to the focal length of visible light g CCD , visible pixel size and infrared pixel size, calculate the infrared focal length of a single pixel under the same field of view, and the camera control board controls the repeated movement of the infrared camera through the PID algorithm until the visible light focal length is 1 / 6 of the infrared focal length; When the UAV is performing a mission, if the infrared lens is the main lens and the visible lens is integrated and tracked, the following steps are performed: when the infrared lens is the main lens and the ground station sends a large field of view or small field of view command, only the infrared lens responds; after the ground station control ends, the infrared lens is in a static state. At this time, the focal length value g of the infrared lens is read out. IR ; According to the focal length value g of the infrared lens IR , according to the visible pixel size and the infrared pixel size, the visible light focal length of a single pixel under the same field of view is calculated. The camera control board controls the repeated movement of the visible light camera through the PID algorithm until the visible light focal length is 1 / 6 of the infrared focal length.

5. The method for stable target tracking combined with ground speed compensation according to claim 1, characterized in that: The fusion tracking method in step S3 specifically includes the following sub-steps: S301. Calculate the local contrast of the visible light image and the infrared image; normalize the local contrast of the visible light image and the infrared image to obtain the weight of the local contrast of the visible light image and the infrared image; S302. Calculate the information entropy of the visible light image and the information entropy of the infrared image, normalize the information entropy of the visible light image and the infrared image, and obtain the weights of the information entropy of the visible light image and the infrared image; S303. Convert the RGB format file of the visible light image to YUV, and make the Y channel brightness value of the visible light image Y CCD and infrared image Y channel brightness value Y IR superposition, pixel-by-pixel fusion tracking, and generation of fusion tracking images Y fused , the expression is as follows: ； In the formula, Y fused represents the fused tracking image, Y CCD represents the Y channel brightness value of the visible light image. Y ir is the Y channel brightness value of the infrared image; represents the visible light image weight based on local contrast, represents the infrared image weight based on local contrast; represents the weight of the visible light image based on information entropy, Represents the infrared image weight based on information entropy.

6. The method for stable target tracking combined with ground speed compensation according to claim 5, characterized in that: The step S301 specifically includes the following sub-steps: S3011. Determine a local area centered on the feature, apply a mean filter to the local area to obtain the local average brightness, calculate the standard deviation of the deviation between the pixel value in the local area and the local average brightness, and take the maximum value of the standard deviation as the local contrast; The calculation formulas for the local standard deviation of visible light images and infrared images are as follows: ； ； In the formula, is the visible image at coordinates ( x , y ), is the infrared image at coordinates ( x , y ), is the local area Ω in the visible light image or infrared image f The average pixel value, |Ω f ∣ represents the local area Ω in the visible light image or infrared image f The total number of pixels; The local contrast expression of a visible light image or infrared image is as follows: CCDLocalContrast=max( ); IRLocalContrast=max( ); S3012. Normalize the local contrast of the visible light image and the infrared image so that the sum of CCDLocalContrast and IRLocalContrast is 1; obtain the weight of the local contrast of the visible light image and the infrared image: ； In the formula, represents the visible light image weight based on local contrast, represents the infrared image weight based on local contrast; The step S302 specifically includes the following sub-steps: S3021. Use a histogram to represent the number of times the gray value i appears in the visible light image or infrared image; for each pixel in the gray image, the gray value or , do the following: ； In the formula, and Represents visible histogram and infrared histogram; i represents grayscale value; represents the indicator function; when ;when ; Represents the grayscale value of the pixel with coordinates (x, y) in the visible image. Represents the grayscale value of the pixel with coordinates (x, y) in the infrared image; S3022. Perform normalization to convert the histogram into a probability distribution by dividing the frequency of each gray value by the total number of pixels in the image: ； ； In the formula, and are the probability distribution of each gray value of the visible light image and infrared image respectively, and are the width and height of the visible light image, respectively. and are the width and height of the infrared image respectively; S3023. Using the Shannon information entropy formula, multiply the probability of each gray value by its logarithm with base 2, and accumulate all 256 possible gray values ​​to calculate the information entropy of the visible light image. H CCD and infrared image information entropy H ir; ； ； S3024. Normalize the information entropy of the visible light image and the infrared image so that H CCD and H ir is 1, and the weights of the information entropy of the visible light image and the infrared image are obtained: ； In the formula, represents the weight of the visible light image based on information entropy, Represents the infrared image weight based on information entropy.

7. The method for stable target tracking combined with ground speed compensation according to claim 6, characterized in that: The correlation tracking algorithm in step S3 specifically includes the following steps: S304. Target region initialization: in the fused tracking image, the initial region of the target is determined by the target detection algorithm, and the initial region is used as the starting point of the tracking algorithm; S305. Template matching and correlation calculation: Use the initial area of ​​the target as a template and search for the target by calculating the correlation between the template and each possible position in the image; the correlation can be calculated by the following formula: ； in, R ( x , y ) is at the fused image position ( x , y ), T ( i , j ) is the pixel value of the fused template image, I ( x + i , y + j ) is the pixel value of the target fused image, μT and μI are the means of the template and target regions, respectively; S306. Peak detection and target positioning: In the correlation graph, the new position of the target is determined by finding the local maximum; if the peak value is higher than the preset threshold, the target is considered to be successfully tracked at this position, and the target model is updated according to the tracking result.

8. The method for stable target tracking combined with ground speed compensation according to claim 7, characterized in that: The method for performing target search area offset according to the ground speed compensation parameters in step S3 specifically comprises the following steps: S307. Compensate value based on ground speed C Calculate the ground speed of the drone in Δt time C Distance moved Δ P offset, the expression is: S308. Determine the initial search area center as the target position currently observed by the drone P current ; Set the search area according to Δ P offset Move, the new search area center is: ； S309. Execute the correlation tracking algorithm in the new search area to search for the target with the maximum probability and calculate the target miss distance Δ P error .

9. The method for stable target tracking combined with ground speed compensation according to claim 1, characterized in that: The step S4 further comprises the following sub-steps: The error input of the tracking loop of the servo system is the miss distance Δ P error ; According to the off-target amount Δ P error Adjust the pointing direction of the optoelectronic turret to reduce deviation and achieve stable tracking: ； In the formula, Δ P correction is the tracking loop control output, which is used to complete the tracking loop closure to achieve accurate tracking of the target. P error represents the off-target amount, Kp , Ki and Kd They are the proportional, integral and differential gains of the tracking loop of the servo control system.

Images