Infrared-based ground-to-air unmanned aerial vehicle detection and tracking method

Abstract

The invention relates to the technical field of photoelectric detection and computer vision, and particularly discloses an infrared-based ground-to-air unmanned aerial vehicle detection and tracking method, which comprises the following steps: acquiring an infrared image sequence of a target airspace, dynamically adjusting the integral time and gain parameters of an infrared image by adopting a self-adaptive gain control algorithm based on a scene temperature histogram, and obtaining a target image sequence; generating an infrared image with enhanced thermal radiation characteristics; carrying out multi-level feature fusion detection on the infrared image by adopting a YOLOv5 improved multi-level global feature fusion network, and generating a saliency map containing the thermal features of the unmanned aerial vehicle rotor; according to the invention, by combining the infrared imaging technology, the improved YOLOv5 multi-level global feature fusion network and the improved BotSort algorithm, the detection precision and tracking stability of the low-altitude small unmanned aerial vehicle are effectively improved; the image quality is optimized through a self-adaptive gain control algorithm, the thermal radiation characteristic of the unmanned aerial vehicle is remarkably enhanced, and the failure problem caused by environmental interference in the traditional technology is avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of optoelectronic detection and computer vision, and particularly relates to a ground-to-air UAV detection and tracking method based on infrared, which is applicable to all-weather monitoring of UAV targets in scenarios such as low-altitude security, border monitoring, and airport protection. Background Art

[0002] Existing UAV detection technologies mainly rely on radar, visible light imaging, or radio frequency signals, and have the following problems:

[0003] Radar system: The detection accuracy for low-altitude small UAVs is insufficient. This is mainly because the radar cross-section (RCS) of UAVs is small (<0.01m 2 ) and the flight altitude is low (<300 meters), making it easy to be interfered by ground clutter and multipath effects, resulting in a high false alarm rate; the cost is high, and large-scale flexible deployment is difficult.

[0004] Visible light imaging: It depends on natural lighting conditions. The signal-to-noise ratio (SNR) drops by more than 50% at night or in a haze environment, and it fails when the target resolution is lower than 10 pixels × 10 pixels; the dynamic target tracking is affected by the jitter of the pan-tilt head, and the trajectory deviation error can reach ±5°.

[0005] Radio frequency monitoring: It depends on the communication signal between the UAV and the control terminal (such as the 2.4GHz / 5.8GHz frequency band), and cannot identify pre-programmed or inertial navigation autonomous UAVs; electromagnetic interference sources (such as Wi-Fi, Bluetooth) in the urban environment cause signal confusion, and the positioning error exceeds 500; it overlaps with the civilian communication frequency band, which is prone to false killings.

[0006] In addition, traditional target detection algorithms face problems such as weak small target features and poor dynamic tracking stability in infrared images. The above problems make it difficult for the existing technologies to meet the core requirements of all-weather monitoring, high-precision identification, and dynamic stable tracking. For example, in the airport scenario, the radar false alarm wrongly triggers the prevention and control system, while the visible light imaging fails at night, forming a security loophole; in the complex electromagnetic environment at the border, the radio frequency monitoring is prone to losing the target; the traditional infrared algorithm misses the threat of micro UAVs (such as the DJI Mavic series).

[0007] Therefore, it is necessary to propose a ground-to-air UAV detection and tracking method based on infrared to solve the problem of insufficient detection accuracy of low-altitude small UAVs in the existing technologies.

[0008] The above information disclosed in this background art is only used to increase the understanding of the background art of the present invention. Therefore, it may include prior arts that are not known to those of ordinary skill in the art. Summary of the Invention

[0009] The object of the present invention is to provide an infrared-based ground-to-air UAV detection and tracking method to solve the problems raised in the above-mentioned background technology.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] An infrared-based ground-to-air UAV detection and tracking method includes:

[0012] Obtain an infrared image sequence of the target airspace, and use an adaptive gain control algorithm based on the scene temperature histogram to dynamically adjust the integration time and gain parameters of the infrared image to generate an infrared image with enhanced thermal radiation characteristics;

[0013] Perform multi-level feature fusion detection on the infrared image using a multi-level global feature fusion network improved based on YOLOv5 to generate a saliency map containing the thermal characteristics of the UAV rotor;

[0014] According to the saliency map, use the improved BotSort algorithm to predict the movement trajectory of the UAV within a preset time period, and determine the three-dimensional space coordinates and movement trajectory parameters of the UAV;

[0015] According to the three-dimensional space coordinates and movement trajectory parameters of the UAV, control the load gimbal to perform adaptive steering, and fuse the laser ranging data for altitude-constrained tracking.

[0016] Preferably, the step of using an adaptive gain control algorithm based on the scene temperature histogram to dynamically adjust the integration time and gain parameters of the infrared image to generate an infrared image with enhanced thermal radiation characteristics includes:

[0017] Statistical histogram of the current frame infrared radiation intensity, and determine the dynamic adjustment threshold according to the histogram distribution;

[0018] Dynamic adjustment threshold formula:

[0019] T = μ + k·σ

[0020] In the formula, μ is the mean value of the current frame infrared radiation intensity, σ is the standard deviation, and k is an empirical coefficient (value range 0.5 - 1.5);

[0021] When the infrared radiation intensity is lower than the threshold, increase the integration time and gain parameters;

[0022] When the infrared radiation intensity is higher than the threshold, reduce the integration time and gain parameters;

[0023] Based on the adjusted infrared radiation intensity histogram, generate an infrared image with enhanced thermal radiation characteristics.

[0024] Preferably, the improved multi-level global feature fusion network based on YOLOv5 is used to perform multi-level feature fusion detection on the infrared image to generate a saliency map containing the thermal characteristics of the drone rotor, including:

[0025] Combine CSPDarknet53 with the improved multi-level global feature fusion network and embed the cross-attention mechanism to extract multi-scale features of the infrared image;

[0026] The cross-attention mechanism generates channel attention weights through global average pooling, and then optimizes the correlation of local features through the spatial pyramid pooling layer;

[0027] Add a spatial information enhancement module after the spatial pyramid pooling layer, and use dilated convolution and adaptive spatial pooling layer to optimize the edge contour features of the drone;

[0028] Introduce a channel information interaction module to dynamically adjust the channel weights through 1×1 convolution to suppress the interference of complex backgrounds;

[0029] Based on the above multi-level feature fusion detection, generate a saliency map containing the thermal characteristics of the drone rotor.

[0030] Preferably, the improved BotSort algorithm is used to predict the motion trajectory of the drone within a preset time period according to the saliency map, and the three-dimensional spatial coordinates and motion trajectory parameters of the drone are determined, including:

[0031] Introduce the Farneback dense optical flow algorithm to compensate for the motion trajectory deviation caused by the jitter of the load pan in real time;

[0032] Based on the joint association matrix that fuses Euclidean distance, SIoU, and H-IoU, optimize the identity jump caused by target occlusion;

[0033] Perform feature matching based on the thermal radiation fingerprint feature library to achieve re-identification of the target after occlusion;

[0034] After the re-identification of the target, obtain the three-dimensional spatial coordinates and motion trajectory parameters of the drone.

[0035] Preferably, the introduction of the Farneback dense optical flow algorithm to compensate for the motion trajectory deviation caused by the jitter of the load pan in real time includes:

[0036] Select one or more fixed reference object areas in the image and calculate the Farneback dense optical flow field;

[0037] Use principal component analysis to extract the main motion direction θ of the optical flow field, and establish a pan angular velocity error model based on the main motion direction θ:

[0038]

[0039]

[0040] Where ω err,x and ω err,y are the angular velocity errors of the load pan in the horizontal and vertical directions respectively, ω i is the weight of the i-th optical flow vector, and u i and v i are the horizontal and vertical components of the optical flow vector;

[0041] Calculate the coordinate compensation amount of the target in the image coordinate system according to the pan angular velocity error model:

[0042]

[0043] Combined with the predicted motion trajectory, use Kalman filtering to smooth the compensated motion coordinates to obtain the final motion trajectory.

[0044] Preferably, the optimization of the identity jump caused by target occlusion based on the joint association matrix of fused Euclidean distance, SIoU, and H-IoU includes:

[0045] Calculate the Euclidean distance, SIoU, and H-IoU in sequence, and perform weighted fusion on the Euclidean distance, SIoU, and H-IoU to obtain a joint association matrix;

[0046] Utilize the comprehensive information in the joint association matrix, combine the predicted motion trajectory and the UAV edge contour features to correct or update the motion trajectory of the target;

[0047] Use the Kalman filtering or particle filtering method to smooth the corrected or updated target motion trajectory to obtain the final motion trajectory.

[0048] Preferably, the steps for constructing the thermal radiation fingerprint feature library include:

[0049] Obtain multiple aerodynamic thermal radiation degradation images of the infrared imaging device to form an image sequence;

[0050] Set a reference image, calculate the difference between each image and the reference image, and generate a difference image for each image;

[0051] Perform two-dimensional surface fitting on the difference image of each image to obtain a two-dimensional surface polynomial corresponding to the heat flux density;

[0052] Surface fitting based on the least squares method:

[0053] f(x,y) = ax 2 + by 2 + cxy + dx + ey + f

[0054] In the formula, a and b respectively represent the curvature distributions of the thermal radiation in the horizontal and vertical directions in the image, c represents the non-linear correlation of the thermal radiation in the diagonal direction, d and e represent the linear change trends of the thermal radiation along the coordinate axes, and f is a constant term, representing the offset of the reference thermal radiation intensity;

[0055] By comprehensively analyzing the coefficients in multiple polynomials, a relational expression of the coefficients of the same terms with respect to the heat flux density is established to form a thermal radiation fingerprint feature library;

[0056] Coefficient extraction and heat flux density relationship formula:

[0057] S = l1Q + h1

[0058] Z = l2Q + h2

[0059] In the formula, Q is the heat flux density, representing the physical quantity of the thermal radiation of the UAV, l1 and l2 are proportionality coefficients, reflecting the strength of the linear relationship between the polynomial coefficients S, Z and the heat flux density Q, and h1 and h2 are intercept terms, representing the reference coefficient values when the heat flux density is zero.

[0060] Preferably, controlling the load gimbal to perform adaptive steering according to the three-dimensional space coordinates and motion trajectory parameters of the UAV, and fusing the laser ranging data for altitude constraint tracking includes:

[0061] Based on the position and attitude of the load gimbal and the internal and external parameters of the infrared imaging device, the three-dimensional space coordinates and motion trajectory parameters of the UAV are converted from the world coordinate system to the load gimbal coordinate system;

[0062] According to the horizontal coordinate and vertical coordinate of the UAV in the load gimbal coordinate system, the corresponding horizontal steering angle and vertical steering angle are calculated respectively;

[0063]

[0064] In the formula, f is the focal length of the infrared imaging device;

[0065] Using the weighted average or Kalman filtering method, the laser ranging data is fused with the altitude of the UAV in the load gimbal coordinate system to generate an altitude constraint;

[0066] According to the altitude constraint, the pitch angle of the load gimbal or the infrared imaging device is adjusted, and the adaptive steering of the load gimbal is realized in combination with the horizontal steering angle and the vertical steering angle.

[0067] Compared with the prior art, the beneficial effects of the present invention are:

[0068] The present invention effectively improves the detection accuracy and tracking stability of low-altitude small unmanned aerial vehicles (UAVs) by combining infrared imaging technology, an improved YOLOv5 multi-level global feature fusion network, and an improved BotSort algorithm. The image quality is optimized through an adaptive gain control algorithm, significantly enhancing the thermal radiation characteristics of the UAV and avoiding the failure problems caused by environmental interference in traditional technologies. At the same time, the Farneback dense optical flow algorithm is used to compensate for the trajectory deviation caused by gimbal jitter, and the height constraint is combined with laser ranging data to ensure all-weather and high-precision dynamic target tracking, significantly improving the detection and protection capabilities of low-altitude small UAVs. Especially in complex environments, such as at night, in haze, etc., it can still maintain efficient monitoring and precise tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flowchart of the infrared-based ground-to-air UAV detection and tracking method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0071] Embodiment 1:

[0072] Please refer to Figure 1 As shown, an infrared-based ground-to-air UAV detection and tracking method includes:

[0073] An infrared image sequence of the target airspace is obtained through an infrared imaging device, and an adaptive gain control algorithm based on the scene temperature histogram is used to dynamically adjust the integration time and gain parameters of the infrared image to generate an infrared image with enhanced thermal radiation characteristics;

[0074] The infrared radiation intensity histogram of the current frame is statistically analyzed, and the dynamic adjustment threshold is determined according to the histogram distribution;

[0075] When the infrared radiation intensity is lower than the threshold, the integration time and gain parameters are increased;

[0076] When the infrared radiation intensity is higher than the threshold, the integration time and gain parameters are decreased;

[0077] Based on the adjusted infrared radiation intensity histogram, an infrared image with enhanced thermal radiation characteristics is generated.

[0078] Furthermore, the integration time and gain parameters are dynamically adjusted through an adaptive gain control algorithm to automatically optimize the exposure effect of the image according to the infrared radiation intensity of the current environment, thereby effectively enhancing the thermal radiation characteristics of the infrared image, improving the clarity and detail performance of the image, and ensuring the image quality especially under different radiation intensities, and enhancing the accuracy of target detection and analysis.

[0079] A multi-level global feature fusion network improved based on YOLOv5 is used to perform multi-level feature fusion detection on the infrared image to generate a saliency map containing the thermal characteristics of the drone rotor;

[0080] CSPDarknet53 is combined with the improved multi-level global feature fusion network, and a cross-attention mechanism is embedded to extract multi-scale features of the infrared image;

[0081] The cross-attention mechanism generates channel attention weights through global average pooling, and then optimizes the relevance of local features through a spatial pyramid pooling layer;

[0082] A spatial information enhancement module is added after the spatial pyramid pooling layer, and dilated convolution and an adaptive spatial pooling layer are used to optimize the edge contour features of the drone;

[0083] A channel information interaction module is introduced, and the channel weights are dynamically adjusted through 1×1 convolution to suppress the interference of complex backgrounds;

[0084] Based on the above multi-level feature fusion detection, a saliency map containing the thermal characteristics of the drone rotor is generated.

[0085] Furthermore, by combining the improved multi-level global feature fusion network with CSPDarknet53 and embedding a cross-attention mechanism, the detection ability of the thermal characteristics of the drone rotor in the infrared image is effectively improved. The application of the cross-attention mechanism and the spatial pyramid pooling layer optimizes the multi-scale fusion of features and the relevance of local features. At the same time, the introduced spatial information enhancement module and channel information interaction module can accurately extract the edge contour of the drone and suppress the interference of complex backgrounds, thereby improving the accuracy and robustness of drone detection.

[0086] According to the saliency map, the improved BotSort algorithm is used to predict the movement trajectory of the drone within a preset time period to determine the three-dimensional spatial coordinates and movement trajectory parameters of the drone;

[0087] The Farneback dense optical flow algorithm is introduced to compensate for the movement trajectory offset caused by the jitter of the load gimbal in real time;

[0088] One or more fixed reference object areas in the image are selected to calculate the Farneback dense optical flow field;

[0089] Extract the main motion direction θ of the optical flow field using principal component analysis, and establish a gimbal angular velocity error model based on the main motion direction θ;

[0090] Calculate the coordinate compensation amount of the target in the image coordinate system according to the gimbal angular velocity error model;

[0091] Furthermore, by combining the improved BotSort algorithm with the saliency map, accurately predict the motion trajectory of the UAV, and use the Farneback dense optical flow algorithm to compensate for the trajectory deviation caused by gimbal jitter in real time. By calculating the optical flow field and using principal component analysis to extract the main motion direction, a gimbal angular velocity error model is established, thereby effectively compensating for the target coordinate error in the image coordinate system, improving the accuracy and stability of the UAV motion trajectory, and achieving more accurate real-time tracking and compensation especially in complex environments.

[0092] Optimize the identity jump caused by target occlusion based on the joint association matrix that fuses Euclidean distance, SIoU, and H-IoU;

[0093] Calculate Euclidean distance, SIoU, and H-IoU in sequence, and perform weighted fusion on Euclidean distance, SIoU, and H-IoU to obtain a joint association matrix;

[0094] Utilize the comprehensive information in the joint association matrix, combine the predicted motion trajectory and the UAV edge contour features to correct or update the target motion trajectory;

[0095] Use the Kalman filter or particle filter method to smooth the corrected or updated target motion trajectory to obtain the final motion trajectory;

[0096] Furthermore, through the joint association matrix that fuses Euclidean distance, SIoU, and H-IoU, the problem of identity jump caused by target occlusion is effectively optimized. By weighted fusing these three metrics and combining the target motion trajectory and the UAV edge contour features, the target motion trajectory can be accurately corrected and updated. Further applying the Kalman filter or particle filter to smooth the corrected trajectory, thereby improving the stability and accuracy of target tracking, and reducing the noise and error in the motion trajectory.

[0097] Perform feature matching based on the thermal radiation fingerprint feature library to achieve re-identification of the target after occlusion;

[0098] Obtain multiple aerodynamic thermal radiation degraded images of the infrared imaging device to form an image sequence;

[0099] Set a reference image, calculate the difference between each image and the reference image to generate the difference image of each image;

[0100] Perform two-dimensional surface fitting on the difference image of each image to obtain the two-dimensional surface polynomial corresponding to the heat flux density;

[0101] Comprehensively analyze the coefficients in multiple polynomials, establish a relationship between the coefficients of the same terms with respect to the heat flux density, and form a thermal radiation fingerprint feature library.

[0102] Furthermore, through feature matching based on the thermal radiation fingerprint feature library, re-identification of the target after occlusion is achieved. By obtaining multiple aerodynamic thermal radiation degraded images and generating difference images, and using two-dimensional surface fitting technology to extract the thermal radiation features under the heat flux density, a thermal radiation fingerprint feature library is finally established. This method effectively captures and analyzes the thermal radiation features of the target, and can accurately identify and track the target even under occlusion or interference conditions, improving the robustness and accuracy of target re-identification.

[0103] According to the three-dimensional space coordinates and motion trajectory parameters of the unmanned aerial vehicle (UAV), control the load gimbal to perform adaptive steering, and fuse the laser ranging data for altitude constraint tracking;

[0104] Based on the position and attitude of the load gimbal and the internal and external parameters of the infrared imaging device, convert the three-dimensional space coordinates and motion trajectory parameters of the UAV from the world coordinate system to the load gimbal coordinate system;

[0105] Calculate the corresponding horizontal steering angle and vertical steering angle according to the horizontal coordinate and vertical coordinate of the UAV in the load gimbal coordinate system respectively;

[0106] Adopt the weighted average or Kalman filtering method to fuse the laser ranging data with the altitude of the UAV in the load gimbal coordinate system to generate an altitude constraint;

[0107] Adjust the pitch angle of the load gimbal or the infrared imaging device according to the altitude constraint, and combine the horizontal steering angle and vertical steering angle to achieve the adaptive steering of the load gimbal.

[0108] Furthermore, by precisely controlling the adaptive steering of the UAV load gimbal and combining the laser ranging data for altitude constraint tracking, the positioning accuracy and stability of the UAV in three-dimensional space are effectively improved. By fusing the three-dimensional space coordinates, motion trajectory of the UAV and the internal and external parameters of the infrared imaging device, precise steering of the load gimbal is achieved to ensure that the device always points to the target. In addition, the weighted average or Kalman filtering method is used to fuse the laser ranging data to optimize the altitude control and enhance the tracking ability and adaptive adjustment performance of the UAV in complex environments.

[0109] Example 2:

[0110] Application example: Application of the infrared-based ground-to-air UAV detection and tracking method in urban low-altitude security

[0111] I. Application Scenario Description

[0112] In the urban low-altitude security scenario, an infrared-based UAV detection and tracking method is adopted to monitor and track the activities of small UAVs over the city in real time. By integrating a high-performance mid-wave infrared camera, a heavy-duty pan-tilt, and an edge computing platform, all-weather, high-precision, and low-latency UAV detection and tracking are achieved.

[0113] II. Architecture Composition and Workflow

[0114] (1) Infrared Image Acquisition

[0115] Equipment: Mid-wave infrared camera

[0116] Working band: 3.7 - 4.8 μm

[0117] Thermal sensitivity: NETD ≤ 15 mK @ 25°C

[0118] Resolution: 1280 × 1024

[0119] Detection distance: 8 km

[0120] Operating temperature: -40°C to +60°C

[0121] Workflow: The infrared camera continuously acquires infrared image sequences of the target airspace, covering the main monitoring areas over the city.

[0122] (2) Adaptive Gain Control Algorithm

[0123] Algorithm: Adaptive gain control algorithm based on the scene temperature histogram.

[0124] Workflow:

[0125] Statistically calculate the infrared radiation intensity histogram of the current frame, and calculate the mean μ and standard deviation σ.

[0126] Determine the dynamically adjusted threshold according to the formula threshold = μ + σ · k (k takes an empirical value, such as 1.0).

[0127] When the infrared radiation intensity is lower than the threshold, increase the integration time and gain parameter; when it is higher than the threshold, decrease the integration time and gain parameter to optimize the infrared image quality.

[0128] (3) Multi-level Global Feature Fusion Network

[0129] Network structure: A multi-level global feature fusion network improved based on YOLOv5, combined with CSPDarknet53, and embedded with a cross-attention mechanism.

[0130] Workflow:

[0131] Feature extraction: Use CSPDarknet53 to extract multi-scale features of infrared images.

[0132] Cross-attention mechanism: Generate channel attention weights through global average pooling, and then optimize the relevance of local features through the spatial pyramid pooling layer.

[0133] Spatial information enhancement module: Use dilated convolution and adaptive spatial pooling layer to optimize the edge contour features of the drone.

[0134] Channel information interaction module: Dynamically adjust the channel suppression weights through 1×1 convolution to reduce background interference.

[0135] Object detection: Generate a saliency map containing the thermal features of the drone's rotor to achieve high-precision object detection.

[0136] (4) Trajectory prediction algorithm

[0137] Algorithm: Improve the BotSort algorithm and combine it with the Farneback dense optical flow algorithm.

[0138] Workflow:

[0139] Motion trajectory prediction: Predict the motion trajectory of the drone within a preset time period based on the saliency map.

[0140] Real-time compensation: Introduce the Farneback dense optical flow algorithm to compensate for the motion trajectory deviation caused by the jitter of the payload gimbal in real time.

[0141] Identity re-identification: Perform feature matching based on the thermal radiation fingerprint feature library to solve the problem of identity jump after target occlusion.

[0142] (5) Gimbal control mechanism

[0143] Device: Heavy-duty gimbal

[0144] Load capacity: ≥20 kg

[0145] Rotation range: 360° continuous rotation horizontally, pitch angle -90° to +60°

[0146] Positioning accuracy: ±0.01°

[0147] Variable speed range: 0.01° to 20° / s stepless speed change

[0148] Protection level: IP66

[0149] Workflow:

[0150] Coordinate transformation: Convert the three-dimensional space coordinates and motion trajectory parameters of the drone from the world coordinate system to the payload gimbal coordinate system.

[0151] Steering angle calculation: Based on the horizontal and vertical coordinates of the drone in the load gimbal coordinate system, calculate the corresponding horizontal steering angle and vertical steering angle respectively.

[0152] Height constraint tracking: Integrate the laser ranging data, use the weighted average or Kalman filtering method to generate height constraints, adjust the pitch angle of the load gimbal, and achieve adaptive steering in combination with the horizontal steering angle and vertical steering angle.

[0153] (6) Edge computing platform

[0154] Device: NVIDIA Jetson AGX Xavier edge computing platform

[0155] Workflow:

[0156] Model deployment: Deploy the lightweight model to the edge platform to achieve low-latency inference (≤50ms).

[0157] Real-time processing: Receive the infrared camera and laser ranging data in real time, construct a spatio-temporally synchronized feature input stream, and support multi-target real-time tracking.

[0158] Video processing capability: Support parallel processing of 4 channels of 1080P@30fps infrared video streams.

[0159] III. Technical effect demonstration

[0160] (1) Detection accuracy

[0161] The detection accuracy of drone targets with a pixel ratio ≤ 5×5 is 92.7%, and the false alarm rate < 0.8%.

[0162] (2) Tracking stability

[0163] In the case of target occlusion or rapid maneuvering scenarios, the trajectory continuity rate ≥ 95%, and the identity jump rate ≤ 14.5%.

[0164] (3) Detection performance

[0165] Inference latency: ≤ 50ms, supporting multi-target real-time tracking.

[0166] Device power consumption: < 60W, suitable for security scenarios with low power requirements.

[0167] Video processing capability: Support parallel processing of 4 channels of 1080P@30fps infrared video streams.

[0168] (4) Capture rate

[0169] In the actual measurement around the airport, the capture rate of drones within 500m height is 98.3%.

[0170] IV. Details for Optimization

[0171] (1) Specification of Equipment Parameters

[0172] Clarify the operating temperature range (-40°C to +60°C) of the mid-wave infrared camera, the protection level (IP66) of the heavy-duty pan-tilt, and the video processing capacity (supporting parallel processing of 4 channels of 1080P@30fps infrared video streams) of the edge computing platform.

[0173] (2) Supplementary Details of Algorithms

[0174] In the adaptive gain control algorithm, further explain the value range (0.5 to 1.5) of the empirical coefficient k in the threshold calculation formula and give a typical value (such as 1.0).

[0175] (3) Performance Improvement

[0176] Highlight the low-power feature (<60W) of the edge computing platform and its advantages in multi-target real-time tracking.

[0177] (4) Expansion of Application Scenarios

[0178] Although the application scenario is described as urban low-altitude security, the high-performance parameters of UAV detection (such as 8 km detection range and 98.3% capture rate) are also applicable to scenarios such as border monitoring and airport protection.

[0179] V. Summary

[0180] This application example demonstrates the specific application of the infrared-based ground-to-air UAV detection and tracking method in the urban low-altitude security scenario. By integrating a high-performance mid-wave infrared camera, a heavy-duty pan-tilt, and an edge computing platform, all-weather, high-precision, and low-latency UAV detection and tracking are achieved. The quantitative data of the technical effects show excellent performance in terms of detection accuracy, tracking stability, and performance, providing effective technical support for urban low-altitude security. At the same time, through detail optimization, the example has been improved in terms of the accuracy of technical parameters, the integrity of algorithm details, and the comprehensiveness of performance.

[0181] Example 3:

[0182] An embodiment of the present invention also provides a computer-readable storage medium. A program of a ground-to-air UAV detection and tracking method based on infrared as described in any one of the above is stored on the computer-readable storage medium. When the program is executed by a processor, each process of the above detection and tracking method embodiment is implemented, and the same technical effects can be achieved. To avoid repetition, it will not be described in detail here. Among them, the computer-readable storage medium is, for example, a read-only memory (ROM for short), a random access memory (RAM for short), a magnetic disk, or an optical disc, etc.

[0183] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0184] In the drawings of the disclosed embodiments of the present invention, only the structures related to the disclosed embodiments of the present invention are involved. Other structures can refer to the general design. Without conflict, the same embodiment and different embodiments of the present invention can be combined with each other.

[0185] The flowcharts shown in the drawings are only illustrative, and do not necessarily include all the contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged. Therefore, the actual execution order may change according to the actual situation.

[0186] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and deformations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An infrared-based ground-to-air UAV detection and tracking method, characterized in that, Including: Obtain an infrared image sequence of the target airspace, and use an adaptive gain control algorithm based on the scene temperature histogram to dynamically adjust the integration time and gain parameters of the infrared image, generating an infrared image with enhanced thermal radiation characteristics; Perform multi-level feature fusion detection on the infrared image using a multi-level global feature fusion network improved based on YOLOv5, generating a saliency map containing the thermal characteristics of the drone rotor; According to the saliency map, use the improved BotSort algorithm to predict the motion trajectory of the drone within a preset time period, determining the three-dimensional spatial coordinates and motion trajectory parameters of the drone; According to the three-dimensional spatial coordinates and motion trajectory parameters of the drone, control the load gimbal to perform adaptive steering and fuse laser ranging data for altitude-constrained tracking.

2. The method for ground-to-air UAV detection and tracking based on infrared according to claim 1, wherein: The step of using an adaptive gain control algorithm based on the scene temperature histogram to dynamically adjust the integration time and gain parameters of the infrared image, generating an infrared image with enhanced thermal radiation characteristics, includes: Statistical histogram of the current frame's infrared radiation intensity, and determine the dynamic adjustment threshold according to the histogram distribution; Dynamic adjustment threshold formula: T = μ + k·σ Wherein, μ is the mean value of the current frame's infrared radiation intensity, σ is the standard deviation, and k is an empirical coefficient; When the infrared radiation intensity is lower than the threshold, increase the integration time and gain parameters; When the infrared radiation intensity is higher than the threshold, reduce the integration time and gain parameters; Based on the adjusted infrared radiation intensity histogram, generate an infrared image with enhanced thermal radiation characteristics.

3. The method for ground-to-air UAV detection and tracking based on infrared according to claim 2, characterized in that: The step of performing multi-level feature fusion detection on the infrared image using a multi-level global feature fusion network improved based on YOLOv5, generating a saliency map containing the thermal characteristics of the drone rotor, includes: Combine CSPDarknet53 with the improved multi-level global feature fusion network and embed a cross-attention mechanism to extract multi-scale features of the infrared image; The cross-attention mechanism generates channel attention weights through global average pooling, and then optimizes the correlation of local features through a spatial pyramid pooling layer; Add a spatial information enhancement module after the spatial pyramid pooling layer, and use dilated convolution and an adaptive spatial pooling layer to optimize the drone edge contour features; Introduce a channel information interaction module to dynamically adjust the channel weights through 1×1 convolution, suppressing the interference of complex backgrounds; Based on the above multi-level feature fusion detection, generate a saliency map containing the thermal characteristics of the drone rotor.

4. The method for ground-to-air UAV detection and tracking based on infrared according to claim 3, characterized in that: The step of using the improved BotSort algorithm to predict the motion trajectory of the drone within a preset time period according to the saliency map, determining the three-dimensional spatial coordinates and motion trajectory parameters of the drone, includes: Introduce the Farneback dense optical flow algorithm to compensate for the motion trajectory offset caused by the jitter of the load gimbal in real time; Based on the joint association matrix that fuses Euclidean distance, SIoU, and H-IoU, optimize the identity jump caused by target occlusion; Perform feature matching based on the thermal radiation fingerprint feature library to achieve re-identification of the target after occlusion; After the target re-identification, obtain the three-dimensional spatial coordinates and motion trajectory parameters of the drone.

5. A ground-to-air UAV detection and tracking method based on infrared according to claim 4, characterized in that: Introducing the Farneback dense optical flow algorithm to compensate for the motion trajectory deviation caused by the jitter of the load gimbal in real time, including: Select one or more fixed reference object areas in the image and calculate the Farneback dense optical flow field; Use principal component analysis to extract the main motion direction θ of the optical flow field, and establish a gimbal angular velocity error model based on the main motion direction θ: where ω err,x and ω err,y are the angular velocity errors of the load gimbal in the horizontal and vertical directions respectively, ω i is the weight of the i-th optical flow vector, u i and v i are the horizontal and vertical components of the optical flow vector; Calculate the coordinate compensation amount of the target in the image coordinate system according to the gimbal angular velocity error model: Combined with the predicted motion trajectory, use Kalman filtering to smooth the compensated motion coordinates to obtain the final motion trajectory.

6. The method for ground-to-air UAV detection and tracking based on infrared according to claim 5, characterized in that: Based on the joint correlation matrix that fuses Euclidean distance, SIoU, and H-IoU, optimize the identity jump caused by target occlusion, including: Calculate Euclidean distance, SIoU, and H-IoU in sequence, and perform weighted fusion on Euclidean distance, SIoU, and H-IoU to obtain a joint correlation matrix; Use the comprehensive information in the joint correlation matrix, combined with the predicted motion trajectory and the UAV edge contour features, to correct or update the motion trajectory of the target; Use the Kalman filtering or particle filtering method to smooth the corrected or updated target motion trajectory to obtain the final motion trajectory.

7. The method for ground-to-air UAV detection and tracking based on infrared according to claim 6, characterized in that: The steps for constructing the thermal radiation fingerprint feature library include: Obtain multiple aerodynamic thermal radiation degradation images of the infrared imaging device to form an image sequence; Set a reference image, calculate the difference between each image and the reference image, and generate a difference image for each image; Perform two-dimensional surface fitting on the difference image of each image to obtain a two-dimensional surface polynomial corresponding to the heat flux density; Surface fitting based on the least squares method: f(x,y) = ax 2 + by 2 + cxy + dx + ey + f In the formula, a and b respectively represent the curvature distributions of thermal radiation in the horizontal and vertical directions in the image, c represents the non-linear correlation of thermal radiation in the diagonal direction, d and e represent the linear change trends of thermal radiation along the coordinate axes, and f is a constant term representing the offset of the reference thermal radiation intensity; Comprehensively analyze the coefficients in multiple polynomials, establish a relationship formula of the same-term coefficients with respect to the heat flux density, and form a thermal radiation fingerprint feature library; Coefficient extraction and relationship formula with heat flux density: S = l1Q + h1 Z = l2Q + h2 In the formula, Q is the heat flux density, representing the physical quantity of UAV thermal radiation, l1 and l2 are proportionality coefficients, reflecting the strength of the linear relationship between the polynomial coefficients S, Z and the heat flux density Q, and h1, h2 are intercept terms, representing the reference coefficient values when the heat flux density is zero.

8. The method for detecting and tracking ground-to-air drones based on infrared according to claim 7, characterized in that: According to the three-dimensional space coordinates and motion trajectory parameters of the UAV, control the load gimbal to perform adaptive steering, and fuse the laser ranging data for altitude-constrained tracking, including: Based on the position and attitude of the load gimbal and the internal and external parameters of the infrared imaging device, convert the three-dimensional space coordinates and motion trajectory parameters of the UAV from the world coordinate system to the load gimbal coordinate system; Calculate the corresponding horizontal steering angle and vertical steering angle according to the horizontal coordinate and vertical coordinate of the UAV in the load gimbal coordinate system respectively; In the formula, f is the focal length of the infrared imaging device; Use the weighted average or Kalman filtering method to fuse the laser ranging data with the altitude of the UAV in the load gimbal coordinate system to generate an altitude constraint; Adjust the pitch angle of the load pan-tilt or infrared imaging device according to the height constraint, and combine the horizontal steering angle and the vertical steering angle to achieve the adaptive steering of the load pan-tilt.

Images