A method and system for detecting and tracking a dual-mode low and slow small unmanned aerial vehicle

Abstract

The present disclosure provides a detection and tracking method and system for a dual-mode low-slow-small unmanned aerial vehicle, the method comprising: synchronously collecting infrared images and visible light images of the same airspace, and obtaining the current light intensity; inputting the infrared images, visible light images and light intensity into a preset dual-mode fusion recognition model to capture the images and coordinates of the unmanned aerial vehicle; and based on an improved BoTSORT tracking algorithm, tracking the unmanned aerial vehicle according to the images and coordinates of the unmanned aerial vehicle. The present disclosure uses a dual-mode fusion model to accurately and quickly fuse and recognize the unmanned aerial vehicle based on visible light images and infrared images, and realizes a closed-loop process of detection and tracking, can adaptively adjust the fusion weight according to the light intensity, realizes accurate identification of unmanned aerial vehicles in various scenes, and at the same time, the lightweight model design can effectively improve the detection speed, allows deployment on edge devices, and expands the use scenarios of unmanned aerial vehicle detection and tracking.

Description

Technical Field

[0001] This disclosure relates to the field of unmanned aerial vehicle (UAV) detection and identification technology, and in particular to a detection and tracking method and system for a dual-modal low-speed small UAV. Background Technology

[0002] Low-altitude, slow-flying, and small unmanned aerial vehicles (UAVs) are characterized by their small size, slow flight speed (easily confused with birds), low flight altitude (easily affected by ground clutter), relatively low noise, and weak infrared signature. These characteristics make them difficult to detect and identify effectively and reliably in complex urban environments, low-altitude airspace, or severe weather conditions.

[0003] The current mainstream detection methods and their existing problems are as follows:

[0004] (1) Radar detection: By emitting electromagnetic waves and receiving the reflected signals from the target, information is obtained, and the distance, speed, and azimuth of the UAV are calculated. Radar detection can achieve long-range, all-weather monitoring. However, radar is susceptible to terrain obstruction and electromagnetic interference, and its detection capability for UAVs made of non-metallic materials is relatively weak.

[0005] (2) Optoelectronic vision: High-resolution optical cameras or infrared cameras are used to acquire images of the UAV, and deep learning algorithms (such as YOLO, C2F-Drone, ByteTrack, etc.) are used for target detection and tracking. Visible light is suitable for daytime detection; infrared can achieve nighttime detection and can acquire more target information. Visual detection is highly intuitive and suitable for low-altitude, close-range UAV detection, but it is greatly affected by weather (fog, rain, snow) and background occlusion (birds, buildings).

[0006] (3) Acoustic detection: High-frequency noise from the UAV rotor and motor is collected by a microphone array, and characteristic frequencies are extracted using voiceprint recognition technology for UAV detection. Acoustic waves are suitable for short-range monitoring in urban or complex electromagnetic environments. However, acoustic waves are easily affected by environmental noise, have limited detection range, and are difficult to handle in swarm UAV scenarios.

[0007] (4) Radio frequency detection: The remote control signals of the UAV (such as Wi-Fi, GPS, data transmission, image transmission and other radio frequency bands) are detected by detection equipment such as spectrum analyzers and radio direction finders for positioning and identification. Radio frequency detection can detect UAVs that have not yet taken off in advance and supports electronic interference countermeasures (such as GPS spoofing and signal blocking), but it is susceptible to interference from Wi-Fi, Bluetooth and other signals in urban environments. Summary of the Invention

[0008] The purpose of this disclosure is to provide a method and system for detecting and tracking dual-modal low-speed small unmanned aerial vehicles (UAVs) to solve the problem that low-speed small UAVs are difficult to effectively identify in the prior art.

[0009] The embodiments of this disclosure adopt the following technical solution: a detection and tracking method for a dual-modal low-speed small unmanned aerial vehicle (UAV), applied to UAV monitoring equipment, comprising: simultaneously acquiring infrared and visible light images in the same airspace and obtaining the current illumination intensity; inputting the infrared image, the visible light image, and the illumination intensity into a preset dual-modal fusion recognition model to capture the image and coordinates of the UAV; wherein, the preset dual-modal fusion recognition model is constructed based on a dual-stream backbone network, the backbone network of the dual-stream backbone network includes a cross-attention fusion module and parallel first and second backbone branches, the first backbone branch is used to process the infrared image, the second backbone branch is used to process the visible light image, each backbone branch includes multiple spatial-depth residual convolutional feature extraction modules, feature aggregation modules, and MViTBv3 self-attention mechanism modules, the cross-attention fusion module adjusts the fusion weights between the first and second backbone branches according to the illumination intensity; and tracking the UAV based on the improved BoTSORT tracking algorithm according to the image and coordinates of the UAV.

[0010] In some embodiments, the spatial-depth residual convolutional feature extraction module is specifically used for: inputting the original feature map into the SPD layer for segmentation based on the downsampling scaling factor to obtain multiple sub-feature maps; concatenating all the sub-feature maps along the channel dimension, and performing a convolution operation with a stride of 1 on the concatenated sub-feature maps to obtain a new feature map; and performing a no-stride convolution on the new feature map with a stride of 3. 3. Convolutional kernels capture local features to obtain a convolutional feature map; the convolutional feature map is then added element-by-element to the original feature map to obtain the output of the spatial-depth residual convolutional feature extraction module.

[0011] In some embodiments, the cross-attention fusion module is specifically used to: normalize the light intensity to obtain a light intensity coefficient; when the light intensity coefficient is greater than a first preset value, increase the fusion weight of the first backbone branch and decrease the fusion weight of the second backbone branch; when the light intensity coefficient is less than or equal to the first preset value and greater than or equal to a second preset value, balance the fusion weight of the first backbone branch and the fusion weight of the second backbone branch; when the light intensity coefficient is less than the second preset value, increase the fusion weight of the second backbone branch and decrease the fusion weight of the first backbone branch.

[0012] In some embodiments, the improved BoTSORT tracking algorithm includes: optimizing the cross-union ratio (CUP) metric in the BoTSORT tracking algorithm using a Focaler-IoU function, wherein the Focaler-IoU function includes:

[0013]

[0014] in, This represents the original value of the intersection-union ratio. This represents the intersection-union ratio after reconstruction. and This is the interval partitioning parameter.

[0015] In some embodiments, after simultaneously acquiring infrared and visible light images in the same spatial domain, the method further includes: performing super-resolution reconstruction processing on the infrared images based on a super-resolution reconstruction network; wherein the super-resolution reconstruction network includes a shallow feature extraction module and multiple consecutively stacked blue... Figure 3 The core network module, the multi-layer feature fusion module, and the reconstruction module, wherein all of the blue Figure 3 The first blue in the core network module Figure 3 The core network module is connected to the reconfiguration module via a long jump connection.

[0016] In some embodiments, after synchronously acquiring infrared and visible light images in the same spatial domain, the method further includes: registering the visible light image and the super-reconstructed infrared image.

[0017] This disclosure also provides a detection and tracking system for a dual-modal low-speed small unmanned aerial vehicle (UAV), comprising: a gimbal for supporting a visible light image acquisition device and an infrared image acquisition device; a light sensor for acquiring the current light intensity; and a processor for implementing the dual-modal low-speed small UAV detection and tracking method described above, to capture images and coordinates of the UAV and track the UAV.

[0018] In some embodiments, the processor is further configured to: adjust the movement direction and / or rotation angle of the gimbal based on the image and coordinates of the UAV.

[0019] In some embodiments, the processor is specifically configured to: calculate the center coordinates of the current visible light image. and the centroid coordinates of the UAV The difference in the x-coordinates between Difference between the vertical and horizontal axes , ;

[0020] exist and At this time, the movement direction of the gimbal is not adjusted;

[0021] exist and At that time, control the gimbal to move horizontally to the right;

[0022] exist and At that time, control the gimbal to move upward and to the right;

[0023] exist and At that time, control the gimbal to move vertically upward;

[0024] exist and At that time, control the gimbal to move upward and to the left;

[0025] exist and At that time, control the gimbal to move horizontally to the left;

[0026] exist and At that time, control the gimbal to move downwards and to the left;

[0027] exist and At that time, control the gimbal to move vertically downwards;

[0028] exist and At that time, control the gimbal to move downwards and to the right;

[0029] in, The preset radius for the movement constraint region.

[0030] In some embodiments, the processor is specifically configured to: take the upper left corner of the visible light image as the origin, the horizontal direction to the right as the positive x-axis, and the vertical direction downward as the positive y-axis, and obtain the coordinates of the detected drone in the image. According to the horizontal field of view of the visible light image acquisition device and vertical field perspective Calculate the horizontal rotation angle of the gimbal with its rotation center as the origin. and the rotation angle in the vertical direction :

[0031]

[0032]

[0033] Where W is the width of the visible light image and H is the height of the visible light image.

[0034] The beneficial effects of this disclosure are as follows: It utilizes a dual-modal fusion model to perform accurate and rapid fusion recognition of UAVs based on visible light images and infrared images, and realizes a closed-loop process of detection and tracking. The fusion weight can be adaptively adjusted according to the light intensity to achieve accurate recognition of UAVs in various scenarios. At the same time, the lightweight model design can effectively improve the detection speed, allow deployment on edge devices, and expand the application scenarios of UAV detection and tracking. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart of the detection and tracking method for a dual-modal low-speed small unmanned aerial vehicle in the first embodiment of this disclosure;

[0037] Figure 2 This is a schematic diagram of the architecture of the super-resolution reconstruction network BKTN in the first embodiment of this disclosure;

[0038] Figure 3 This is a schematic diagram of the specific structure of ETKB, TKA Block, BSRB-I and BSConv-I in the first embodiment of this disclosure;

[0039] Figure 4 This is a schematic diagram of an architecture of a preset dual-modal fusion recognition model in the first embodiment of this disclosure;

[0040] Figure 5 This is a schematic diagram of the CAFM module in the first embodiment of this disclosure;

[0041] Figure 6 This is a schematic diagram of the SPD-RC module in the first embodiment of this disclosure;

[0042] Figure 7 This is a schematic diagram of the structure of the MViTBv3 self-attention mechanism module in the first embodiment of this disclosure;

[0043] Figure 8 This is a schematic diagram of the system hardware architecture of the detection and tracking system for a dual-modal low-speed small unmanned aerial vehicle in the second embodiment of this disclosure. Detailed Implementation

[0044] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0045] To address the problem of the difficulty in effectively and reliably detecting and identifying low-speed, small unmanned aerial vehicles (UAVs) in existing technologies, the first embodiment of this disclosure provides a detection and tracking method for dual-modal low-speed, small UAVs. This method is mainly applied to UAV monitoring equipment, including but not limited to any electronic device with visible light and infrared image capture capabilities. Its flowchart is shown below. Figure 1 As shown, it mainly includes steps S10 to S30:

[0046] S10 simultaneously acquires infrared and visible light images of the same spatial domain and obtains the current light intensity.

[0047] During the operation of the monitoring equipment, infrared and visible light images within the same spatial domain are simultaneously acquired using its configured visible light image acquisition devices (e.g., RGB cameras) and infrared image acquisition devices (e.g., IR cameras) to ensure temporal and spatial consistency of the two types of image data, facilitating subsequent fusion processing. Simultaneously, the ambient light intensity can be synchronously acquired using sensors such as light sensors, serving as a reliability reference for both visible light and infrared images, and adaptively adjusting the fusion weights of the two images during subsequent fusion and recognition processes.

[0048] In practical implementations, the resolution of infrared images is typically lower than that of visible light images. Furthermore, combined with downsampling operations in subsequent model processing, key feature information is easily lost when detecting small target drones, affecting detection accuracy. To improve the resolution of infrared images and enhance detection performance, this embodiment performs super-resolution reconstruction processing on the acquired infrared images to expand the pixel resolution of small targets.

[0049] Specifically, this embodiment utilizes a super-resolution reconstruction network to perform super-resolution reconstruction processing on the infrared image. Figure 2 This diagram illustrates the architecture of the super-resolution reconstruction network BKTN in this embodiment, which mainly includes a shallow feature extraction module BSConv-I and multiple consecutively stacked blue... Figure 3The network module ETKB, the multi-layer feature fusion module Concat, and the reconstruction module, wherein the reconstruction module consists of a 3×3 standard convolutional layer Conv3 and a sub-pixel convolutional layer Sub-Pixel, the specific structure of the above modules is as follows. Figure 3 As shown. In the shallow feature extraction stage, shallow features of the infrared image (IR) are extracted and mapped to a high-dimensional feature space. This process can be represented as:

[0050]

[0051] in, This represents the shallow feature extraction module BSConv-I, combined with Figure 3 (a) BSConv-I consists of a 1×1 convolution and an Inception Depth-Wise convolution, then passed through several stacked ETKB blocks. The process of extracting deep features can be represented as follows:

[0052]

[0053] in, This represents the k-th ETKB block. and Let represent the input and output features of the k-th ETKB block, respectively. Then, features generated at different depths are fused and mapped, and refined using BSConv-I. The multi-layer feature fusion process can be represented as:

[0054]

[0055] in, This represents a multi-layer feature fusion module. These are the features output after fusion. Simultaneously, to better utilize the input information, residual learning and long-skip connections are used in the reconstruction process, which can be represented as:

[0056]

[0057] in, This represents the completed reconstructed image. The reconstruction module consists of 3×3 standard convolutional layers and subpixel convolutional layers. This network uses... The loss function, after adjustment and optimization, can be expressed as:

[0058]

[0059] It should be noted that infrared images can be super-reconstructed with reference to the resolution of visible light images to ensure the consistency of resolution between the two as much as possible.

[0060] Furthermore, after super-resolution reconstruction of the infrared image, image registration between the reconstructed infrared image and the visible light image can be performed. This embodiment adopts an image registration method based on camera calibration to optimize the registration and subsequent data processing speed. During actual registration, the IR camera and RGB camera are combined and fixed, and calibration boards at different positions are photographed. OpenCV is then used for camera calibration to obtain the IR and RGB camera parameters. Subsequently, using the obtained RGB camera extrinsic parameter matrix M and intrinsic parameter matrix K, the RGB pixel coordinate system is converted to the world coordinate system. The conversion formula is as follows:

[0061]

[0062] in, This represents the actual position of the object in the world coordinate system. This is the extrinsic parameter matrix of the RGB camera. This is the intrinsic parameter matrix of the RGB camera. The scaling factor for RGB cameras. This represents the object's position in the RGB pixel coordinate system. Then, the object's coordinates in the world coordinate system are converted to pixel coordinates in the IR camera using the intrinsic parameter matrix K and extrinsic parameter matrix M of the IR camera. The conversion formula is:

[0063]

[0064] in, The scaling factor for the IR camera. This represents the object's position in the IR pixel coordinate system. This is the intrinsic parameter matrix of the IR camera. Here is the extrinsic parameter matrix of the IR camera. This represents the actual object position in the world coordinate system. Through these two transformations, pixels from the RGB camera are mapped to the infrared image. OpenCV is used to detect the corner coordinates of the checkerboard pattern in both the RGB and IR images. The corner coordinates in the RGB image are then mapped to their positions in the infrared image using camera coordinate transformation. The mean absolute error and standard deviation of the mapped coordinate positions compared to the actual pixel coordinates in the IR image are used as evaluation metrics for this method. The mean absolute error and standard deviation are calculated using...

[0065] S20 inputs infrared images, visible light images, and light intensity into a preset dual-modal fusion recognition model to capture images and coordinates of the drone.

[0066] To address the characteristics of low-altitude, slow-moving, and small unmanned aerial vehicles (UAVs), this embodiment constructs a pre-defined dual-modal fusion recognition model for UAV position detection and recognition. The pre-defined dual-modal fusion recognition model is built upon a dual-stream backbone network. The backbone network of the dual-stream backbone network includes a cross-attention fusion module and parallel first and second backbone branches. The first backbone branch processes infrared images, and the second backbone branch processes visible light images. Each backbone branch includes multiple spatial-depth residual convolutional feature extraction modules (SPD-RC), feature aggregation modules (C2f), and MViTBv3 self-attention mechanism modules. The cross-attention fusion module (CAFM) adjusts the fusion weights between the first and second backbone branches based on the illumination intensity.

[0067] Figure 4 A schematic diagram of an architecture for a pre-defined dual-modal fusion recognition model is shown, such as... Figure 4 It can be seen that its backbone network has two backbone branches. The left branch corresponds to the first backbone branch and is used to process infrared images, while the right branch corresponds to the second backbone branch to process visible light images. The two backbone branches have the same structure, each including four consecutive sets of SPD-RC modules and C2f modules. At the same time, MViTBv3 modules are set between the third and fourth sets of SPD-RC modules and C2f modules. Furthermore, the two backbone branches achieve cross-attention fusion through multiple CAFM modules.

[0068] Specifically, the design of the CAFM module is as follows: Figure 5 As shown. Its implementation principle is as follows:

[0069] (1) Let the input characteristics of the infrared channel be: The input characteristics of the visible light channel are After dimensionality reduction using 1×1 convolution (denoted as Conv1×1), low-dimensional features are obtained:

[0070]

[0071] Where H is the feature map height, W is the feature map width, and C is the number of channels in the feature map; it should be noted that the output dimension of the low-dimensional feature is... ,and , The dimension of the input feature;

[0072] (2) For the dimensionality-reduced features, local features are extracted through dynamic convolution (denoted as DConv3×3), and a query is generated through linear transformation. ),key( ),value( ) and query ( ),key( ),value( Then calculate cross-attention:

[0073]

[0074]

[0075] (3) Feature fusion and output:

[0076]

[0077] Figure 6 The diagram shows the structure of the SPD-RC module, which, based on SPD-Conv, continues to generate new feature maps through stride-free convolution and constructs residual connections. This maintains the image resolution while preserving the flow of feature information and reducing interference from redundant information.

[0078] Specifically, the SPD layer maps all pixel information of each spatial block in the input feature map to the depth dimension, preserving information in the channel dimension and avoiding information loss. First, the original feature map is input into the SPD layer, and then... The downsampling scaling factor segments the original feature map into The formula for each sub-feature map is as follows:

[0079]

[0080]

[0081]

[0082] Then, each sub-feature map is concatenated and rearranged along the channel dimension. The output feature map is then subjected to a convolution operation with a stride of 1 to obtain a new feature map. This new feature map is then subjected to a convolution operation without stride, using a 3×3 kernel to capture local features. Finally, the convolution-processed feature map is added element-wise to its original feature map, which preserves image details and spatial information and improves the expressive power of the features.

[0083] Furthermore, this embodiment adds the MViTBv3 module to the deep feature extraction network of the backbone network. This module is a lightweight network architecture that combines the advantages of CNN and Transformer, overcoming the limitations of traditional CNN in capturing global information and processing long sequence data. Compared with ordinary Vision Transformer (ViT) network, it has fewer parameters and less computation. Figure 7 The diagram illustrates the structure of the MViTBv3 self-attention mechanism module, which consists of three parts: a local feature representation module, a global feature representation module, and a feature fusion module. Input feature map. First, the input is fed into the local feature representation module, where a 3×3 depthwise separable convolution is used to extract local features within the neighborhood while preserving spatial information. Then, a 1×1 convolution is used to map the feature map to a d (number of channels) dimensional space, resulting in the feature map. Global features are extracted using an Unfold-Transformer-Fol structure. The Unfold operation divides the feature map into N non-overlapping patches, which are then transformed into feature vectors through linear mapping. The input is fed into the Transformer for global feature modeling as follows:

[0084]

[0085] Feature map output after self-attention mechanism The fold operation recombines the processed sequences to obtain feature maps. The output feature map is projected from d dimensions back to the original dimension through a 1×1 convolution. The local and global feature representation modules are then concatenated along the feature dimension, and a 1×1 convolution is used to reduce the dimensionality of the resulting feature map. The dimensionality-reduced feature map is fused with the input feature map X through residual connections to obtain the final feature map Y.

[0086] In this embodiment, the C2f module works as follows: the input feature map is first expanded through a 1×1 convolution (Conv1), typically with the number of output channels being twice that of the input, enhancing the feature representation capability; the Concat module is used to concatenate feature maps of different levels (such as low-resolution high-semantic features and high-resolution low-semantic features) along the channel dimension to form a richer feature representation, enhancing the model's ability to perceive target size and details; the Upsample module is used for sampling; SPPF (Spatial Pyramid Pooling Fast) is an improved module for multi-scale feature fusion, enhancing the model's adaptability to changes in target size through fast spatial pyramid pooling.

[0087] In some embodiments, the CAFM module adjusts the weights during cross-fusion based on the acquired light intensity. Specifically, under sufficient lighting conditions, the visible light channel has a relatively high weight; while in low-light or nighttime environments, the infrared channel has a correspondingly increased weight. This adaptive weight adjustment mechanism ensures the system's stability and robustness under different environmental conditions. Finally, based on the determined channel weights, the system fuses the infrared and visible light images to generate a fused image containing the UAV target. In practical implementation, the light intensity can first be normalized to obtain a light intensity coefficient within the range [0,1]. It can be considered The higher the value, the higher the light intensity, and the better the visible light image capture; conversely, the lower the value, the better the image quality. The smaller the value, the lower the light intensity, and the better the infrared image capture effect. Based on the relationship between the light intensity coefficient and pre-defined first and second preset values, the fusion weights of the two backbone branches are adaptively adjusted. For example, when the light intensity coefficient is greater than the first preset value, the fusion weight of the first backbone branch is increased, and the fusion weight of the second backbone branch is decreased; when the light intensity coefficient is less than or equal to the first preset value but greater than or equal to the second preset value, the fusion weights of the first and second backbone branches are balanced; when the light intensity coefficient is less than the second preset value, the fusion weight of the second backbone branch is increased, and the fusion weight of the first backbone branch is decreased. It should be noted that the specific magnitude of increasing or decreasing the fusion weight of a branch can be adjusted according to the magnitude of the light intensity coefficient. The closer the light intensity coefficient is to the upper or lower limit value, the greater the corresponding adjustment in fusion weight. Furthermore, the first and second preset values ​​can be set independently based on the camera's image capture effect and fusion requirements; for example, the first preset value could be 0.7, and the second preset value 0.4.

[0088] The S30, based on the improved BoTSORT tracking algorithm, tracks the drone using images and coordinates.

[0089] The BoTSORT tracking algorithm is a combination of target detection and multi-target tracking. It improves tracking accuracy by fusing motion features and appearance information. In this embodiment, the Focaler-IoU function is used to adjust the intersection-union ratio (IoU) in the BoTSORT tracking algorithm. The measurement method is optimized, specifically including:

[0090]

[0091] in, This represents the original value of the intersection-union ratio. This represents the optimized intersection-union ratio. and The interval partitioning parameter takes values ​​between [0,1]. The Focaler-IoU function allows the loss function to be applied within a certain interval. The system is sensitive to value, allowing it to focus more on samples with moderate overlap between the predicted and ground truth bounding boxes. This not only improves tracking accuracy for small targets but also enhances the algorithm's robustness to occlusion and environmental interference during tracking. Specifically, the device first performs target detection on the fused image, identifying the drone's position and size. Then, using a bottom-up ranking tracking algorithm combined with the Focaler-IoU metric, it calculates the deviation between the current target position and the predicted position. Based on this deviation, the system calculates the horizontal angle, vertical angle, and distance the monitoring device needs to move, thus achieving precise drone tracking.

[0092] This embodiment's dual-modal tracking method for low-altitude, slow-moving, and small UAVs overcomes the limitations of single-modal tracking in complex environments by fusing infrared and visible light image modalities. The introduction of the SPD-RC and CAFM fusion modules significantly enhances feature extraction and fusion capabilities. Simultaneously, an adaptive weight adjustment mechanism based on illumination intensity enables the system to adapt to target tracking tasks under different lighting conditions. The improved Focaler-IoU metric further enhances the accuracy and robustness of the tracking algorithm, enabling the system to achieve stable and accurate tracking of low-altitude, slow-moving, and small UAVs.

[0093] The second embodiment of this disclosure provides a detection and tracking system for a dual-modal, low-altitude, slow-speed, small unmanned aerial vehicle (UAV). This system can effectively identify and track low-altitude, low-speed, small UAVs under various environmental conditions. It mainly includes: a gimbal for supporting visible light image acquisition equipment and infrared image acquisition equipment; a light sensor for acquiring the current light intensity; and a processor that implements the dual-modal, low-altitude, slow-speed, small UAV detection and tracking method provided in the first embodiment of this disclosure to capture images and coordinates of the UAV and track it.

[0094] In some embodiments, the system may further include a temperature sensor for acquiring ambient temperature data, which can assist in the calibration and processing of infrared images, improving infrared imaging quality; it may also include a GPS receiver for acquiring the system's own precise location information, providing geographic reference for target positioning and trajectory recording. The gimbal is used to simultaneously acquire infrared and visible light images and track drones. For example, it can employ a two-axis or three-axis design, supporting 360° horizontal rotation and -90° to +90° vertical pitch, capable of covering an all-around monitoring area. The infrared camera operates in the 8-14μm wavelength range, effectively capturing target thermal radiation information at night or in low-light conditions; the visible light camera operates in the visible light band, providing high-resolution detail information. The gimbal has a high-precision servo motor drive system, enabling smooth and precise motion control, ensuring image stability during tracking.

[0095] The processor can be implemented on a processor platform with neural network model implementation capabilities to complete the UAV detection and tracking method provided in the first embodiment of this disclosure. Furthermore, the processor is also used to adjust the movement direction and / or movement angle of the gimbal according to the coordinate changes of the UAV in two adjacent frames of images to complete the continuous tracking and shooting of the target UAV.

[0096] In some embodiments, the processor can determine whether it is necessary to move the gimbal to center the drone within the current frame based on the relationship between the drone's position in the current frame and the position of the center of the frame, thereby assisting in drone aiming. Specifically, the processor can calculate the center coordinates of the current visible light image based on the current visible light image and the drone coordinates determined by the detection and tracking method. and the centroid coordinates of the UAV The difference in the x-coordinates between Difference between the vertical and horizontal axes ,in, Then, the gimbal movement direction is controlled and adjusted according to the following conditions:

[0097] exist and At this time, do not adjust the movement direction of the gimbal;

[0098] exist and At that time, control the gimbal to move horizontally to the right;

[0099] exist and At that time, control the gimbal to move upward and to the right;

[0100] exist and At that time, control the gimbal to move vertically upwards;

[0101] exist and At that time, control the gimbal to move upward and to the left;

[0102] exist and At that time, control the gimbal to move horizontally to the left;

[0103] exist and At that time, control the gimbal to move downwards and to the left;

[0104] exist and At that time, control the gimbal to move vertically downwards;

[0105] exist and At that time, control the gimbal to move downwards and to the right;

[0106] in, The preset radius of the movement constraint area means that when the drone's position is not within the area with the center of the screen as the origin and radius R, the gimbal movement direction needs to be adjusted to meet the drone's position tracking requirements.

[0107] Furthermore, the processor can achieve precise aiming of the drone by adjusting the rotation angle of the gimbal, helping to fulfill potential drone strike requirements. Specifically, the processor uses the upper left corner of the visible light image as the origin, the horizontal direction to the right as the positive x-axis, and the vertical direction downwards as the positive y-axis, to obtain the coordinates of the detected drone in the image. Subsequently, based on the horizontal field of view of the visible light image acquisition device... and vertical field perspective Calculate the horizontal rotation angle of the gimbal with its rotation center as the origin. and the rotation angle in the vertical direction as follows:

[0108]

[0109]

[0110] Where W is the width of the visible light image and H is the height of the visible light image.

[0111] In a preferred embodiment, the processor also integrates an image stabilization algorithm that can compensate for image jitter caused by system vibration or wind, further improving image quality and target tracking stability.

[0112] In some embodiments, the system of this embodiment may further include a cloud platform. The cloud platform is used to store and display identification information, and adopts a distributed architecture, possessing high reliability and scalability. The cloud platform includes a data storage system, a data analysis system, and a user interface system. The data storage system is used to store the identification information of the UAV, including data such as UAV type, flight trajectory, speed, and altitude. The data analysis system can mine and analyze historical data, identify abnormal flight patterns, and predict potential threats. The user interface system provides an intuitive visual interface, displaying the UAV's location, trajectory, and attribute information in real time, and supports multi-terminal access, including PCs, mobile devices, and large-screen display systems. The cloud platform also has an alarm function, which can automatically send alarm information to relevant personnel when an unauthorized UAV intrusion is detected. In a preferred embodiment, the cloud platform also supports multi-system networking to achieve regional joint defense, allowing multiple tracking systems to share target information, coordinate tracking, and expand the monitoring range.

[0113] Figure 7This diagram illustrates the system hardware architecture of a dual-modal low-altitude, slow-moving, small unmanned aerial vehicle (UAV) detection and tracking system. The processor utilizes a Jetson TX2 as its core processing platform, providing powerful AI computing capabilities and rich peripheral interfaces for image processing and target recognition. The Jetson TX2 controls the gimbal via a serial port and connects to external sensors through I / O interfaces to acquire information such as illumination and position.

[0114] This embodiment of the dual-modal low-altitude, slow-moving, and small unmanned aerial vehicle (UAV) detection and tracking system achieves all-weather, high-precision tracking of UAVs by combining infrared and visible light dual-modal imaging technology, advanced image processing algorithms, and cloud platform technology. The system is suitable for security monitoring in areas such as airports, borders, and important facilities, providing an effective technical means for UAV intrusion detection and defense.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A detection and tracking method for a dual-modal low-speed small unmanned aerial vehicle (UAV), characterized in that, The application is applied to an unmanned aerial vehicle monitoring device, comprising: synchronously collecting infrared images and visible light images of the same airspace and obtaining current illumination intensity; inputting the infrared images, the visible light images and the illumination intensity into a preset dual-modal fusion recognition model to capture images and coordinates of the unmanned aerial vehicle; wherein the preset dual-modal fusion recognition model is constructed based on a dual-flow backbone network, a backbone network of the dual-flow backbone network comprises a cross-attention fusion module and parallel first and second backbone branches, the first backbone branch is used for processing infrared images, the second backbone branch is used for processing visible light images, each backbone branch comprises a plurality of spatial-depth residual convolution feature extraction modules, a feature aggregation module and an MViTBv3 self-attention mechanism module, and the cross-attention fusion module adjusts fusion weights between the first and second backbone branches according to the illumination intensity; tracking the unmanned aerial vehicle according to the images and coordinates of the unmanned aerial vehicle based on an improved BoTSORT tracking algorithm.

2. The detection tracking method of claim 1, wherein, The spatial-depth residual convolution feature extraction module is specifically configured to: input an input original feature map into an SPD layer for segmentation based on a down-sampling scale factor to obtain a plurality of sub-feature maps; splicing all the sub-feature maps along a channel dimension and performing convolution operation on the spliced sub-feature maps with a step of 1 to obtain a new feature map; The new feature map is subjected to stepless convolution to obtain a feature map with 3 3 The convolution kernel captures local features to obtain a feature map after convolution processing; element-by-element adding the convolution-processed feature map and the original feature map to obtain an output result of the spatial-depth residual convolution feature extraction module.

3. The detection tracking method of claim 1, wherein, The cross-attention fusion module is specifically configured to: perform normalization processing on the illumination intensity to obtain an illumination intensity coefficient; when the illumination intensity coefficient is greater than a first preset value, increasing the fusion weight of the first backbone branch and reducing the fusion weight of the second backbone branch; when the illumination intensity coefficient is less than or equal to the first preset value and greater than or equal to a second preset value, balancing the fusion weight of the first backbone branch and the fusion weight of the second backbone branch; when the illumination intensity coefficient is less than the second preset value, increasing the fusion weight of the second backbone branch and reducing the fusion weight of the first backbone branch.

4. The detection tracking method of claim 1, wherein, The improved BoTSORT tracking algorithm comprises: optimizing the intersection-over-union measurement method in the BoTSORT tracking algorithm by using a Focaler-IoU function, the Focaler-IoU function comprising: wherein, represents the original value of the intersection over union, represents the optimized intersection over union, and is an interval division parameter.

5. The detection tracking method according to any one of claims 1 to 4, characterized in that, after the synchronous collection of the infrared images and the visible light images of the same airspace, further comprising: performing super-resolution reconstruction processing on the infrared images based on a super-resolution reconstruction network; wherein the super-resolution reconstruction network comprises a shallow feature extraction module, a plurality of continuously stacked blueprint tri-core network modules, a multi-layer feature fusion module and a reconstruction module, wherein a first blueprint tri-core network module in all the blueprint tri-core network modules is connected to the reconstruction module by long jump.

6. The detection tracking method of claim 5, wherein, after the synchronous collection of the infrared images and the visible light images of the same airspace, further comprising: registering the visible light images and the super-resolution reconstructed infrared images.

7. A dual mode low slow small unmanned aircraft detection and tracking system characterized by, comprising: A gimbal, configured to carry a visible light image acquisition device and an infrared image acquisition device; An illumination sensor, configured to acquire a current illumination intensity; A processor, configured to implement the detection and tracking method of the dual-mode low-slow-small unmanned aerial vehicle according to any one of claims 1 to 6 to capture an image and coordinates of the unmanned aerial vehicle and track the unmanned aerial vehicle.

8. The detection tracking system of claim 7, wherein, The processor is further configured to: adjust a moving direction and / or a rotating angle of the gimbal according to the image and the coordinates of the unmanned aerial vehicle.

9. The detection tracking system of claim 8, wherein, The processor is specifically configured to: calculating a difference in a horizontal coordinate between a center coordinate of the current visible light image and a center of mass coordinate of the drone a difference in a horizontal coordinate and a difference in a vertical coordinate , ; In and when the yaw angle is greater than the pitch angle, the moving direction of the gimbal is not adjusted. In and when the gimbal is controlled to move horizontally to the right; In and when the pan-tilt is controlled to move to the upper right. In and when the gimbal is controlled to move vertically upward; In and when the angle of the gimbal is greater than the angle threshold, the gimbal is controlled to move to the left upper side. When and the gimbal is controlled to move horizontally to the left; In and when the pan-tilt is controlled to move to the left lower side; In and when the gimbal is controlled to move vertically downward; In and when the pan-tilt is controlled to move to the right lower side; wherein, is the preset radius of the movement constraint area.

10. The detection tracking system of claim 8, wherein, The processor is specifically configured to: Taking the upper left corner of the visible light image as the origin, the horizontal right direction as the positive direction of the x-axis, and the vertical downward direction as the positive direction of the y-axis, the coordinates of the detected unmanned aerial vehicle in the image are obtained as ; According to the horizontal field angle of view of the visible light image acquisition device and the vertical field angle of view , the rotation angle of the horizontal direction of the pan-tilt and the rotation angle of the vertical direction of the pan-tilt are calculated with the rotation center of the pan-tilt as the origin: wherein W represents a width of the visible light image, and H represents a height of the visible light image.

Images