Anti-unmanned aerial vehicle intelligent identification and tracking system based on multi-source data fusion

Abstract

The invention provides an anti-unmanned aerial vehicle intelligent identification and tracking system based on multi-source data fusion, and relates to the technical field of anti-unmanned aerial vehicle detection, and the system comprises a multi-source data preprocessing module which is used for outputting preprocessed multi-source data; the target detection module is used for carrying out unmanned aerial vehicle target detection on visual data in the preprocessed multi-source data and outputting a detection result containing a bounding box position, confidence and morphological characteristics; the target tracking module is used for performing unmanned aerial vehicle target tracking based on the target detection result and outputting a tracking result; and the fusion decision module is used for confirming the target identity based on the tracking result and the preprocessed multi-source data and outputting a final recognition result. The technical problems of low detection precision of small targets, difficulty in distinguishing similar targets, inaccurate 3D motion prediction, difficulty in re-identification after long-time shielding and the like in the prior art can be solved, and accurate identification, stable tracking and intelligent decision making of the unmanned aerial vehicle target are realized.

Description

Technical Field

[0001] This invention relates to the field of anti-drone detection technology, and in particular to an intelligent anti-drone identification and tracking system based on multi-source data fusion. Background Technology

[0002] Current target detection technologies in anti-drone systems mainly rely on single or simple combinations of sensor devices, such as radar, photoelectric, and radio detection. Existing target detection algorithms have significant shortcomings when dealing with small targets like drones: First, small targets occupy a very small proportion of the image, and key feature information is easily lost after multiple downsampling by convolutional neural networks; second, drones are highly similar in shape to targets such as birds and kites, making it difficult for traditional detection algorithms to effectively distinguish them; third, factors such as dust, changes in lighting, and ground cover in complex environments severely affect detection accuracy.

[0003] In terms of target tracking technology, most existing multi-target tracking algorithms are designed based on 2D planar motion models, which cannot accurately predict the trajectory of UAVs in 3D space. Meanwhile, traditional appearance feature extraction methods are poorly adaptable to changes in the appearance of UAVs under different angles and lighting conditions, leading to issues such as ID switching and tracking loss during tracking. Especially after prolonged occlusion, relying solely on visual information is insufficient for accurate target re-identification, limiting the robustness of the tracking system.

[0004] Chinese invention patent CN115932834A discloses a target detection method for anti-drone systems based on multi-source heterogeneous data fusion. This method achieves target detection by fusing data from radar, radio detection, ADS-B, and IFF modules, and finally uses inter-frame difference for image detection. However, this method only performs simple fusion at the data level and lacks in-depth algorithm optimization, especially in improving the target detection and tracking algorithms themselves, making it difficult to meet the requirements for accurate identification and stable tracking of drone targets in complex environments. Summary of the Invention

[0005] In view of this, the present invention provides an anti-drone intelligent identification and tracking system based on multi-source data fusion. By improving the YOLOv8 detection network and StrongSORT tracking algorithm, and combining the synergistic advantages of visual, sound and radar multi-source data, it solves the technical problems in the prior art such as low detection accuracy of small targets, difficulty in distinguishing similar targets, inaccurate 3D motion prediction, and difficulty in re-identification after long-term occlusion, so as to achieve accurate identification, stable tracking and intelligent decision-making of drone targets.

[0006] The technical solution of this invention is implemented as follows: This invention provides an intelligent anti-drone identification and tracking system based on multi-source data fusion, comprising: The multi-source data preprocessing module is used to collect multi-source data, including visual data, sound data and radar data, and to perform time synchronization and coordinate transformation on the multi-source data, and output the preprocessed multi-source data. The target detection module is used to perform UAV target detection on visual data in preprocessed multi-source data, including an improved YOLOv8 network model, which outputs detection results including bounding box position, confidence score and morphological features. The target tracking module is used for UAV target tracking based on target detection results. It includes a motion predictor, an appearance feature extractor, and a data correlator. The motion predictor predicts the target's motion state, the appearance feature extractor extracts the target's appearance features, and the data correlator correlates and matches the detection results with existing trajectories based on the predicted motion state and appearance features, and outputs the tracking results. The fusion decision module is used to confirm the target identity based on the tracking results and preprocessed multi-source data. It includes a behavior feature analyzer, a multi-source confidence fusion unit, and a hierarchical confirmer. The hierarchical confirmer verifies the target identity step by step based on the behavior features extracted by the behavior feature analyzer and the fusion results of the multi-source confidence fusion unit, and outputs the final identification result.

[0007] Preferably, in the multi-source data preprocessing module, time synchronization processing establishes a unified time reference and timestamps the data from each sensor. in, It is the synchronized timestamp. It is the sensor's original timestamp. It is the offset of each sensor relative to the reference time; Coordinate transformation processing converts the spatial coordinates of different sensors to a unified global coordinate system.

[0008] Preferably, the improved YOLOv8 network model includes an image preprocessing enhancement unit, a backbone feature extraction unit, a neck feature fusion unit, a head detection unit, and a post-processing unit, wherein: The image preprocessing enhancement unit uses CLAHE for local contrast enhancement and a bilateral filter to remove noise; the backbone feature extraction unit uses CSPDarknet53 to extract hierarchical features from the preprocessed and enhanced image; the neck feature fusion unit adds shallow feature direct connection paths on the basis of standard FPN and PAN; the head detection head includes three detection heads, each of which includes a bounding box regression branch, a classification branch, and a fine-grained morphological feature recognition branch; the post-processing unit is used to perform NMS filtering and coordinate transformation on the prediction results.

[0009] Preferably, the processing steps for improving the YOLOv8 network model include: The input image is subjected to CLAHE local contrast enhancement and bilateral filter denoising. The preprocessed and enhanced image was subjected to hierarchical feature extraction using CSPDarknet53 to obtain multi-layer features F1, F2, F3, F4, and F5. Feature fusion is performed on multi-layer features. A feature pyramid is constructed through top-down feature propagation via FPN and bottom-up feature aggregation via PAN. During the fusion process, the first layer features of the Backbone are transformed by a 1×1 convolutional layer to adjust the number of channels to match the top layer features of the FPN. Then, they are downsampled by a 3×3 convolutional layer with a stride of 2, and finally weighted and fused with the top layer features of the FPN. in, It is an enhanced feature after fusion. These are the directly connected features after dimension adjustment. yes Convolutional layer This is the first layer feature of the backbone. It is the top-level feature of FPN. These are the downsampled features. These are the weights for shallow features; Multi-branch prediction is performed based on the enhanced features after fusion. The bounding box regression branch, classification branch, and fine-grained morphological feature recognition branch output the bounding box regression result, class probability, and morphological feature vector, respectively. The morphological feature vector is 4-dimensional and corresponds to the number of rotors, fuselage shape, size ratio, and degree of motion blur, respectively. Calculate the confidence level of the multi-branch prediction results: in: It is the enhanced final detection confidence level. It is the probability of drone category. It is the first The probability of a morphological feature. It is the sigmoid activation function; Decode the bounding box and convert the output bounding box regression results from relative coordinates to absolute coordinates; Duplicate detections were removed by NMS screening to obtain the final detection results.

[0010] Preferably, the total loss function of the improved YOLOv8 network model includes standard detection loss and morphological feature loss: in, It is the total loss function. It is the bounding box regression loss. It is the IoU predicted loss. It is a loss due to drone classification. It is the distribution focus loss. It is a loss of morphological features. It is the first A true label for each morphological feature , , , represents the weights of each loss, and i represents the i-th detector head.

[0011] Preferably, the target tracking module includes: The motion predictor receives trajectory history information, identifies the target motion pattern through a motion pattern recognition network, uses an adaptive Kalman filter to predict the target's position and velocity in the current frame, and outputs the predicted bounding box and prediction confidence. The appearance feature extractor is used to extract the position and appearance features of the target from the detection results of the current frame, and generates the target descriptor by adopting a dual-scale feature extraction strategy with enhanced environment awareness. The data correlator receives the prediction results from the motion predictor and the target descriptor from the appearance feature extractor. It establishes the correspondence between the trajectory and the detected target through multimodal correlation cost calculation and outputs the tracking results.

[0012] Preferably, the motion pattern recognition network receives historical motion information of the trajectory and outputs four motion modes: uniform linear motion, maneuvering evasion motion, hovering observation motion, and circling flight motion. The input feature vector is: in, It is the first 3D velocity vector at time step It is the first 3D acceleration vector at time t, Indicates the size of the history window; The adaptive Kalman filter dynamically adjusts the process noise covariance matrix Q and the observation noise covariance matrix R based on the motion pattern recognition results.

[0013] Preferably, the data correlator adopts a multimodal correlation cost calculation strategy, which establishes the correspondence between trajectory and detected target by integrating visual correlation cost, sound correlation cost and radar correlation cost. The visual correlation cost is calculated by combining position and appearance information, the sound correlation cost is calculated based on acoustic feature matching, and the radar correlation cost is calculated based on 3D position and distance. The comprehensive correlation cost is obtained by adaptively weighting and fusing the correlation costs of each modality.

[0014] Preferably, the target tracking module also includes a trajectory management strategy, including: the trajectory survival time is dynamically adjusted according to the motion mode and confidence level; uniform linear motion trajectories maintain a basic survival time; maneuvering evasive motion trajectories reduce survival time; hovering observation motion trajectories increase survival time; and roving flight motion trajectories moderately increase survival time; a prediction update strategy is adopted under occlusion conditions, during trajectory mismatch, only the position information predicted by the Kalman filter is updated, and the appearance features are not updated; the trajectory deletion condition is that continuous mismatch exceeds the maximum survival time, and a trajectory re-identification mechanism is included.

[0015] Preferably, the graded verification device uses a three-level verification mechanism to confirm the identity of the target, including: the first level of verification performs preliminary screening based on the comprehensive confidence level, and marks targets with a comprehensive confidence level exceeding the preliminary threshold as suspected drones; the second level of verification combines the results of behavioral feature analysis to perform secondary verification on targets that simultaneously meet the confidence level requirement and the behavioral feature requirement; the third level of verification performs final verification on targets that have passed the first two levels of verification, calculates the final verification score by combining all information, and when the verification score exceeds the final threshold, the target is confirmed as a drone and the final identification result is output.

[0016] The present invention has the following advantages over the prior art: (1) By constructing a collaborative working mechanism of a multi-source data preprocessing module, a target detection module, a target tracking module and a fusion decision module, this invention achieves deep fusion of visual, sound and radar data, effectively solves the problem of insufficient detection accuracy and poor robustness of a single sensor in complex environments, and improves the target recognition accuracy and tracking stability of the anti-drone system. (2) By adding a fine-grained morphological feature recognition branch, the improved YOLOv8 network model can effectively extract key features such as the number of rotors, fuselage shape, size ratio and motion ambiguity of the UAV. Combined with the improved confidence calculation method, it improves the ability of UAV to distinguish from similar targets such as birds and kites, and reduces false detection and false negative detection. (3) The feature fusion technology of shallow feature direct connection path directly transmits the first layer features of the Backbone to the top of the feature pyramid, retains rich details, effectively improves the problem of feature information loss in small target detection, and improves the detection performance of long-distance and small-sized UAV targets; (4) The motion pattern recognition network, combined with the adaptive Kalman filter, identifies four typical motion patterns: uniform linear motion, maneuvering evasion motion, hovering observation motion, and roving flight motion. It dynamically adjusts the filter parameters, improves the prediction accuracy of complex motion trajectories of UAVs, and enhances the adaptability and stability of the tracking system. (5) The multimodal association cost calculation data correlator establishes a more accurate correspondence between trajectory and detection target by integrating visual, sound and radar information, effectively reducing the probability of ID switching and target loss during the tracking process, and showing stronger robustness, especially in occlusion and interference environments. (6) The trajectory management strategy dynamically adjusts the trajectory survival time according to different motion modes and confidence levels, and adopts a prediction update strategy to handle occlusion states, thereby avoiding feature pollution problems and improving the stability of long-term tracking and the accuracy of re-identification. Attached Figure Description

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a schematic diagram of the improved YOLOv8 network structure of the present invention; Figure 3 This is a schematic diagram of the target tracking module of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, this invention provides an intelligent anti-drone identification and tracking system based on multi-source data fusion, comprising: The multi-source data preprocessing module is used to collect multi-source data, including visual data, sound data and radar data, and to perform time synchronization and coordinate transformation on the multi-source data, and output the preprocessed multi-source data. The target detection module is used to perform UAV target detection on visual data in preprocessed multi-source data, including an improved YOLOv8 network model, which outputs detection results including bounding box position, confidence score and morphological features. The target tracking module is used for UAV target tracking based on target detection results. It includes a motion predictor, an appearance feature extractor, and a data correlator. The motion predictor predicts the target's motion state, the appearance feature extractor extracts the target's appearance features, and the data correlator correlates and matches the detection results with existing trajectories based on the predicted motion state and appearance features, and outputs the tracking results. The fusion decision module is used to confirm the target identity based on the tracking results and preprocessed multi-source data. It includes a behavior feature analyzer, a multi-source confidence fusion unit, and a hierarchical confirmer. The hierarchical confirmer verifies the target identity step by step based on the behavior features extracted by the behavior feature analyzer and the fusion results of the multi-source confidence fusion unit, and outputs the final identification result.

[0021] Anti-drone systems refer to comprehensive defense systems that utilize technologies such as detection, tracking, identification, and countermeasures to monitor, interfere with, deceive, control, or destroy threatening drones. In terms of technical implementation, anti-drone systems involve the coordinated operation of multiple sensors and subsystems, primarily including various detection technologies such as radar detection, photoelectric identification, radio eavesdropping, and acoustic detection, as well as target detection and tracking algorithms based on deep learning. However, existing anti-drone systems still face technical challenges in areas such as small target detection, adaptability to complex environments, and stability in multi-target tracking. In particular, they suffer from problems such as the ease with which small drone targets are missed in complex backgrounds, difficulty in distinguishing them from similar targets like birds, and difficulty in re-identification after prolonged occlusion. This invention addresses these technical problems in existing technologies by improving YOLOv8 and StrongSORT and combining the collaborative advantages of multi-source data, such as low accuracy in small target detection, difficulty in distinguishing similar targets, inaccurate 3D motion prediction, and difficulty in re-identification after prolonged occlusion.

[0022] Specifically, the technical approach of this invention is as follows: Addressing the technical problems in existing anti-drone systems, such as insufficient detection accuracy of single sensors, loss of small target features, inaccurate 3D motion prediction, and difficulty in re-identification under long-term occlusion, this invention constructs an intelligent anti-drone identification and tracking system based on multi-source data fusion. This system achieves time synchronization and coordinate unification of visual, audio, and radar data through a multi-source data preprocessing module. It employs an improved YOLOv8 network model, adding a fine-grained morphological feature recognition branch to the detection head and shallow feature direct connection paths to the Neck network to enhance small target detection and similar target differentiation capabilities. Based on an improved StrongSORT algorithm, it achieves accurate 3D motion prediction through a motion pattern recognition network and an adaptive Kalman filter. It utilizes multimodal association cost calculation and intelligent trajectory management strategies to improve tracking stability and occlusion recovery capabilities. Finally, it achieves accurate target identity confirmation through a three-level verification mechanism in the fusion decision module. This deeply optimizes detection and tracking performance at the algorithm level, and combined with collaborative verification of multi-source data, improves the overall identification accuracy and tracking stability of the anti-drone system.

[0023] Specifically, in one embodiment of the present invention, the multi-source data preprocessing module first establishes a unified time base and performs timestamp calibration on the data from each sensor. Since the internal clocks of different sensors may differ, a time synchronization algorithm is needed to eliminate time deviations. In specific implementation, the system sets a reference time. As a benchmark, the synchronization time of the data from each sensor is calculated as follows: in: It is the synchronized timestamp. It is the sensor's original timestamp. This represents the offset of each sensor relative to the reference time. Offset The system calibration process determines that during system initialization, the offset is calculated by sending a synchronization signal to each sensor and recording the response time.

[0024] Based on basic time synchronization, the system establishes a time sliding window mechanism to perform temporal fusion of multiple frames of data within the window. The window size is set to W=5 frames, and the time decay constant is τ=2. Multi-frame data information is fused using a weighted average method. For visual data, the fusion formula is: For visual data, information from multiple images is fused using a weighted average method: For audio data, multiple frames of audio signals are fused using a time-domain sliding window: For radar data, measurement accuracy is improved through multi-frame data fusion: in: Let τ = 2 be the time decay weight, and let I(t−k), A(t−k), and R(t−k) be the visual, audio, and radar data at time t−k, respectively. Weight normalization ensures that the sum of the fused weights is 1. .

[0025] To address the issue of different sensors having their own independent coordinate systems, the multi-source data preprocessing module transforms the spatial coordinates of each sensor to a unified global coordinate system. The transformation relationship from sensor coordinates to global coordinates is as follows: in: These are 3D coordinates in the global coordinate system. These are 3D coordinates in the sensor coordinate system. It is the rotation matrix from the sensor to the global coordinate system. It is the translation vector from the sensor to the global coordinate system.

[0026] Rotation matrix Translation vector This is obtained through the sensor calibration process. After the system is installed, multi-point calibration is performed using a calibration target within the sensor's observation range. The coordinate transformation parameters are solved using the least squares method to ensure that the same target measured by different sensors has a consistent spatial position in the global coordinate system.

[0027] In this embodiment, the preprocessing module also performs format unification processing on various types of sensor data. Visual data is normalized to a fixed size of 1280×720 pixels, with pixel values ​​normalized to the range of [0,1], while maintaining the aspect ratio of the original image. Images that do not conform to the target size are adjusted by padding or cropping. Audio data is uniformly sampled at 44.1kHz, with a fixed time window of 200ms. For audio data with inconsistent sampling rates, a resampling algorithm is used to adjust to a uniform sampling rate; for audio data with inconsistent time window lengths, truncation or zero padding is used to adjust to a fixed length. After preprocessing such as pre-emphasis and windowing, the audio data is converted into a frequency domain feature representation. Radar data is uniformly formatted as range-azimuth-elevation, with a range accuracy set to 0.1m and an angle accuracy set to 0.1°. For radar data with inconsistent accuracy, interpolation or quantization is used to adjust to uniform accuracy. Radar data also needs to undergo clutter filtering and target extraction processing to remove ground clutter and other interference signals, while retaining effective target echo information.

[0028] Specifically, in one embodiment of the present invention, the target detection module implements UAV target detection based on an improved YOLOv8 network model. The module receives preprocessed visual data output by the multi-source data preprocessing module and outputs detection results including bounding box position, confidence level and morphological features.

[0029] like Figure 2 As shown, in this embodiment, the improved YOLOv8 network model includes an image preprocessing enhancement unit, a backbone feature extraction unit, a neck feature fusion unit, a head detection head, and a post-processing unit, wherein: The image preprocessing enhancement unit uses CLAHE for local contrast enhancement and a bilateral filter to remove noise; the backbone feature extraction unit uses CSPDarknet53 to extract hierarchical features from the preprocessed and enhanced image; the neck feature fusion unit adds shallow feature direct connection paths on the basis of standard FPN and PAN; the head detection head includes three detection heads, each of which includes a bounding box regression branch, a classification branch, and a fine-grained morphological feature recognition branch; the post-processing unit is used to perform NMS filtering and coordinate transformation on the prediction results.

[0030] The process of improving the YOLOv8 network model includes: The input image is subjected to CLAHE local contrast enhancement and bilateral filter denoising. Specifically, the image preprocessing enhancement unit performs quality enhancement on the input image. First, it uses CLAHE (Contrast Limited Adaptive Histogram Equalization) for local contrast enhancement. The CLAHE algorithm enhances local contrast through adaptive histogram equalization, effectively improving image quality in complex environments. The processing procedure is as follows: in: It is the original input image. This is the image after CLAHE processing. The CLAHE algorithm's cropping limit is set to 2.0, and the grid size is set to 8×8.

[0031] Then, a bilateral filter is used to remove noise while preserving edge information. The bilateral filter can remove noise while maintaining the sharpness of image edges. The filtering process is as follows: in: This is the final preprocessed image after bilateral filtering. It is the diameter of the filter. It is the spatial standard deviation. It is the standard deviation of grayscale.

[0032] The preprocessed and enhanced image was subjected to hierarchical feature extraction using CSPDarknet53 to obtain multi-layer features F1, F2, F3, F4, and F5. Specifically, the Backbone feature extraction unit uses the CSPDarknet53 network to perform hierarchical feature extraction on the preprocessed and enhanced image, obtaining multi-scale feature maps. CSPDarknet53 reduces computational complexity while maintaining feature expressiveness through cross-stage partial connections. The feature extraction process is as follows: in: These are feature maps from different stages, with sizes as follows: , and These are the height and width of the input image.

[0033] Feature fusion is performed on multi-layer features. A feature pyramid is constructed through top-down feature propagation via FPN and bottom-up feature aggregation via PAN. During the fusion process, the first layer features of the Backbone are transformed by a 1×1 convolutional layer to adjust the number of channels to match the top layer features of the FPN. Then, they are downsampled by a 3×3 convolutional layer with a stride of 2, and finally weighted and fused with the top layer features of the FPN. in, It is an enhanced feature after fusion. These are the directly connected features after dimension adjustment. yes Convolutional layer This is the first layer feature of the backbone. It is the top-level feature of FPN. These are the downsampled features. These are the weights for shallow features; The role of the Neck network is to effectively fuse multi-scale features and construct a feature pyramid structure to achieve the detection of targets at different scales.

[0034] The standard YOLOv8 employs a bidirectional feature fusion architecture combining FPN (Feature Pyramid Network) and PAN (Path Aggregation Network). First, FPN upsampling fusion is performed to pass deeper semantic information to shallower layers. in This represents the convolution operation. This indicates an upsampling operation. Then, PAN downsampling fusion is performed to transfer detailed information from shallower layers to deeper layers. in: This is the feature map after PAN processing. This indicates a downsampling operation.

[0035] The improvement of this invention lies in adding a shallow feature direct connection path specifically for small target detection to the traditional FPN-PAN architecture. Since UAVs often appear as small targets with very few pixels when detecting targets at long distances, the traditional feature pyramid structure easily loses crucial details of these small targets during multiple downsampling and upsampling processes. To address this issue, this invention establishes a path in the Neck network starting from the first layer of the Backbone... To the top of the feature pyramid A direct connection path ensures that high-resolution features of small targets can be directly transmitted to the detection head.

[0036] In the specific implementation process, firstly... Perform dimensional adaptation, through Convolutional layer Adjust the channel dimensions: yes Convolutional layers achieve dimensionality transformation: , yes The number of channels, yes The number of channels.

[0037] In order to Space dimensions alignment is required. Further spatial dimension processing is performed. This is done using a 3×3 convolutional layer with a stride of 2. Perform downsampling, adjusting its spatial dimensions from H / 4×W / 4 to H / 8×W / 8: After completing the dimension adaptation, the processed shallow features Compared with the original FPN features Weighted fusion is performed. The feature fusion process employs an adaptive weight allocation mechanism based on small target density. The system first analyzes the small target density of the current input image, calculated using the following formula: in and These represent the number of smaller goals and the total goal, respectively. and These represent the area proportions of the small target and the total target, respectively. Based on the density of the small targets, the shallow layer feature weights are dynamically adjusted as follows: This adaptive weighting mechanism ensures that in scenes with dense small targets, shallow high-resolution features can receive greater weight, thereby better preserving the feature information of small targets.

[0038] The directly connected features are weighted and fused with the original FPN features to obtain the enhanced feature map: in These are enhanced features resulting from fusion, used to replace the original ones. The subsequent PAN processing is then performed. This feature map, which has undergone shallow information enhancement, will continue to participate in the PAN downsampling and fusion process, ultimately generating three feature maps for detection.

[0039] Multi-branch prediction is performed based on the enhanced features after fusion. The bounding box regression branch, classification branch, and fine-grained morphological feature recognition branch output the bounding box regression result, class probability, and morphological feature vector, respectively. The morphological feature vector is 4-dimensional and corresponds to the number of rotors, fuselage shape, size ratio, and degree of motion blur, respectively. Specifically, YOLOv8 has three detection heads: the first, second, and third detection heads. Each detection head is responsible for processing a feature map of a specific scale (P3, P4, P5) to achieve the detection of targets of different sizes.

[0040] YOLOv8's detector heads employ a decoupled head design, meaning each detector head internally uses a decoupled head structure. Each decoupled head separates the prediction task into two independent branches: a bounding box regression branch and a classification branch. The standard YOLOv8 decoupled head structure is as follows: in: Output the bounding box regression results. Output the class probability.

[0041] The improvement of this invention lies in adding a third branch to each decoupling head, in addition to the original two branches. This third branch is the fine-grained morphological feature recognition branch. in: Output 4D morphological feature vector These correspond to the number of rotors, fuselage shape, size ratio, and degree of motion ambiguity, respectively.

[0042] In this invention, each decoupling head contains three parallel output branches: a regression branch, a classification branch, and a morphological feature branch. This multi-branch prediction architecture significantly improves the accuracy of identifying UAV targets while maintaining the original YOLOv8 detection performance, effectively solving the problem of distinguishing UAVs from similar targets such as birds and kites.

[0043] The morphological feature recognition branch is implemented using a lightweight convolutional sub-network structure. This branch receives feature maps from the Neck network as input and processes them through three... Convolutional layers are stacked to extract morphological features. The first convolutional layer adjusts the number of channels in the input feature map to 128, the second convolutional layer further adjusts the number of channels to 64, and the third convolutional layer outputs 4 channels, corresponding to the four morphological feature dimensions. Each convolutional layer is followed by batch normalization and ReLU activation function to ensure the network's training stability and feature representation ability.

[0044] The specific meanings of the four morphological feature dimensions are as follows: It indicates the number of rotors and identifies the presence of rotors by detecting periodic structures within the target area; It indicates the shape characteristics of the fuselage, and determines whether it is a man-made object by analyzing the geometry and symmetry of the target; It represents the size proportion characteristics, and evaluates the geometric features of the target by calculating the aspect ratio and compactness of the target; It represents the degree of motion blur and determines the motion state of the target by analyzing the blur kernel features of the image.

[0045] Calculate the confidence level of the multi-branch prediction results: in: It is the enhanced final detection confidence level. It is the probability of drone category. It is the first The probability of a morphological feature. It is a sigmoid activation function; this fusion method ensures that only targets that simultaneously meet the requirements of category classification and morphological features can obtain high confidence, thereby improving the accuracy of UAV target recognition.

[0046] Decode the bounding box and convert the output bounding box regression results from relative coordinates to absolute coordinates; Duplicate detections were removed by NMS screening to obtain the final detection results.

[0047] Specifically, the post-processing unit is responsible for processing the prediction results, including bounding box decoding, NMS filtering, and coordinate transformation.

[0048] Convert the relative coordinates output by the network to absolute coordinates using Anchor-Free mode: in: It is the bounding box in the absolute coordinate system. These are the relative coordinates output by the network. These are grid coordinates. It is the downsampling factor.

[0049] Using the calculated enhanced confidence level All decoded bounding boxes are filtered. First, detection boxes with confidence scores below a preset threshold are filtered out. Then, redundant overlapping boxes targeting the same object are eliminated using the Non-Maximum Suppression (NMS) algorithm, and the box with the highest confidence score is retained as the final detection result. in: It is the first The set of frame detection results. It is the first One test result, These are the bounding box coordinates. It's the confidence level. It is a category label.

[0050] In this embodiment, the total loss function of the improved YOLOv8 network model includes standard detection loss and morphological feature loss: in: It is the total loss function. It is the bounding box regression loss (using CIou Loss). It is a classification loss (using BCE Loss). It is the distribution focus loss. It is a loss of morphological features. , , , represents the weights of each loss, and i represents the i-th detector head.

[0051] The morphological feature loss is expressed as cross-entropy: in: It is the first The true label (0 or 1) for each morphological feature. It is the first The predicted probability of each morphological feature.

[0052] Specifically, in one embodiment of the present invention, the target tracking module includes: The motion predictor receives trajectory history information, identifies the target motion pattern through a motion pattern recognition network, uses an adaptive Kalman filter to predict the target's position and velocity in the current frame, and outputs the predicted bounding box and prediction confidence. The appearance feature extractor is used to extract the position and appearance features of the target from the detection results of the current frame, and generates the target descriptor by adopting a dual-scale feature extraction strategy with enhanced environment awareness. The data correlator receives the prediction results from the motion predictor and the target descriptor from the appearance feature extractor. It establishes the correspondence between the trajectory and the detected target through multimodal correlation cost calculation and outputs the tracking results.

[0053] like Figure 3 As shown, the target tracking module implements multi-target tracking based on the improved StrongSORT algorithm. The complete process includes five stages: trajectory prediction, feature extraction, data association, trajectory update, and trajectory management.

[0054] Phase 1: Trajectory Prediction To address the diverse motion patterns of unmanned aerial vehicles (UAVs) in complex environments, this invention designs an adaptive motion pattern recognition mechanism based on neural networks. For each existing trajectory, the system first analyzes its historical motion patterns and then dynamically adjusts the parameters of the Kalman filter.

[0055] The system establishes a lightweight motion pattern classification network. This network receives historical motion information of the trajectory as input and outputs the motion pattern category at the current moment. The input feature vector is defined as follows: in, It is the first 3D velocity vector at time step It is the first 3D acceleration vector at time t, Indicates the size of the history window.

[0056] The network classifies motion patterns using three fully connected layers, outputting four typical UAV motion patterns: Uniform Motion, Maneuvering Motion, Hovering Motion, and Patrol Motion. The motion pattern prediction probability is calculated as follows: in: It is the first Motion patterns at all times These correspond to four different sports modes. , , It is the network weight matrix. , , It is the bias vector.

[0057] This embodiment uses an 8-dimensional state vector to process 3D motion. The state vector is defined as follows: in, These are the pixel center coordinates. These are depth coordinates. These are the width and height of the bounding box. It is a 3D velocity component.

[0058] Based on motion pattern recognition results, the system dynamically adjusts the process noise covariance matrix Q and the observation noise covariance matrix R of the Kalman filter. For different motion patterns, the system employs different combinations of noise parameters: in, and These are the process noise and observation noise covariance matrices corresponding to the k-th motion mode, respectively. Specific parameters are set as follows: Uniform linear motion: , ; Maneuvering evasive maneuvers: ; Hovering to observe motion: ; Touring flight sports: .

[0059] The state transition equation adopts an adaptive form: Wherein, the state transition matrix Adjust according to the exercise mode: in, It is the velocity coupling matrix related to the motion mode. This is the velocity decay matrix. The parameter settings for different motion modes are as follows: Uniform linear motion: ; Maneuvering evasive maneuvers: ; Hovering to observe motion: ; Touring flight sports: .

[0060] Phase Two: Feature Extraction for Enhanced Environmental Perception The feature extraction process is adaptively adjusted based on environmental conditions. The system first assesses the impact of the current environment on different sensors, and then dynamically adjusts the feature extraction strategy.

[0061] The system first assesses the impact of the current environment on feature extraction: in: , These are the signal-to-noise ratios of the audio signal and the radar signal, respectively. , , It's about adjusting parameters. It is the signal-to-noise ratio threshold. It is a comprehensive image quality index, calculated by weighting the image's contrast, sharpness, and noise level.

[0062] Based on the environmental assessment results, the system dynamically adjusts the scale parameters for feature extraction. For each detection result in the current frame, the system employs an environment-adaptive dual-scale strategy: Cropping target regions of different sizes from the detection frame: The feature extraction process uses a pre-trained ResNet-50 network: The final features are obtained through weighted fusion: Phase 3: Multi-level Adaptive Data Association Based on environmental conditions and target motion characteristics, the system dynamically adjusts the weights of different modal information to achieve more robust data association.

[0063] Multimodal association cost calculation: Visual association cost combines location and appearance information: It is Mahalanobis distance. It is the cosine distance. , .

[0064] Sound association cost is based on acoustic feature matching: Radar correlation cost is based on 3D location distance: Adaptive weight fusion: The overall associated cost is calculated through adaptive weight fusion: Phase 4: Status Update of Motion Pattern Perception Based on the identified motion patterns, the system adopts corresponding state update strategies to improve the accuracy and stability of tracking.

[0065] Observation equation design: Combining visual and radar observations, the observation equation is: Where: observation matrix Designed as follows: Adaptive Kalman Update: Kalman gain calculation employs adaptive observation noise covariance: State update equation: Covariance update: Feature update strategy: Weights for feature updates are adjusted based on motion patterns: Phase 5: Intelligent Trajectory Management Based on motion patterns and environmental conditions, the system employs an intelligent trajectory management strategy to improve the stability of long-term tracking.

[0066] Trajectory Management Strategy: To improve occlusion recovery capabilities, a strategy of extending trajectory lifespan is adopted. Trajectories remain in prediction state even when there are more than 30 consecutive mismatches, with a maximum lifespan of 100 frames. During the mismatch period, only the positional information of the Kalman prediction is updated, without updating appearance features to avoid feature contamination. New appearance features are written only after a re-match.

[0067] Trajectory lifecycle management: The system adjusts the survival time of the trajectory based on the motion pattern: in: It is the base survival time. It's the amount of adjustment for the sports mode. This is the confidence level adjustment. The motion mode adjustment is set to: uniform linear motion. Maneuvering and evasive maneuvers: Hovering to observe motion: Touring flight sports: The confidence level adjustment is calculated based on the overall confidence level. , It is the overall confidence level, and the adjustment range is... .

[0068] Occlusion prediction and handling: The system predicts potential occlusion by analyzing motion trajectories and environmental information: in: It is a motion trend vector. It is a scene structure vector. It is a vector of historical occlusion information. It is the sigmoid function.

[0069] When the occlusion probability exceeds the threshold of 0.6, the system, based on the original 30-frame mismatch judgment, starts the enhanced prediction mode in advance, using a larger process noise covariance matrix. .

[0070] The system adopts a trajectory status management mechanism, including status transition logic such as creating a new trajectory, confirming a trajectory, and deleting a trajectory.

[0071] Trajectory Initialization: For unmatched detection results, the system initializes a new trajectory. The state vector of the new trajectory is initialized as follows: ;in: This indicates the location and size of the detection result, with the velocity initialized to 0. The new trajectory is set to "pending confirmation," requiring subsequent continuous matching to verify its validity.

[0072] Track Confirmation: A new track needs to be matched continuously for a certain number of frames to be confirmed as a valid track. The confirmation conditions are: Only confirmed trajectories are output to the final tracking results. This mechanism effectively filters out false trajectories caused by false detections.

[0073] Under occlusion conditions, the system employs the following strategy: 1. Only use the Kalman filter for position prediction, without updating appearance features, to avoid feature contamination: ; 2. The trajectory remains active and continues to attempt predictions and associations until a successful rematch is achieved or the maximum survival time is exceeded. 3. When a trajectory successfully rematches, normal state and feature updates resume.

[0074] Track deletion: The track deletion strategy determines when to remove tracks that have not matched for a long time from the system. This invention employs a strategy to extend track lifespan: This strategy significantly extends the survival time of the trajectory compared to traditional methods and improves the ability to re-identify the target after long-term occlusion.

[0075] Track lifecycle management: Each track has a complete lifecycle state machine, including: Initialization state: Newly created track, waiting for confirmation; Confirmation state: Confirmed valid track, tracking normally; Occlusion state: Mismatch exceeding 30 frames, prediction only, no update; Deletion state: Mismatch exceeding 100 frames, ready for deletion.

[0076] Final output: The tracking results include trajectory ID, 3D position, motion pattern, and confidence level. ;in: It is a trajectory identifier. It is a state vector. It's in sports mode. It is the overall confidence level.

[0077] Specifically, in one embodiment of the present invention, the behavior feature analyzer extracts behavior features by analyzing the target's historical motion trajectory, mainly including two dimensions: trajectory smoothness and motion regularity. Trajectory smoothness reflects the continuity and stability of the target's motion. The system measures the smoothness of motion by calculating the degree of change in the velocity vector between adjacent time points. For UAV targets, normal cruise flight typically exhibits high trajectory smoothness, generally greater than 0.7; while during maneuvering evasive flight, the smoothness drops to the range of 0.3-0.6. The system uses a sliding window mechanism to analyze the velocity changes in the most recent 8 frames and obtains the smoothness index through normalization. Motion regularity assesses the consistency of the target's motion direction. The system analyzes the changing trend of the motion direction angle and calculates the standard deviation and frequency of change of direction. Due to their artificial control characteristics, UAVs maintain a certain degree of motion regularity even during maneuvering flight, which is significantly different from naturally flying objects such as birds. The system sets a regularity threshold of 0.6; targets exceeding this value are considered to have artificial control characteristics. The system weights and combines trajectory smoothness and motion regularity to form a comprehensive behavior feature score. The trajectory smoothness is weighted at 0.6, and the motion regularity is weighted at 0.4. This weighting is based on statistical analysis of the UAV's motion characteristics and can effectively distinguish the UAV from other flying targets.

[0078] The multi-source confidence fusion engine integrates confidence information from visual, acoustic, and radar sensors, dynamically adjusting the weights of each sensor based on current environmental conditions. Visual confidence is based on the enhanced confidence output from the target detection module, adjusted in conjunction with trajectory stability. The system considers factors such as the number of consecutive trajectory matching frames and detection box stability to correct the initial confidence. A higher number of consecutive trajectory matching frames results in a corresponding increase in visual confidence, reflecting the stability of target tracking. Acoustic confidence is based on the matching degree between acoustic features and a pre-trained UAV acoustic model. The system extracts acoustic features from the target area, including spectral and temporal features, and matches them with UAV acoustic templates in the database. Confidence assessment of the acoustic modality is obtained through similarity calculation and threshold judgment. Radar confidence comprehensively considers position consistency and radar cross-sectional area matching. Position consistency is assessed by comparing the deviation between the radar detection position and the visually predicted position; a smaller deviation indicates higher confidence. Radar cross-sectional area matching compares the detected echo intensity with typical UAV RCS features to assess the target's radar feature conformity. The system dynamically adjusts the weights of each sensor based on environmental assessment factors. In well-lit environments, visual sensors are given higher weight; in complex acoustic environments, sound sensors are given lower weight; and in environments with electromagnetic interference, the weight of radar sensors is adjusted. The final overall confidence score is calculated using a weighted average to ensure full utilization of high-quality sensor information.

[0079] The hierarchical verification device uses a three-level verification mechanism to confirm the identity of the target, improving the reliability of the identification level by level.

[0080] Level 1 Validation: Confidence Level Screening; Level 1 validation performs a preliminary screening based on overall confidence level. The system sets an initial threshold of 0.5, marking targets exceeding this threshold as "suspected drones," while targets below the threshold are directly excluded. This level of validation primarily serves as a coarse screening, filtering out targets that clearly do not meet the criteria.

[0081] Level 2 Validation: Behavioral Feature Analysis; Level 2 validation combines the results of behavioral feature analysis to conduct a more detailed analysis of targets that have passed Level 1 validation. The system requires targets to simultaneously meet both confidence and behavioral feature requirements, i.e., the behavioral feature score must exceed a threshold of 0.6. Only targets that pass both confidence and behavioral feature validation can proceed to Level 3 validation; other targets are marked as "to be observed".

[0082] Level 3 Verification: Final Confirmation; Level 3 verification performs final confirmation, calculating a final confirmation score based on all available information. This score considers four dimensions: fusion confidence, behavioral feature score, trajectory length, and multi-frame consistency, with weights of 0.4, 0.3, 0.2, and 0.1, respectively. When the final confirmation score exceeds a threshold of 0.75, the system confirms the target as a drone; otherwise, it continues to observe, awaiting more information for further judgment.

[0083] The fusion decision module outputs complete target information, including the target's unique identifier, overall confidence level, final confirmation score, current position and velocity, complete motion trajectory, and behavioral characteristics.

[0084] The system is capable of handling abnormal situations. When a sensor malfunction is detected, the weight of the faulty sensor is automatically set to zero, and the fusion result is recalculated. Under extreme environmental conditions, the system dynamically lowers the confirmation threshold to ensure that reasonable detection performance is maintained even under harsh conditions.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A counter-drone intelligent identification and tracking system based on multi-source data fusion, characterized in that, include: The multi-source data preprocessing module is used to collect multi-source data, including visual data, sound data and radar data, and to perform time synchronization and coordinate transformation on the multi-source data, and output the preprocessed multi-source data. The target detection module is used to perform UAV target detection on visual data in preprocessed multi-source data, including an improved YOLOv8 network model, which outputs detection results including bounding box position, confidence score and morphological features. The target tracking module is used for UAV target tracking based on target detection results. It includes a motion predictor, an appearance feature extractor, and a data correlator. The motion predictor predicts the target's motion state, the appearance feature extractor extracts the target's appearance features, and the data correlator correlates and matches the detection results with existing trajectories based on the predicted motion state and appearance features, and outputs the tracking results. The fusion decision module is used to confirm the target identity based on the tracking results and preprocessed multi-source data. It includes a behavior feature analyzer, a multi-source confidence fusion unit, and a hierarchical confirmer. The hierarchical confirmer verifies the target identity step by step based on the behavior features extracted by the behavior feature analyzer and the fusion results of the multi-source confidence fusion unit, and outputs the final identification result.

2. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 1, characterized in that, In the multi-source data preprocessing module, time synchronization processing establishes a unified time reference and timestamps the data from each sensor. in, It is the synchronized timestamp. It is the sensor's original timestamp. It is the offset of each sensor relative to the reference time; Coordinate transformation processing converts the spatial coordinates of different sensors to a unified global coordinate system.

3. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 1, characterized in that, The improved YOLOv8 network model includes an image preprocessing enhancement unit, a backbone feature extraction unit, a neck feature fusion unit, a head detection unit, and a post-processing unit, wherein: The image preprocessing enhancement unit uses CLAHE for local contrast enhancement and a bilateral filter to remove noise; the backbone feature extraction unit uses CSPDarknet53 to extract hierarchical features from the preprocessed and enhanced image; the neck feature fusion unit adds shallow feature direct connection paths on the basis of standard FPN and PAN; the head detection head includes three detection heads, each of which includes a bounding box regression branch, a classification branch, and a fine-grained morphological feature recognition branch; the post-processing unit is used to perform NMS filtering and coordinate transformation on the prediction results.

4. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 3, characterized in that, The process of improving the YOLOv8 network model includes: The input image is subjected to CLAHE local contrast enhancement and bilateral filter denoising. The preprocessed and enhanced image was subjected to hierarchical feature extraction using CSPDarknet53 to obtain multi-layer features F1, F2, F3, F4, and F5. Feature fusion is performed on multi-layer features. A feature pyramid is constructed through top-down feature propagation via FPN and bottom-up feature aggregation via PAN. During the fusion process, the first layer features of the Backbone are transformed by a 1×1 convolutional layer to adjust the number of channels to match the top layer features of the FPN. Then, they are downsampled by a 3×3 convolutional layer with a stride of 2, and finally weighted and fused with the top layer features of the FPN. in, It is an enhanced feature after fusion. These are the directly connected features after dimension adjustment. yes Convolutional layer This is the first layer feature of the backbone. It is the top-level feature of FPN. These are the downsampled features. These are the weights for shallow features; Multi-branch prediction is performed based on the enhanced features after fusion. The bounding box regression branch, classification branch, and fine-grained morphological feature recognition branch output the bounding box regression result, class probability, and morphological feature vector, respectively. The morphological feature vector is 4-dimensional and corresponds to the number of rotors, fuselage shape, size ratio, and degree of motion blur, respectively. Calculate the confidence level of the multi-branch prediction results: in: It is the enhanced final detection confidence level. It is the probability of drone category. It is The probability of a morphological feature. It is the sigmoid activation function; Decode the bounding box and convert the output bounding box regression results from relative coordinates to absolute coordinates; Duplicate detections were removed by NMS screening to obtain the final detection results.

5. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 4, characterized in that, The improved YOLOv8 network model's total loss function includes standard detection loss and morphological feature loss: in, It is the total loss function. It is the bounding box regression loss. It is the IoU predicted loss. It is a loss due to drone classification. It is the distribution focus loss. It is a loss of morphological features. It is A true label for each morphological feature , , , represents the weights of each loss, and i represents the i-th detector head.

6. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 1, characterized in that, The target tracking module includes: The motion predictor receives trajectory history information, identifies the target motion pattern through a motion pattern recognition network, uses an adaptive Kalman filter to predict the target's position and velocity in the current frame, and outputs the predicted bounding box and prediction confidence. The appearance feature extractor is used to extract the position and appearance features of the target from the detection results of the current frame, and generates the target descriptor by adopting a dual-scale feature extraction strategy with enhanced environment awareness. The data correlator receives the prediction results from the motion predictor and the target descriptor from the appearance feature extractor. It establishes the correspondence between the trajectory and the detected target through multimodal correlation cost calculation and outputs the tracking results.

7. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 6, characterized in that, The motion pattern recognition network receives historical motion information about trajectories and outputs four motion modes: uniform linear motion, maneuvering evasion motion, hovering observation motion, and circling flight motion. The input feature vector is: in, It is 3D velocity vector at time step It is 3D acceleration vector at time t, Indicates the size of the history window; The adaptive Kalman filter dynamically adjusts the process noise covariance matrix Q and the observation noise covariance matrix R based on the motion pattern recognition results.

8. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 6, characterized in that, The data correlator adopts a multimodal correlation cost calculation strategy. It establishes the correspondence between trajectory and detected target by integrating visual correlation cost, sound correlation cost and radar correlation cost. The visual correlation cost is calculated by combining position and appearance information, the sound correlation cost is calculated based on acoustic feature matching, and the radar correlation cost is calculated based on 3D position and distance. The comprehensive correlation cost is obtained by adaptively weighting and fusing the correlation costs of each modality.

9. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 6, characterized in that, The target tracking module also includes trajectory management strategies, including: trajectory survival time is dynamically adjusted based on motion mode and confidence level; uniform linear motion trajectories maintain a basic survival time; maneuvering evasive motion trajectories reduce survival time; hovering observation motion trajectories increase survival time; and roving flight motion trajectories moderately increase survival time. Under occlusion conditions, a prediction update strategy is adopted, updating only the position information predicted by the Kalman filter during trajectory mismatch, without updating appearance features. Trajectory deletion is conditional upon consecutive mismatches exceeding the maximum survival time, and includes a trajectory re-identification mechanism.

10. The anti-drone intelligent identification and tracking system based on multi-source data fusion according to claim 1, characterized in that, The graded verification system employs a three-level verification mechanism to confirm target identity, including: Level 1 verification performs preliminary screening based on comprehensive confidence level, marking targets with comprehensive confidence levels exceeding a preliminary threshold as suspected drones; Level 2 verification combines behavioral feature analysis results to perform secondary verification on targets that simultaneously meet both confidence level and behavioral feature requirements; Level 3 verification performs final verification on targets that have passed the first two levels of verification, calculating a final verification score based on all information, and when the verification score exceeds the final threshold, the target is confirmed as a drone and the final identification result is output.

Images