A UAV Ground Target Detection and Tracking Method Based on Multi-Scale Complementary Feature Aggregation

Abstract

This invention discloses a UAV ground target detection and tracking method based on multi-scale complementary feature aggregation, belonging to the fields of computer vision and UAV perception. This method explicitly decouples the detection head from the re-identification feature extraction module and performs targeted modeling. After completing target detection in UAV aerial images, the constructed re-identification feature extraction module uses a high-resolution feature map as the sole input to calculate the appearance features of each target box in the detection results. This fully utilizes the local texture and edge detail information in the high-resolution features, reducing the problem of background texture and interference from neighboring targets when directly extracting appearance embeddings from low-resolution high-semantic feature maps. Thus, it enhances the identity discrimination capability of small-scale targets without significantly increasing system complexity. Then, the extracted appearance features are used for trajectory association, which can effectively improve the identity preservation capability in multi-target tracking in dense scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and UAV perception, and more specifically, relates to a UAV ground target detection and tracking method based on multi-scale complementary feature aggregation. Background Technology

[0002] Drones can flexibly collect data and provide unique perspectives, significantly improving the comprehensiveness of scene perception. With the continuous development of drone technology, camera-equipped drones have been widely used in various fields such as fire safety and intelligent traffic management. However, drone scene understanding algorithms based on target detection and multi-target tracking still face many challenges.

[0003] Drone aerial images typically have a wide field of view, with target objects occupying only a small area of ​​the image. Information on target edges, textures, and local structures is weak, and the flexible shooting angles of gimbal cameras can cause significant scale variations in the target, further increasing the difficulty of target detection. Since multi-target tracking usually relies on detection results for subsequent trajectory association, detection box position deviations and missed target detections directly affect trajectory continuity, leading to trajectory breaks or incorrect matches. Therefore, the detection stage needs to possess good small target representation capabilities, multi-scale adaptability, and complex background suppression capabilities to provide stable and reliable target candidates for subsequent trajectory association.

[0004] In multi-target tracking, the accuracy of trajectory association depends not only on the location of the bounding boxes but also on the ability to distinguish the identities of different targets. Existing post-detection tracking methods typically first obtain the bounding boxes of the current frame through a detection network, and then match the current detection results with historical trajectories based on the degree of overlap, positional relationship, or appearance features of the bounding boxes. For typical UAV aerial photography scenarios such as traffic roads, parking lots, and intersections, there are a large number of targets that are densely distributed, with similar appearances and scales, and they may occlude or intersect with each other. In such cases, relying solely on geometric information such as the degree of overlap of the bounding boxes for trajectory association is prone to mismatches due to the proximity of the detected boxes, the similarity of the target appearances, or the short-term occlusion of the targets, leading to identity switching and trajectory discontinuity.

[0005] To enhance target identity preservation, some multi-target tracking methods introduce re-identification features. Re-identification features provide identity discrimination information beyond geometric location for target association, playing a crucial role when targets are occluded, intersecting, or close together. However, existing re-identification feature extraction methods still have certain limitations. One type of method directly relies on low-resolution, high-semantic feature maps in the detection network to extract appearance embeddings. Low-resolution feature maps contain limited effective information corresponding to the target region, making it difficult to fully preserve target edges, local textures, and spatial details. Furthermore, they are prone to incorporating background textures and information from neighboring targets, resulting in insufficient discriminative ability in appearance embedding. Another type of method utilizes higher-resolution features to simultaneously perform target detection and re-identification feature extraction, but the feature requirements of detection and re-identification tasks are not entirely consistent. The re-identification branch is still susceptible to feature bias in the detection task, background response, and interference from neighboring targets, leading to insufficient discriminative power for small-scale target identity features. In scenarios with dense small targets, such as those encountered by drones, this problem further weakens the stability of trajectory association, making it easier for targets to switch identities when intersecting, occluded, or briefly adjacent.

[0006] Therefore, designing a detection and tracking scheme with strong adaptability, good tracking stability, and good real-time performance is an urgent technical challenge to be solved, addressing issues such as small target size, large scale variation, strong interference from complex backgrounds, and difficulty in maintaining identity in dense multi-target scenes in UAV images. Summary of the Invention

[0007] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a UAV ground target detection and tracking method based on multi-scale complementary feature aggregation. This solves the problems in existing UAV ground target detection and multi-target tracking technologies, such as weak stability of multi-scale target detection, susceptibility of re-identified features to interference from background and nearby targets, and insufficient ability to distinguish identities through trajectory association in dense scenes.

[0008] To achieve the above objectives, according to a first aspect of the present invention, a UAV ground target detection and tracking system based on multi-scale complementary feature aggregation is provided, comprising: The convolution module is used to extract features from aerial images captured by the drone to obtain image features P1; Spatial semantic feature complementary unit, used to extract multi-scale features from P1 to obtain multi-scale features P2, P3, and P4; A progressive receptive field extension unit is used to sequentially perform local context enhancement and global context enhancement on P4 to obtain the global enhanced feature P5; The multi-scale feature splitting and aggregation unit includes first to fourth splitting and aggregation modules, which are respectively used to process the feature spliced ​​from P5 and P3 to obtain the first fused feature P6, process the feature spliced ​​from P6 and P2 to obtain the second fused feature P7, process the feature spliced ​​from P6 and P7 to obtain the third fused feature P8, and process the feature spliced ​​from P8 and P5 to obtain the fourth fused feature P9. The detection head is used to predict the target category and target bounding box position in the aerial image based on P8 and P9. The re-identification feature extraction module is used to calculate the bounding boxes for each target based on P7. appearance features ; The re-identification feature extraction module includes: a deep convolutional layer, a pointwise convolutional layer, a channel recalibration layer, and an embedding mapping layer; after local encoding processing by the deep convolutional layer, P7 is input to the pointwise convolutional layer to compress and reassemble the channel dimensions to obtain intermediate features. Channel recalibration layer Weighted features are obtained by recalibrating the channels. Embedded mapping layer according to Calculate each target box appearance features ; The trajectory association module is used to calculate the appearance similarity between the appearance features of each target box in the aerial image and the appearance features corresponding to the historical trajectory, and to construct an association cost based on the appearance similarity to complete the matching between the current detection result and the historical trajectory.

[0009] According to a second aspect of the present invention, an electronic device is provided, comprising: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0010] According to a third aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to perform the method as described in the first aspect.

[0011] According to a fourth aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0012] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: The system provided by this invention explicitly decouples the detection head from the re-identification feature extraction module and performs targeted modeling. After completing target detection in UAV aerial images, the re-identification feature extraction module is constructed. Using a high-resolution feature map as the sole input, it calculates the appearance features of each target box in the detection results. This fully utilizes the local texture and edge detail information in the high-resolution features, reducing the problem of background texture and interference from neighboring targets when directly extracting appearance embeddings from low-resolution, high-semantic feature maps. Thus, it enhances the ability to identify small-scale targets without significantly increasing system complexity. The extracted appearance features are then used for trajectory association, which can effectively improve the ability to maintain the identity of multiple targets in dense scenes.

[0013] The training method provided by this invention first trains the target detection network to obtain stable target localization capabilities and multi-scale feature representation capabilities, providing reliable detection results for subsequent trajectory association. Then, pre-trained detection weights are loaded, and the re-identification feature extraction module is trained or fine-tuned separately to make appearance features more suitable for target identity differentiation. This phased training approach can enhance the discriminative ability of re-identification features while maintaining detection performance, reducing mutual interference between the detection task and identity feature learning, thereby improving overall tracking performance. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the structure of a UAV ground target detection and tracking system based on multi-scale complementary feature aggregation provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the spatial semantic feature complementary module structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the progressive receptive field extension unit structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the re-identification feature extraction module and detection head provided in an embodiment of the present invention; Figure 5 The image shows the tracking results provided in an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0016] This invention provides a UAV ground target detection and tracking system based on multi-scale complementary feature aggregation, such as... Figure 1As shown, it includes: The convolution module is used to extract features from aerial images captured by the drone to obtain image features P1 with a resolution of p1.

[0017] Specifically, the convolutional module is used to simultaneously perform feature extraction and feature map downsampling of the input image. As an example, the convolutional module can use a 3×3 convolutional layer with a stride of 2.

[0018] Preferably, before the aerial image is input to the convolution module, the aerial image is preprocessed, including data augmentation, scale unification and normalization, to obtain a standardized input image.

[0019] The spatial semantic feature complementarity unit includes three sequentially connected spatial semantic feature complementarity modules, which are used to extract multi-scale features P2, P3, and P4 with resolutions of p2, p3, and p4, respectively.

[0020] Specifically, the spatial semantic feature complementarity modules all adopt a structure in which spatial feature branches and semantic feature branches are explicitly decoupled and dynamically complemented to enhance the multi-scale feature representation of small targets.

[0021] The spatial semantic feature complementarity module includes a spatial branch and a semantic branch, which respectively output spatial features. With semantic features Furthermore, it utilizes dynamic complementary modulation and fusion of spatial attention and channel attention to fuse and output features. satisfy:

[0022] in, For element-wise modulation operators, Let be the channel attention mapping function, satisfying For spatial attention mapping function, For the fusion mapping function, and the fusion mapping This is a dynamic weighted summation.

[0023] The channel attention mapping Spatial attention mapping They respectively satisfy:

[0024]

[0025] in, For aggregation operators of spatial dimensions, Generate a mapping function for the channel weights. Generate a mapping function for the spatial weights. This is the normalized activation mapping function.

[0026] like Figure 2 As shown, the input features directly enter the two branches of the spatial-semantic feature complementarity module for spatial and semantic feature extraction. Both branches utilize reparameterized convolutions for core feature extraction. Then, the spatial branch obtains the features. ( For the number of channels, (The feature map) is generated into a size of [size missing] through 3x3 convolution and sigmoid operation. A single-channel spatial mask ranging from 0 to 1, which is related to semantic branch features. Element-by-element multiplication yields Semantic branch features The spatial dimension is compressed using Global Average Pooling (GAP), followed by pointwise convolution and sigmoid to generate channel weight vectors, which are then applied to the spatial branch features to generate channel-modulated features. .

[0027] The formula is as follows:

[0028] The last two branches are dynamically weighted to obtain the modulated output characteristics. Weight As the feature map size decreases, the formula is as follows:

[0029] A progressive receptive field extension unit is used to sequentially perform local context enhancement and global context enhancement on P4 to obtain the global enhanced feature P5.

[0030] Specifically, the progressive receptive field extension unit includes a local context enhancement module and a C2PSA module connected in sequence; the local context enhancement module is used to perform local context enhancement on P4 to obtain local enhanced features. The C2PSA module is used for local enhancement features. Global context enhancement is performed to obtain the global enhanced feature P5.

[0031] The local context enhancement module is jointly composed of SPPF and large kernel attention, and the C2PSA module is the C2PSA module in YOLO11. The overall mapping relationship is expressed as:

[0032]

[0033] in, The input is a high-dimensional semantic feature (i.e., P4). Features enhanced by local context The output feature (i.e., P5) is the result of global self-attention modeling. This is a mapping for modeling the local context, which is jointly constructed by SPPF and large kernel attention. Model the global context mapping composed of C2PSA.

[0034] a) Local context enhancement like Figure 3 As shown, for input features Dimensionality reduction is performed along the channel dimension, compressing the number of channels from C to C / 2, resulting in dimensionality-reduced features. This reduces computational load while filtering redundant information. Then, based on... Using core size The max-pooling layer is used for sequential triple pooling to generate four sets of features with different receptive fields (corresponding to receptive field sizes of 5x5, 9x9, 13x13, and 17x17, respectively). These four sets of features are then concatenated along the channel dimension to form a multi-scale fused feature. , This indicates max pooling with a kernel size of 5. Indicates continuous operation Submax pooling.

[0035]

[0036] for Large-scale separable kernel attention (LSKA) is introduced. Features are obtained after channel normalization Then, by decomposing a convolutional kernel of size k = 3 into 1xk and kx1 stripe separable convolutions and performing bidirectional convolution operations, the encoding of the target basic spatial association is completed. Then, with a void ratio of 2 and a core size of 2... The large-kernel strip convolution is also decomposed into two convolutions: 1 x 5 and 5 x 1. The last two bidirectional depthwise separable strip convolution operations yield an equivalent receptive field of kernel k = 11, enabling the encoding of long-range spatial dependencies. This achieves the effect of capturing long-range dependencies of the target without significantly increasing computational cost, generating large-kernel attention features. Finally, the number of feature channels is restored to the original dimension, and the final local enhancement feature is output. .

[0037] as follows:

[0038]

[0039]

[0040]

[0041] in , These are bar-shaped depth-separable convolution kernels. This is a convolution operation. These are separable strip convolution kernels with separable hole depths. The mask obtained after 1x1 convolution encoding can achieve feature enhancement and background suppression of the target region.

[0042] b) Global Context Modeling Global context modeling is performed using the C2PSA module. The C2PSA module models the relationships between different spatial locations in the feature map through a global self-attention mechanism, thus supplementing the shortcomings of large kernel attention, which primarily focuses on local region modeling, and obtaining the final globally enhanced features. .

[0043]

[0044] Therefore, the final output of the entire progressive receptive field expansion module is: (i.e., P5) represents the feature representation after further global relation modeling based on local context enhancement.

[0045] The multi-scale feature splitting and aggregation unit includes first to fourth splitting and aggregation modules, which are respectively used to process the features spliced ​​from P5 and P3 to obtain the first fused feature P6, process the features spliced ​​from P6 and P2 to obtain the second fused feature P7, process the features spliced ​​from P6 and P7 to obtain the third fused feature P8, and process the features spliced ​​from P8 and P5 to obtain the fourth fused feature P9.

[0046] Specifically, each splitting and aggregation module has different inputs, but the function is the same: to split and aggregate the multi-scale fusion features of the input to suppress redundancy and preserve details, such as... Figure 4 As shown, the splitting process generates a multi-branch feature set from the fused features. The output features are obtained through aggregation and compression. ,satisfy:

[0047] in, For channel-dimensional splicing operators, This refers to the channel compression and remapping function. And the branch feature set... It includes four categories: channel calibration branch, spatial coding branch and detail preservation branch, and dynamic enhancement branch.

[0048] The first to fourth split-aggregation modules of the multi-scale feature splitting and aggregation unit are used to process the feature pyramids P2, P3, and P5 obtained after passing through the backbone feature extraction network (including convolutional modules, spatial semantic feature complementation units, and progressive receptive field expansion units), which have resolutions of p2, p3, and p4 respectively. That is, P2, P3, and P5 correspond to 1 / 4, 1 / 8, and 1 / 16 of the input image resolution, respectively. The first to fourth split-aggregation modules then perform bidirectional feature fusion on these feature pyramids from bottom to top and from top to bottom. Before fusing feature maps of different resolutions, sampling is required, and the resulting fused features are obtained by stitching together the sampled data.

[0049] In each splitting and aggregation module, the input features are first expanded in channel dimension using pointwise convolution to generate basic intermediate features. Then, they undergo a multi-branch processing flow.

[0050] (a) Basic channel calibration branch The input features are subjected to pointwise convolution to restore and calibrate the channel dimensions, resulting in calibrated features. Redundant features in the channel dimension are eliminated.

[0051]

[0052] (b) Lightweight Spatial Coding Branch Composed of two 1x1 convolutions at the beginning and end, and a depthwise separable convolution with k=3 in the middle, spatial feature encoding is completed with extremely low computational cost, finally yielding... This approach preserves the spatial details of small targets while significantly reducing the computational cost of spatial convolution.

[0053] . (c) Original feature branching By dividing the input features into two equal parts along the channel dimension, the original high-resolution detail features without additional operations are preserved, avoiding the loss of small target information caused by repeated downsampling.

[0054] . (d) Dynamically enhanced branches Implemented using stacked FasterNet Blocks (FBNs). The FBN module first divides the input features into two groups based on the number of channels: C and C / 4. Only the features with C / 4 channels are subjected to 3x3 convolutions, while the other three groups retain their original distribution. Then, the two groups are concatenated along the channel dimension. First, the channels are expanded using 1x1 convolutions, and then convolutions without normalization and activation are used to restore the original channel dimension. This multilayer perceptron-like design dynamically weights the channels, adaptively adjusting the feature weights to enhance effective target features.

[0055]

[0056]

[0057] In the formula, n>1, For the input of the FBN module, For 3x3 partial convolutions, spatial blending is performed only on 1 / 4 of the channels; Implement dynamic weight allocation along the channel dimension. Apply DropPath to the MLP output. During training, randomly set elements of the feature tensor to zero with a probability of 0.1 to achieve random depth regularization and prevent overfitting. During inference, scale the features by 0.9 to ensure that the output expectation remains unchanged.

[0058] (e) Aggregation and Compression Stage All the above branches are concatenated to form a multi-branch feature set, and then the channel dimension is compressed to the target dimension through 1 x 1 convolution to complete the efficient aggregation of features. The overall process can be represented as follows.

[0059]

[0060] In the backbone feature extraction network, the feature maps are downsampled only to p4. Therefore, during bidirectional feature fusion, feature maps at resolutions p2 to p4 are fused. The high-resolution p2 feature map P7 can incorporate more spatial detail information and is subsequently applied to the re-identification feature extraction module. However, for the detection head, introducing a p2 layer would significantly increase the post-processing computation. Therefore, the detection head only processes feature maps at resolutions p3 and p4.

[0061] The detection head is used to perform bounding box regression and classification prediction based on multi-scale features P8 and P9, and outputs the target category and target box location prediction in the drone aerial image.

[0062] The detection head uses an existing structure, such as the detection head of YOLO11.

[0063] The re-identification feature extraction module is used to calculate the bounding boxes of each target based on the high-resolution feature map P7. appearance features That is, P7 does not input the detection head to predict the target category and the target box position, but instead serves as the input to the re-identification feature extraction module to generate appearance features, which are used to characterize the identity of the target box.

[0064] The re-identification feature extraction module includes: a deep convolutional layer, a pointwise convolutional layer, a channel recalibration layer, and an embedding mapping layer; after local encoding processing by the deep convolutional layer, P7 is input to the pointwise convolutional layer to compress and reassemble the channel dimensions to obtain intermediate features. Channel recalibration layer Weighted features are obtained by recalibrating the channels. Embedded mapping layer according to Calculate each target box appearance features .

[0065] Specifically, the re-identification feature extraction module employs a joint structure of local texture encoding and channel recalibration. For the input high-resolution feature map P7, depthwise convolution is first used to locally encode the features, and then pointwise convolution is used to compress and reorganize the channel dimensions.

[0066] in, For depthwise convolution, a 3×3 convolutional layer can be used. To achieve pointwise convolution, a 1×1 convolutional layer can be used.

[0067] To further enhance the response to the discriminative channel and suppress background noise, intermediate features... Perform channel recalibration:

[0068]

[0069] After obtaining the weighted features, the features are adjusted to the set output dimension through pointwise convolution, and finally the final identity feature vector (i.e. appearance feature vector) is obtained through the embedding mapping layer. And perform normalization:

[0070] That is, in the re-identification feature extraction module, for each target bounding box b output by the detection head, its appearance features are obtained from the high-resolution embedded feature map P7 through coordinate mapping.

[0071] The trajectory association module is used to calculate the association cost between the aerial image and the historical trajectory based on the appearance features of each target box, and to complete the trajectory association by combining motion information or geometric overlap information.

[0072] The trajectory association module adopts Bytetrack's two-stage association paradigm, with re-identified features used to calculate the association cost separately. Let the j-th detection result... The bounding box and appearance features are denoted as follows: and Then the appearance similarity between targets i and j is... and geometric similarity It can be represented as:

[0073] The corresponding association cost is expressed as:

[0074] In the actual tracking process, the appearance similarity or association cost is used together with the motion prediction result or geometric overlap information to detect the matching between the result and the historical trajectory. The Hungarian algorithm is used to obtain the global optimal matching result, thereby completing the trajectory association and outputting the tracking result with identification.

[0075] This invention provides a training method for a UAV ground target detection and tracking system based on multi-scale complementary feature aggregation as described in any of the above embodiments, comprising: S1, the convolutional module, spatial semantic feature complementarity unit, progressive receptive field expansion unit, multi-scale feature splitting and aggregation unit, and detection head are trained using the first dataset; The first dataset includes positive and negative sample images and their class labels, and the ground truth bounding boxes in the positive sample images; the loss function used during training... , , 、 All are loss coefficients. , , These are the classification loss, DFL loss, and Gaussian combined distance, respectively.

[0076] Specifically, the training supervision of the convolutional module, spatial semantic feature complementarity unit, progressive receptive field expansion unit, multi-scale feature splitting and aggregation unit, and detection head includes joint optimization of label assignment and loss. Label assignment uses Gaussian Combined Distance (GCD) to characterize the matching relationship between predicted and ground truth bounding boxes, and together with classification confidence, constructs an alignment metric to filter positive samples. The loss function consists of classification loss and regression loss, where the regression loss includes distributed bounding box regression loss (DFL) and localization loss based on Gaussian distance, thereby improving regression stability and model robustness in complex scale scenarios.

[0077] For the prediction box GT box (i.e., truth box) The center coordinates, width, and height are calculated and defined as follows: ,

[0078] ,

[0079] set up >0 prevents division by zero. Define the center-of-symmetry difference term and the scale difference term:

[0080]

[0081] Thus, the GCD distance is obtained. Similarity mapping :

[0082] During model training, the label assignment parameters are adaptively adjusted based on the scale information of the real targets. Smaller-scale targets are assigned a higher number of positive candidate samples and / or a higher localization similarity weight to enhance the supervision strength of small targets and improve assignment stability. The distance threshold is measured as follows:

[0083] in ∈[0,1] varies with the target scale The changes mean that small-scale targets are more dependent on GCD similarity, while large-scale targets are more dependent on IoU similarity. In the experiment, a piecewise function was used, with the largest pixel area less than a threshold of 0.01. Set to 0, and set to 1 for pixels whose area ratio is greater than the threshold.

[0084] Then, the label assignment process is carried out, and the label assignment is based on classification confidence. Construct an alignment metric based on location similarity. By selecting a set of positive samples, calculate the alignment metric between the anchor point and the ground truth (GT). :

[0085] Where s is the classification score and o is the location similarity. and We assign weight values, for example, 1 and 6 respectively. This ensures that anchors with high classification confidence and high localization accuracy are prioritized for localization. The top k anchors with the highest alignment metrics are selected as candidate anchors; in this experiment, k is set to 13. Anchor centers within the ground truth (GT) generate the final positive samples. For multiple assignment conflicts, the GT with the highest overlap is assigned to the predicted value.

[0086] The loss function for network training is as follows: The loss in network training consists of classification loss and regression loss. (1) Classification loss Classification loss is used to optimize the model's prediction of the target class. Standard binary cross-entropy loss is used. N is the total number of samples. The true labels are 1 positive sample and 0 negative samples. To predict probabilities, This is the Sigmoid function.

[0087]

[0088] (2) Regression loss (a) DFL loss The DFL loss is used to supervise the accuracy of the bounding box distribution. , which represents the left and right grid indices corresponding to consecutive positions of the target. represents the left and right weights. P is the predicted distribution logits of the bounding box regression. This is the cross-entropy loss function.

[0089]

[0090] (b) Gaussian combination distance loss Gaussian combined distance loss is used in conjunction with label assignment. The weight of the k-th positive sample is obtained by weighting the confidence scores in the label assignment. The GCD similarity of the k-th positive sample is calculated using the same method as the label assignment. This is the index for positive samples.

[0091]

[0092] (3) Overall loss function The overall loss function is as follows, where , 、 These represent the loss coefficients. By strengthening the localization weights and fusing GCD Gaussian modeling, the regression accuracy and model robustness in complex-scale scenarios are significantly improved while addressing the gradient vanishing problem for non-overlapping targets.

[0093]

[0094] S2, to ensure detection performance, based on the pre-trained weights of the detection network (including convolutional modules, spatial semantic feature complementarity units, progressive receptive field expansion units, multi-scale feature splitting and aggregation units, and detection head) obtained in S1, the detection network parameters are kept fixed or updated with a small learning rate, and the re-identification feature extraction module is trained using a second dataset containing identity information.

[0095] The second dataset includes positive and negative sample images and their category labels, as well as ground truth bounding boxes and their identity information in the positive sample images. Different ID numbers can be used to represent the identity information of different ground truth bounding boxes. Consistent IDs within the same frame indicate positive samples with the same identity; otherwise, they are negative samples with different identities. The identity of a sample is determined based on its ID during loss calculation.

[0096] Specifically, for the re-identification feature extraction module, this invention employs a method of loading pre-trained detection weights and then training the re-identification feature extraction module separately to obtain optimal tracking performance. The loss of the re-identification feature extraction module... We employ triplet loss. Let the nth sample image in the second dataset be... i The target bounding box of each detection result (A detection result includes a target category and a bounding box) The appearance features are as follows: Therefore, in the embedding space, triplet sets of real target bounding box samples, positive target bounding box samples with the same identity, and negative target bounding box samples with different identities are constructed, and their embedding vectors are respectively... , and The triplet loss is defined as follows:

[0097] Where m represents the interval parameter. M represents the number of triples constructed during training. This loss requires that the distance between the anchor box and the positive sample be as small as possible, while the distance between the anchor box and the negative sample be at least greater than the interval of the positive sample distance m, thereby forming a more compact and separable identity embedding space.

[0098] The workflow of the system provided by this invention will be further illustrated below with a specific example.

[0099] (1) Data input and preprocessing: For RGB three-channel images captured by UAV aerial photography, the original image is first loaded and Mosaic data augmentation is applied. Then, it is uniformly scaled to 640x640 size and finally normalized preprocessing is performed to obtain a standardized tensor that meets the requirements of the model backbone input.

[0100] (2) Core feature extraction: The spatial semantic feature complementary unit adopts a structure that explicitly decouples and complements spatial and semantic information to extract features from the input image at multiple scales; the spatial branch is used to extract the spatial location and structural details of the target, and the semantic branch is used to extract high-level semantic feature information. The features of the two branches are dynamically modulated and fused through spatial attention and channel attention to enhance the retention of small target features during the layer-by-layer propagation of the network.

[0101] (3) Progressive receptive field expansion: The high-level semantic features are first aggregated with local context through the Spatial Pyramid Pooling Fast Structure (SPPF), and a large kernel attention mechanism is superimposed on the SPPF output to expand the local context range and enhance the local spatial dependency modeling ability to obtain local enhanced features. Then, a global self-attention module is connected to the local enhanced features to perform global receptive field modeling, thus forming a progressive receptive field expansion process from local to global, so as to improve the context perception ability of scale-changing targets and enhance the stability of small target detection.

[0102] (4) Multi-scale feature splitting and aggregation: After dimensional expansion, the fused features are split into channel calibration branch, lightweight spatial coding branch, original feature preservation branch and dynamic enhancement branch to suppress redundancy caused by sampling fusion and preserve high-resolution details. Then, the outputs of each branch are concatenated and compressed by pointwise convolution to obtain the output features, thereby highlighting key target features and enhancing the response of small targets without significantly increasing computational overhead.

[0103] (5) Detection head prediction: To balance real-time processing efficiency, only the two lower-resolution feature maps are fed into the detection head for bounding box prediction. The detection results are as follows: Figure 5 As shown, the detection head employs a dual-branch design that decouples category and regression, ultimately generating category predictions and discrete distribution results of bounding boxes. The position of the bounding boxes is confirmed by the expected values ​​of the discrete distributions in four directions.

[0104] (6) Re-identification feature extraction: The re-identification feature extraction module uses the high-resolution feature layer as the only input to calculate the appearance features of each target box.

[0105] (7) Trajectory Association and Output: During the tracking process, the extracted appearance features are used to calculate the appearance similarity between the current frame detection result and the historical trajectory, and trajectory association is completed by combining motion information or geometric overlap information. The target detection and tracking results with identification are output, such as... Figure 5 As shown.

[0106] The system was trained and validated using the publicly available UAV target detection benchmark datasets VisDrone2019 and UAVDT. The original annotation format was uniformly converted to the YOLO standard training format; the officially defined training and validation sets were used to ensure consistent scene distribution. During the training phase, data augmentation strategies such as random horizontal flipping, random cropping, and brightness / contrast adjustment were employed to improve the model's generalization ability.

[0107] As an example, the training parameters are designed as follows: the initial learning rate is set to 0.01, and a cosine annealing scheduler is used to dynamically adjust the learning rate, which decays according to the cosine function after each iteration, eventually dropping to 1e-5 of the initial value; the batch size is set to 16 to fully utilize GPU memory and improve training efficiency; the number of training epochs is 300, with the first 5 epochs being warm-up training, where the learning rate is gradually increased from 1e-4 to 0.01 to avoid model oscillation caused by an excessively high initial learning rate; SGD is selected as the optimizer, with momentum set to 0.937 and weight decay set to 0.0005 to suppress model overfitting.

[0108] This invention provides a UAV ground target detection and tracking method based on multi-scale complementary feature aggregation, comprising: The aerial image of the UAV to be detected is input into the UAV ground target detection and tracking system based on multi-scale complementary feature aggregation as described in any of the above embodiments to obtain the detection and tracking results.

[0109] This invention provides an electronic device, including: a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the training method or detection and tracking method as described in any of the above embodiments.

[0110] This invention provides a computer-readable storage medium storing computer instructions for causing a processor to execute a training method or a detection and tracking method as described in any of the above embodiments.

[0111] This invention provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the training method or detection and tracking method as described in any of the above embodiments.

[0112] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A UAV ground target detection and tracking system based on multi-scale complementary feature aggregation, characterized in that, include: The convolution module is used to extract features from aerial images captured by the drone to obtain image features P1; Spatial semantic feature complementary unit, used to extract multi-scale features from P1 to obtain multi-scale features P2, P3, and P4; A progressive receptive field extension unit is used to sequentially perform local context enhancement and global context enhancement on P4 to obtain the global enhanced feature P5; The multi-scale feature splitting and aggregation unit includes first to fourth splitting and aggregation modules, which are respectively used to process the feature spliced ​​from P5 and P3 to obtain the first fused feature P6, process the feature spliced ​​from P6 and P2 to obtain the second fused feature P7, process the feature spliced ​​from P6 and P7 to obtain the third fused feature P8, and process the feature spliced ​​from P8 and P5 to obtain the fourth fused feature P9. The detection head is used to predict the target category and target bounding box position in the aerial image based on P8 and P9. The re-identification feature extraction module is used to calculate the bounding boxes for each target based on P7. appearance features ; The re-identification feature extraction module includes a deep convolutional layer, a pointwise convolutional layer, a channel recalibration layer, and an embedding mapping layer. After local encoding processing by the deep convolutional layer, P7 is input to the pointwise convolutional layer to compress and reassemble the channel dimensions to obtain intermediate features. Channel recalibration layer Weighted features are obtained by recalibrating the channels. Embedded mapping layer according to Calculate each target box appearance features ; The trajectory association module is used to calculate the appearance similarity between the appearance features of each target box in the aerial image and the appearance features corresponding to the historical trajectory, and to construct an association cost based on the appearance similarity to complete the matching between the current detection result and the historical trajectory.

2. The system as described in claim 1, characterized in that, intermediate features The calculation formula is: in, For depthwise convolution, For pointwise convolution; Weighted features The calculation formula is: in, , For global average pooling, For standard convolution, This is the Sigmod activation function; Each target box appearance features The calculation formula is: in, To target bounding box exist Bilinear interpolation mapping is performed at the corresponding positions in the matrix.

3. The system as described in claim 1 or 2, characterized in that, The appearance similarity , For the first i Individual test results target box Appearance characteristics For the first j Individual test results target box Appearance features.

4. A training method for a UAV ground target detection and tracking system based on multi-scale complementary feature aggregation as described in any one of claims 1-3, characterized in that, include: S1, the convolutional module, spatial semantic feature complementarity unit, progressive receptive field expansion unit, multi-scale feature splitting and aggregation unit, and detection head are trained using the first dataset; The first dataset includes positive and negative sample images and their class labels, and the ground truth bounding boxes in the positive sample images; the loss function used during training... , , 、 All are loss coefficients. , , These are classification loss, Gaussian combined distance loss, and DFL loss, respectively. S2, load the weight coefficients of the convolutional module, spatial semantic feature complementary unit, progressive receptive field expansion unit, multi-scale feature splitting and aggregation unit and detection head trained by S1, fix the weight coefficients, and train the re-identification feature extraction module using the second dataset; The second dataset includes positive and negative sample images and their class labels, ground truth bounding boxes in positive sample images and their identity information; the loss function used during training... Where m represents the interval parameter, , , , These represent the true bounding boxes, positive samples of bounding boxes with the same identity, and negative samples of bounding boxes with different identities, respectively, and M is the number of triplets of true bounding boxes, positive samples of bounding boxes with the same identity, and negative samples of bounding boxes with different identities.

5. A method for UAV ground target detection and tracking based on multi-scale complementary feature aggregation, characterized in that, include: The aerial image of the UAV to be detected is input into the UAV ground target detection and tracking system based on multi-scale complementary feature aggregation as described in any one of claims 1-3 to obtain the detection and tracking results.

6. An electronic device, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the training method as described in claim 4 or the detection and tracking method as described in claim 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to execute the training method of claim 4 or the detection and tracking method of claim 5.

8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the training method of claim 4 or the detection and tracking method of claim 5.

Images