A multi-unmanned aerial vehicle oriented differential information perception single target tracking method
By constructing a single-target tracking method based on the differential information perception of multiple UAVs, and utilizing differential information perception fusion and cross-source attention focusing modules, the accuracy and robustness issues of UAV visual tracking in complex scenarios are solved, and stable target tracking results are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DALIAN MARITIME UNIVERSITY
- Filing Date
- 2026-03-20
- Publication Date
- 2026-06-23
AI Technical Summary
In complex scenarios, UAV visual tracking technology suffers from problems such as low tracking accuracy, insufficient robustness, and inadequate information utilization. In particular, under extreme conditions such as occlusion and drastic changes in lighting, tracking interruption and target loss are likely to occur.
A single-target tracking method based on differential information perception for multiple UAVs is constructed. Through a differential information perception fusion module and a cross-source attention focusing module, multi-scale differential feature fusion and adaptive feature enhancement are achieved. The multi-scale differential fusion module and the cross-source attention focusing module are used to optimize target search features, and the UAV template sharing mechanism is combined for target localization and prediction.
It improves the target tracking accuracy and robustness of UAVs in complex scenarios, effectively copes with occlusion and changes in lighting, reduces tracking drift, and achieves stable target tracking.
Smart Images
Figure CN122265883A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a single-target tracking method for multi-UAV differential information perception. Background Technology
[0002] Unmanned aerial vehicle (UAV) visual tracking technology, as a highly flexible, wide-coverage, and fast-response intelligent sensing equipment technology, can perform continuous target tracking tasks in complex scenarios such as intelligent security, disaster relief, and environmental monitoring. During operation, UAV visual tracking requires real-time capture of target visual features, continuous estimation of target position, and transmission to the tracking control system to achieve stable target tracking. However, factors such as dynamic scene changes, drastic changes in target appearance, occlusion, and lighting fluctuations increase the difficulty of UAV visual tracking, leading to insufficient stability and accuracy in target tracking. Therefore, researching efficient, stable, and accurate multi-UAV, multi-view visual tracking methods to address the problems of low tracking accuracy, insufficient robustness, and inadequate information utilization in existing methods is of significant research value and importance for promoting the practical application of UAV visual tracking technology.
[0003] Multi-view visual perception and fusion technology can learn to represent the complete visual features of a target, thereby achieving accurate target tracking. Therefore, it has become the mainstream research direction in the field of UAV visual tracking. Existing multi-UAV single-target tracking algorithms are mainly divided into traditional multi-view tracking methods that rely on preset camera geometry and optimized algorithms designed for dynamic UAV scenarios. Traditional methods that rely on preset camera geometry, while capable of basic multi-view information fusion, suffer from poor adaptability and insufficient flexibility, failing to cope with dynamic changes in viewpoint and position during UAV flight. Optimized algorithms designed for dynamic UAV scenarios include ASNet and TransMDOT. Although these methods have advantages such as template sharing and high information utilization efficiency, they lack the ability to suppress cross-view background interference, easily leading to tracking drift. While CRM series algorithms alleviate background interference through consistent representation learning, they fail to quantify the degree of feature differences between viewpoints, resulting in poor targeting of target feature enhancement and difficulty in fully mining prior information in template features and differential features between viewpoints. Furthermore, in real-world drone tracking scenarios, the complex target tracking environment, occlusion, and extreme conditions such as drastic changes in lighting often lead to problems such as tracking interruption and target loss in existing algorithms. Therefore, there is an urgent need for an efficient multi-drone single-target tracking method to solve the existing problems and thus meet the practical application requirements of drone visual tracking. Summary of the Invention
[0004] This invention provides a single-target tracking method based on differential information perception for multiple unmanned aerial vehicles (UAVs) to overcome the aforementioned technical problems.
[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A single-target tracking method based on differential information perception for multiple UAVs specifically includes the following steps: S1: Acquire target images and search images containing the tracked target in a scenario of coordinated tracking by several drones; Image sample sets are obtained by annotating bounding boxes of the tracked targets in the target image and the search image; S2: Construct a single-target tracking model for multi-UAV differential information perception, the model including an input module, a differential information perception fusion module and a prediction module; The input module is used to input image samples from the image sample sets corresponding to any two UAVs into the difference information perception and fusion module; The difference information perception and fusion module is used to extract pixel features from the target image and search image through the backbone network to obtain template features and search features. For any two UAVs, the search feature corresponding to one UAV is defined as a single-branch search feature, and the template feature corresponding to the other UAV is defined as a cross-UAV shared template feature. Through the constructed multi-scale difference fusion module, after obtaining the initial difference features based on the cross-UAV shared template features and the single-branch search features, the initial difference features are encoded at multiple scales to obtain multi-scale difference features, and difference fusion weights are obtained based on the multi-scale difference features. Through the constructed cross-source attention focusing module, optimized target search features are obtained based on the single-branch search features and the difference fusion weights. The prediction module is used to obtain self-similarity based on optimized target search features and template features. At the same time, through the constructed UAV template sharing mechanism, it obtains cross-similarity based on cross-UAV shared template features and optimized target search features, and selects the similarity response as the final target location and prediction based on cross-similarity and self-similarity. S3: Train the constructed differential information perception single target tracking model for multiple UAVs based on the image sample set to obtain the optimal model; implement differential information perception single target tracking for multiple UAVs based on the optimal model.
[0006] Furthermore, the method for obtaining difference fusion weights through the constructed multi-scale difference fusion module in S2 specifically includes: The initial difference features are obtained based on cross-UAV shared template features and single-branch search features:
[0007] In the formula: Indicates the initial difference characteristics. This represents the aligned template features after adaptive pooling and channel projection, i.e., the shared template features across drones. This indicates a single-branch search feature; By constructing three parallel deep-dilated convolutional branches with different dilation rates, the initial differential features are encoded at multiple scales to obtain the multi-scale differential features:
[0008] In the formula: Indicates the expansion rate 3×3 depth-dilated convolution operation; Represents the ReLU activation function. This represents the multi-scale differential features extracted by three deep-dilated convolution branches; The difference fusion weights are obtained based on multi-scale difference features as follows:
[0009] In the formula: This represents a standard 3×3 convolutional layer. This indicates element-wise multiplication. This represents the difference fusion weight.
[0010] Furthermore, a method for obtaining optimized target search features based on single-branch search features and difference fusion weights, through the constructed cross-source attention focusing module, specifically includes: According to the preset A window of a certain size performs non-overlapping block operations on the single-branch search features to obtain the window block feature set. ; Simultaneously, global contour feature extraction is performed on the single-branch search features to obtain a global pooled feature set. for:
[0011] In the formula: This represents the input single-branch search features of the cross-source attention focusing module. Indicates a linear projection layer. Represents the Gaussian error linear activation function. This indicates a global average pooling operation. Representation layer normalization, Represents the global pooling feature set; Window segmentation feature set With global pooling feature set Defined as a dual-path feature set, and the attention similarity matrix calculated based on the dual-path feature set is as follows:
[0012]
[0013] In the formula: Representing coordinates The corresponding L2-normalized pixel query vector within the window, This represents a learnable query embedding. , These represent the L2-normalized key vectors of the windowed features and the global pooling features, respectively. , These represent the attention similarity matrices of the dual-path feature sets; Set the positional bias of the learnable corresponding window block path for each of the two path feature sets. Position bias relative to the global pooling path for:
[0014] In the formula: Indicates the position offset of the window block path Position bias relative to the global pooling path The complete positional offset after splicing; Based on complete position offset Weighting with differences Adaptive optimization of attention weights is performed to obtain the final dual-path attention weights. for:
[0015] In the formula: This represents the final dual-path attention weights. This indicates element-wise multiplication. Position offset based on window block path Position bias relative to the global pooling path The final attention weights are split into weights for the window block paths based on the sequence length of the window block features and global pooling features. Weights of the global pooling path Simultaneously, based on the weight of the window block path... Weights of the global pooling path The optimization target features are obtained as follows:
[0016] In the formula: Indicates the target feature to be optimized; This represents a value vector representing the corresponding window block path; This represents the value vector corresponding to the global pooling path.
[0017] Furthermore, the method for selecting the similarity response for final target location localization and prediction based on cross-similarity and self-similarity specifically includes: Optimize target features Updated to single-branch search feature ,for In a collaborative tracking scenario involving multiple drones, the first The self-similarity of the UAV tracking branch is calculated from its own template features and single-branch search features, and its expression is:
[0018] In the formula: Indicates the first The self-similarity of the drone, Indicates the first Target template image of a drone, Indicates the first Search images from drones, ; Indicates the first Template features of a drone; Through the constructed UAV template sharing mechanism, the cross-similarity is obtained based on the cross-UAV shared template features and the optimized target search features:
[0019] In the formula: Indicates cross-similarity. Indicates the first Template features extracted from a drone, Indicates the first Single-branch search features extracted by a drone. ; The similarity response used for final target location and prediction is selected based on cross-similarity and self-similarity.
[0020] Furthermore, the method for selecting the similarity response as the final location and prediction of the target location is as follows: set a similarity threshold, determine the size between the cross similarity and the similarity threshold and the self-similarity, if the cross similarity is greater than the similarity threshold and also greater than the self-similarity, then the cross similarity is used as the similarity response for the final location and prediction of the target location; otherwise, the self-similarity is used as the similarity response for the final location and prediction of the target location.
[0021] Furthermore, the methods for obtaining the optimal model in S3 specifically include: S31: Randomly divide the image sample set into a training set and a validation set according to a preset ratio; S32: Train the constructed multi-UAV differential information perception single target tracking model based on the training set to obtain the trained tracking model; S33: Based on the constructed composite loss function, the trained tracking model is validated using the validation set; That is, to determine whether the output of the trained tracking model has converged; If the output of the trained tracking model converges, then the trained tracking model is confirmed to be the optimal model. Otherwise, based on the backpropagation method, the weight parameters of the trained tracking model are adaptively adjusted, and step S32 is repeated until the weight parameters of the trained tracking model that have converged are confirmed to be the optimal weight parameters, and the tracking model is reconstructed to obtain the optimal model.
[0022] Furthermore, the constructed composite loss function is a weighted function of the KL divergence regression loss used to optimize the target bounding box localization accuracy and the hinge loss used to distinguish the target from the background region.
[0023] Beneficial effects: This invention provides a single-target tracking method based on the differential information perception of multiple UAVs. It addresses the problems of insufficient information utilization, inadequate differential information mining, and poor robustness of tracking in extreme scenarios in existing tracking algorithms. By constructing a single-target tracking model based on the differential information perception of multiple UAVs, it can achieve deep fusion of differential information from multiple UAV perspectives and adaptive feature enhancement, thereby improving the accuracy and robustness of target tracking in complex scenarios. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a flowchart of the single-target tracking method for multi-UAV differential information perception according to the present invention; Figure 2 This is a schematic diagram of the structure of the target tracking model CDMTrack constructed in this embodiment; Figure 3 This is a schematic diagram of the multi-scale difference fusion module (MDFU) described in this embodiment; Figure 4 This is a schematic diagram of the cross-source attention focusing module CSF described in this embodiment; Figure 5 This is a visualization example of the DDSOT dataset constructed as described in this embodiment; Figure 6 This is the core block diagram of the single-target tracking method in this embodiment. Detailed Implementation
[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0027] This embodiment provides a single-target tracking method based on differential information perception for multiple unmanned aerial vehicles (UAVs), such as... Figure 1 and Figure 6 As shown, the specific steps include: S1: Obtain target images and search images containing the tracked target in a cooperative tracking scenario involving several drones; annotate the bounding boxes of the tracked target in the target images and search images to obtain an image sample set; specifically, construct the Dual-Drone Single Object Tracking dataset (DDSOT) for model training, validation, and performance evaluation. Figure 5 As shown, this embodiment uses two DJI drones as data acquisition devices, simultaneously controlled by professional pilots, to acquire video sequences in a real outdoor scene. The acquired raw video samples are manually screened to remove low-quality sequences that are blurry, overexposed, or lack tracking targets. Python and OpenCV are used to extract frames from the selected, valid synchronized videos to obtain raw image samples. The LabelImg annotation tool is used to perform fine-grained annotations on the images in the sequences, with the annotations being the bounding box coordinates of the targets. Finally, the DDSOT dataset is constructed. This dataset includes multiple types of tracked targets such as pedestrians, bicycles, tricycles, and cars, covering three typical lighting scenarios: daytime, dusk, and nighttime. It includes typical challenging scenarios for multi-drone tracking, such as occlusion, rapid viewpoint changes, dramatic lighting changes, and motion blur. It contains a total of 16 sets of synchronized dual-view video clips and 9370 high-resolution 1920×1080 images.
[0028] S2: Construct a single-target tracking model for multi-UAV differential information perception, such as... Figure 2 As shown, the model includes an input module, a difference information perception and fusion module, and a prediction module; The input module is used to input image samples from the image sample sets corresponding to any two UAVs into the difference information perception and fusion module; The difference information perception and fusion module is used to extract pixel features from the target image and the search image through the backbone network ResNet to obtain template features and search features. For any two UAVs, the search feature corresponding to one UAV is defined as a single-branch search feature, and the template feature corresponding to the other UAV is defined as a cross-UAV shared template feature. Through the constructed multi-scale difference fusion module, after obtaining the initial difference features based on the cross-UAV shared template feature and the single-branch search feature, the initial difference features are encoded in a multi-scale manner to obtain multi-scale difference features, and the difference fusion weights are obtained based on the multi-scale difference features. The method for obtaining the difference fusion weights in this embodiment is as follows: Figure 3 As shown, the specific steps include: The initial difference features are obtained based on cross-UAV shared template features and single-branch search features:
[0029] In the formula: Indicates the initial difference characteristics. This represents the aligned template features after adaptive pooling and channel projection; This represents a single-branch search feature, which is used to characterize the difference between the shared template and the original features of the search region.
[0030] In this embodiment, a multi-scale difference fusion module (MDFU) is introduced. This module takes cross-UAV shared template features and single-branch search region features as input. First, it performs spatial alignment and channel projection on the template features, and then performs difference and absolute value operations with the search features to obtain the initial difference features.
[0031] After the initial differential features are generated, MDFU constructs a multi-scale receptive structure using dilated convolutions with different dilation rates. This expands the receptive field without increasing the number of model parameters. Specifically, it encodes the initial differential features at multiple scales by constructing three parallel deep dilated convolution branches with different dilation rates, obtaining the multi-scale differential features as follows:
[0032] In the formula: Indicates the expansion rate A 3×3 depth-dilated convolution operation with the same number of groups as the feature channel dimension; Represents the ReLU activation function. This represents the multi-scale differential features extracted by the three deep-hole convolution branches under different receptive fields; In this embodiment, standard 3×3 convolution transformation is applied to the multi-scale differential features of each depth-dilated convolution branch to achieve semantic space alignment. Then, element-wise multiplication is used to enhance the high-response regions of multi-scale co-occurrence. Finally, the differential fusion weights are normalized using the Sigmoid activation function to obtain the final differential fusion weights:
[0033] In the formula: This represents a standard 3×3 convolutional layer. This indicates element-wise multiplication. This represents the difference fusion weights of the MDFU output. These weights will be used as attention guidance signals to input the subsequent cross-source attention focusing module, i.e., the CSF module, to achieve adaptive enhancement of the target features.
[0034] By constructing a cross-source attention focusing module, the optimized target search features are obtained based on single-branch search features and difference fusion weights, such as... Figure 4 As shown, it specifically includes: According to the preset A window of a certain size performs non-overlapping block operations on the single-branch search features to obtain the window block feature set. And divide the window into feature sets The window block features are used as local fine features; at the same time, global contour features are extracted from the single-branch search features to obtain a global pooled feature set. for:
[0035] In the formula: This represents the input single-branch search features of the cross-source attention focusing module. Indicates a linear projection layer. Represents the Gaussian error linear activation function. This indicates a global average pooling operation. Representation layer normalization, This represents the global pooling feature set, used to capture the global contour and semantic information of the target.
[0036] In this embodiment, the designed cross-source attention focusing module CSF (Cross-Source Fusion) is connected after the MDFU module. This module searches for the original features of the region in a single branch. Difference fusion weights with MDFU output For dual inputs, a dual-path feature set is first constructed from the input feature map to separate and extract local fine features from global contour features. This includes a window-block feature set. Press the input feature map The dimensions are divided into non-overlapping blocks, with the coordinates of the top-left corner of each block as the reference. As a unique identifier, the feature dimension of each block is . Global pooling feature set By constructing a series of operations including "linear projection-GELU activation-average pooling-layer normalization", feature dimension compression and global information extraction are achieved.
[0037] Window segmentation feature set With global pooling feature set Defined as a dual-path feature set, and the attention similarity matrix calculated based on the dual-path feature set is as follows:
[0038]
[0039] In the formula: Representing coordinates The corresponding L2-normalized pixel query vector within the window, This represents a learnable query embedding. , These represent the L2-normalized key vectors of the windowed features and the global pooling features, respectively. , These represent the attention similarity matrices for the dual-path feature sets. In this embodiment, the attention similarity matrices are calculated separately for each dual-path feature set. First, the query vector and key vector are L2 normalized, and then a learnable query embedding is incorporated into the normalized query vector. This reduces the model's single dependence on the original input query.
[0040] Set the positional bias of the learnable corresponding window block path for each of the two path feature sets. Position bias relative to the global pooling path for:
[0041] In the formula: Indicates the position offset of the window block path Position bias relative to the global pooling path The complete positional offset after splicing; In this embodiment, a learnable temperature parameter is introduced. With sequence length scaling factor ( , (Representing the height and width of the global pooling feature), query-key relevance is measured by scaling cosine similarity. The dual-path similarity matrices are concatenated and then superimposed with positional biases. The basic attention weights are obtained through Softmax activation and then fused with the difference weights output by MDFU. Element-wise multiplication enables adaptive optimization of attention weights, i.e., based on the complete positional bias. Weighting with differences Adaptive optimization of attention weights is performed to obtain the final dual-path attention weights. for:
[0042] In the formula: This represents the final dual-path attention weights. This indicates element-wise multiplication. Position offset based on window block path Position bias relative to the global pooling path The final attention weights are split into weights for the window block paths based on the sequence length of the window block features and global pooling features. Weights of the global pooling path Simultaneously, based on the weight of the window block path... Weights of the global pooling path The optimization target features are obtained as follows:
[0043] In the formula: Indicates the target feature to be optimized; This represents a value vector representing the corresponding window block path; This represents the value vector corresponding to the global pooling path. In this embodiment, the final attention weights are split into the sequence lengths of the window block features and the global pooling features. , , respectively with the value vector of the corresponding path , After multiplication and summation, the local fine texture features and global contour features are fused to output the target feature optimized by the CSF module. This feature is an optimized feature after target enhancement and background suppression, which will be input into the similarity calculation stage for subsequent target localization and tracking.
[0044] The prediction module is used to obtain self-similarity based on optimized target search features and template features. Simultaneously, through a constructed UAV template sharing mechanism, it obtains cross-similarity based on cross-UAV shared template features and optimized target search features. Finally, based on cross-similarity and self-similarity, it selects a similarity response for final target location and prediction, specifically including: In this embodiment, the DiMP discriminative tracking model is used as the baseline framework. A tracking branch is constructed for each UAV participating in cooperative tracking to achieve parallel feature processing of multiple UAVs. The core objective of visual target tracking is to search for the optimal target position by minimizing the objective function. This process can be regarded as an end-to-end parameter learning problem. The objective function formula is as follows:
[0045] In the formula: Indicates the tracking model at the feature location The prediction results at the location, This indicates the actual label corresponding to this location. This represents the residual calculation term, used to quantify the deviation between the prediction result and the true label; This represents the filter weight parameters of the model. This represents the sample influence factor, used to distinguish training weights from different samples. The regularization weights are used to constrain the parameter space complexity and prevent model overfitting. In this embodiment, based on the principle of the aforementioned objective function, the target image can be... and search images Inputting the backbone network for feature extraction yields template features. and search features Discriminative trackers transform target localization into a template matching problem, generating a response map by calculating the cross-correlation similarity between template features and search features:
[0046] In the formula: This represents a two-dimensional cross-correlation operation. This represents the global offset of the similarity value. Represents the identity matrix. This means assigning the same offset value to all feature locations. This represents the final similarity response map, where the peak position corresponds to the optimal center position of the target within the search area; Optimize target features Updated to single-branch search feature ,for In a collaborative tracking scenario involving multiple drones, the first The self-similarity of the UAV tracking branch is calculated from its own template features and single-branch search features, and its expression is:
[0047] In the formula: Indicates the first The self-similarity of the drone, Indicates the first Target template image of a drone, Indicates the first Search images from drones, ; Indicates the first Template features of a drone; Through the constructed UAV template sharing mechanism, the cross-similarity is obtained based on the cross-UAV shared template features and the optimized target search features:
[0048] In the formula: Indicates cross-similarity. Indicates the first Template features extracted from a drone, Indicates the first Single-branch search features extracted by a drone. ; The similarity response used for final target location localization and prediction is selected based on cross-similarity and self-similarity. Specifically, by setting a similarity threshold, the cross-similarity is compared with the threshold and the self-similarity. If the cross-similarity is greater than the threshold and also greater than the self-similarity, then the cross-similarity is used as the similarity response for final target location localization and prediction; otherwise, the self-similarity is used as the similarity response for final target location localization and prediction.
[0049] S3: Train the constructed multi-UAV differential information perception single target tracking model based on the image sample set to obtain the optimal model. Specific steps include: S31: Randomly divide the image sample set into a training set and a validation set according to a preset ratio; S32: Train the constructed multi-UAV differential information perception single target tracking model based on the training set to obtain the trained tracking model; S33: Based on the constructed composite loss function, the trained tracking model is validated using the validation set; the constructed composite loss function is a weighted function of KL divergence regression loss used to optimize the target bounding box localization accuracy and hinge loss used to distinguish the target from the background region. That is, to determine whether the output of the trained tracking model has converged; If the output of the trained tracking model converges, then the trained tracking model is confirmed to be the optimal model. Otherwise, based on the backpropagation method, the weight parameters of the trained tracking model are adaptively adjusted, and step S32 is repeated until the weight parameters of the trained tracking model that have converged are confirmed to be the optimal weight parameters and the tracking model is reconstructed to obtain the optimal model. Based on the optimal model, achieve single-target tracking with differential information perception for multiple UAVs.
[0050] Specifically, this embodiment obtains the optimal model weights by performing offline training and optimization on the server side. During the model training phase, data augmentation techniques such as horizontal flipping, brightness jittering, and random scaling are used to process the training set images to improve the model's generalization ability.
[0051] The model training outputs the bounding box locations of targets in the image and their corresponding classification scores. The loss function employs a multi-loss weighted joint optimization strategy, where the bounding box regression loss is KL divergence regression (KLRegression) to optimize the localization accuracy of the target bounding boxes, and the classification loss is LBHinge loss with a threshold of 0.05 to distinguish targets from background regions. The optimizer is the AdamW optimizer, with differentiated learning rates set for different network modules, and a StepLR learning rate scheduler configured. Every 15 training epochs, the learning rate of all modules is decayed to 0.2 times its original value. In each training iteration, the preprocessed and data-augmented image is input into the model for forward propagation, and the weighted joint loss function value is calculated. The gradients of each layer's parameters are calculated using the backpropagation algorithm, and the model parameters are updated based on the gradient descent strategy of the AdamW optimizer. This process continues for multiple rounds of training until the model's performance on the validation set converges. During training, a validation set is used to validate the model and check its performance on unseen data. Optimal weights are determined based on the model's performance on the validation set. When the model's performance on the validation set no longer improves or begins to decline, training can be stopped, and the corresponding model parameters are selected as the optimal weights. The trained model with the optimal weights is then deployed on a high-performance computing device on a drone to perform forward inference, achieving robust single-target tracking in multi-drone collaborative scenarios.
[0052] Compared with existing technologies, the beneficial effects of the method described in this embodiment are as follows: addressing the problems of insufficient information utilization, inadequate difference information mining, and poor robustness of tracking in extreme scenarios in existing tracking algorithms, by constructing a single-target tracking model for multi-UAV difference information perception, it is possible to achieve deep fusion of UAV multi-view difference information and adaptive feature enhancement, thereby improving the accuracy and robustness of target tracking in complex scenarios.
[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A single-target tracking method based on differential information perception for multiple unmanned aerial vehicles (UAVs), characterized in that, Specifically, the following steps are included: S1: Acquire target images and search images containing the tracked target in a scenario of coordinated tracking by several drones; Image sample sets are obtained by annotating bounding boxes of the tracked targets in the target image and the search image; S2: Construct a single-target tracking model for multi-UAV differential information perception, the model including an input module, a differential information perception fusion module and a prediction module; The input module is used to input image samples from the image sample sets corresponding to any two UAVs into the difference information perception and fusion module; The difference information perception and fusion module is used to extract pixel features from the target image and the search image through the backbone network to obtain template features and search features. For any two drones, the search feature corresponding to one drone is defined as a single-branch search feature, and the template feature corresponding to the other drone is defined as a cross-drone shared template feature. By constructing a multi-scale difference fusion module, initial difference features are obtained based on cross-UAV shared template features and single-branch search features. Then, multi-scale encoding is performed on the initial difference features to obtain multi-scale difference features, and difference fusion weights are obtained based on the multi-scale difference features. By constructing a cross-source attention focusing module, optimized target search features are obtained based on single-branch search features and difference fusion weights. The prediction module is used to obtain self-similarity based on optimized target search features and template features. At the same time, through the constructed UAV template sharing mechanism, it obtains cross-similarity based on cross-UAV shared template features and optimized target search features, and selects the similarity response as the final target location and prediction based on cross-similarity and self-similarity. S3: Train the constructed differential information perception single target tracking model for multiple UAVs based on the image sample set to obtain the optimal model; implement differential information perception single target tracking for multiple UAVs based on the optimal model.
2. The single-target tracking method for multi-UAV differential information perception according to claim 1, characterized in that, The method for obtaining difference fusion weights in S2 through the constructed multi-scale difference fusion module specifically includes: The initial difference features are obtained based on cross-UAV shared template features and single-branch search features: In the formula: Indicates the initial difference characteristics. This represents the aligned template features after adaptive pooling and channel projection, i.e., the shared template features across drones. This indicates a single-branch search feature; By constructing three parallel deep-dilated convolutional branches with different dilation rates, the initial differential features are encoded at multiple scales to obtain the multi-scale differential features: In the formula: Indicates the expansion rate 3×3 depth-dilated convolution operation; Represents the ReLU activation function. This represents the multi-scale differential features extracted by three deep-dilated convolution branches; The difference fusion weights are obtained based on multi-scale difference features as follows: In the formula: This represents a standard 3×3 convolutional layer. This indicates element-wise multiplication. This represents the difference fusion weight.
3. The single-target tracking method for multi-UAV differential information perception according to claim 2, characterized in that, The method for obtaining optimized target search features based on single-branch search features and difference fusion weights through the constructed cross-source attention focusing module includes: According to the preset A window of a certain size performs non-overlapping block operations on the single-branch search features to obtain the window block feature set. ; Simultaneously, global contour feature extraction is performed on the single-branch search features to obtain a global pooled feature set. for: In the formula: This represents the input single-branch search features of the cross-source attention focusing module. Indicates a linear projection layer. Represents the Gaussian error linear activation function. This indicates a global average pooling operation. Representation layer normalization, Represents the global pooling feature set; Window segmentation feature set With global pooling feature set Defined as a dual-path feature set, and the attention similarity matrix calculated based on the dual-path feature set is as follows: In the formula: Representing coordinates The corresponding L2-normalized pixel query vector within the window, This represents a learnable query embedding. , These represent the L2-normalized key vectors of the windowed features and the global pooling features, respectively. , These represent the attention similarity matrices of the dual-path feature sets; Set the positional bias of the learnable corresponding window block path for each of the two path feature sets. Position bias relative to the global pooling path for: In the formula: Indicates the position offset of the window block path Position bias relative to the global pooling path The complete positional offset after splicing; Based on complete position offset Weighting with differences Adaptive optimization of attention weights is performed to obtain the final dual-path attention weights. for: In the formula: This represents the final dual-path attention weights. This indicates element-wise multiplication. Position offset based on window block path Position bias relative to the global pooling path The final attention weights are split into weights for the window block paths based on the sequence length of the window block features and global pooling features. Weights of the global pooling path Simultaneously, based on the weight of the window block path... Weights of the global pooling path The optimization target features are obtained as follows: In the formula: Indicates the target feature to be optimized; This represents a value vector representing the corresponding window block path; This represents the value vector corresponding to the global pooling path.
4. The single-target tracking method for multi-UAV differential information perception according to claim 3, characterized in that, Methods for selecting the similarity response used for final target location localization and prediction based on cross-similarity and self-similarity include: Optimize target features Updated to single-branch search feature ,for In a collaborative tracking scenario involving multiple drones, the first The self-similarity of the UAV tracking branch is calculated from its own template features and single-branch search features, and its expression is: In the formula: Indicates the first The self-similarity of the drone, Indicates the first Target template image of a drone, Indicates the first Search images from drones, ; Indicates the first Template features of a drone; Through the constructed UAV template sharing mechanism, the cross-similarity is obtained based on the cross-UAV shared template features and the optimized target search features: In the formula: Indicates cross-similarity. Indicates the first Template features extracted from a drone, Indicates the first Single-branch search features extracted by a drone. ; The similarity response used for final target location and prediction is selected based on cross-similarity and self-similarity.
5. A single-target tracking method for multi-UAV differential information perception according to claim 4, characterized in that, The method for selecting the similarity response for the final location and prediction of the target position is as follows: set a similarity threshold, determine the size between the cross similarity and the similarity threshold and the self-similarity, if the cross similarity is greater than the similarity threshold and also greater than the self-similarity, then the cross similarity is used as the similarity response for the final location and prediction of the target position; otherwise, the self-similarity is used as the similarity response for the final location and prediction of the target position.
6. A single-target tracking method for multi-UAV differential information perception according to claim 5, characterized in that, The methods for obtaining the optimal model in S3 specifically include: S31: Randomly divide the image sample set into a training set and a validation set according to a preset ratio; S32: Train the constructed multi-UAV differential information perception single target tracking model based on the training set to obtain the trained tracking model; S33: Based on the constructed composite loss function, the trained tracking model is validated using the validation set; That is, to determine whether the output of the trained tracking model has converged; If the output of the trained tracking model converges, then the trained tracking model is confirmed to be the optimal model. Otherwise, based on the backpropagation method, the weight parameters of the trained tracking model are adaptively adjusted, and step S32 is repeated until the weight parameters of the trained tracking model that have converged are confirmed to be the optimal weight parameters, and the tracking model is reconstructed to obtain the optimal model.
7. A single-target tracking method for multi-UAV differential information perception according to claim 6, characterized in that, The constructed composite loss function is a weighted function of the KL divergence regression loss used to optimize the target bounding box localization accuracy and the hinge loss used to distinguish the target from the background region.