A UAV target tracking method and system based on collaborative attention and serial channel cross-correlation
By employing collaborative attention and cascaded channel cross-correlation methods, the feature mismatch problem caused by target scale variations and background interference in UAV target tracking was solved, achieving more efficient and accurate target tracking.
Patent Information
- Application Number
- CN202410944594.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-07-15
AI Technical Summary
In UAV target tracking, there are feature mismatch issues caused by changes in target scale, occlusion, and background interference, which affect tracking performance and positioning accuracy.
A method based on collaborative attention and serial channel cross-correlation is adopted. Features are extracted through Siamese network, the intrinsic correlation between template and search features is enhanced by collaborative attention module, and cross-correlation operation is performed by serial channel cross-correlation module. Target localization is then performed in conjunction with anchorless tracking head network.
It improves the robustness and positioning accuracy of target tracking, and can effectively solve the feature mismatch problem in complex scenarios such as target scale changes and background interference, thus achieving more efficient target tracking.
Smart Images

Figure CN118941985B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for tracking unmanned aerial vehicles (UAVs) targets based on collaborative attention and serial channel cross-correlation. Background Technology
[0002] Target tracking is an important research direction in the field of computer vision. With the development of information technology, target tracking has received increasing attention and application in human-computer interaction, autonomous driving, national defense and security, and video surveillance. In recent years, unmanned aerial vehicles (UAVs) have been widely used in civilian and military fields due to their small size, flexibility, intelligence, and autonomy, such as atmospheric environmental monitoring, disaster relief, reconnaissance, enemy target tracking, and battlefield intelligence gathering. Target tracking is particularly crucial in reconnaissance and search operations. Therefore, deep learning-based target tracking algorithms are increasingly being used in UAVs. The complexity of UAV visual target tracking scenarios and the variability of moving targets make target feature extraction and target model building difficult, posing a significant challenge to target tracking performance.
[0003] Because target tracking technology has a wide range of applications in UAVs, it has become a research hotspot both domestically and internationally in recent years, resulting in the emergence of many excellent algorithms. At the same time, many challenging datasets have appeared, and the difficulty of these datasets is gradually increasing. This has also put forward new requirements for the improvement and innovation of target tracking algorithms in order to cope with target tracking problems in complex and ever-changing scenarios.
[0004] Target tracking algorithms are broadly classified into two categories: generative target tracking algorithms and discriminative target tracking algorithms. The first category includes algorithms such as particle filtering, mean shift, and Kalman filtering. The second category comprises discriminative target tracking algorithms, which incorporate a detection step, focusing on distinguishing the tracked object from the background. A detector is obtained through offline pre-training or online learning, and this detector is used to differentiate between background and foreground targets, thereby determining the position of the tracked object. Therefore, discriminative algorithms are generally more robust than generative ones. Within discriminative target tracking algorithms, research on single-target tracking algorithms based on correlation filtering and deep learning is currently a major trend. Summary of the Invention
[0005] The purpose of this invention is to provide a UAV target tracking method and system based on collaborative attention and serial channel cross-correlation, which has strong tracking performance and high positioning accuracy. On the one hand, it can solve the problem of not being able to accurately locate the target when the target scale changes or the target is occluded. On the other hand, it can solve the problem of feature mismatch when the tracked target is subject to background interference or interference from similar targets.
[0006] The technical solution to achieve the purpose of this invention is: a UAV target tracking method based on collaborative attention and serial channel cross-correlation, comprising the following steps:
[0007] Step 1: Crop the first frame of the video to obtain the input of the template branch, and crop the subsequent frame images to obtain the input of the search branch. The first frame and subsequent frame images are processed by the Siamese network to extract features, and then the features are interacted through the collaborative attention module to obtain the template feature map and the search feature map.
[0008] Step 2: Input the template feature map and search feature map obtained in Step 1 into the serial channel cross-correlation module to perform cross-correlation operation and obtain the output feature map;
[0009] Step 3: Input the output feature map obtained in Step 2 into the anchorless tracking head network. The classification branch obtains the classification score of each pixel in the feature map, the centrality branch obtains the centrality score between each pixel and the target, and the regression branch obtains the target box information corresponding to each pixel.
[0010] Step 4: Multiply the classification score and centrality score of each pixel in Step 3, select the pixel with the highest score, and determine the target box information of the pixel through the regression branch, thereby determining the location of the target in the current frame.
[0011] Step 5: Repeat steps 1 through 4 until the position of the last frame in the video is tracked.
[0012] A UAV target tracking system based on collaborative attention and serial channel cross-correlation is disclosed. This system implements the aforementioned UAV target tracking method based on collaborative attention and serial channel cross-correlation. The system includes a feature extraction module, a cross-correlation operation module, a tracking head network module, a target position determination module, and a tracking module, wherein:
[0013] The feature extraction module crops the first frame of the video to obtain the input of the template branch and crops the subsequent frame images to obtain the input of the search branch. It then performs feature extraction on the first and subsequent frame images through a Siamese network and performs feature interaction through a collaborative attention module to obtain the template feature map and the search feature map.
[0014] The cross-correlation module takes the template feature map and the search feature map obtained by the feature extraction module and inputs them into the serial channel cross-correlation module to perform cross-correlation operation and obtain the output feature map.
[0015] The tracking head network module inputs the output feature map obtained by the cross-correlation operation module into the anchorless tracking head network. The classification branch obtains the classification score of each pixel in the feature map, the centrality branch obtains the centrality score between each pixel and the target, and the regression branch obtains the target box information corresponding to each pixel.
[0016] The target location determination module multiplies the classification score and centrality score of each pixel in the tracking head network module, selects the pixel with the highest score, and determines the target box information of the pixel through the regression branch, thereby determining the location of the target in the current frame;
[0017] The tracking module repeats the processing steps of the feature extraction module, cross-correlation operation module, tracking head network module, and target location determination module until the position of the last frame in the video is tracked.
[0018] A mobile terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the UAV target tracking method based on collaborative attention and serial channel cross-correlation.
[0019] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the UAV target tracking method based on cooperative attention and serial channel cross-correlation.
[0020] Compared with the prior art, the significant advantages of this invention are:
[0021] (1) The last three Block structures of the pre-trained weighted ResNet50 were adopted. A collaborative attention module was added after each of the three Block structures to deeply mine the intrinsic correlation between template features and search features, and to emphasize the effectiveness of the interaction information between the two branches in enhancing the appearance expression of the model, so as to better identify the target.
[0022] (2) The cross-correlation module of the serial channel is used to perform cross-correlation operation, thereby learning more obvious features. This effectively solves the problem of feature mismatch when the tracking target is subject to background interference, similar target interference, etc., thereby improving the tracking performance of the model. Attached Figure Description
[0023] Figure 1 This is a flowchart of the UAV target tracking method based on collaborative attention and serial channel cross-correlation of the present invention.
[0024] Figure 2 This is a schematic diagram of the UAV target tracking method based on collaborative attention and serial channel cross-correlation of the present invention.
[0025] Figure 3 This is a schematic diagram of the feature interaction process performed by the collaborative attention module in this invention.
[0026] Figure 4 This is a schematic diagram of the process of cross-correlation of series channels in this invention.
[0027] Figure 5 This is a comparison curve showing the accuracy of the method of the present invention and existing tracking algorithms in the DTB70 in the embodiments.
[0028] Figure 6 This is a comparison curve of the success rate of the method of the present invention and the existing tracking algorithm in DTB70 in the embodiment.
[0029] Figure 7 This is a schematic diagram of the tracking results in the embodiment. Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Based on the embodiments of the present invention, all other instances obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention.
[0031] This invention provides a UAV target tracking method based on collaborative attention and serial channel cross-correlation, comprising the following steps:
[0032] Step 1: Crop the first frame of the video to obtain the input of the template branch, and crop the subsequent frame images to obtain the input of the search branch. The first frame and subsequent frame images are processed by the Siamese network to extract features, and then the features are interacted through the collaborative attention module to obtain the template feature map and the search feature map.
[0033] Step 2: Input the template feature map and search feature map obtained in Step 1 into the serial channel cross-correlation module to perform cross-correlation operation and obtain the output feature map;
[0034] Step 3: Input the output feature map obtained in Step 2 into the anchorless tracking head network. The classification branch obtains the classification score of each pixel in the feature map, the centrality branch obtains the centrality score between each pixel and the target, and the regression branch obtains the target box information corresponding to each pixel.
[0035] Step 4: Multiply the classification score and centrality score of each pixel in Step 3, select the pixel with the highest score, and determine the target box information of the pixel through the regression branch, thereby determining the location of the target in the current frame.
[0036] Step 5: Repeat steps 1 through 4 until the position of the last frame in the video is tracked.
[0037] As a specific example, the Siamese network in step 1 is a ResNet50 with shared weights. It uses the last three Block modules of ResNet50, combines the Siamese network with the collaborative attention module, and adds a collaborative attention module after each Block module.
[0038] As a specific example, step 1 is as follows:
[0039] Step 1.1: Crop the first frame of the video with the target as the center and use it as the input for the template branch; crop the subsequent frames and use them as the input for the search branch.
[0040] The initial template feature map is obtained by extracting features from the first frame image using a twin network. The initial search feature map is obtained by extracting features from subsequent frame images.
[0041] Where R represents the real number field; N represents the number of samples processed at one time; W T H represents the width of the template feature map. T W represents the height of the template feature map. I H represents the width of the search feature map. I The width of the search feature map is represented by ; C represents the number of channels in the feature map.
[0042] Step 1.2: After the template branch is processed by a 1×1 convolution kernel k(●), tensor reshaping is performed to obtain the first feature map. The search branch is processed by a 1×1 convolution kernel g(●) and then tensor reshaping is performed to obtain the second feature map. The first feature map Second feature map Perform matrix multiplication to obtain intermediate feature maps.
[0043] Step 1.3: After the template branch is processed by a 1×1 convolution kernel g(●), tensor reshaping is performed to obtain the third feature map. The search branch is processed by a 1×1 convolution kernel g(●) and then tensor reshaping is performed to obtain the fourth feature map. The obtained third feature map Fourth feature map Compare with the intermediate feature maps from step 1.2 respectively. Perform matrix multiplication to obtain the template branch modulation feature map. and search branch modulation feature map
[0044] Step 1.4: Process the template branch modulation feature map from Step 1.3. Tensor reshaping is performed to obtain the fifth feature map. Search branch modulation feature map Tensor reshaping is performed to obtain the sixth feature map.
[0045] Fifth feature map Sixth feature map By applying 1×1 convolution kernels q(●) and w(●) respectively, the channel dimension is increased to C, resulting in the fifth feature map after channel dimension transformation. The sixth feature map after channel dimension transformation
[0046] Using the initial template feature map and the fifth feature map after transforming the channel dimensions Perform matrix addition to obtain the intrinsic template-related feature map ψ(x; z);
[0047] Using the initial search feature map and the sixth feature vector after transforming the channel dimensions Perform matrix addition to obtain the intrinsic search-related features ψ(z;x);
[0048] Step 1.5: Compress the obtained intrinsic template-related features ψ(x; z) using both max pooling and global average pooling operations. Then, activate them through a fully connected MLP layer. Next, use a sigmoid operation to normalize the importance of each channel to a 0-1 range. Finally, multiply the sigmoided channel parameters (normalized to 0-1) with the intrinsic template-related features ψ(x; z) and the intrinsic search-related features ψ(z; x) respectively to obtain the template feature map. and search feature map
[0049] As a specific example, step 2 involves inputting the template feature map and the search feature map obtained in step 1 into the serial channel cross-correlation module for cross-correlation operation to obtain the output feature map, as detailed below:
[0050] Step 2.1: Transfer the template feature map and search feature map Perform one Conv+BN+ReLU operation each, followed by the first cross-correlation operation. The formula is:
[0051]
[0052] Where Conv represents the convolution operation in deep learning, which filters the input data through a convolution kernel; BN represents the standardization operation for features; ReLU is the activation function; φ first The feature map representing the first cross-correlation; XCorr dw This indicates the similarity between computational features;
[0053] Step 2.2: The first cross-correlation feature map φ obtained in Step 2.1... first Perform a single convolutional (Conv) operation, followed by two consecutive Conv+BN+ReLU operations for upsampling, with padding set to 3 and 2 respectively, to obtain the processed features. Padding refers to the process of adding extra pixels to the edges of the input feature map during convolution operations in order to control the size of the output feature map.
[0054] Step 2.3: Process the features obtained in Step 2.2. The search feature map obtained in step 1.5 Set a weighted fusion strategy to obtain the fused search features. The formula is:
[0055]
[0056] Wherein, λ1 and λ2 are set to 0.8 and 0.2 respectively;
[0057] Step 2.4: Obtain the template feature map obtained in Step 1.5. Template features are obtained through the ECA channel attention mechanism. Then the template feature map Template features Adding them together yields the template features needed for the second cross-correlation.
[0058] Step 2.5: Perform the second cross-correlation operation, the formula is:
[0059]
[0060] φ second Feature map representing the second cross-correlation;
[0061] Step 2.6: The feature map φ of the first cross-correlation... first The feature map φ of the second cross-correlation second Add them together to obtain the output feature map.
[0062] As a specific example, the classification branch in step 3 is used to distinguish the targets, and the regression branch is used to obtain scale information of the target size;
[0063] The final response feature map obtained by the classification branch is A. cls A cls ∈R h×w×2 Response feature map A cls The size is h×w, and 2 represents the response feature map A. cls The number of channels, A cls Includes the probability that a pixel is a positive sample or a negative sample;
[0064] The final response feature map obtained by the regression branch is A. reg A reg ∈R h×w×4 Response feature map A reg The size is h×w, 4 represents the number of channels in the feature map, A reg The distance from each pixel (i,j) to each border is denoted as t. (i,j) = (l,t,r,b), where l,t,r,b represent the distances from pixel (i,j) to the left, top, right, and bottom borders.
[0065] As a specific example, the classification branch uses the cross-entropy loss function, the regression branch uses the -lnIOU loss function, and the final loss L of the entire network is expressed as:
[0066] L=λ4L cls +λ5L cen +λ6L reg
[0067] Where λ4, λ5, and λ6 are set to 1, 1, and 3 respectively, L cls L cen and L reg These represent classification loss, centering loss, and regression loss, respectively.
[0068] As a specific example, the response feature map of the centrality branch described in step 3 is A. cen A cen ∈R h×w×1 The response feature map is A. cen The size is h×w, where 1 represents the number of channels in the response map, and contains the centrality score of each pixel. The accurate target score is obtained by multiplying the centrality score by the classification score.
[0069] The true centrality value C(i,j) is:
[0070]
[0071] in:
[0072]
[0073] Where l, t, r, b represent the feature map points P obtained from the search frame, respectively. feature The search patch point P corresponding to (i,j) search (i,j) is the distance between the left, top, right, and bottom boundaries of the bounding box defined in the first frame;
[0074] P feature (i,j) are points in the current feature map, P feature (i,j) in the search patch P search There is a corresponding position, denoted as P. search (i,j); P search (i,j) are points in the search patch, P search (i,j) and feature map P feature The corresponding points in the data are related;
[0075] The loss function for centrality is:
[0076]
[0077] This invention also provides a UAV target tracking system based on collaborative attention and serial channel cross-correlation. This system is used to implement the aforementioned UAV target tracking method based on collaborative attention and serial channel cross-correlation. The system includes a feature extraction module, a cross-correlation operation module, a tracking head network module, a target position determination module, and a tracking module, wherein:
[0078] The feature extraction module crops the first frame of the video to obtain the input of the template branch and crops the subsequent frame images to obtain the input of the search branch. It then performs feature extraction on the first and subsequent frame images through a Siamese network and performs feature interaction through a collaborative attention module to obtain the template feature map and the search feature map.
[0079] The cross-correlation module takes the template feature map and the search feature map obtained by the feature extraction module and inputs them into the serial channel cross-correlation module to perform cross-correlation operation and obtain the output feature map.
[0080] The tracking head network module inputs the output feature map obtained by the cross-correlation operation module into the anchorless tracking head network. The classification branch obtains the classification score of each pixel in the feature map, the centrality branch obtains the centrality score between each pixel and the target, and the regression branch obtains the target box information corresponding to each pixel.
[0081] The target location determination module multiplies the classification score and centrality score of each pixel in the tracking head network module, selects the pixel with the highest score, and determines the target box information of the pixel through the regression branch, thereby determining the location of the target in the current frame;
[0082] The tracking module repeats the processing steps of the feature extraction module, cross-correlation operation module, tracking head network module, and target location determination module until the position of the last frame in the video is tracked.
[0083] The present invention also provides a mobile terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the UAV target tracking method based on collaborative attention and serial channel cross-correlation.
[0084] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the UAV target tracking method based on collaborative attention and serial channel cross-correlation.
[0085] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0086] Example
[0087] Combination Figure 1 , Figure 2 This embodiment presents a UAV target tracking method based on collaborative attention and serial channel cross-correlation, comprising the following steps:
[0088] Step 1: Cropped image of the first frame of the video to obtain the input of the template branch, and cropped image of subsequent frames to obtain the input of the search branch. Features are extracted from the first and subsequent frames using a Siamese network, and then interacted with by a collaborative attention module to obtain template features and search features, such as... Figure 3 As shown, the details are as follows:
[0089] Furthermore, the Siamese network is a ResNet50 with shared weights, using the last three Block modules of ResNet50, combining the Siamese network with a collaborative attention module, and adding a collaborative attention module after each Block module;
[0090] Step 1.1: Extract features from the first frame image using a Siamese network to obtain the initial template feature map. The initial search feature map is obtained by extracting features from subsequent frame images.
[0091] In the template branch, two feature vectors with a channel dimension of C / 2 are obtained by using 1×1 convolution kernels g(●) and k(·), respectively.
[0092] In the search branch, two feature vectors with a channel dimension of C / 2 are obtained by using 1×1 convolution kernels g(●) and k(·), respectively.
[0093] Where R represents the real number field; N represents the number of samples processed at one time; W T H represents the width of the template feature map. T W represents the height of the template feature map. I H represents the width of the search feature map. I The width of the search feature map is represented by ; C represents the number of channels in the feature map.
[0094] Step 1.2: After the template branch is processed by a 1×1 convolution kernel k(·), tensor reshaping is performed to obtain the first feature map. The search branch is processed by a 1×1 convolution kernel g(●) and then tensor reshaping is performed to obtain the second feature map. The first feature map Second feature map Perform matrix multiplication to obtain intermediate feature maps.
[0095] Step 1.3: After the template branch is processed by a 1×1 convolution kernel g(●), tensor reshaping is performed to obtain the third feature map. The search branch is processed by a 1×1 convolution kernel g(●) and then tensor reshaping is performed to obtain the fourth feature map. The obtained third feature map Fourth feature map Compare with the intermediate feature maps from step 1.2 respectively. Perform matrix multiplication to obtain the template branch modulation feature map. and search branch modulation feature map
[0096] Step 1.4: Process the template branch modulation feature map from Step 1.3. Tensor reshaping is performed to obtain the fifth feature map. Search branch modulation feature map Tensor reshaping is performed to obtain the sixth feature map.
[0097] Fifth feature map Sixth feature map By applying 1×1 convolution kernels q(·) and w(·) respectively, the channel dimension is increased to C, resulting in the fifth feature map after channel dimension transformation. The sixth feature map after channel dimension transformation
[0098] Using the initial template feature map and the fifth feature map after transforming the channel dimensions Perform matrix addition to obtain the intrinsic template-related feature map ψ(x; z);
[0099] Using the initial search feature map and the sixth feature vector after transforming the channel dimensions Perform matrix addition to obtain the intrinsic search-related features ψ(z;x);
[0100] Step 1.5: Compress the obtained intrinsic template-related features ψ(x; z) using both max pooling and global average pooling operations. Then, activate them through a fully connected MLP layer. Next, use a sigmoid operation to normalize the importance of each channel to a 0-1 range. Finally, multiply the sigmoided channel parameters (normalized to 0-1) with the intrinsic template-related features ψ(x; z) and the intrinsic search-related features ψ(z; x) respectively to obtain the template feature map. and search feature map
[0101] Step 2: Input the template feature map and search feature map obtained in Step 1 into the serial channel cross-correlation module for cross-correlation operation to obtain the output feature map, such as... Figure 4 The details are as follows:
[0102] template feature map and search feature map Perform one Conv+BN+ReLU operation each, followed by the first cross-correlation operation. The formula is:
[0103]
[0104] Where Conv represents the convolution operation in deep learning, which filters the input data through a convolution kernel; BN represents the standardization operation for features; ReLU is the activation function; φ first The feature map representing the first cross-correlation; XCorr dw This indicates the similarity between computational features;
[0105] Step 2.2: The first cross-correlation feature map φ obtained in Step 2.1... first Perform a single convolutional (Conv) operation, followed by two consecutive Conv+BN+ReLU operations for upsampling, with padding set to 3 and 2 respectively, to obtain the processed features. Padding refers to the process of adding extra pixels to the edges of the input feature map during convolution operations in order to control the size of the output feature map.
[0106] Step 2.3: Process the features obtained in Step 2.2. The search feature map obtained in step 1.5 Set a weighted fusion strategy to obtain the fused search features. The formula is:
[0107]
[0108] Wherein, λ1 and λ2 are set to 0.8 and 0.2 respectively;
[0109] Step 2.4: Obtain the template feature map obtained in Step 1.5. Template features are obtained through the ECA channel attention mechanism. Then the template feature map Template features Adding them together yields the template features needed for the second cross-correlation.
[0110] Step 2.5: Perform the second cross-correlation operation, the formula is:
[0111]
[0112] φ second Feature map representing the second cross-correlation;
[0113] Step 2.6: The feature map φ of the first cross-correlation... first The feature map φ of the second cross-correlation second Add them together to obtain the output feature map.
[0114] Step 3: Input the output feature map obtained in Step 2 into the anchorless tracking head network. The classification branch obtains the classification score of each pixel in the feature map, the centrality branch obtains the centrality score between each pixel and the target, and the regression branch obtains the target box information corresponding to each pixel.
[0115] Furthermore, the tracking head network includes two branches: a classification branch and a regression branch. The classification branch is used to distinguish targets, and the regression branch is used to obtain scale information about the size of the targets.
[0116] The final response feature map obtained by the classification branch is A. cls A cls ∈R h×w×2 Response feature map A cls The size is h×w, and 2 represents the response feature map A. cls The number of channels, A cls Includes the probability that a pixel is a positive sample or a negative sample;
[0117] The final response feature map obtained by the regression branch is A.reg A reg ∈R h×w×4 Response feature map A reg The size is h×w, 4 represents the number of channels in the feature map, A reg The distance from each pixel (i,j) to each border is denoted as t. (i,j) = (l,t,r,b), where l,t,r,b represent the distances from pixel (i,j) to the left, top, right, and bottom borders.
[0118] Furthermore, the classification branch uses the cross-entropy loss function, the regression branch uses the -lnIOU loss function, and the final loss L of the entire network is expressed as:
[0119] L=λ4L cls +λ5L cen +λ6L reg
[0120] Where λ4, λ5, and λ6 are set to 1, 1, and 3 respectively, L cls L cen and L reg These represent classification loss, centering loss, and regression loss, respectively.
[0121] Step 4: Multiply the classification score and centrality score of each pixel in Step 3, select the pixel with the highest score, and determine the target box information of the pixel through the regression branch, thereby determining the location of the target in the current frame.
[0122] Furthermore, the response characteristic map of the centrality branch is A. cen A cen ∈R h×w×1 The response feature map is A. cen The size is h×w, where 1 represents the number of channels in the response map, and contains the centrality score of each pixel. The accurate target score is obtained by multiplying the centrality score by the classification score.
[0123] The true centrality value C(i,j) is:
[0124]
[0125] in:
[0126]
[0127] Where l, t, r, b represent the feature map points P obtained from the search frame, respectively. feature The search patch point P corresponding to (i,j) search (i,j) is the distance between the left, top, right, and bottom boundaries of the bounding box defined in the first frame;
[0128] P feature (i,j) are points in the current feature map, P feature (i,j) in the search patch P search There is a corresponding position, denoted as P. search (i,j); P search (i,j) are points in the search patch, P search (i,j) and feature map P feature The corresponding points in the data are related;
[0129] The loss function for centrality is:
[0130]
[0131] Step 5: Repeat steps 1 through 4 until the position of the last frame in the video is tracked.
[0132] Furthermore, ResNet50 is a deep learning network architecture; Block is a network block structure unique to ResNet; Padding is the kernel padding size; Conv is the convolution operation; and XCorr... dw This is a deep cross-correlation operation; the IoU is the result of dividing the overlapping part of two regions by the two set parts; the DTB70 is a dataset used to test tracker performance.
[0133] Furthermore, the overall network includes a feature extraction module, a collaborative attention module, a cascaded cross-correlation module, and a tracking head network. The feature extraction module consists of two ResNet50 modules with shared weights. The collaborative attention module interacts with template features and search features to enhance feature representation. The cascaded cross-correlation module obtains basic matching information of the target object in the search region. The tracking head network consists of classification and regression branches for target localization and tracking. The Siamese network structure of this invention is as follows: Figure 2 As shown.
[0134] This embodiment uses the DTB70 dataset to verify the tracking results of the present invention. The DTB70 dataset contains 70 fully annotated high-definition video datasets and benchmarks captured from a low-altitude aerial perspective. It includes various attributes such as aspect ratio variation, background clutter, camera motion, fast motion, complete occlusion, illumination variation, low resolution, out-of-view, partial occlusion, similar targets, scale variation, and viewpoint variation, which can better test the tracker's performance.
[0135] The first frame of the video is cropped to a size of 127×127×3 centered on the target and used as the input of the template branch. Subsequent frames are cropped to a size of 255×255×3 and used as the input of the search branch respectively. After passing through the feature extraction network, features are extracted from the first frame and subsequent frames.
[0136] This invention uses a collaborative attention module for feature interaction in the last three blocks of ResNet50 to obtain processed template features. and search features
[0137] template features and search features The input to the serial channel undergoes a cross-correlation operation, resulting in two cross-correlation operations, as shown below.
[0138] The features after cross-correlation are input into the tracking head network to calculate the classification and regression branches. The classification branch uses the cross-entropy loss function, and the regression branch uses the -lnIOU loss function. The target boxes use the coordinates (x0, y0) and (x1, y1) of the top left and bottom right corners. The distance from any point (x, y) in the search region to the four target bounding boxes is represented as: l = x - x0, t = y - y0, r = x1 - x, b = y1 - y.
[0139] The difference between the true bounding box and the predicted box is calculated using the -lnIOU loss function, which is then used to regress the target box.
[0140] The tracker of this invention was evaluated against trackers such as Ocean, SiamFC++, SiamAPN, SiamAPN++, SiamCAR, and SiamGAT, and the final results are as follows. Figure 5 and Figure 6 As shown, the tracking effect is as follows Figure 7 As shown.
[0141] Depend on Figure 5 , Figure 6 , Figure 7 As can be seen, this invention utilizes a collaborative attention module and a serial channel cross-correlation module to accurately extract the features of the target, thereby obtaining a more robust tracking model. It can better solve problems such as target scale changes and occlusion, thus achieving accurate and robust tracking results.
[0142] The above examples are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that obvious changes or modifications derived from the spirit of the present invention are still within the protection scope of the present invention.
Claims
1. A UAV target tracking method based on collaborative attention and serial channel cross-correlation, characterized in that, Includes the following steps: Step 1: Crop the first frame of the video to obtain the input of the template branch, and crop the subsequent frame images to obtain the input of the search branch. The first frame and subsequent frame images are processed by the Siamese network to extract features, and then the features are interacted through the collaborative attention module to obtain the template feature map and the search feature map. Step 2: Input the template feature map and search feature map obtained in Step 1 into the serial channel cross-correlation module to perform cross-correlation operation and obtain the output feature map; Step 3: Input the output feature map obtained in Step 2 into the anchorless tracking head network. The classification branch obtains the classification score of each pixel in the feature map, the centrality branch obtains the centrality score between each pixel and the target, and the regression branch obtains the target box information corresponding to each pixel. Step 4: Multiply the classification score and centrality score of each pixel in Step 3, select the pixel with the highest score, and determine the target box information of the pixel through the regression branch, thereby determining the location of the target in the current frame. Step 5: Repeat steps 1 to 4 until the position of the last frame in the video is tracked. The Siamese network in step 1 is a ResNet50 with shared weights. It uses the last three Blocks of ResNet50 and combines the Siamese network with the collaborative attention module by adding a collaborative attention module after each Block. Step 2 involves inputting the template feature map and search feature map obtained in Step 1 into the serial channel cross-correlation module for cross-correlation operation to obtain the output feature map, as detailed below: Step 2.1: Transfer the template feature map and search feature map Perform once each The operation is performed first, followed by the first cross-correlation operation, with the formula as follows: in, This represents the convolution operation in deep learning, which uses a convolution kernel to filter the input data. This indicates an operation that standardizes features; It is an activation function; Feature maps representing the first cross-correlation; This indicates the similarity between computational features; Step 2.2: Calculate the first cross-correlation feature map obtained in Step 2.
1. Perform one convolution The operation, followed by two consecutive... The operation is used to perform upsampling. Setting them to 3 and 2 respectively yields the processed features. ; In convolution operations, extra pixels are padded around the edges of the input feature map to control the size of the output feature map. Step 2.3: Process the features obtained in Step 2.
2. The search feature map obtained in step 1.5 Set a weighted fusion strategy to obtain the fused search features. The formula is: in, and Set them to 0.8 and 0.2 respectively; Step 2.4: Obtain the template feature map obtained in Step 1.
5. Template features are obtained through the ECA channel attention mechanism. Then the template feature map Template features Adding them together yields the template features needed for the second cross-correlation. ; Step 2.5: Perform the second cross-correlation operation, the formula is: Feature map representing the second cross-correlation; Step 2.6: Convert the feature maps of the first cross-correlation. Feature maps of the second cross-correlation Add them together to obtain the output feature map. .
2. The UAV target tracking method based on collaborative attention and serial channel cross-correlation according to claim 1, characterized in that, Step 1 is described in detail as follows: Step 1.1: Crop the first frame of the video with the target as the center and use it as the input for the template branch; crop the subsequent frames and use them as the input for the search branch. The initial template feature map is obtained by extracting features from the first frame image using a twin network. , The initial search feature map is obtained by extracting features from subsequent frame images. , ; in Represents the real number field; Indicates the number of samples processed at one time; Indicates the width of the template feature map; Indicates the height of the template feature map; Indicates the width of the search feature map; Indicates the width of the search feature map; Indicates the number of channels in the feature map; Step 1.2: The template branch is processed by a 1×1 convolution kernel. After processing, tensor reshaping is performed to obtain the first feature map. The search branch passes through a 1×1 convolution kernel. After processing, tensor reshaping is performed to obtain the second feature map. , the first feature map Second feature map Perform matrix multiplication to obtain intermediate feature maps. ; Step 1.3: The template branch is processed by a 1×1 convolution kernel. After processing, tensor reshaping is performed to obtain the third feature map. The search branch passes through a 1×1 convolution kernel. After processing, tensor reshaping is performed to obtain the fourth feature map. The obtained third feature map Fourth feature map Compare with the intermediate feature maps from step 1.2 respectively. Perform matrix multiplication to obtain the template branch modulation feature map. and search branch modulation feature map ; Step 1.4: Process the template branch modulation feature map from Step 1.
3. Tensor reshaping is performed to obtain the fifth feature map. Search branch modulation feature map Tensor reshaping is performed to obtain the sixth feature map. ; Fifth feature map Sixth Feature Map Each is processed by a 1×1 convolution kernel. and Increase the channel dimension to The fifth feature map after transforming the channel dimensions is obtained. The sixth feature map after channel dimension transformation ; Using the initial template feature map and the fifth feature map after transforming the channel dimensions Perform matrix addition to obtain the intrinsic template-related feature map. ; Using the initial search feature map and the sixth feature vector after transforming the channel dimensions Perform matrix addition to obtain the intrinsic search-related features. ; Step 1.5: Obtain the intrinsic template-related features Global features are compressed using both max pooling and global average pooling operations, and then activated by fully connected MLP layers. Next, a sigmoid operation is used to normalize the importance of each channel to a 0-1 range. Finally, the sigmoid-normalized channel parameters are correlated with the intrinsic template-related features. Features related to intrinsic search Multiply to obtain the template feature map and search feature map .
3. The UAV target tracking method based on collaborative attention and serial channel cross-correlation according to claim 2, characterized in that, In step 3, the classification branch is used to distinguish the targets, and the regression branch is used to obtain scale information about the size of the targets. The final response feature map obtained by the classification branch is: , Response feature map The size is ,2 represents the response feature map The number of channels, Includes the probability that a pixel is a positive sample or a negative sample; The response feature map ultimately obtained by the regression branch is: , Response feature map The size is 4 represents the number of channels in the feature map. Includes each pixel The distance to each border is expressed as... , Represents pixels The distance to the left, top, right, and bottom borders.
4. The UAV target tracking method based on collaborative attention and serial channel cross-correlation according to claim 3, characterized in that, The classification branch uses the cross-entropy loss function, and the regression branch uses... Loss function, the final loss of the entire network. Represented as: in, , and Set them to 1, 1, and 3 respectively. , and These represent classification loss, centering loss, and regression loss, respectively.
5. The UAV target tracking method based on collaborative attention and serial channel cross-correlation according to claim 4, characterized in that, The response feature map of the centrality branch mentioned in step 3 is as follows: , The response feature map is The size is 1 indicates the number of channels in the response map, which includes the centrality score of each pixel. The accurate target score is obtained by multiplying the centrality score by the classification score. Centrality True Value for: in: in, These represent the feature map points obtained from the search frame. Corresponding search patch points The distance between the left, top, right, and bottom boundaries of the bounding box defined in the first frame; It is a point in the current feature map. In search patch There is a corresponding position in it, which is represented as ; It is a point in the search patch. With feature map The corresponding points in the data are related; The loss function for centrality is: 。 6. A UAV target tracking system based on collaborative attention and serial channel cross-correlation, characterized in that, This system is used to implement the UAV target tracking method based on collaborative attention and serial channel cross-correlation as described in any one of claims 1 to 5. The system includes a feature extraction module, a cross-correlation operation module, a tracking head network module, a target position determination module, and a tracking module, wherein: The feature extraction module crops the first frame of the video to obtain the input of the template branch and crops the subsequent frame images to obtain the input of the search branch. It then performs feature extraction on the first and subsequent frame images through a Siamese network and performs feature interaction through a collaborative attention module to obtain the template feature map and the search feature map. The cross-correlation module takes the template feature map and the search feature map obtained by the feature extraction module and inputs them into the serial channel cross-correlation module to perform cross-correlation operation and obtain the output feature map. The tracking head network module inputs the output feature map obtained by the cross-correlation operation module into the anchorless tracking head network. The classification branch obtains the classification score of each pixel in the feature map, the centrality branch obtains the centrality score between each pixel and the target, and the regression branch obtains the target box information corresponding to each pixel. The target location determination module multiplies the classification score and centrality score of each pixel in the tracking head network module, selects the pixel with the highest score, and determines the target box information of the pixel through the regression branch, thereby determining the location of the target in the current frame; The tracking module repeats the processing steps of the feature extraction module, cross-correlation operation module, tracking head network module, and target location determination module until the position of the last frame in the video is tracked.
7. A mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the UAV target tracking method based on collaborative attention and serial channel cross-correlation as described in any one of claims 1 to 5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the UAV target tracking method based on collaborative attention and serial channel cross-correlation as described in any one of claims 1 to 5.
