Unmanned aerial vehicle ground target tracking method based on feature compression and feature coding network

Abstract

The application discloses a UAV target tracking method based on feature compression and feature coding network. In the target feature modeling aspect, the method proposes a feature compression network and a feature coding network based on attention. Compared with other tracking methods, the method uses time information to make up for the lack of spatial information caused by the low resolution of the UAV target, and increases the robustness to extreme environments such as motion blur and target occlusion. In terms of tracking speed, compared with other attention-based tracking methods, the method compresses the features, reduces the computational complexity and storage space requirements, and is more friendly to UAV platforms with limited computing power.

Description

Technical Field

[0001] This invention pertains to the application of artificial intelligence in the field of computer vision technology, specifically a method for tracking ground targets captured by drones using feature compression and feature encoding networks. Background Technology

[0002] Visual target tracking is a widely researched topic in computer vision. Due to the small size and ease of operation of unmanned aerial vehicle (UAV) platforms, UAV-based ground target tracking has been widely used in applications such as moving target analysis, geographic surveying, and maritime rescue. Although existing template-matching-based tracking methods can track moving targets, UAV platforms face complex and changing natural environments during flight, presenting the following challenges compared to fixed or low-speed camera platforms:

[0003] Problem 1: Ground targets captured by UAVs are characterized by small scale and low resolution, resulting in insufficient spatial information. However, existing visual target tracking methods rely on template matching based on spatial features, which makes them less robust to UAV-based ground tracking.

[0004] Question 2: Due to the drastic attitude changes experienced by drones during flight, additional target deformation and occlusion occur, leading to changes in camera perspective. Existing methods construct a similarity response map between the template image and the search region image through cocorrelation operations. However, cocorrelation is a local operation, which prevents it from aggregating features that are spatially distant, resulting in low robustness to the aforementioned disturbances. Summary of the Invention

[0005] The purpose of this invention is to propose a UAV ground target tracking method based on feature compression and feature encoding networks to address the problems existing in existing methods. This method uses a feature compression network to compress different spatial features of objects in time, thereby reducing the storage space requirements and computational complexity of the tracking method while utilizing temporal features. The method also utilizes an attention-based feature encoding network to more fully capture the dependencies between template features, current search region features, and previous search region features, thereby obtaining a more semantically informative search region feature response vector.

[0006] The specific technical solution for achieving the objective of this invention is as follows:

[0007] A UAV ground target tracking method based on feature compression and feature encoding networks includes the following steps:

[0008] Step 1: For a given template image, feature extraction is performed using a feature extraction backbone network to obtain the template feature vector; specifically: the template image... Inputting the backbone network yields the template feature vector f z H z0 W z0 Input the height and width of the template image;

[0009] Step 2: For the search image in frame t of the video, feature extraction is performed using a feature extraction backbone network to obtain the feature vector of the search region in frame t; specifically, the search image in frame t is... Inputting the backbone network yields the feature vector of the search region in frame t. Where H x0 W x0 To search for the height and width of the image;

[0010] Step 3: Input the template feature vector extracted in Step 1 into the template feature compression network to obtain the compressed template feature vector. Specifically:

[0011] First, consider the template feature vector f. z Perform a flattening operation in the spatial dimension to obtain the flattened template feature vector f. z1 ; for f z1 The formula for calculating multi-head cross-attention is as follows:

[0012]

[0013] in It is f z1 The positional encoding, where dz is the dimension of each feature in the template feature vector, N z This represents the number of features in the template feature vector. Compress the embedding vector for the template. The dimension is This represents the number of template features after compression. To compress the template feature vector, The dimension is MultiHead is a multi-head attention mechanism; the calculation formula for multi-head attention is as follows:

[0014]

[0015] Where Q, K, and V are the inputs to the multi-head attention mechanism, W is the learning parameter matrix, Cat(.) represents the spatial dimension concatenation operation, and H... i Let represent the calculation result of the i-th attention, which is calculated as follows:

[0016]

[0017] Where softmax(.) is the probability normalization function. To learn the parameter matrix;

[0018] Step 4: Extract the feature vector of the search region in frame t-1. Input the search region feature compression network to obtain the compressed search region feature vector. Specifically:

[0019] First of all By performing a flattening operation in the spatial dimension, we obtain... right The formula for calculating multi-head cross-attention is as follows: Where P res Yes Position code, P res The dimension is d x N represents the dimension of each feature in the feature vector of the search region. x This represents the number of features in the feature vector of the search region. Compress the embedding vector for the search region. The dimension is This represents the number of features in the compressed search region. To compress the feature vector of the search region, The dimension is MultiHead is a multi-head attention mechanism;

[0020] Step 5: Compress the template feature vector obtained in Step 3. Step 2 extracts the feature vector of the search region in frame t. and the compressed search region feature vector obtained in step 4 Inputting the attention-based feature search network yields the feature response vector f of the search region. res Specifically:

[0021] The attention-based feature search network consists of a feature encoding network and a feature decoding network; firstly, the feature vectors of the search region are processed. Compressed search region feature vector and compressed template feature vector In space dimension

[0022] The concatenation operation is performed at different degrees to obtain the input f of the feature encoding network. feat First, for f feat Perform the following multi-head self-attention calculation: Then The input to the feedforward network yields the output of the feature encoding network. in Decompose it in the spatial dimension to obtain the encoded template feature vector. Encoded compressed search region feature vector and the encoded search region feature vector Specifically f enc =Cat(f temp f tem f scr The feature encoding network encodes a template feature vector f. temp and the feature vector f of the search region of the encoding scr For output;

[0023] The feature decoding network uses an encoded template feature vector f temp and the feature vector f of the search region of the encoding scr For input; followed by f temp For key-value input, f scr Cross-attention is calculated for the query value input, specifically as follows:

[0024] f scr1 =f scr +MultiHead(f temp +P temp f temp +P temp f scr ),

[0025] Where P temp It is f temp Position encoding, MultiHead is a multi-head attention mechanism; then f scr1 The input to the feedforward network yields the feature response vector f of the search region output by the feature decoding network. res The calculation is as follows: f res =f scr1 +FFN(f scr1 ), where FFN is a feedforward network;

[0026] Step 6: Transfer the feature response vector f of the search region res Inputting the prediction network yields the tracking results; specifically:

[0027] First, the feature response vector of the search region is analyzed. Perform a dimensional transformation to obtain a feature response map of the search region with a spatial dimension of 2. Where H x0 W x0 H represents the height and width of the feature response map of the search region. x0 =W x0 H x0 ×W x0 =N x The prediction network is a fully convolutional neural network, specifically: fmap =FCN(f res2 ), where FCN(.) is a fully convolutional neural network, This is the output of a fully convolutional neural network; subsequently, f... map Perform a split along the feature dimension to obtain the probability map of the top-left corner. Probability diagram of the bottom right corner Finally, by calculating the expected value of the probability maps of the top left and bottom right corners, the tracking result is obtained, and its coordinates are: in These are the x-coordinate of the top left corner, the y-coordinate of the top left corner, the x-coordinate of the bottom right corner, and the y-coordinate of the bottom right corner, respectively; the mathematical expectation is calculated as follows: in This is the summation symbol.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] The attention-based feature compression network proposed in this invention compresses different spatial features of objects in time. Its advantages are: 1. It uses different spatial features of objects in time to enhance the representation ability of spatial features and suppresses the low spatial feature representation ability of ground targets captured by UAVs due to low resolution; 2. In addition, because different spatial features in time are compressed, this method has lower requirements for memory space and computing power compared with other tracking methods based on spatiotemporal information, thus making it more suitable for UAV platforms.

[0030] The attention-based feature search network proposed in this invention establishes a dependency relationship between template features, current search region features, and previous search region features, enabling the method to better construct search region feature response vectors. Advantage 3: Compared with existing search region feature response vector construction methods, the feature search network proposed in this method can still obtain accurate similarity response maps between templates and search regions when the target undergoes deformation, occlusion, or changes in camera viewpoint, providing more accurate target information for the generation of subsequent tracking results.

[0031] The prediction network proposed in this invention obtains the final tracking result by predicting the probability distributions of the top-left and bottom-right corner points of the target. Advantage 4: Compared to anchor-box-based methods, the prediction network proposed in this invention does not require pre-setting anchor boxes. Compared to anchor-box-less methods based on multilayer perceptrons, the prediction network proposed in this invention reduces the number of network parameters and directly outputs the target tracking result without complex post-processing. Attached Figure Description

[0032] Figure 1 This is a flowchart of the present invention;

[0033] Figure 2 This is a diagram of the attention-based template feature compression network structure according to an embodiment of the present invention;

[0034] Figure 3 This is a diagram of the attention-based search region feature compression network structure according to an embodiment of the present invention;

[0035] Figure 4 This is a diagram of the attention-based feature encoding network structure according to an embodiment of the present invention;

[0036] Figure 5 This is a diagram of the attention-based feature decoding network structure according to an embodiment of the present invention;

[0037] Figure 6 This is a diagram of the prediction network structure in an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0039] See Figure 1 The overall process of this invention is as follows:

[0040] Step 1: For a given template image, use a feature extraction backbone network to extract features and obtain the template feature vector;

[0041] Step 2: For the search image in frame t of the video, use the feature extraction backbone network to extract features and obtain the feature vector of the search region in frame t;

[0042] Step 3: Input the template feature vector extracted in Step 1 into the template feature compression network to obtain the compressed template feature vector.

[0043] Step 4: Extract the feature vector of the search region in frame t-1. Input the search region feature compression network to obtain the compressed search region feature vector.

[0044] Step 5: Compress the template feature vector obtained in Step 3. Step 2 extracts the feature vector of the search region in frame t. and the compressed search region feature vector obtained in step 4 Inputting the attention-based feature search network yields the feature response vector f of the search region. res ;

[0045] Step 6: Transfer the feature response vector f of the search region res Input the data into the prediction network to obtain the tracking results;

[0046] See Figure 2 Step 3 specifically includes:

[0047] First, consider the template feature vector f. z Perform a flattening operation in the spatial dimension to obtain the flattened template feature vector f. z1 ; for f z1 The formula for calculating multi-head cross-attention is as follows:

[0048]

[0049] in It is f z1 Position encoding, d z N represents the dimension of each feature in the template feature vector. z This represents the number of features in the template feature vector. Compress the embedding vector for the template. The dimension is This represents the number of template features after compression. To compress the template feature vector, The dimension is MultiHead is a multi-head attention mechanism; the calculation formula for multi-head attention is as follows:

[0050]

[0051] Where Q, K, and V are the inputs to the multi-head attention mechanism, W is the learning parameter matrix, Cat(.) represents the spatial dimension concatenation operation, and H... i Let represent the calculation result of the i-th attention, which is calculated as follows:

[0052]

[0053] Where softmax(.) is the probability normalization function. To learn the parameter matrix.

[0054] See Figure 3 Step 4 specifically includes:

[0055] First of all By performing a flattening operation in the spatial dimension, we obtain... right The formula for calculating multi-head cross-attention is as follows: Where P res Yes Position code, P res The dimension is d x N represents the dimension of each feature in the feature vector of the search region. x This represents the number of features in the feature vector of the search region. Compress the embedding vector for the search region. The dimension is This represents the number of features in the compressed search region. To compress the feature vector of the search region, The dimension is MultiHead is a multi-head attention mechanism.

[0056] See Figure 4 The feature encoding network processing in step 5 specifically includes:

[0057] The attention-based feature search network consists of a feature encoding network and a feature decoding network; firstly, the feature vectors of the search region are processed. Compressed search region feature vector and compressed template feature vector In space dimension

[0058] The concatenation operation is performed at different degrees to obtain the input f of the feature encoding network. feat First, for f feat Perform the following multi-head self-attention calculation: Then The input to the feedforward network yields the output of the feature encoding network. in Decompose it in the spatial dimension to obtain the encoded template feature vector. Encoded compressed search region feature vector and the encoded search region feature vector Specifically f enc =Cat(f temp f tem f scr The feature encoding network encodes a template feature vector f. temp and the feature vector f of the search region of the encoding scr This is the output.

[0059] See Figure 5 The feature decoding network processing in step 5 specifically includes:

[0060] The feature decoding network uses an encoded template feature vector f temp and the feature vector f of the search region of the encoding scr For input; followed by f temp For key-value input, fscr Cross-attention is calculated for the query value input, specifically as follows:

[0061] f scr1 =f scr +MultiHead(f temp +P temp f temp +P temp f scr ),

[0062] Where P temp It is f temp Position encoding, MultiHead is a multi-head attention mechanism; then f scr1 The input is fed forward to the feature decoding network, which outputs the feature response vector f of the search region. res The calculation is as follows: f res =f scr1 +FFN(f scr1 ), where FFN is a feedforward network.

[0063] See Figure 6 Step 6 specifically includes:

[0064] First, the feature response vector of the search region is analyzed. Perform a dimensional transformation to obtain a feature response map of the search region with a spatial dimension of 2. Where H x0 W x0 H represents the height and width of the feature response map of the search region. x0 =W x0 H x0 ×W x0 =N x The prediction network is a fully convolutional neural network, specifically: f map =FCN(f res2 ), where FCN(.) is a fully convolutional neural network, This is the output of a fully convolutional neural network; subsequently, f... map Perform a split along the feature dimension to obtain the probability map of the top-left corner. Probability diagram of the bottom right corner Finally, the tracking result is obtained by calculating the expected value of the probability maps of the top left and bottom right corners. in These are the x-coordinate of the top left corner, the y-coordinate of the top left corner, the x-coordinate of the bottom right corner, and the y-coordinate of the bottom right corner, respectively; the mathematical expectation is calculated as follows: in This is the summation symbol.

[0065] Furthermore, to verify the effectiveness of the proposed method, ablation experiments were conducted on the UAV123 dataset, and the experimental results were compared with the performance indicators of industry-leading tracking methods (see Table 1).

[0066] This invention underwent ablation experiments on the UAV123 dataset. The UAV123 dataset contains 123 video sequences and over 110,000 frames of images, making it one of the representative datasets in the field of UAV target tracking. The values ​​marked in bold black in the table indicate the best performance of this invention under that metric. As shown in Table 1, the abbreviation for this invention is FCNT. To evaluate the effectiveness of the proposed feature compression network and feature search network, in the ablation experiments, uncompressed template feature vectors were used to replace the compressed template feature vectors output by the proposed template feature compression network, and search region feature vectors were used to replace the compressed search region feature vectors output by the proposed search region feature compression network. Cocorrelation operations were used to replace the proposed feature search network. In the table, FCNT-baseline is used as an abbreviation for the method without using the search region feature compression network, template feature compression network, and feature search network; FCNT1 is used as an abbreviation for the method using the search region feature compression network and template feature compression network but without using the feature search network; FCNT2 is used as an abbreviation for the method without using the search region feature compression network and template feature compression network but using the feature search network; and FCNT is used as an abbreviation for the method using the search region feature compression network, template feature compression network, and feature search network.

[0067] Compared to the current mainstream SiamFC method, this invention improves the success rate by 15.9% and the accuracy by 14.8%. The feature compression network and feature search network proposed in this invention improve the performance of the method to a certain extent, and experimental results verify the effectiveness of the proposed method.

[0068] Table 1 Comparison of success rate and accuracy between this invention and other advanced methods

[0069]

[0070]

[0071] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of the inventive concept are included in this invention and are protected by the appended claims.

Claims

1. A UAV ground target tracking method based on feature compression and feature encoding networks, characterized in that, Includes the following steps: Step 1: For a given template image, use a feature extraction backbone network to extract features and obtain the template feature vector; Specifically, this involves: the template image Inputting the backbone network yields the template feature vector f z H z0 W z0 Input the height and width of the template image; Step 2: For the search image in frame t of the video, feature extraction is performed using a feature extraction backbone network to obtain the feature vector of the search region in frame t; specifically, the search image in frame t is... Inputting the backbone network yields the feature vector of the search region in frame t. Where H x0 W x0 To search for the height and width of the image; Step 3: Input the template feature vector extracted in Step 1 into the template feature compression network to obtain the compressed template feature vector. Specifically: First, consider the template feature vector f. z Perform a flattening operation in the spatial dimension to obtain the flattened template feature vector f. z1 ; for f z1 The formula for calculating multi-head cross-attention is as follows: in It is f z1 The positional encoding, where dz is the dimension of each feature in the template feature vector, N z This represents the number of features in the template feature vector. Compress the embedding vector for the template. The dimension is This represents the number of template features after compression. To compress the template feature vector, The dimension is MultiHead is a multi-head attention mechanism; the calculation formula for multi-head attention is as follows: Where Q, K, and V are the inputs to the multi-head attention mechanism, W is the learning parameter matrix, Cat(.) represents the spatial dimension concatenation operation, and H... i Let represent the calculation result of the i-th attention, which is calculated as follows: Where softmax(.) is the probability normalization function. To learn the parameter matrix; Step 4: Extract the feature vector of the search region in frame t-1. Input the search region feature compression network to obtain the compressed search region feature vector. Specifically: First of all By flattening the object in the spatial dimension, we obtain... right The formula for calculating multi-head cross-attention is as follows: Where P res Yes Position code, P res The dimension is d x N represents the dimension of each feature in the feature vector of the search region. x This represents the number of features in the feature vector of the search region. Compress the embedding vector for the search region. The dimension is This represents the number of features in the compressed search region. To compress the feature vector of the search region, The dimension is MultiHead is a multi-head attention mechanism; Step 5: Compress the template feature vector obtained in Step 3. Step 2 extracts the feature vector of the search region in frame t. and the compressed search region feature vector obtained in step 4 Inputting the attention-based feature search network yields the feature response vector f of the search region. res Specifically: The attention-based feature search network consists of a feature encoding network and a feature decoding network; firstly, the feature vectors of the search region are processed. Compressed search region feature vector and compressed template feature vector Performing a concatenation operation in the spatial dimension yields the input f to the feature encoding network. feat First, for f feat Perform the following multi-head self-attention calculation: Then The input to the feedforward network yields the output of the feature encoding network. in Decompose it in the spatial dimension to obtain the encoded template feature vector. Encoded compressed search region feature vector and the encoded search region feature vector Specifically f enc =Cat(f temp f tem f scr The feature encoding network encodes a template feature vector f. temp and the feature vector f of the search region of the encoding scr For output; The feature decoding network uses an encoded template feature vector f temp and the feature vector f of the search region of the encoding scr For input; followed by f temp For key-value input, f scr Cross-attention is calculated for the query value input, specifically as follows: f scr1 =f scr +MultiHead(f temp +P temp ,f temp +P temp ,f scr ), Where P temp It is f temp Position encoding, MultiHead is a multi-head attention mechanism; then f scr1 The input to the feedforward network yields the feature response vector f of the search region output by the feature decoding network. res The calculation is as follows: f res =f scr1 +FFN(f scr1 ), where FFN is a feedforward network; Step 6: Transfer the feature response vector f of the search region res Inputting the prediction network yields the tracking results; specifically: First, the feature response vector of the search region is analyzed. Perform a dimensional transformation to obtain a feature response map of the search region with a spatial dimension of 2. Where H x0 W x0 H represents the height and width of the feature response map of the search region. x0 =W x0 H x0 ×W x0 =N x The prediction network is a fully convolutional neural network, specifically: f map =FCN(f res2 ), where FCN(.) is a fully convolutional neural network, This is the output of a fully convolutional neural network; subsequently, f... map Perform a split along the feature dimension to obtain the probability map of the top-left corner. Probability diagram of the bottom right corner Finally, by calculating the expected value of the probability maps of the top left and bottom right corners, the tracking result is obtained, and its coordinates are: in These are the x-coordinate of the top left corner, the y-coordinate of the top left corner, the x-coordinate of the bottom right corner, and the y-coordinate of the bottom right corner, respectively; the mathematical expectation is calculated as follows: in This is the summation symbol.

Images