Underwater target tracking method and system based on adaptive cross attention

Abstract

The application discloses a kind of underwater target tracking method and system based on adaptive cross attention.The application designs underwater target tracking algorithm SACAM for underwater complex environment, by using main network combined with adaptive cross attention module ACAM, search branch and template branch respectively share the weight after passing through channel attention, so that two branches influence each other to achieve template adaptation, improve the reliability of feature extraction;Through feature hierarchical aggregation, shallow and deep features are fused;Through the improved anchor-free frame strategy, the tracked target is classified and the bounding box is predicted.The experiment shows that the application can be used for target tracking task in complex underwater environment stably and reliably, provides a reliable method for real-time and effective underwater target tracking task, has high tracking success rate and practicality.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing, specifically relating to an underwater target tracking method and system based on adaptive cross-attention. Background Technology

[0002] Traditional underwater target tracking methods are mainly divided into sonar image-based and photoelectric sensor-based underwater target detection and tracking, depending on the imaging method. After acquiring specific information about the object being tracked (distance, orientation, frequency, etc.), filtering algorithms are used to continuously estimate and predict the object's velocity, position, acceleration, and other states. Due to the special nature of the underwater environment, nonlinear filtering algorithms are generally used for object state estimation. With the development of optical vision equipment and deep learning, target tracking methods based on natural images and deep learning are also gradually being applied.

[0003] Currently, deep learning-based target tracking technology in terrestrial environments is developing rapidly. With the upgrade of convolutional neural networks, the feature extraction capabilities of neural networks have far surpassed those of traditional hand-crafted features, and single-target tracking based on Siamese networks has also made significant progress in recent years. Among them, the end-to-end SiamFC algorithm has attracted more attention from researchers due to its balance between speed and accuracy, and is regarded as the pioneering work in the field of single-target tracking based on Siamese networks. Many subsequent algorithms have also improved upon it. In bounding box regression, SiamRPN introduces the Region Candidate Network (RPN) to achieve target boundary regression using anchor boxes; SiamFC++ introduces an anchor-free regression strategy, achieving more robust bounding box prediction. In the backbone network, SiamRPN++ successfully introduces deeper networks into target tracking tasks, solving the translation invariance caused by padding.

[0004] In engineering applications, the underwater environment differs significantly from the terrestrial environment. Problems such as the high degree of freedom of movement of underwater organisms, abundant underwater background clutter, and frequent obstruction are quite serious. Therefore, achieving both real-time and accurate underwater target tracking remains one of the challenges in engineering implementation. Summary of the Invention

[0005] Objective: This invention aims to provide an underwater target tracking method and system based on adaptive cross-attention (ACAM). Addressing the motion blur and significant deformation issues exhibited by some targets during high-speed underwater movement, the method utilizes the features extracted by the backbone network through an ACAM module to achieve better feature extraction results. Furthermore, to address the issues of multiple underwater object classes and partial occlusion, a multi-layer feature fusion method is employed to obtain features containing rich shallow and deep information. Finally, to address the problems of target rotation and scale changes, an improved anchor-free boundary prediction strategy is proposed to achieve accurate regression of target boundaries. Ultimately, this method enables real-time tracking of underwater organisms with high success rates and tracking speeds.

[0006] Technical solution: To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0007] An underwater target tracking method based on adaptive cross-attention includes the following steps:

[0008] Feature extraction: The template frame features T and search frame features S extracted from multiple levels of the backbone network are respectively passed through the corresponding adaptive cross-attention (ACAM) modules to obtain the features after the interaction between the two branches at each level; the calculation method of the ACAM module for specific template frame features T and search frame features S includes: obtaining weights M by passing T and S through channel attention respectively. CT and M CS New features T1 and S1 are generated by multiplying features T and S by the channel attention weights, respectively; weights M are obtained by applying spatial attention to T1 and S1. ST and M SS New features T2 and S2 are generated by multiplying features T1 and S1 with spatial attention weights, respectively. The channel attention weights of the two branches are swapped to allow for adaptive guidance. New features T1 and S2 are generated by multiplying features T and S with the swapped channel attention weights, respectively. CS and S CT ;T2 and T CS Add them together to obtain the affected feature T3, then combine S2 with S CT The effect is obtained by adding them together, resulting in feature S3.

[0009] Feature fusion: Features obtained from multiple levels through the ACAM module are aggregated hierarchically to obtain fused template features and search features;

[0010] Target prediction: The fused template features and search features are input into the classification head and regression head to obtain the classification response map and regression response map, respectively. The optimized FCOS algorithm is used to predict the anchor-free bounding box to obtain the tracking result. The optimized FCOS algorithm uses a circular region centered on the target's center position as the positive and negative sample classification method. Each pixel in the classification response map corresponds to a search sub-window, and each score represents the confidence of the target in the corresponding sub-window. The regression response map consists of four matrices of the same scale as the classification response map, and the four maps represent the distances from the center position of the current pixel to the four boundaries.

[0011] Preferably, the hierarchical aggregation uses max pooling to make features at different levels have the same scale, splices them along the channel dimension, and finally uses convolution to adjust the number of channels to obtain the fused output features.

[0012] As a preferred method, the ACAM module calculates the features as follows:

[0013]

[0014] Where σ(·) is the Sigmoid activation function and δ(·) is the ReLU activation function. and For the two levels of parameters in the template branch MLP, and This refers to the two levels of parameters in the search branch of the MLP.

[0015] The new features T1 and S1 generated by multiplying the original branch input features by the channel attention weights are calculated as follows:

[0016]

[0017] The newly generated features are fed into the spatial attention module, where the input features, after being fused by average pooling and max pooling, are convolved to obtain the spatial location weights. The specific calculation method is as follows:

[0018]

[0019] Where Conv is the convolution operation and Concat is the concatenation operation.

[0020] The new feature maps of the two branches obtained through spatial attention are denoted as T2 and S2, respectively, and are calculated as follows:

[0021]

[0022] Channel attention weight sharing allows the template branch and the search branch to influence each other. By exchanging the weights of the two branches, they can adaptively guide each other to obtain more reliable results. The calculation method is as follows:

[0023]

[0024] Therefore, the template branch and search branch of the ACAM module are respectively processed by the adaptive cross-attention module, and the final output is as follows:

[0025]

[0026] Preferably, this invention uses ResNet-50 as the backbone network. The last three layers of features of the template branch and the search branch are passed through the ACAM module, and the scale is adjusted by pooling. After concatenation in the channel dimension, the final output features containing rich deep semantic information and shallow detail information are obtained by convolution.

[0027] The last three layers of features from the template branch and the search branch are fed into the ACAM module to obtain features after adaptive cross-attention, calculated as follows:

[0028]

[0029] Since the output feature scales of the last three residual blocks in ResNet-50 are not the same (the scale of the output feature blocks decreases as they come later, and the number of channels increases), max pooling is used on the shallow features to ensure that the outputs of the third, fourth, and fifth layers have the same scale. Next, the adjusted features from the three layers are concatenated along the channel dimension, and finally, a convolution operation is used to adjust the number of channels to obtain the output features that fuse the information from the three layers.

[0030]

[0031] As a preferred option, in the improved FCOS algorithm, the radius of the circular region...

[0032] R is defined as follows:

[0033] R = min(g) w g h ) / β

[0034] Where g w g h Let β be the width and height from the center point to the ground truth, respectively, and β ∈ [0.8, 0.85].

[0035] Based on the same inventive concept, this invention provides an underwater target tracking system based on adaptive cross-attention, comprising:

[0036] The feature extraction module is used to pass the template frame features T and search frame features S extracted from multiple levels of the backbone network through the corresponding adaptive cross-attention (ACAM) modules to obtain the features after the interaction between the two branches at each level. The calculation method of the ACAM module for specific template frame features T and search frame features S includes: obtaining weights M by passing T and S through channel attention. CT and M CS New features T1 and S1 are generated by multiplying features T and S by the channel attention weights, respectively; weights M are obtained by applying spatial attention to T1 and S1. ST and M SS New features T2 and S2 are generated by multiplying features T1 and S1 with spatial attention weights, respectively. The channel attention weights of the two branches are swapped to allow for adaptive guidance. New features T1 and S2 are generated by multiplying features T and S with the swapped channel attention weights, respectively. CS and S CT ;T2 and T CS Add them together to obtain the affected feature T3, then combine S2 with S CT The effect is obtained by adding them together, resulting in feature S3.

[0037] The feature fusion module is used to aggregate features obtained from multiple levels through the ACAM module to obtain fused template features and search features.

[0038] The system includes a prediction module, which inputs the fused template features and search features into the classification head and regression head to obtain classification response maps and regression response maps. An optimized FCOS algorithm is used to predict anchor-free bounding boxes to obtain tracking results. The optimized FCOS algorithm uses a circular region centered on the target's center position as the positive and negative sample classification method. Each pixel in the classification response map corresponds to a search sub-window, and each score represents the confidence level of the target within the corresponding sub-window. The regression response map consists of four matrices of the same scale as the classification response map, representing the distance from the center position of each pixel to the four boundaries.

[0039] Based on the same inventive concept, the present invention provides a computer system including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when loaded onto the processor, implements the steps of the underwater target tracking method based on adaptive cross-attention.

[0040] Beneficial effects: Compared with the prior art, the present invention has the following technical effects:

[0041] 1. This invention utilizes the Adaptive Cross-Attention (ACAM) module to achieve adaptive template adjustment. After inputting the current layer template features and search features, the channel attention mechanism is used to obtain the channel attention weights of the two branches and share them with the other branch. This allows for the acquisition of channel-enhanced template features and search features, making full use of the non-linear relationship between the template branch and the search branch to obtain more reliable features.

[0042] 2. This invention utilizes a feature hierarchical aggregation module to hierarchically aggregate output features at different scales, obtaining final features that include deep semantic features and shallow detailed information. Shallow features possess rich detailed information, including edges and colors; deep information contains rich high-level semantic information, suitable for blurred and deformed scenes. Hierarchical aggregation of these two types of features yields features with stronger expressiveness and richer information.

[0043] 3. This invention utilizes an improved FCOS strategy to achieve accurate bounding box prediction based on anchorless boxes. The improved strategy restricts the selection of positive and negative samples in the FCOS strategy. Previous FCOS strategies regarded samples within the bounding box as positive samples, which introduced a lot of noise. Therefore, a circular region centered on the target center is used to restrict the selection of positive samples, thereby obtaining more reliable training parameters during training.

[0044] 4. Test results on three datasets show that the SACAM underwater target tracking algorithm based on adaptive cross-attention can not only achieve real-time tracking of targets, but also effectively cope with the complex underwater environment of high-degree-of-freedom movement of underwater organisms and underwater background clutter, and has a high tracking success rate. Attached Figure Description

[0045] Figure 1 This is a structural diagram of the underwater target tracking framework according to an embodiment of the present invention.

[0046] Figure 2 This is a structural diagram of the adaptive cross-attention (ACAM) model in an embodiment of the present invention.

[0047] Figure 3 This is a structural diagram of the state prediction model in an embodiment of the present invention.

[0048] Figure 4 This is a schematic diagram illustrating the improved positive and negative sample selection in an embodiment of the present invention.

[0049] Figure 5 The following are comparison charts showing the tracking performance of the SACAM algorithm in the OTB100 dataset according to an embodiment of the present invention, where (a) is a comparison chart of tracking success rate and (b) is a comparison chart of tracking accuracy.

[0050] Figure 6The following are comparison charts showing the tracking performance of the SACAM algorithm in 40 underwater datasets according to an embodiment of the present invention, where (a) is a comparison chart of tracking success rate and (b) is a comparison chart of tracking accuracy.

[0051] Figure 7 The image shows a comparison of the tracking performance of the SACAM algorithm in this invention under 11 tracking challenge factors, where (a) to (k) correspond to illumination change, out-of-plane rotation, scale change, occlusion, deformation, motion blur, fast motion, in-plane rotation, partial out-of-view, background blur, and low resolution, respectively.

[0052] Figure 8 This is a comparison of some actual tracking results of the SACAM algorithm in an underwater dataset according to an embodiment of the present invention. The images in different rows represent different scenes, and multiple images in the same row represent different image frames. Detailed Implementation

[0053] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings:

[0054] like Figure 1-3 As shown in the figure, the underwater target tracking method based on adaptive cross-attention disclosed in this invention has the following main steps:

[0055] Step 1: The template frame features and search frame features extracted from multiple layers by the backbone network are respectively passed through the corresponding adaptive cross-attention (ACAM) modules. This embodiment of the invention uses a relatively deep ResNet-50 as the backbone network and passes the features from the last three layers through the adaptive cross-attention (ACAM) module. Previous target tracking tasks used the backbone network to extract template frame and search frame features separately and performed cross-correlation operations to obtain the feature response map. However, this operation isolates the template frame and search frame, limiting the tracker's ability to capture the non-linear relationship between the template frame and search frame, and preventing the search branch and template branch from influencing each other.

[0056] The tracking framework of the ACAM module is as follows: Figure 2 As shown, in the ACAM module, the template branch (search branch) shares the weights after channel attention with the search branch (template branch), allowing the two branches to influence each other to achieve template adaptation. T and S are the template frame features and search frame features extracted by the backbone network, respectively, with dimensions of h×w×C and H×W×C. These features are then input into the channel attention module, where average pooling and max pooling are used to spatially aggregate the template frame and search frame features, generating... and Two distinct spatial features are input into a fully connected MLP layer, and the output features are summed pixel-by-pixel to generate 1×1×C channel attention weights M. C The attention weights of the template frame channels are denoted as M.CT The channel attention weights for the search frame are denoted as M. CS The calculation of channel attention weights can be represented as follows:

[0057]

[0058]

[0059] Where σ(·) is the Sigmoid activation function and δ(·) is the ReLU activation function. and For the two levels of parameters in the template branch MLP, and For the two levels of parameters in the search branch MLP.

[0060] The new features T1 and S1 generated by multiplying the original branch input features by the channel attention weights are calculated as follows:

[0061]

[0062] The newly generated features are fed into the spatial attention module. The input features, after being fused by average pooling and max pooling, are then convolved to obtain the spatial location weights. The specific calculation method is as follows:

[0063]

[0064] Where Conv stands for convolution operation and Concat is the Concat function in Python.

[0065] Let T2 and S2 be the new feature maps of the two branches obtained through spatial attention, respectively, and represent them as follows:

[0066]

[0067] Channel attention weight sharing allows the template branch and the search branch to influence each other. By exchanging the weights of the two branches, they can adaptively guide each other to obtain more reliable results. The calculation method is as follows:

[0068]

[0069] Therefore, the template branch and search branch of the ACAM module are respectively processed by the adaptive cross-attention module, and the final output is as follows:

[0070]

[0071] Step 2: The features obtained from multiple layers through the ACAM module are aggregated hierarchically to obtain fused template features and search features. In this embodiment, the last three layers of features from the template branch and the search branch are processed through the ACAM module, and the scale is adjusted by pooling. After concatenation along the channel dimension, convolution is used to obtain the final output features containing rich deep semantic information and shallow detail information.

[0072] The last three layers of features from the template branch and the search branch are fed into the ACAM module to obtain features after adaptive cross-attention, calculated as follows:

[0073]

[0074] Since the output feature scales of the last three residual blocks in ResNet-50 are not the same (the scale of the output feature blocks decreases as they come later, and the number of channels increases), max pooling is used on the shallow features to ensure that the outputs of the third, fourth, and fifth layers have the same scale. Next, the adjusted features from the three layers are concatenated along the channel dimension, and finally, a convolution operation is used to adjust the number of channels to obtain the output features that fuse the information from the three layers.

[0075]

[0076] Step 3: Predict anchorless bounding box using optimized FCOS:

[0077] The block diagram of the state prediction module is as follows: Figure 3 As shown, the template features and search features after feature fusion are used as module inputs and input into the classification head and regression head to obtain the classification response map and regression response map.

[0078] The classification branch maps each pixel in the classification response map to a sub-window in the input image, and each score in the classification response map directly gives the confidence of the target in the response sub-window. The regression response contains four response maps, and the output of each point in each map is the distance from the corresponding point in the original search image to each edge of the target bounding box. The four maps represent the distances from the corresponding image in the search image to the four edges of the bounding box. However, experiments have shown that FCOS produces many low-quality bounding boxes at locations far from the target center. This is because classification confidence and localization accuracy are not well correlated, and directly using classification scores to select the final tracking box leads to a decrease in localization accuracy. For example, in multi-class underwater target tracking scenarios, many similar fish may appear in adjacent image patches, requiring more accurate localization of the tracked fish. The penalty method used in SiamFC is to add a cosine window to the response map; however, the object's position is not always in the center of the image, and directly adding a cosine window sometimes worsens the effect. Therefore, a "center-ness" module to suppress responses at greater distances is added to FCOS, calculated as follows:

[0079]

[0080] Where l, t, r, and b are the distances from the corresponding position in the input image to the four sides of the ground-truth bounding box, respectively. The value of center-ness is between 0 and 1. During testing, the predicted center-ness is multiplied by the corresponding classification response to obtain the final classification response.

[0081] In the traditional FCOS algorithm, during training, pixels in the feature map are mapped back to the original input map. Pixels falling within the ground truth are considered positive samples, otherwise negative samples. This approach introduces ambiguous samples in multi-object detection and tracking tasks, but not in single-object tracking tasks. Since the ground truth bounding box annotation inevitably introduces some noise, a circular region is used here for positive and negative sample classification, such as... Figure 4 As shown, when a pixel enters the region centered at the target's center and R is half its radius, it is considered a positive sample; otherwise, it is considered a negative sample. This method provides more accurate classification of positive samples. The definition of R is as follows:

[0082] R = min(g) w h h ) / β

[0083] Where g w g h These are the width and height from the center point to the ground truth, respectively, and β∈[0.8,0.85]. In this example, it is set to 0.8, which is the optimal value obtained from the experiment.

[0084] During the overall training process, the loss function is the sum of the three output branches, including classification loss, regression loss, and center-ness loss, calculated as follows:

[0085]

[0086] in:

[0087] N pos This represents the number of positive samples;

[0088] p x，y This represents the predicted class score at point (x, y) in the classification response map;

[0089] This represents the true category label corresponding to the point (x, y) in the classification response map;

[0090] t x，y This represents the predicted bounding box information of the target at point (x, y) in the regression response plot;

[0091] This represents the true bounding box information of the target at point (x, y) in the regression response map;

[0092] s x，y This represents the predicted center-ness at point (x, y) in the classification response map;

[0093] This represents the true center-ness of the point (x, y) in the classification response map;

[0094] The value is set to 1 when the classification response map matches a positive sample and 0 when it matches a negative sample.

[0095] Classification loss L cls Using BCE Loss and Focal Loss, all samples are included in the calculation; regression loss L reg Using GIoU Loss, only positive samples are included in the calculation; center-ness loss L centerness Using BCE Loss, only positive samples are included in the calculation.

[0096] In the tracking performance test, the tracking algorithm was first trained using the SACAM algorithm pre-trained on the ImageNet dataset with ResNet-50, and then trained offline on four commonly used datasets: YouTube-BB, ILSVRC2015-VID, GOT-10K, and COCO. After offline training, corresponding tests were performed. The template image size was 127×127, and the search image size was 255×255. The optimization method used was stochastic gradient descent. The entire training process consisted of 50 rounds, with each round using a 6×10... 5 Image pairs. In the first 10 rounds of training, at 10... -3 The warm-up learning rate trains the anchorless network and freezes the backbone network. Starting from round 11, the backbone network is unfrozen, and the entire network is trained end-to-end, with the learning rate starting at 5×10⁻⁶. -3 Decaying exponentially to 1×10 -5 The weight decay and momentum are set to 1×10. -3 And 0.9. To verify the tracking performance of the SACAM algorithm, this section conducted experiments on OTB100, VOT2018, and a self-built underwater single-target tracking benchmark dataset.

[0097] like Figure 5 As shown in the figure, on the OTB100 dataset, the proposed SACAM outperforms most mainstream algorithms such as SiamCAR and SiamRPN++ with a success rate of 69.7%, second only to the corner detection-based CGACD algorithm. Due to the use of a deeper ResNet-50 backbone network and hierarchical aggregation of 3-5 layers of features, SACAM's success rate is 1.5% higher than SiamFC++ using GoogleNet and 2.1% higher than Ocean using the fourth layer of ResNet-50 as output. In terms of accuracy, due to the advantage of the anchor-free regression strategy, which is not limited by anchor boxes when regressing bounding boxes, less background information is introduced into the results, resulting in more accurate bounding boxes. SACAM leads the anchor-based SiamRPN++ by 0.9% with an accuracy of 91.5%.

[0098] Table 1 shows the comparison results of each algorithm on the VOT2018 dataset.

[0099] method SiamRPN SiamFC++ SiamRPN++ SiamBAN DiMP Ocean SACAM A 0.586 0.587 0.600 0.590 0.597 0.592 0.602 R 0.276 0.183 0.234 0.178 0.153 0.117 0.141 EAO 0.383 0.426 0.414 0.447 0.440 0.489 0.477

[0100] The tracking results on the VOT2018 dataset are compared. The main reference metrics are accuracy (A), robustness (R), and EAO (Earning Objective of Tracking). The top three metrics in each category are marked in red, green, and blue, respectively. As shown in the table, the SACAM algorithm proposed in this chapter ranks first with a success rate of 60.2%, leading SiamRPN++ (based on anchor box and feature fusion strategy) by 0.2% and DiMP (based on online update strategy) by 0.5%. Robustness (R) reflects the number of tracking losses; SACAM ranks second with an EAO of 0.141, which is less robust than the Ocean algorithm. The more tracking losses in a video sequence, the more smaller video sequences are divided into, resulting in a lower EAO. SACAM ranks second with an EAO of 0.477, second only to the Ocean algorithm.

[0101] exist Figure 6 Among the algorithms, the top three are SACAM, SiamRPN++, and Ocean, as presented in this chapter. In terms of accuracy, SACAM leads the other tracking algorithms with 86.8%, exceeding SiamRPN++ (ranked second) by 4.3% and Ocean (ranked third) by 6.9%. Regarding success rate, SACAM ranks first with 73.6%, exceeding SiamRPN++ (ranked second) by 0.9% and Ocean (ranked third) by 2.4%.

[0102] In terms of tracking speed, SACAM has a much higher computational load than SiamFC and SiamRPN due to the multi-feature fusion, but it can still achieve real-time target tracking at 27.36fps.

[0103] from Figure 7 As can be seen, SACAM ranks first in tracking success rate for both illumination changes and background clutter attributes, exceeding the second-ranked tracking algorithm by at least 1%. This is attributed to SACAM's feature hierarchical aggregation module, which allows the output features to fully utilize shallow detail posts and high-level semantic features. Regarding scale change attributes, SACAM leads the second-ranked tracker by 0.7%, and both top-ranked algorithms are based on an anchor-free strategy, demonstrating the effectiveness of anchor-free algorithms in handling scale changes. In deformation, in-plane rotation, and rapid motion attributes, SACAM leads the second-ranked tracker by 3.2%, 0.8%, and 1.1%, respectively. When the target shape changes or experiences high-speed movement within a short period, the classification regression in the state prediction module can still locate the tracked target, and state regression can also detect large displacements within a short time. Overall, the SACAM tracking algorithm achieves good tracking results across 11 different attributes, and is particularly suitable for underwater scenes with background clutter, illumination changes, deformation, and scale changes.

[0104] exist Figure 8 It is also clear from the data that when facing problems such as underwater high-degree-of-freedom motion and background clutter, the SACAM tracking algorithm has a better tracking speed and tracking success rate than other tracking algorithms.

[0105] Based on the same inventive concept, this invention discloses an underwater target tracking system based on adaptive cross-attention (ACAM), comprising: a feature extraction module, used to pass template frame features T and search frame features S extracted from multiple levels of the backbone network through corresponding adaptive cross-attention ACAM modules to obtain features after mutual influence between the two branches at each level; a feature fusion module, used to perform hierarchical aggregation of the features obtained from multiple levels through the ACAM module to obtain fused template features and search features; and a prediction module, used to input the fused template features and search features into a classification head and a regression head to obtain a classification response map and a regression response map, and use an optimized FCOS algorithm to perform anchorless bounding box boundary prediction to obtain the tracking result. Specific implementation details are as described in the above method embodiments and will not be repeated here.

[0106] Based on the same inventive concept, an embodiment of the present invention discloses a computer system including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the steps of the underwater target tracking method based on adaptive cross-attention.

[0107] Although the above embodiments have described the present invention in detail, in actual implementation, details can still be supplemented and equivalent substitutions can be made according to actual operation. It should be noted that these technical solutions that supplement details and make equivalent substitutions to the claims of the present invention are all considered to be within the scope of protection of the present invention.

Claims

1. A method for underwater target tracking based on adaptive cross-attention, characterized in that, The steps include the following: The template frame features T extracted from multiple levels of the backbone network are respectively input into corresponding adaptive cross-attention ACAM modules with the search frame features S, to obtain features after mutual influence of two branches at each level; the calculation of the ACAM module for specific template frame features T and search frame features S includes: passing T and S through channel attention to obtain weights M CT and M CS , multiplying the features T and S by the channel attention weights to generate new features T1 and S1; passing T1 and S1 through spatial attention to obtain weights M ST and M SS , multiplying the features T1 and S1 by the spatial attention weights to generate new features T2 and S2; exchanging the channel attention weights of the two branches to adaptively guide the two branches, multiplying the features T and S by the exchanged channel attention weights to generate new features T CS and S CT ; adding T2 and T CS to obtain the influenced feature T3, and adding S2 and S CT to obtain the influenced feature S3; The hierarchical aggregation is performed on the features obtained by the ACAM module to obtain fused template features and search features; The fused template features and search features are input into a classification head and a regression head to obtain a classification response map and a regression response map, and an optimized FCOS algorithm is used for anchor-free bounding box prediction to obtain a tracking result; the optimized FCOS algorithm adopts a circular region with a target center position as a center as a positive and negative sample classification mode; each pixel point in the classification response map corresponds to a search sub-window, and each score represents the confidence of a target in the corresponding sub-window; the regression response map is four matrices with the same scale as the classification response map, and each matrix represents the distance from the center position of a corresponding pixel point to four boundaries.

2. The adaptive cross-attention based underwater target tracking method of claim 1, wherein, The hierarchical aggregation is performed on the features obtained by the ACAM module to obtain fused template features and search features; 3. The adaptive cross-attention based underwater target tracking method of claim 1, wherein, The new features T1 and S1 generated by multiplying the original branch input features and the channel attention weight are calculated as follows: where σ(·) is a Sigmoid activation function, and δ(·) is a Relu activation function, and are two layers of parameters in the template branch MLP, and are two layers of parameters in the search branch MLP; The newly generated features are input into a spatial attention module, the input features after average pooling and maximum pooling are fused and convolved to obtain a spatial position weight, and the specific calculation mode is as follows: where Conv is a convolution operation, and Concat is a concatenation operation; The new feature maps of the two branches through the spatial attention are denoted as T2 and S2, and the calculation mode is as follows: The channel attention weight sharing can enable the template branch and the search branch to influence each other, and the two branches can be adaptively guided by exchanging the weights of the two branches to obtain more reliable results, and the calculation mode is as follows: The template branch and the search branch of the ACAM module are output finally through the adaptive cross-attention module as follows: ResNet-50 is used as a backbone network, and the last three features are input into the adaptive cross-attention ACAM module, and the three output features obtained through the ACAM module are hierarchically aggregated, and the specific calculation mode is as follows:

4. The adaptive cross-attention based underwater target tracking method of claim 1, wherein, The three inputs of the hierarchical aggregation are the last three features in the ResNet-50, and the calculation mode after the ACAM module is as follows: The radius R of the circular region is defined as follows: wherein T 3 , S 3 , T 4 , S 4 , T 5 , S 5 are the third, fourth, and fifth layer template frame features and search frame features extracted by ResNet-50, respectively; max pooling is used to make the outputs of the third and fourth layers and the output of the fifth layer have the same scale, then the three adjusted features are spliced in the channel dimension, and finally a convolution operation is used to adjust the number of channels to obtain the output feature of the fusion of three layers of information:

5. The adaptive cross-attention based underwater target tracking method according to claim 1, wherein, The steps include the following: R = min(g w ,g h ) / β where g w , h h are the width and height of the center point to ground-truth, β ∈ [0.8, 0.85].

6. An underwater target tracking system based on adaptive cross-attention, characterized in that, The features fusion module is configured to perform hierarchical aggregation on the features obtained by the ACAM module to obtain fused template features and search features; The feature extraction module is used for extracting the template frame features T of multiple levels of the backbone network and the search frame features S through corresponding adaptive cross attention ACAM modules respectively, to obtain the features of the two branches after mutual influence of each level; the calculation method of the ACAM module for specific template frame features T and search frame features S includes: passing T and S through channel attention to obtain weights M CT and M CS , multiplying the features T and S by the channel attention weights to generate new features T1 and S1; passing T1 and S1 through spatial attention to obtain weights M sT and M sS , multiplying the features T1 and S1 by the spatial attention weights to generate new features T2 and S2; exchanging the channel attention weights of the two branches to adaptively guide, multiplying the features T and S by the exchanged channel attention weights to generate new features T CS and S CT ; adding T2 and T CS to obtain the influenced feature T3, and adding S2 and S CT to obtain the influenced feature S3; The prediction module is configured to input the fused template features and search features into a classification head and a regression head to obtain a classification response map and a regression response map, and an optimized FCOS algorithm is used for anchor-free bounding box prediction to obtain a tracking result; the optimized FCOS algorithm adopts a circular region with a target center position as a center as a positive and negative sample classification mode; each pixel point in the classification response map corresponds to a search sub-window, and each score represents the confidence of a target in the corresponding sub-window; the regression response map is four matrices with the same scale as the classification response map, and each matrix represents the distance from the center position of a corresponding pixel point to four boundaries. ​ 7. The adaptive cross-attention based underwater target tracking system of claim 6, wherein, The feature fusion module adopts a maximum pooling manner to make features of different levels have the same scale, and splices in the channel dimension, and finally uses a convolution operation to adjust the number of channels to obtain the fused output feature.

8. The adaptive cross-attention based underwater target tracking system of claim 6, wherein, The calculation manner of the ACAM module for obtaining the feature is as follows: where σ(·) is a Sigmoid activation function, and δ(·) is a Relu activation function, and are two-layer parameters in the template branch MLP, and are two-layer parameters in the search branch MLP; The new feature T1, S1 generated after the original branch input feature is multiplied by the channel attention weight is calculated as follows: The newly generated feature is sent to the spatial attention module, the input features after average pooling and maximum pooling are fused and then convolved to obtain the spatial position weight, and the specific calculation manner is as follows: Wherein, Conv is a convolution operation, and Concat is a splicing operation. The new feature maps of the two branches through the spatial attention are respectively denoted as T2 and S2, and the calculation manner is as follows: The channel attention weight sharing can make the template branch and the search branch influence each other, and through the exchange of the weights of the two branches, the two branches are adaptively guided to obtain more reliable results, and the calculation manner is as follows: The template branch and the search branch of the ACAM module are respectively output through the adaptive cross-attention module, and the final output is as follows:

9. The adaptive cross-attention based underwater target tracking system of claim 1, wherein, ResNet-50 is used as the backbone network, and the last three layers of features are passed through the adaptive cross-attention ACAM module, and the three output features after the ACAM module are aggregated in layers, and the specific calculation manner is as follows: The three inputs of the layered aggregation are the last three layers of features in the ResNet-50, and the calculation after the ACAM module is as follows: wherein T 3 , S 3 , T 4 , S 4 , T 5 , S 5 are the third, fourth, and fifth layer template frame features and search frame features extracted by ResNet-50, respectively; max pooling is used to make the outputs of the third and fourth layers and the output of the fifth layer have the same scale, then the three adjusted features are spliced in the channel dimension, and finally a convolution operation is used to adjust the number of channels to obtain the output feature of the fusion of three layers of information:

10. A computer system comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program comprises computer program code configured to cause the processor to perform the method of any one of claims 1 to 9. The computer program is loaded into the processor to realize the steps of the underwater target tracking method based on adaptive cross-attention according to any one of claims 1-5.

Images