An end-to-end single target tracking method based on underwater clue-driven prompt learning

Abstract

The application provides an end-to-end single target tracking method based on underwater clue-driven prompt learning, and belongs to the field of computer vision. Mainly comprising: modulating the template image and the search image in a channel mode by using a channel attention mechanism in a clue feature generation network; obtaining underwater object texture features by marking the local interaction between the modulated features by using a spatial dimension attention mechanism in the clue feature generation network; obtaining underwater clue prompt features by performing a multi-head cross attention operation; inputting the template and search features and the underwater clue prompt features into a fusion module; obtaining underwater foreground texture features by continuing to multiply the template and search features with the correlation score in the matrix corresponding position; inputting the underwater clue prompt features and the template and search features into the backbone Transformer layer of the tracker to interact information and calculate the prediction box information. The application solves the problem that the existing single target tracker structure cannot be trained end-to-end in the underwater scene.

Description

Technical Field

[0001] The present invention relates to technical fields such as marine environment perception and image data processing, and specifically to an end-to-end single target tracking method based on underwater cue-driven prompt learning. Background Art

[0002] Cue learning is an emerging technique in natural language processing that aims to address the adaptability and generalization issues of pre-trained language models for specific tasks. It introduces cues to guide the model to better understand and execute the task, thereby improving model performance. Based on this concept, cue learning has shown great potential in the field of multimodal tracking and vision. In multimodal tracking, in addition to common visual and textual information, data from multiple modalities can be introduced as cues. Using a cue learning framework, appropriate cues are designed to guide the model to learn the associations between different modalities, thereby achieving a more comprehensive understanding of the scene and target, and improving tracking accuracy and robustness. Currently, some research, based on this modality transfer concept, uses dark cues as cues to assist mainstream target tracking methods in scene transfer in dark scenes, thereby achieving improved accuracy. However, the prevailing approach in current underwater single-target tracking methods is to "enhance first, then track." This approach makes architectures designed based on this concept difficult to train in an end-to-end manner and is complex to design. Therefore, two key issues remain: how to effectively apply cue learning to single-target tracking for underwater scene transfer and how to efficiently extract underwater texture information as auxiliary cues for the backbone tracking network. Summary of the Invention

[0003] In light of the shortcomings of existing technologies, this paper provides an end-to-end single-target tracking method based on underwater cue-driven prompt learning. This paper introduces variants of the channel-wise and spatial-attention mechanisms, along with the auxiliary concept of prompt learning, to better address the challenges of end-to-end training of underwater single-target tracking networks and the difficulty in extracting effective underwater information.

[0004] The technical means adopted in the present invention are as follows:

[0005] An end-to-end single target tracking method based on underwater cue-driven prompt learning includes the following steps:

[0006] S1. Acquire underwater video data to be tracked, a search image, and a template image; construct a clue feature generation network, wherein the clue feature generation network includes a channel attention mechanism, a spatial attention mechanism, and a multi-head cross attention mechanism for clue extraction;

[0007] S2. utilizing a channel attention mechanism of a clue feature generation network to modulate the template image and the search image in a channel-wise manner, thereby generating a modulation feature that highlights the texture of foreground objects in the underwater image;

[0008] S3, using the spatial dimension attention mechanism in the clue feature generation network to mark the local interaction between the modulation features to obtain the texture features of the underwater object;

[0009] S4, constructing a carrier embedding for storing texture information of underwater scene objects, wherein the embedding is initially set to all 1s;

[0010] S5. Use the initialized embedding as the query tag and the underwater object texture feature as the key tag and value tag of the multi-head cross attention mechanism to perform a multi-head cross attention operation to obtain the underwater clue prompt feature;

[0011] S6. Input the template, search features, and underwater clue prompt features into a fusion module, wherein the fusion module is used to perform a matrix multiplication operation between the features to obtain a correlation score, wherein the template and search features are obtained by: processing the template image and the search image through a PatchEmbedding operation and then feeding them into a Transformer backbone for processing;

[0012] S7, using the template and the search feature to continue to perform matrix multiplication operation on the corresponding position of the correlation score to obtain the underwater foreground texture feature;

[0013] S8. Input the underwater clue prompt features, template and search features into the backbone Transformer layer of the tracker for information interaction, and use the central regression head to calculate the predicted box information of the final result.

[0014] Furthermore, the template image and the search image are modulated in a channel-wise manner using a channel attention mechanism of a clue feature generation network, thereby generating a modulation feature that highlights the texture of foreground objects in the underwater image, including:

[0015] S201, cutting the sizes of the template image and the search image in the image based on the hyperparameter setting;

[0016] S202, based on the cropped template and the search image information, perform channel dimension information attention processing and calculate the channel dimension attention weight, including the feature X0 processed by the cropping and PatchEmbedding operation or the feature X0 not processed by the clue feature generation network. i Assign to channel feature X c :

[0017] X c =X i or X c=X0(1)

[0018] where X i Represents the template and search image features output by the i-th layer Transformer backbone;

[0019] S203, first perform average pooling on the template and search image features, the formula is:

[0020]

[0021] Where h and w represent the height and width of the feature after it is converted into a feature map;

[0022] S204, the formula for calculating the maximum pooling is:

[0023]

[0024] S205. The formula for concatenating the features obtained by average pooling and maximum pooling in the dimension of the number of tags is:

[0025]

[0026] S206: The concatenated result is reduced in dimension through a multi-layer perceptron, and first processed through a fully connected layer:

[0027]

[0028] Where δ() represents the ReLU activation function, W0 is the weight matrix of the fully connected layer in the multilayer perceptron, and B0 is the bias matrix of the fully connected layer in the multilayer perceptron;

[0029] M2=δ(W1M1+B1) (6)

[0030]

[0031] Among them, M2 represents the output result of the second fully connected layer;

[0032] S207, input the channel attention processed feature map M into the Sigmoid activation function layer to obtain the channel attention weight, the formula is:

[0033]

[0034] S208. Based on the obtained channel attention weight, the channel attention weight is multiplied by the channel feature bit by bit to obtain the feature map adjusted by the channel attention. The formula is:

[0035]

[0036] Further, the local interaction between the labels in the modulation feature is marked by using a spatial dimension attention mechanism in the clue feature generation network to obtain the underwater object texture feature, including:

[0037] S301, obtain average pooling and maximum pooling features of the modulation feature;

[0038] S302, splice the features after average pooling and maximum pooling Divide into four equally sized feature maps Input each divided feature map into a multi-layer perceptron for local feature interaction to obtain a spatially processed feature Then, the four small feature maps are spliced into one large feature map according to the original splitting order Here, N represents the total number of labels in the feature;

[0039] S303, the obtained feature map X s Flatten to the shape of (B, C, N) and input to the Sigmoid activation function layer to calculate the spatial attention weight β as follows:

[0040]

[0041] S304, multiply the spatial attention weight with the feature processed by the channel attention to obtain the underwater object texture feature:

[0042]

[0043] Further, a carrier embedding for storing underwater scene target texture information is constructed, including:

[0044] S401, in the first layer processing, the embedding initialized to all 1, the dimension information is consistent with the feature processed by the main network except the token number, and the embedding information input in the subsequent layer processing is the new embedding feature output by the previous layer;

[0045] S402, this embedding serves as a carrier to preserve all underwater object texture features in a frame of picture and propagates in each clue extraction layer in a self-recurrent manner, and interacts and propagates to preserve the most accurate information.

[0046] Further, the initialized embedding is used as a query label, and the underwater object texture feature is used as a key label and a value label of a multi-head cross attention mechanism to perform a multi-head cross attention operation to obtain an underwater clue prompt feature, including:

[0047] S501, the sources of the query label, the key label and the value label of the multi-head cross attention are as shown in the formula:

[0048]

[0049] in is an initialized learnable, autoregressive embedding, It represents the clue features extracted by the clue extraction module in the previous layer;

[0050] S502: Perform a multi-head cross attention operation on the underwater object texture features processed by the clue feature generation network and the learnable embedding. The multi-head cross attention formula is as follows:

[0051] p n =MultiHead(Q,K,V)=Concat(H1,...,H n )W 0 (13)

[0052]

[0053] Where W i Q ,W i K ,W i V ,W O are all learnable parameters, n is the number of layers in the network;

[0054] S503. Normalize the clue results obtained based on multi-head cross attention and use the residual structure. The formula is as follows:

[0055]

[0056] in represents the clue features after normalization and residual processing, p n Represents the subsequent underwater cue features processed by multi-headed cross-attention.

[0057] Furthermore, the template, search features, and underwater clue prompt features are input into a fusion module, which is used to perform matrix multiplication operations between features to obtain a correlation score, including:

[0058] S601: Perform a dot product operation on the underwater clue prompt feature, the template, and the search feature to obtain a correlation score. The formula is as follows:

[0059] A tt n n=p n ×X n (16)

[0060] where p n Represents underwater clue prompt features, X n Represents templates and search features.

[0061] Furthermore, the template and search features are used to continue to perform matrix multiplication operations with the correlation scores at corresponding positions to obtain underwater foreground texture features, including:

[0062] S701, based on the dot product operation, the correlation score is multiplied bit by bit with the template and the search feature, and the residual structure is used to obtain the underwater foreground texture feature. The formula is as follows:

[0063] S n =Attn n *X n (17)

[0064] Among them S n represents the underwater foreground texture feature, X n Represents template and search features, Attn n Represents the relevance score.

[0065] Furthermore, the underwater clue features, template, and search features are input into the tracker’s backbone Transformer layer for information interaction. The final result is used to calculate the predicted box information using the central regression head, including:

[0066] S801. The formula for information interaction is as follows:

[0067]

[0068] Among them, X0 represents the template and search features, S0 represents the underwater foreground texture features extracted by the first layer clue feature generation network, The template and search features representing the enhanced foreground texture are ready to be input into the first layer of the backbone Transformer encoder. Represents the result output by the n-th layer backbone Transformer encoder, N represents the total number of layers of the backbone Transformer, and LN represents the layer normalization operation;

[0069] S802: Use the center-based head to predict the center of gravity position and scale of the target, take the position with the highest classification score as the position of the target, and calculate the final tracking result by combining the offset size and the bounding box size.

[0070] Compared with the prior art, the present invention has the following advantages:

[0071] This paper proposes an end-to-end single-target tracking method based on underwater cue-driven prompt learning. This method abandons the "enhance first, then track" architecture of existing underwater single-target trackers, draws on the idea of ​​prompt learning for scene migration, and constructs an end-to-end tracking architecture for application in complex underwater scenes. In addition, this method uses a learnable, autoregressive embedding marker to preserve texture information, which greatly reduces the number of model parameters, reduces computer resource consumption, and improves model inference speed. In summary, this method realizes a highly effective and robust end-to-end underwater single-target tracking algorithm while ensuring low computational cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0073] Figure 1 This is a flow chart of an end-to-end single target tracking method based on underwater cue-driven prompt learning in an embodiment of the present invention.

[0074] Figure 2 This is an architecture diagram of the end-to-end single target tracking method based on underwater scene clues in an example of the present invention.

[0075] Figure 3 This is a diagram of an underwater clue extraction module based on underwater scene clue prompts in an example of the present invention. DETAILED DESCRIPTION

[0076] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0077] Existing underwater single-object trackers follow an "augment first, then track" architecture. This separate enhancement and tracking approach cannot build an end-to-end trainable visual network. To address this problem, we propose a new architecture called underwater cue-based tracking.

[0078] like Figure 1As shown, the present application provides an end-to-end single target tracking method based on underwater clue-driven prompt learning, which includes an underwater scene clue generation and fusion step. The clue generation utilizes spatial and channel attention mechanisms to process underwater image data, enhancing the similarity of labeled features and weakening irrelevant labels, and then processes the features through a self-attention mechanism, taking the processed features as keys and values to initialize the learnable embedding as a query to pass underwater texture information; the fusion step fuses the clues with underwater image features through an attention mechanism to focus on the features of the foreground object. The present application solves the problem that the existing single target tracker structure cannot be trained end-to-end in an underwater scene. Specifically, the method includes the following steps:

[0079] S1, obtaining underwater video data to be tracked, search images and template images; a clue feature generation network is constructed, which includes a channel attention mechanism, a spatial attention mechanism and a multi-head cross-attention mechanism for clue extraction.

[0080] Specifically, the input of the single target tracking task generally includes two parts, one of which is the template image, and the other is the search image. The template image is the first frame image of the given tracking video, and the position information (x, y, w, h) of the given target box in the template image is known. The search image is one of all subsequent frames in a video sequence with the template image. The purpose of the single target tracking task is to find the position information of the known target in the template image in the search image.

[0081] S2, modulating the template image and the search image in a channel manner by using the channel attention mechanism of the clue feature generation network, thereby producing modulation features that highlight the texture of the foreground object in the underwater image. This step includes:

[0082] S201, cropping the size of the template and search images in the image based on the hyperparameter settings. First, calculate the size of the cropping area according to the real box, determine the top-left corner and bottom-right corner coordinates of the cropping area, and calculate the size that needs to be filled. Then, crop the image to obtain the information input into the network. The cropping process of the template image is the same as that of the search image. In the inference stage, the prediction value obtained by using the last frame image to pass through the model is used as the real box for the cropping process of the next frame.

[0083] S202, performing channel dimension information attention processing based on the cropped template and search image information, calculating channel dimension attention weights, including setting the features X0 processed through cropping and PatchEmbedding operation or the features X i assigned to the channel features X c :

[0084] X c =X i or Xc =X0(1)

[0085] where X i Represents the template and search image features output by the i-th layer Transformer backbone, such as Figure 2 The X1, X2, etc. shown here refer to the features obtained through the Transformer layer processing. The Transformer layers are the Transformer backbone.

[0086] S203, first perform average pooling on the features, the formula is:

[0087]

[0088] Where h and w represent the height and width of the feature after it is converted into feature graph form.

[0089] S204, perform maximum pooling on the features, the formula is:

[0090]

[0091] S205. The formula for concatenating the features obtained by average pooling and maximum pooling in the dimension of the number of tokens is:

[0092]

[0093] S206. The concatenated result is reduced in dimension through a multi-layer perceptron. The features are first processed through a fully connected layer. The formula is:

[0094] M1=δ(W0M cat +B0) (5)

[0095] Where δ represents the ReLU activation function, W0 represents the weight matrix of the fully connected layer in the multilayer perceptron, B0 represents the bias matrix of the fully connected layer in the multilayer perceptron, and M1 represents the output result of the first fully connected layer.

[0096] M2=δ(W1M1+B1) (6)

[0097] M=LayerNorm(M2+M cat ) (7)

[0098] Where δ represents the ReLU activation function, W1 represents the weight matrix of the fully connected layer in the multilayer perceptron, B1 represents the bias matrix of the fully connected layer in the multilayer perceptron, and M2 represents the output of the second fully connected layer, that is, the output of the multilayer perceptron. The output is then connected using a residual structure and layer normalization is performed.

[0099] S207, input the channel attention processed feature map M into the Sigmoid activation function layer to obtain the channel attention weight α∈(0,1), the formula is:

[0100]

[0101] S208: Based on the obtained channel attention weight α, the channel attention weight α is multiplied bit by bit with the original input feature map X to obtain the feature map adjusted by the channel attention. The formula is:

[0102]

[0103] S3, using the spatial dimension attention mechanism in the clue feature generation network to mark the local interaction between the modulation features to obtain the texture features of underwater objects. Specifically, Figure 2 As shown in the figure, the input template image and search image are preprocessed separately, and the pixel images are converted into 16*16 pixel tokens as the unit for subsequent processing. The tokens are locally interacted to generate underwater object texture features. This step includes:

[0104] S301. Obtain average pooling and maximum pooling features of features.

[0105] S302, concatenate the features after average pooling and maximum pooling Divide into four feature maps of equal size: up, down, left, and right Each divided feature map is input into the multi-layer perceptron for local feature interaction to obtain the spatially processed features. Then the four small feature maps are reassembled into a large feature map according to the original split order Here N represents the total number of tokens in the feature.

[0106] S303, the obtained feature map X s Flatten it into a ([B, C, N]) shape and input it into the Sigmoid activation function layer. The formula for calculating the spatial attention weight β is as follows:

[0107]

[0108] S304: Multiply the spatial attention weight by the feature processed by the channel attention bit by bit to obtain the feature processed by both the channel attention and the spatial attention, that is, the foreground texture feature.

[0109]

[0110] S4, constructing a carrier embedding for storing underwater scene target texture information, wherein the embedding is initially set to all 1s. This step includes:

[0111] S401. In the first layer of processing, a learnable embedding initialized to all 1s is used. Its dimensional information is consistent with the features processed by the backbone network except for the number of tokens. The embedded information input in subsequent layers is the new embedded feature output from the previous layer.

[0112] S402: This embedding, serving as a carrier for retaining all key information within a frame, is propagated through each cue extraction layer in an autoregressive manner, interactively propagating to retain the most accurate information. The key information refers to the texture features of foreground objects. Furthermore, this method of using a single labeled embedding to preserve the information within a frame is a first for underwater tracking tasks, significantly reducing the computational resource consumption inherent in multi-head cross-attention.

[0113] S5. Use the initialized embedding as the query tag and the underwater object texture features as the key tag and value tag of the multi-head cross attention mechanism to obtain the underwater clue prompt features. This step includes:

[0114] S501. The sources of the query tag, key tag, and value tag of multi-head cross attention are shown in the formula:

[0115]

[0116] in is an initialized learnable, autoregressive embedding, It represents the clue features extracted by the clue extraction module in the previous layer.

[0117] S502: Perform a multi-head cross attention operation on the underwater object texture features processed by the clue feature generation network and the learnable embedding. The multi-head cross attention formula is as follows:

[0118] p n =MultiHead(Q,K,V)=Concat(H1,...,H n )W 0 (13)

[0119]

[0120] Where W i Q ,W i K ,W i V ,W O are all learnable parameters, and n is the number of layers in the network.

[0121] S503: Normalize the clue results obtained based on multi-head cross attention and use the residual structure. The formula is as follows:

[0122]

[0123] in represents the clue features after normalization and residual processing, p n Represents the subsequent cue features processed by multi-head cross attention.

[0124] S6. Input the template, search features, and underwater clue prompt features into a fusion module, and the fusion module is used to perform matrix multiplication operations between features to obtain a correlation score.

[0125] The template and search features are obtained by processing the template image and search image through the PatchEmbedding operation and then sending them to the Transformer backbone for processing. Specifically, the template image and search image undergo the PatchEmbedding operation (simply put, the entire image is divided into 16*16 blocks, for example, 256*256 pixels are divided into 16*16 pixel blocks, for a total of 256 blocks). The results of the template image division and the search image division are then spliced ​​together, followed by feature mapping using convolution, and then input into the Transformer backbone for feature extraction and relationship interaction.

[0126] This step includes:

[0127] S601: Perform a dot product operation on the underwater clues extracted by the underwater clue extraction module and the original features to obtain a correlation score. The formula is as follows:

[0128] A tt n n=p n ×X n (16)

[0129] where p n Represents underwater clues, X n Represents features that have not been processed.

[0130] S7. Use the template and search features to continue to perform matrix multiplication operations with the correlation score at corresponding positions to obtain underwater foreground texture features. This step includes:

[0131] S701, based on the dot product operation, the correlation score is multiplied bit by bit with the feature that has not been processed, and the residual structure is used to obtain the underwater foreground texture feature. The formula is as follows:

[0132] S n=Attn n * X n (17)

[0133] wherein S n represents the underwater foreground texture feature, and the feature contains information of the template and the search image.

[0134] S8, the underwater clue prompt feature and the template and search feature are input into the backbone Transformer layer of the tracker to interact information, and the final result is calculated by using the center regression head to calculate the prediction box information. This step includes:

[0135] S801, based on the obtained underwater foreground texture feature, the underwater foreground texture feature and the features after the template and the search image are input into the backbone Transformer layer of the tracker to interact information, and the formula is as follows:

[0136]

[0137] wherein X0 represents the template and search features of the input template and search image after preliminary convolution processing, S0 represents the underwater foreground texture feature extracted by the first layer clue feature generation network, represent the template and search features of the reinforced foreground texture, ready to be input into the first layer backbone Transformer encoder, represent the results output by the nth layer backbone Transformer encoder, N represents the total number of layers of the backbone Transformer, and LN represents the layer normalization operation.

[0138] S802, the center-based head is used to predict the center of gravity position and scale of the target, the position with the highest classification score is taken as the position of the target, and the final tracking result is calculated by combining the offset size and the bounding box size.

[0139] The application innovatively uses the idea of prompt learning, and carries out fine tuning operation for the tracking backbone. Specifically, for the backbone part, the network parameters are obtained by pre-training through a large number of data sets collected on land. These land data sets cover a variety of scenes and target types, which can enable the backbone network to learn a general feature representation.

[0140] After completing the pre-training of the land data set, the model is migrated to the underwater data set environment for formal end-to-end training. In the whole training process, in order to ensure that the general features learned by the backbone part are not affected, the parameters of the backbone part are frozen. This means that the parameters of the backbone part will not change during the training process, thereby providing a stable feature extraction basis for the model.

[0141] At the same time, the parameters updated in real time focus on cue feature generation and fusion. In underwater environments, the extraction and fusion of cue features are crucial for accurate target tracking. By continuously adjusting these parameters during training, the model can better adapt to the specific characteristics of the underwater environment, such as the effects of light refraction and water disturbances on target appearance and motion. This allows for more effective extraction and fusion of cue features relevant to the target, ultimately achieving high-precision target tracking.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An end-to-end single target tracking method based on underwater cue-driven prompt learning, characterized by: The following steps are involved: S1. Acquire underwater video data to be tracked, a search image, and a template image; construct a clue feature generation network, wherein the clue feature generation network includes a channel attention mechanism, a spatial attention mechanism, and a multi-head cross attention mechanism for clue extraction; S2. utilizing a channel attention mechanism of a clue feature generation network to modulate the template image and the search image in a channel-wise manner, thereby generating a modulation feature that highlights the texture of the foreground object in the underwater image; S3. Using the spatial dimension attention mechanism in the clue feature generation network to mark the local interaction between the modulation features to obtain the texture features of the underwater object, including: S301, obtaining the average pooling and maximum pooling features of the modulation features, S302, the features after average pooling and maximum pooling ([B, C, , ]) is divided into four feature maps of equal size ([B, C, / 2, / 2]), each divided feature map is input into the multi-layer perceptron for local feature interaction to obtain the spatially processed features ([B,1, / 4, / 4]), and then reassemble the four small feature maps into a large feature map according to the original split order ([B, C, , ]), where N represents the total number of tags in the feature, S303, the obtained feature map X s Flattened to the shape of ([B, C, N]) and input into the Sigmoid activation function layer to calculate the spatial attention weight The formula is as follows: （10） S304: Multiply the spatial attention weight by the feature processed by the channel attention bit by bit to obtain the texture feature of the underwater object: （11）； S4, constructing a carrier embedding for storing texture information of underwater scene objects, wherein the embedding is initially set to all 1s; S5. Use the initialized embedding as the query tag and the underwater object texture feature as the key tag and value tag of the multi-head cross attention mechanism to perform a multi-head cross attention operation to obtain the underwater clue prompt feature; S6. Input the template, search features, and underwater clue prompt features into a fusion module, wherein the fusion module is used to perform a matrix multiplication operation between the features to obtain a correlation score, wherein the template and search features are obtained by: processing the template image and the search image through a PatchEmbedding operation and then feeding them into a Transformer backbone for processing; S7, using the template and the search feature to continue to perform matrix multiplication operation with the correlation score at corresponding positions to obtain underwater foreground texture features; S8: Input the underwater clue features, template, and search features into the tracker's backbone Transformer layer for information interaction. The final result is used to calculate the predicted box information using the central regression head, including: S801. The formula for information interaction is as follows: （18） in X 0 represents the template and search feature, S 0 represents the underwater foreground texture features extracted by the first layer clue feature generation network, The template and search features representing the enhanced foreground texture are ready to be input into the first layer of the backbone Transformer encoder. Representatives from n The result of the layer backbone Transformer encoder output, N represents the total number of layers of the backbone Transformer, LN represents the layer normalization operation, S802: Use the center-based head to predict the center of gravity position and scale of the target, take the position with the highest classification score as the position of the target, and calculate the final tracking result by combining the offset size and the bounding box size.

2. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that The template image and the search image are modulated in a channel-wise manner using a channel attention mechanism of a clue feature generation network, thereby generating modulation features that highlight the texture of foreground objects in the underwater image, including: S201, cutting the sizes of the template image and the search image in the image based on the hyperparameter setting; S202, based on the cropped template and search image information, perform channel dimension information attention processing and calculate the channel dimension attention weight, including the features processed by the cropping and PatchEmbedding operations. X 0 or features not processed by the clue feature generation network X i Assign to channel features X c : or (1) in X i Representatives from i The template and search image features output by the Transformer backbone layer; S203, first perform average pooling on the template and search image features, the formula is: （2） in h , w Respectively represent the height and width of the feature after it is converted into feature graph form; S204, the formula for calculating the maximum pooling is: （3） S205. The formula for concatenating the features obtained by average pooling and maximum pooling in the dimension of the number of tags is: （4） S206: The concatenated result is reduced in dimension through a multi-layer perceptron, and first processed through a fully connected layer: （5） in () represents the ReLU activation function, W 0 is the weight matrix of the fully connected layer in the multi-layer perceptron, B 0 is the bias matrix of the fully connected layer in the multilayer perceptron; （6） （7） in, M 2 represents the output result of the second fully connected layer; S207, feature map of channel attention processing M Input into the Sigmoid activation function layer to obtain the channel attention weight, the formula is: （8） S208. Based on the obtained channel attention weight, the channel attention weight is multiplied by the channel feature bit by bit to obtain the feature map adjusted by the channel attention. The formula is: （9）。 3. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that: Construct a carrier embedding for storing texture information of underwater scene targets, including: S401: In the first layer, an embedding initialized to all 1s is used. Its dimensional information, except for the number of tokens, is consistent with the features processed by the backbone network. The embedding information input by subsequent layers is the new embedding feature corresponding to the output of the previous layer. S402: This embedding is used as a carrier to retain the texture features of all underwater objects in a frame of the image and is propagated in each clue extraction layer in an autoregressive manner, and interactively propagated to retain the most accurate information.

4. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that The initialized embedding is used as the query tag, and the underwater object texture features are used as the key tags and value tags of the multi-head cross attention mechanism. The multi-head cross attention operation is performed to obtain the underwater clue prompt features, including: S501. The sources of the query tag, key tag, and value tag of multi-head cross attention are shown in the formula: （12） in is an initialized learnable, autoregressive embedding, It represents the clue features extracted by the clue extraction module in the previous layer; S502: Perform a multi-head cross attention operation on the underwater object texture features processed by the clue feature generation network and the learnable embedding. The multi-head cross attention formula is as follows: （13） （14） in are all learnable parameters, n is the number of layers in the network; S503: Normalize the clue results obtained based on multi-head cross attention and use the residual structure. The formula is as follows: （15） in Represents the clue features after normalization and residual processing, Represents the subsequent underwater cue features processed by multi-headed cross-attention.

5. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that: The template, search features, and underwater clue prompt features are input into a fusion module, which is used to perform matrix multiplication operations between features to obtain a correlation score, including: S601: Perform a dot product operation on the underwater clue prompt feature, the template, and the search feature to obtain a correlation score. The formula is as follows: （16） in p n Represents underwater clue prompt features, X n Represents templates and search features.

6. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that The template and search features are used to continue to multiply the corresponding positions of the matrix with the correlation score to obtain the underwater foreground texture features, including: S701, based on the dot product operation, the correlation score is multiplied bit by bit with the template and the search feature, and the residual structure is used to obtain the underwater foreground texture feature. The formula is as follows: （17） in S n Represents the underwater foreground texture features, X n Represents templates and search features, Represents the relevance score.

Images