End-to-end single target tracking method based on underwater clue driving prompt learning

Abstract

The invention provides an end-to-end single target tracking method based on underwater clue driving prompt learning, and belongs to the field of computer vision. The method mainly comprises the following steps: modulating a template image and a search image in a channel mode by utilizing a channel attention mechanism in a clue feature generation network; local interaction between marks is carried out on the modulation features by utilizing a spatial dimension attention mechanism in a clue feature generation network so as to acquire underwater object texture features; performing multi-head cross attention operation to obtain underwater clue prompt features; inputting the template, the search features and the underwater clue prompt features into a fusion module; performing matrix corresponding position multiplication operation on the template and the search feature and the correlation score to obtain an underwater foreground texture feature; and inputting the underwater clue prompt features, the template and the search features into a trunk Transform layer of a tracker for information interaction, and calculating prediction frame information. According to the invention, the problem that an existing single-target tracker structure cannot carry out end-to-end training in an underwater scene is solved.

Description

Technical Field

[0001] The present invention relates to the technical fields of marine environment perception, image data processing, etc., and specifically to an end-to-end single target tracking method based on underwater cue-driven prompt learning. Background Art

[0002] Cue learning is an emerging technology in the field of natural language processing. It aims to solve the adaptability and generalization problems of pre-trained language models in specific tasks. It introduces cue information to guide the model to better understand and perform tasks, thereby improving model performance. Based on this idea, the application of cue learning in the field of multimodal tracking vision has shown great development potential. In multimodal tracking, in addition to common visual and text information, more modal data can be introduced as cue information. The cue learning framework is used to design appropriate cues to guide the model to learn the association between different modalities, so as to more comprehensively understand the scene and target and improve the accuracy and robustness of tracking. At present, some studies are based on the idea of ​​modal transfer. In dark scenes, dark clues are used as cue information to assist mainstream target tracking methods to perform scene transfer and thus achieve the effect of improving accuracy. However, the general idea of ​​current underwater single target tracking methods is "enhancement first and then tracking". It is obvious that the structure designed based on this idea is difficult to train in an end-to-end form, and the structural design is complex. Therefore, how to explore an effective application of cue learning for underwater scene transfer in the field of single target tracking and how to efficiently extract underwater texture information for auxiliary cueing for the backbone tracking network are two very critical issues. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides an end-to-end single target tracking method based on underwater cue-driven prompt learning. The present invention introduces variants of the channel attention mechanism and the spatial attention mechanism, and introduces the auxiliary idea of ​​prompt learning to better solve the problem that the underwater single target tracking network cannot be trained end-to-end and it is difficult to extract effective underwater information.

[0004] The technical means adopted by the present invention are as follows:

[0005] An end-to-end single target tracking method based on underwater cue-driven prompt learning includes the following steps:

[0006] S1. Acquire underwater video data to be tracked, search images, and template images; construct a clue feature generation network, wherein the clue feature generation network includes a channel attention mechanism, a spatial attention mechanism, and a multi-head cross attention mechanism for clue extraction;

[0007] S2, using the channel attention mechanism of the clue feature generation network to modulate the template image and the search image in a channel manner, thereby generating a modulation feature that highlights the texture of the foreground object in the underwater image;

[0008] S3, using the spatial dimension attention mechanism in the clue feature generation network to mark the local interaction between the modulation features to obtain the texture features of the underwater object;

[0009] S4, constructing a carrier embedding for storing underwater scene target texture information, wherein the embedding is initially set to all 1s;

[0010] S5, using the initialized embedding as the query tag, and the underwater object texture feature as the key tag and value tag of the multi-head cross attention mechanism, and performing the multi-head cross attention operation to obtain the underwater clue prompt feature;

[0011] S6, inputting the template, search features and underwater clue prompt features into a fusion module, wherein the fusion module is used to perform a matrix multiplication operation between features to obtain a correlation score, wherein the template and search features are obtained by: processing the template image and the search image through a PatchEmbedding operation, and then sending them to a Transformer trunk for processing;

[0012] S7, using the template and the search feature to continue to perform matrix multiplication operation on the corresponding position of the correlation score to obtain the underwater foreground texture feature;

[0013] S8. Input the underwater clue prompt features, template and search features into the backbone Transformer layer of the tracker for information interaction, and use the central regression head to calculate the prediction box information of the final result.

[0014] Furthermore, the template image and the search image are modulated in a channel manner using a channel attention mechanism of a clue feature generation network, thereby generating a modulation feature that highlights the texture of a foreground object in the underwater image, including:

[0015] S201, cutting the sizes of the template image and the search image in the image based on the hyperparameter setting;

[0016] S202, performing channel dimension information attention processing based on the cropped template and the search image information, calculating the channel dimension attention weight, including adding the feature X0 processed by the cropping and PatchEmbedding operation or the feature X0 not processed by the clue feature generation network i Assign to channel feature X c :

[0017] X c =X i or X c=X0(1)

[0018] Where X i Represents the template and search image features output by the i-th layer Transformer backbone;

[0019] S203, firstly, average pooling is performed on the template and search image features, and the formula is:

[0020]

[0021] Where h and w represent the height and width of the feature after it is converted into a feature graph;

[0022] S204, the formula for calculating the maximum pooling is:

[0023]

[0024] S205. The formula for concatenating the features obtained by average pooling and maximum pooling in the dimension of the number of markers is:

[0025]

[0026] S206, the concatenated result is reduced in dimension through a multi-layer perceptron, and first processed through a fully connected layer:

[0027]

[0028] Where δ() represents the ReLU activation function, W0 is the weight matrix of the fully connected layer in the multilayer perceptron, and B0 is the bias matrix of the fully connected layer in the multilayer perceptron;

[0029] M2=δ(W1M1+B1) (6)

[0030]

[0031] Among them, M2 represents the output result of the second fully connected layer;

[0032] S207, input the feature map M processed by the channel attention into the Sigmoid activation function layer to obtain the channel attention weight, the formula is:

[0033]

[0034] S208. Based on the obtained channel attention weight, the channel attention weight is multiplied by the channel feature bit by bit to obtain a feature map adjusted by the channel attention. The formula is:

[0035]

[0036] Furthermore, the spatial dimension attention mechanism in the clue feature generation network is used to mark the local interaction between the modulation features to obtain the texture features of the underwater object, including:

[0037] S301, obtaining average pooling and maximum pooling features of modulation features;

[0038] S302: Concatenate the features after average pooling and maximum pooling Divide into four feature maps of equal size: up, down, left, and right Each divided feature map is input into the multi-layer perceptron for local feature interaction to obtain the spatially processed features. Then the four small feature maps are reassembled into a large feature map according to the original split order. Here N represents the total number of tags in the feature;

[0039] S303, the obtained feature map X s Flatten it into a shape of ([B, C, N]) and input it into the Sigmoid activation function layer. The formula for calculating the spatial attention weight β is as follows:

[0040]

[0041] S304, multiply the spatial attention weight by the feature processed by the channel attention bit by bit to obtain the texture feature of the underwater object:

[0042]

[0043] Furthermore, constructing a carrier embedding for storing texture information of underwater scene targets includes:

[0044] S401, in the first layer processing, an embedding initialized to all 1s is used, and its dimensional information is consistent with the features processed by the backbone network except for the number of tokens, and the embedding information input in the subsequent layer processing is the new embedding feature corresponding to the output of the previous layer;

[0045] S402, this embedding is used as a carrier to retain the texture features of all underwater objects in a frame of the picture, and is propagated in each clue extraction layer in the form of autoregression, and interactively propagated to retain the most accurate information.

[0046] Furthermore, the initialized embedding is used as the query tag, and the underwater object texture feature is used as the key tag and value tag of the multi-head cross attention mechanism, and the multi-head cross attention operation is performed to obtain the underwater clue prompt features, including:

[0047] S501. The sources of the query tag, key tag, and value tag of multi-head cross attention are shown in the formula:

[0048]

[0049] in is an initialized learnable, autoregressive embedding, It represents the clue features extracted by the clue extraction module in the previous layer;

[0050] S502, performing a multi-head cross attention operation on the underwater object texture features processed by the clue feature generation network and the learnable embedding, and the multi-head cross attention formula is as follows:

[0051] p n =MultiHead(Q,K,V)=Concat(H1,...,H n )W 0 (13)

[0052]

[0053] Where W i Q ,W i K ,W i V ,W O are all learnable parameters, n is the number of layers in the network;

[0054] S503, normalize the clue results obtained based on multi-head cross attention and use the residual structure. The formula is as follows:

[0055]

[0056] in represents the clue feature after normalization and residual processing, p n Represents the subsequent underwater cue features processed by multi-headed cross-attention.

[0057] Furthermore, the template, the search features, and the underwater clue prompt features are input into a fusion module, and the fusion module is used to perform a matrix multiplication operation between the features to obtain a correlation score, including:

[0058] S601, performing a dot product operation on the underwater clue prompt feature, the template and the search feature to obtain a correlation score, the formula is as follows:

[0059] A tt n n=p n ×X n (16)

[0060] where p n Represents underwater clue prompt features, X n Represents template and search features.

[0061] Furthermore, the template and the search features are used to continue to perform matrix multiplication operations with the correlation scores at corresponding positions to obtain underwater foreground texture features, including:

[0062] S701, based on the dot product operation, the correlation score is multiplied bit by bit with the template and the search feature, and the underwater foreground texture feature is obtained using the residual structure. The formula is as follows:

[0063] S n =Attn n *X n (17)

[0064] Where S n represents the underwater foreground texture feature, X n Represents template and search features, Attn n Represents the relevance score.

[0065] Furthermore, the underwater clue features, templates, and search features are input into the tracker’s backbone Transformer layer for information interaction, and the final result is used to calculate the prediction box information using the central regression head, including:

[0066] S801. The formula for information interaction is as follows:

[0067]

[0068] Among them, X0 represents the template and search features, S0 represents the underwater foreground texture features extracted by the first layer clue feature generation network, The template and search features representing the enhanced foreground texture are ready to be input into the first layer of the backbone Transformer encoder. Represents the result output by the nth layer backbone Transformer encoder, N represents the total number of layers of the backbone Transformer, and LN represents the layer normalization operation;

[0069] S802: Use the center-based head to predict the center of gravity position and scale of the target, take the position with the highest classification score as the position of the target, and calculate the final tracking result by combining the offset size and the bounding box size.

[0070] Compared with the prior art, the present invention has the following advantages:

[0071] The present invention proposes an end-to-end single target tracking method based on underwater cue-driven prompt learning. This method abandons the "enhance first and then track" architecture of existing underwater single target trackers, draws on the idea of ​​prompt learning for scene migration, and constructs an end-to-end tracking architecture for application in complex underwater scenes. In addition, this method uses a learnable, autoregressive embedding marker to save texture information, which greatly reduces the number of model parameters, reduces computer resource consumption, and improves the model reasoning speed. In short, this method realizes an end-to-end underwater single target tracking algorithm with high effectiveness and robustness while ensuring the reduction of computational consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0073] Figure 1 The present invention is a flow chart of an end-to-end single target tracking method based on underwater cue-driven prompt learning in an embodiment of the present invention.

[0074] Figure 2 This is an architecture diagram of an end-to-end single target tracking method based on underwater scene clues in an example of the present invention.

[0075] Figure 3 This is a diagram of an underwater clue extraction module based on underwater scene clue prompts in an example of the present invention. DETAILED DESCRIPTION

[0076] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0077] Existing underwater single-target trackers follow an "enhancement-then-tracking" architecture. This separate enhancement and tracking approach cannot build an end-to-end trainable visual network. To address this problem, we propose a new architecture called underwater cue-based tracking.

[0078] like Figure 1As shown, the present invention provides an end-to-end single target tracking method based on underwater cue-driven prompt learning, which includes underwater scene cue generation and fusion steps. Clue generation uses spatial and channel attention mechanisms to process underwater image data, enhances the similarity of marker features and weakens irrelevant markers, and then processes them through an autoregressive attention mechanism, using the processed features as keys and values ​​to initialize learnable embeddings as queries to pass underwater texture information; the fusion step fuses the clues with underwater image features through an attention mechanism to focus on foreground object features. The present invention solves the problem that existing single target tracker structures cannot be trained end-to-end in underwater scenes. Specifically, the method includes the following steps:

[0079] S1. Acquire underwater video data to be tracked, search images, and template images; construct a clue feature generation network, wherein the clue feature generation network includes a channel attention mechanism, a spatial attention mechanism, and a multi-head cross attention mechanism for clue extraction.

[0080] Specifically, the input of a single target tracking task generally consists of two parts, one of which is a template image and the other is a search image. The template image is the first frame of a given tracking video, and the location information (x, y, w, h) of a given target box in the template image is known. The search image is one of all subsequent frames in a video sequence with the template image. The purpose of a single target tracking task is to find the location information of a known target in the template image in the search image.

[0081] S2, using the channel attention mechanism of the clue feature generation network to modulate the template image and the search image in a channel manner, thereby generating a modulation feature that highlights the texture of the foreground object in the underwater image. This step includes:

[0082] S201, based on the hyperparameter settings, the template and search image size in the image are cropped. First, the cropping area size is calculated based on the true frame, and the coordinates of the upper left corner and lower right corner of the cropping area are determined, and the size to be filled is calculated. Then the image is cropped to obtain the information input to the network. The cropping process of the template image is the same as the search image processing. In the inference stage, the predicted value obtained by the model prediction of the previous frame image is used as the true frame of the next frame cropping processing.

[0083] S202, performing channel dimension information attention processing based on the cropped template and the search image information, calculating the channel dimension attention weight, including adding the feature X0 processed by the cropping and PatchEmbedding operation or the feature X0 not processed by the clue feature generation network i Assign to channel feature X c :

[0084] X c =X i or Xc =X0(1)

[0085] Where X i Represents the template and search image features output by the i-th layer Transformer backbone, such as Figure 2 The X1, X2, etc. shown here refer to the features obtained after processing by the Transformer layer, and the Transformer layers are the Transformer backbone.

[0086] S203, first perform average pooling on the features, the formula is:

[0087]

[0088] Among them, h and w represent the height and width of the feature after it is converted into feature graph form.

[0089] S204, perform maximum pooling on the features, the formula is:

[0090]

[0091] S205. The formula for concatenating the features obtained by average pooling and maximum pooling in the dimension of the number of tokens is:

[0092]

[0093] S206, the concatenated result is reduced in dimension through a multi-layer perceptron, and firstly processed through a fully connected layer, the formula is:

[0094] M1=δ(W0M cat +B0) (5)

[0095] Where δ represents the ReLU activation function, W0 represents the weight matrix of the fully connected layer in the multilayer perceptron, B0 represents the bias matrix of the fully connected layer in the multilayer perceptron, and M1 represents the output result of the first fully connected layer.

[0096] M2=δ(W1M1+B1) (6)

[0097] M=LayerNorm(M2+M cat ) (7)

[0098] Among them, δ represents the ReLU activation function, W1 represents the weight matrix of the fully connected layer in the multilayer perceptron, B1 represents the bias matrix of the fully connected layer in the multilayer perceptron, and M2 represents the output result of the second fully connected layer, that is, the output result of the multilayer perceptron. Then, the output result is connected using a residual structure and layer normalization is performed.

[0099] S207, input the feature map M of the channel attention processing into the Sigmoid activation function layer to obtain the channel attention weight α∈(0,1), the formula is:

[0100]

[0101] S208: Based on the obtained channel attention weight α, the channel attention weight α is multiplied bit by bit with the original input feature map X to obtain the feature map adjusted by the channel attention. The formula is:

[0102]

[0103] S3, using the spatial dimension attention mechanism in the clue feature generation network to mark the local interaction between the modulation features to obtain the texture features of the underwater object. Specifically, Figure 2 As shown, the input template image and the search image are preprocessed respectively, and the image composed of pixels is converted into a token composed of 16*16 pixels as a unit for subsequent processing. The tokens are locally interacted to generate underwater object texture features. This step includes:

[0104] S301, obtaining average pooling and maximum pooling features of features.

[0105] S302: Concatenate the features after average pooling and maximum pooling Divide into four feature maps of equal size: up, down, left, and right Each divided feature map is input into the multi-layer perceptron for local feature interaction to obtain the spatially processed features. Then the four small feature maps are reassembled into a large feature map according to the original split order. Here N represents the total number of tokens in the feature.

[0106] S303, the obtained feature map X s Flatten it into a shape of ([B, C, N]) and input it into the Sigmoid activation function layer. The formula for calculating the spatial attention weight β is as follows:

[0107]

[0108] S304, multiplying the spatial attention weight by the feature processed by the channel attention bit by bit to obtain the feature processed by both the channel attention and the spatial attention, that is, the foreground texture feature.

[0109]

[0110] S4, constructing a carrier embedding for storing underwater scene target texture information, wherein the embedding is initially set to all 1s. This step includes:

[0111] S401, in the first layer of processing, a learnable embedding initialized to all 1s is used, whose dimensional information is consistent with the features processed by the backbone network except the number of tokens, and the embedded information input in the subsequent layers is the new embedded feature corresponding to the output of the previous layer.

[0112] S402, this embedding is used as a carrier to retain all the key information in a frame of the picture, and propagates in each clue extraction layer in the form of autoregression, and interactively propagates to retain the most accurate information. The key information refers to the texture features of the foreground target. At the same time, this method of using a marker embedding to save the information of a frame of the image is used for the first time to process tracking tasks in underwater scenes. This processing method greatly reduces the consumption of computing resources caused by the original multi-head cross attention.

[0113] S5, using the initialized embedding as the query tag, and the underwater object texture feature as the key tag and value tag of the multi-head cross attention mechanism, and performing a multi-head cross attention operation to obtain underwater clue prompt features. This step includes:

[0114] S501. The sources of the query tag, key tag, and value tag of multi-head cross attention are shown in the formula:

[0115]

[0116] in is an initialized learnable, autoregressive embedding, It represents the clue features extracted by the clue extraction module in the previous layer.

[0117] S502, performing a multi-head cross attention operation on the underwater object texture features processed by the clue feature generation network and the learnable embedding, and the multi-head cross attention formula is as follows:

[0118] p n =MultiHead(Q,K,V)=Concat(H1,...,H n )W 0 (13)

[0119]

[0120] Where W i Q ,W i K ,W i V ,W O are all learnable parameters, and n is the number of layers in the network.

[0121] S503, normalize the clue results obtained based on multi-head cross attention and use the residual structure. The formula is as follows:

[0122]

[0123] in represents the clue feature after normalization and residual processing, p n Represents the subsequent cue features processed by multi-head cross attention.

[0124] S6. Input the template, search features and underwater clue prompt features into a fusion module, and the fusion module is used to perform matrix multiplication operations between features to obtain a correlation score.

[0125] The template and search features are obtained by processing the template image and the search image through the PatchEmbedding operation and then sending them to the Transformer backbone for processing. Specifically, the template image and the search image are processed through the PatchEmbedding operation (in simple terms, a whole image is divided into 16*16 blocks, for example, 256*256 pixels are divided into 16*16 pixel blocks, a total of 256 blocks.), and then the template image division results and the search image division results are spliced, and then convolution is used for feature mapping, and then input into the Transformer backbone for feature extraction and relationship interaction.

[0126] This step includes:

[0127] S601, based on the underwater clues extracted by the underwater clue extraction module and the original features, a dot product operation is performed on the two to obtain a correlation score, and the formula is as follows:

[0128] A tt n n=p n ×X n (16)

[0129] where p n Represents underwater clues, X n Represents features that have not been processed.

[0130] S7, using the template and the search feature to continue to perform matrix multiplication operations with the correlation score at corresponding positions to obtain underwater foreground texture features. This step includes:

[0131] S701, based on the dot product operation, the correlation score is multiplied bit by bit with the feature that has not been processed, and the underwater foreground texture feature is obtained using the residual structure. The formula is as follows:

[0132] S n=Attn n * X n (17)

[0133] Where S n Represents the underwater foreground texture feature, which contains the information of the template and the search image.

[0134] S8, input the underwater clue prompt features, template and search features into the tracker's backbone Transformer layer for information interaction, and use the central regression head to calculate the prediction box information. This step includes:

[0135] S801, based on the obtained underwater foreground texture features, the underwater foreground texture features and the features after the template and the search image are input into the backbone Transformer layer of the tracker for information interaction. The formula is as follows:

[0136]

[0137] Where X0 represents the template and search features of the input template and search image after preliminary convolution processing, S0 represents the underwater foreground texture features extracted by the first layer of clue feature generation network, The template and search features representing the enhanced foreground texture are ready to be input into the first layer of the backbone Transformer encoder. Represents the result output by the nth layer backbone Transformer encoder, N represents the total number of backbone Transformer layers, and LN represents the layer normalization operation.

[0138] S802: Use the center-based head to predict the center of gravity position and scale of the target, take the position with the highest classification score as the position of the target, and calculate the final tracking result by combining the offset size and the bounding box size.

[0139] The present invention innovatively uses the idea of ​​prompt learning to perform fine-tuning operations on the tracking backbone. Specifically, for the backbone part, its network parameters are obtained by pre-training with a large number of data sets collected on land. These land data sets cover a wide variety of scenes and target types, enabling the backbone network to learn general feature representations.

[0140] After completing the pre-training of the land dataset, the model was migrated to the underwater dataset environment and formal end-to-end training was carried out. During the entire training process, in order to ensure that the common features learned by the backbone part are not affected, we freeze the parameters of the backbone part. This means that during the training process, the parameters of the backbone part will not change, thus providing a stable feature extraction basis for the model.

[0141] At the same time, the parameters updated in real time are concentrated in the clue feature generation and fusion part. In underwater environments, the extraction and fusion of clue features are crucial for accurate target tracking. By continuously adjusting these parameters during the training process, the model can better adapt to the particularities of the underwater environment, such as the effects of light refraction, water disturbance and other factors on the target's appearance and movement, and then more effectively extract and fuse clue features related to the tracking target, ultimately achieving high-precision target tracking.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An end-to-end single target tracking method based on underwater cue-driven prompt learning, characterized in that: The following steps are involved: S1. Acquire underwater video data to be tracked, search images, and template images; construct a clue feature generation network, wherein the clue feature generation network includes a channel attention mechanism, a spatial attention mechanism, and a multi-head cross attention mechanism for clue extraction; S2, using the channel attention mechanism of the clue feature generation network to modulate the template image and the search image in a channel manner, thereby generating a modulation feature that highlights the texture of the foreground object in the underwater image; S3, using the spatial dimension attention mechanism in the clue feature generation network to mark the local interaction between the modulation features to obtain the texture features of the underwater object; S4, constructing a carrier embedding for storing underwater scene target texture information, wherein the embedding is initially set to all 1s; S5, using the initialized embedding as the query tag, and the underwater object texture feature as the key tag and value tag of the multi-head cross attention mechanism, and performing the multi-head cross attention operation to obtain the underwater clue prompt feature; S6, inputting the template, search features and underwater clue prompt features into a fusion module, wherein the fusion module is used to perform a matrix multiplication operation between features to obtain a correlation score, wherein the template and search features are obtained by: processing the template image and the search image through a PatchEmbedding operation, and then sending them to a Transformer trunk for processing; S7, using the template and the search feature to continue to perform matrix multiplication operation on the corresponding position of the correlation score to obtain the underwater foreground texture feature; S8. Input the underwater clue prompt features, templates and search features into the backbone Transformer layer of the tracker for information interaction, and use the central regression head to calculate the prediction box information of the final result.

2. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that: The template image and the search image are modulated in a channel manner using a channel attention mechanism of a clue feature generation network, thereby generating a modulation feature that highlights the texture of a foreground object in the underwater image, including: S201, cutting the sizes of the template image and the search image in the image based on the hyperparameter setting; S202, performing channel dimension information attention processing based on the cropped template and the search image information, calculating the channel dimension attention weight, including adding the feature X0 processed by the cropping and PatchEmbedding operation or the feature X0 not processed by the clue feature generation network i Assign to channel feature X c : X c =X i or X c =X0(1) Where X i Represents the template and search image features output by the i-th layer Transformer backbone; S203, firstly, average pooling is performed on the template and search image features, and the formula is: Where h and w represent the height and width of the feature after it is converted into a feature graph; S204, the formula for calculating the maximum pooling is: S205. The formula for concatenating the features obtained by average pooling and maximum pooling in the dimension of the number of markers is: S206, the concatenated result is reduced in dimension through a multi-layer perceptron, and first processed through a fully connected layer: Where δ() represents the ReLU activation function, W0 is the weight matrix of the fully connected layer in the multilayer perceptron, and B0 is the bias matrix of the fully connected layer in the multilayer perceptron; M2=δ(W1M1+B1) (6) Among them, M2 represents the output result of the second fully connected layer; S207, input the feature map M processed by the channel attention into the Sigmoid activation function layer to obtain the channel attention weight, the formula is: S208. Based on the obtained channel attention weight, the channel attention weight is multiplied by the channel feature bit by bit to obtain a feature map adjusted by the channel attention. The formula is:

3. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that: The spatial dimension attention mechanism in the clue feature generation network is used to mark the local interaction between the modulation features to obtain the texture features of the underwater object, including: S301, obtaining average pooling and maximum pooling features of modulation features; S302: Concatenate the features after average pooling and maximum pooling Divide into four feature maps of equal size: up, down, left, and right Each divided feature map is input into the multi-layer perceptron for local feature interaction to obtain the spatially processed features. Then the four small feature maps are reassembled into a large feature map according to the original split order. Here N represents the total number of tags in the feature; S303, the obtained feature map X s Flatten it into a shape of ([B, C, N]) and input it into the Sigmoid activation function layer. The formula for calculating the spatial attention weight β is as follows: S304, multiply the spatial attention weight by the feature processed by the channel attention bit by bit to obtain the texture feature of the underwater object:

4. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that: Construct a carrier embedding for storing texture information of underwater scene objects, including: S401, in the first layer processing, an embedding initialized to all 1s is used, and its dimensional information is consistent with the features processed by the backbone network except for the number of tokens, and the embedding information input in the subsequent layer processing is the new embedding feature corresponding to the output of the previous layer; S402, this embedding is used as a carrier to retain the texture features of all underwater objects in a frame of the picture, and is propagated in each clue extraction layer in the form of autoregression, and interactively propagated to retain the most accurate information.

5. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that: The initialized embedding is used as the query tag, and the underwater object texture features are used as the key tag and value tag of the multi-head cross attention mechanism. The multi-head cross attention operation is performed to obtain the underwater clue prompt features, including: S501. The sources of the query tag, key tag, and value tag of multi-head cross attention are shown in the formula: in is an initialized learnable, autoregressive embedding, It represents the clue features extracted by the clue extraction module in the previous layer; S502, performing a multi-head cross attention operation on the underwater object texture features processed by the clue feature generation network and the learnable embedding, and the multi-head cross attention formula is as follows: p n =MultiHead(Q,K,V)=Concat(H1,...,H n )W 0 (13) H i =Attention(QW i Q ,KW i K ,VW i V ) (14) Where W i Q ,W i K ,W i V ,W O are all learnable parameters, n is the number of layers in the network; S503, normalize the clue results obtained based on multi-head cross attention and use the residual structure. The formula is as follows: in represents the clue feature after normalization and residual processing, p n Represents the subsequent underwater cue features processed by multi-headed cross-attention.

6. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1, characterized in that: The template and search features and underwater clue prompt features are input into a fusion module, and the fusion module is used to perform matrix multiplication operations between features to obtain a correlation score, including: S601, performing a dot product operation on the underwater clue prompt feature, the template and the search feature to obtain a correlation score, the formula is as follows: Attn n =p n ×X n (16) where p n Represents underwater clue prompt features, X n Represents template and search features.

7. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that: The template and search features are used to continue to multiply the corresponding positions of the matrix with the correlation score to obtain the underwater foreground texture features, including: S701, based on the dot product operation, the correlation score is multiplied bit by bit with the template and the search feature, and the underwater foreground texture feature is obtained using the residual structure. The formula is as follows: S n = Attn n * X n (17) Where S n represents the underwater foreground texture feature, X n Represents template and search features, Attn n Represents the relevance score.

8. The end-to-end single target tracking method based on underwater cue-driven prompt learning according to claim 1 is characterized in that: The underwater clue features, templates, and search features are input into the tracker’s backbone Transformer layer for information interaction, and the final result is used to calculate the prediction box information using the central regression head, including: S801. The formula for information interaction is as follows: Among them, X0 represents the template and search features, S0 represents the underwater foreground texture features extracted by the first layer clue feature generation network, The template and search features representing the enhanced foreground texture are ready to be input into the first layer of the backbone Transformer encoder. Represents the result output by the nth layer backbone Transformer encoder, N represents the total number of backbone Transformer layers, and LN represents the layer normalization operation; S802: Use the center-based head to predict the center of gravity position and scale of the target, take the position with the highest classification score as the position of the target, and calculate the final tracking result by combining the offset size and the bounding box size.

Images