Underwater fish identification method based on TransNeXt

Abstract

The invention relates to the technical field of image recognition, in particular to an underwater fish recognition method based on TransNeXt, which comprises the following steps: 1) preprocessing an underwater fish school image data set in an MMSegment framework, and dividing the underwater fish school image data set into a training set and a test set; (2) an improved TransNeXt model which is trained in advance on the basis of an Ade20K data set and parameters of the improved TransNeXt model are migrated to an MMSegment framework; 3) performing iterative training on the improved TransNeXt model by using the training set and the test set; and 4) inputting an underwater fish image to be detected into the trained improved TransNeXt model, and outputting an identification result. The method can improve the precision of fish recognition in a complex underwater environment, achieves the accurate recognition of underwater fishes, and is suitable for further popularization and application.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an underwater fish recognition method based on TransNeXt. Background Art

[0002] Currently, most fish farming adopts cage farming. Most of the statistical methods of fish distribution are based on manual measurement after artificial fishing. A small number of them use sonar technology to utilize the propagation characteristics of sound waves to evaluate the distribution of fish in the water. The former is inefficient and can easily affect the physiological condition of fish. Although the latter can efficiently obtain data in larger waters, it is not effective in complex environments such as deep-sea cage farming.

[0003] Therefore, developing a non-invasive, real-time and accurate fish identification method is crucial for further automation and intelligence in the aquaculture industry. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides an underwater fish identification method based on TransNeXt, which can realize accurate identification of fish in complex underwater environments.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] An underwater fish recognition method based on TransNeXt, comprising:

[0007] 1): Preprocess the underwater fish image dataset in the MMSegmentation framework and divide it into training set and test set;

[0008] 2): Migrate the improved TransNeXt model and its parameters pre-trained on the Ade20K dataset to the MMSegmentation framework;

[0009] 3): Use the training set and test set to iteratively train the improved TransNeXt model;

[0010] 4): Input the underwater fish image to be detected into the trained improved TransNeXt model and output the recognition result.

[0011] As a possible implementation, further, the preprocessing step of the input image in step 1 includes:

[0012] The underwater fish images of different sizes are uniformly adjusted to (512, 512, 3) through MMSegmentaion, the number of channels is 3, and the dataset is divided into training set and test set in a ratio of 9:1.

[0013] As a possible implementation, further, the encoder structure in the improved TransNeXt model described in step 2) has a total of 4 stages; among which, stages 1-3 are centered on the AA module, and stage 4 is centered on the multi-head attention mechanism MHA module;

[0014] The main workflow of each stage in the model is as follows: First, the input features are downsampled through the Overlapping Patch Embedding module, and then enter the first LN layer and the core layer for calculation. The calculated features are fused with the input features after downsampling in the first layer to obtain transition features; the transition features are then calculated in the second LN normalization layer and the ConvGLU layer, and the obtained features are fused with the transition features and output. The output features will be used as the input features of the next stage to continue similar operations;

[0015] After the four-stage encoder processing, a decoder with a similar structure to the encoder is connected. The difference from the encoder is that the decoder performs upsampling through the Patch Expanding module. After the PPM processing, the decoder superimposes the output features of each stage with the downsampled features of the corresponding stage after processing by the CAGF module as the input of the next stage;

[0016] Finally, the Final Patch Expanding module is used to restore the feature resolution back to the resolution of the original image, and the detection head is used to process the object or object part in the image, and the segmentation result is output.

[0017] As a possible implementation, further, the AA module includes a pixel focused attention mechanism PFA, a query vector embedding QE and a position attention mechanism PA;

[0018] The pixel-focused attention mechanism PFA is a dual-channel design, which includes a sliding window attention mechanism and a pooling attention mechanism, including:

[0019] Given an input feature with dimension C and size H×W Define d as the output dimension of the attention head, define a set of pixels in the sliding window centered on pixel (i, j) as ρ(i, j), and for a sliding window of size k×k, ‖ρ(i, j)‖=k 2 ;

[0020] The pixel point after pooling of the feature map is defined as σ(X), and the size of the feature map after pooling is defined as H p ×W p , then ‖σ(X)‖=H p W p ; Pixel focus attention PFA can be expressed by the following formula:

[0021]

[0022]

[0023] A (i,j)～ρ(i,j) ,A (i,j)～σ(X) =Split(A (i,j) ,[k 2 ,‖σ(X)‖])

[0024] PFA(X (i,j) )=A (i,j)～ρ(i,j) V ρ(i,j) +A (i,j)～σ(X) V σ(i,j)

[0025] In the formula, Q represents the query vector; K represents the key vector; B represents the relative position bias; S represents the attention score; A represents the attention weight; V represents the value vector; Concat function represents the tensor concatenation operation; Softmax function represents the Softmax activation function; Split function represents the tensor splitting operation.

[0026] As a possible implementation, further, the downsampling method of the model - activation and pooling, can be expressed by the following formula:

[0027] σ(X)=LayerNorm(AvgPool(GELU(Linear(X))))

[0028] In the above formula, the Linear function represents the linear transformation operation through the fully connected layer; the GELU function represents the Gaussian error linear unit activation function; the AvgPool function represents the average pooling operation;

[0029] The LayerNorm function represents the layer normalization operation PFA. It adds learnable QE to all query vector tokens in the traditional QKV attention mechanism to obtain the information aggregation effect, which can be expressed by the following formula:

[0030]

[0031] As a possible implementation, further, the model adopts a method of obtaining attention weights by interacting a learnable key vector with a query vector from an input—a QLV attention mechanism, specifically:

[0032] Introduce a set of learnable tokens in each attention head Let the token interact with the query vector to obtain additional dynamic position bias and add it to the attention weight A (i,j)～ρ(i,j) , which can be expressed by the following formula:

[0033] PFA(X (i,j) )=(A (i,j)～ρ(i,j) +Q (i,j) T)V ρ(i,j) +A (i,j)～σ(X) V σ(i,j) .

[0034] As a possible implementation, the model further adopts a length-scaled cosine attention mechanism, and the attention head dimensions of the query vector and the key vector are l2-normalized and defined as and Both have size 1;

[0035] Let λ≈τlogn, where τ is a learnable variable initialized to

[0036] The length ratio cosine attention mechanism formula is as follows:

[0037] N(i,j)=‖ρ(i,j)‖+‖σ(X)‖-‖μ(i,j)‖

[0038]

[0039] In the above formula, N represents the number of key vectors that can effectively interact with the query vector after removing the masked token; μ(i,j) represents the masked token;

[0040] The model uses logarithmically spaced continuous position bias Log-CPB on the pooling path, according to Q (i,j) and K σ(X) The relative spatial coordinate Δ (i,j)～σ(X) To calculate the relative position deviation B (i,j)～σ(X) , on the sliding window path, the TransNeXt model directly uses the learnable relative position deviation B (i,j)～ρ(X) .

[0041] The aggregate attention mechanism AA can be expressed by the following formula:

[0042]

[0043] B (i,j) =Concat(B (i,j)～ρ(i,j) ,Log-CPB(S (i,j)～σ(X) ))

[0044] A (i,j) =Softmax(τlog N*Concat(S (i,j)～ρ(i,j) ,S (i,j)～σ(X) )+B(i,j) )

[0045] A (i,j)～ρ(i,j) ,A (i,j)～σ(X) =Split(A (i,j) )with size[k 2 ,H p W p ]

[0046]

[0047] As a possible implementation, further, the calculation formula of the multi-head attention mechanism MHA is as follows:

[0048] MHA(Q,K,V)=Concat(Head1,Head2…Head n )W 0

[0049]

[0050] In the above formula, W 0 is the output matrix; Head is the self-attention mechanism formula in each attention head.

[0051] As a possible implementation, further, the workflow of the Patch Expanding module is as follows:

[0052] Before upsampling, a linear layer is applied to the input features to double the feature dimension. Then a shuffle operation is used to double the resolution of the input features and reduce the feature dimension to one quarter.

[0053] In order to restore the feature resolution back to the resolution of the original image, the Final Patch Expanding module first applies a linear layer to the input features to increase the feature dimension to 16 times the original dimension, and then directly expands the resolution of the input features by four times through a rearrangement operation.

[0054] As a possible implementation, further, the workflow of the CAGF module is as follows:

[0055] The local features of the i-th stage are defined as The global feature is In order to ensure the correspondence between local features and global features, sinusoidal position encoding PE is introduced to generate a unique code for each position in the sequence, which can be expressed by the following formula:

[0056]

[0057] In the above formula, pos represents the position in the sequence;

[0058] After position encoding, local features and weight matrix W q Multiply the linear map to the query vector

[0059] The global features are respectively related to the weight matrix W k and W v Multiply the linear map to the key vector Sum value vector Among them C h is the length of the hidden vector in the mapping sequence;

[0060] Q f With K g The attention map obtained by cross product can represent the correlation between all spatial positions of local features and global features, and then the attention map is combined with V g The attention map gate obtained by cross product represents the semantic feature significance of each position in the local feature, which can be used to filter irrelevant noise in the local feature. The feature map obtained by dot multiplication of the local feature and the gate feature map is superimposed on the global feature and then a 1×1 convolution is performed to output the final feature.

[0061] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0062] 1) Multi-scale input expansion performance: Compared with dot product attention, the length-scaled cosine attention mechanism adopted in the present invention can produce more moderate attention weights and effectively enhance the aggregated attention mechanism, which can effectively curb the weakening of output confidence, making the model have better large-scale image expansion performance than the pure convolutional network model.

[0063] 2) Global information aggregation capability: The present invention adopts the aggregated attention mechanism to effectively simulate the continuous movement of the eyeball, which is highly consistent with the focal perception mode of biological vision, and can achieve more natural visual perception. This enables the model to have visual priors comparable to convolutional network models. At the same time, it can solve the shortcoming of the traditional ViT model that relies too much on stacking layers to improve information exchange, and effectively avoids the performance degradation caused by excessive model depth. At the same time, the U-Net-like design further highlights the advantages of the aggregated attention mechanism.

[0064] 3) Local feature capture capability: The convolutional gated linear unit greatly improves the local feature capture capability that most traditional ViT models do not have, realizes the attention of the channel mixer with less computation, and effectively enhances the robustness of the model.

[0065] 4) Filtering the noise of local features: After passing through the CAGF module, the coverage of the prominent semantics becomes more precise and irrelevant details in the background are also suppressed. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0067] Figure 1 This is a brief flowchart of the underwater fish recognition method based on TransNeXt.

[0068] Figure 2 This is a schematic diagram of the network structure and key modules of the underwater fish recognition method based on TransNeXt.

[0069] Figure 3 It is a schematic diagram of the structure of the Cross-Attention Gated Fusion (CAGF) module. DETAILED DESCRIPTION

[0070] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0071] Refer to the attached Figure 1 As shown, this embodiment provides an underwater fish identification method based on TransNeXt, including:

[0072] 1): In the MMSegmentation framework, the underwater fish image dataset is preprocessed and divided into a training set and a test set. The preprocessing steps of the input image include:

[0073] The underwater fish images of different sizes were uniformly adjusted to (512, 512, 3) through the data processing method provided by MMSegmentaion, and the number of channels was 3. The dataset was divided into training set and test set in a ratio of 9:1.

[0074] 2): Migrate the improved TransNeXt model and its parameters pre-trained on the Ade20K dataset to the MMSegmentation framework;

[0075] 3): Use the training set and test set to iteratively train the improved TransNeXt model;

[0076] 4): Input the underwater fish image to be detected into the trained improved TransNeXt model and output the recognition result.

[0077] In step 2), a TransNeXt improved model structure is constructed, as shown in the attached Figure 2 As shown in the figure, the encoder structure in the model has a total of 4 stages, among which stages 1-3 are centered on the AA module, and stage 4 is centered on the multi-head attention mechanism MHA (Multi-HeadAttention) module.

[0078] The main workflow of each stage in the model is as follows: First, the input features are downsampled through the Overlapping Patch Embedding module, and then enter the first LN layer and the core layer for calculation in turn. The calculated features are fused with the input features after downsampling in the first layer to obtain transition features; the transition features are then calculated in the second LN normalization layer and the ConvGLU layer in turn, and the obtained features are fused with the transition features and output. The output features will be used as the input features of the next stage to continue similar operations.

[0079] After the four-stage encoder processing, a decoder with a similar structure to the encoder is connected. The difference from the encoder is that the decoder performs upsampling through the Patch Expanding module. After the PPM processing, the decoder superimposes the output features of each stage with the downsampled features of the corresponding stage processed by the CAGF module as the input of the next stage.

[0080] Finally, the Final Patch Expanding module is used to restore the feature resolution back to the resolution of the original image, and the detection head is used to process the object or object part in the image, and the segmentation result is output.

[0081] The above AA includes three important components: pixel-focused attention mechanism PFA (Pixel-focused Attention), query vector embedding QE (Query Embedding) and positional attention mechanism PA (Positional Attention);

[0082] The pixel-focused attention mechanism PFA is a dual-channel design that includes a sliding window attention mechanism and a pooling attention mechanism, specifically including:

[0083] Given an input feature with dimension C and size H×W Define d as the output dimension of the attention head, define a set of pixels in the sliding window centered on pixel (i, j) as ρ(i, j), and for a sliding window of size k×k, ‖ρ(i, j)‖=k2 The pixel point after pooling of the feature map is defined as σ(X), and the size of the feature map after pooling is defined as H p ×W p , then ‖σ(x)‖=H p W p Pixel Focus Attention PFA can be expressed by the following formula:

[0084]

[0085] A (i,j)～ρ(i,j) ,A (i,j)～σ(X) =Split(A (i,j) ,[k 2 ,‖σ(X)‖])

[0086] PFA(X (i,j) )=A (i,j)～ρ(i,j) V ρ(i,j) +A (i,j)～σ(X) V σ(i,j)

[0087] In the formula, Q represents the query vector; K represents the key vector; B represents the relative position bias; S represents the attention score; A represents the attention weight; V represents the value vector; Concat function represents the tensor concatenation operation; Softmax function represents the Softmax activation function; Split function represents the tensor splitting operation.

[0088] In order to use the linear complexity model of PFA for large-scale image reasoning, the downsampling method adopted by the model - activation and pooling can be expressed by the following formula:

[0089] σ(X)=LayerNorm(AvgPool(GELU(Linear(X))))

[0090] In the above formula, the Linear function represents the linear transformation operation through the fully connected layer; the GELU function represents the Gaussian error linear unit activation function; the AvgPool function represents the average pooling operation;

[0091] The LayerNorm function represents the layer normalization operation PFA. By adding a learnable QE to all query vector tokens in the traditional QKV attention mechanism, the information aggregation effect can be achieved with minimal overhead, which can be expressed by the following formula:

[0092]

[0093] In order to make the query vector learn more implicit relevant information, the model uses the Query-Learnable-Value (QLV) attention mechanism, which is a method of obtaining attention weights by interacting with the query vector from the input. Specifically:

[0094] Introduce a set of learnable tokens in each attention head Let these tokens interact with the query vector to obtain additional dynamic position bias and add to the attention weight A (i,j)～ρ(i,j) , which can be expressed by the following formula:

[0095] PFA(X (i,j) )=(A (i,j)～ρ(i,j) +Q (i,j) T)V ρ(i,j) +A (i,j)～σ(X) V σ(i,j)

[0096] The model uses the length-scaled cosine attention mechanism, and the attention head dimensions of the query vector and the key vector are l2-normalized and defined as and Both have size 1.

[0097] Let λ≈τlog n, where τ is a learnable variable initialized to The length ratio cosine attention mechanism formula is as follows:

[0098] N(i,j)=‖ρ(i,j)‖+‖σ(X)‖-‖μ(i,j)‖

[0099]

[0100] In the above formula, N represents the number of key vectors that can effectively interact with the query vector after removing the masked token; μ(i,j) represents the masked token.

[0101] To further improve the extrapolation ability of PFA, the TransNeXt model uses a two-layer MLP structure with ReLU in the pooling path, called logarithmically spaced continuous position bias (Log-CPB). (i,j) and K σ(X) The relative coordinates of space Δ (i,j)～σ(X) To calculate the relative position deviation B (i,j)～σ(X) , on the sliding window path, the TransNeXt model directly uses the learnable relative position deviation B (i,j)～ρ(X) .

[0102] Combining the above methods and technologies, an aggregate attention mechanism (AA) is proposed, which can be expressed by the following formula:

[0103]

[0104]

[0105] B (i,j) =Concat(B (i,j)～ρ(i,j) ,Log-CPB(S (i,j)～σ(X) ))

[0106] A (i,j) =Softmax(τlog N*Concat(S (i,j)～ρ(i,j) ,S (i,j)～σ(X) )+B (i,j) )

[0107] A (i,j)～ρ(i,j) ,A (i,j)～σ(X) =Split(A (i,j) )with size[k 2 ,H p W p ]

[0108]

[0109] The calculation formula of the above multi-head attention mechanism MHA is as follows:

[0110] MHA(Q,K,V)=Concat(Head1,Head2…Head n )W 0

[0111]

[0112] In the above formula, W 0 is the output matrix; Head is the self-attention mechanism formula in each attention head.

[0113] The workflow of the above Patch Expanding module is: before upsampling, a linear layer is applied to the input features to double the feature dimension; then a shuffle operation is used to double the resolution of the input features and reduce the feature dimension to one quarter. The Final Patch Expanding module is: restore the feature resolution back to the resolution of the original image, first apply a linear layer to the input features to increase the feature dimension to 16 times the original dimension, and then directly expand the resolution of the input features by four times through a shuffle operation.

[0114] Refer to the attached Figure 3 As shown, the workflow of the CAGF module is: define the local features of the i-th stage as The global feature is In order to ensure the correspondence between local features and global features, sinusoidal position encoding PE (Positional Encoding) is introduced to generate a unique code for each position in the sequence, which can be expressed by the following formula:

[0115]

[0116] In the above formula, pos represents the position in the sequence;

[0117] After position encoding, local features and weight matrix W q Multiply the linear map to the query vector

[0118] The global features are respectively related to the weight matrix W k and W v Multiply the linear map to the key vector Sum value vector Among them C h is the length of the hidden vector in the mapping sequence;

[0119] Q f With K g The attention map obtained by cross product can represent the correlation between all spatial positions of local features and global features, and then the attention map is combined with V g The attention map gate obtained by cross product represents the semantic feature significance of each position in the local feature, which can be used to filter irrelevant noise in the local feature. The feature map obtained by dot multiplication of the local feature and the gate feature map is superimposed on the global feature and then a 1×1 convolution is performed to output the final feature.

[0120] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for underwater fish recognition based on TransNeXt, characterized in that: include: 1): Preprocess the underwater fish image dataset in the MMSegmentation framework and divide it into training set and test set; 2): Migrate the improved TransNeXt model and its parameters pre-trained on the Ade20K dataset to the MMSegmentation framework; 3): Use the training set and test set to iteratively train the improved TransNeXt model; 4): Input the underwater fish image to be detected into the trained improved TransNeXt model and output the recognition result.

2. The underwater fish identification method based on TransNeXt according to claim 1, characterized in that: The preprocessing steps of the input image in step 1 include: The underwater fish images of different sizes are uniformly adjusted to (512, 512, 3) through MMSegmentaion, the number of channels is 3, and the dataset is divided into training set and test set in a ratio of 9:

1.

3. The underwater fish identification method based on TransNeXt according to claim 1, characterized in that: The encoder structure in the improved TransNeXt model described in step 2) has a total of 4 stages; among them, stages 1-3 are centered on the AA module, and stage 4 is centered on the multi-head attention mechanism MHA module; The main workflow of each stage in the model is as follows: First, the input features are downsampled through the Overlapping Patch Embedding module, and then enter the first LN layer and the core layer for calculation. The calculated features are fused with the input features after downsampling in the first layer to obtain transition features; the transition features are then calculated in the second LN normalization layer and the ConvGLU layer, and the obtained features are fused with the transition features and output. The output features will be used as the input features of the next stage to continue similar operations; After the four-stage encoder processing, a decoder with a similar structure to the encoder is connected. The difference from the encoder is that the decoder performs upsampling through the Patch Expanding module. After the PPM processing, the decoder superimposes the output features of each stage with the downsampled features of the corresponding stage after processing by the CAGF module as the input of the next stage; Finally, the resolution of the feature is restored to the resolution of the original image through the FinalPatch Expanding module, and the detection head is used to process the object or object part in the image to complete the segmentation and output the segmentation result.

4. The underwater fish identification method based on TransNeXt according to claim 3 is characterized in that: The AA module includes a pixel focused attention mechanism PFA, a query vector embedding QE and a position attention mechanism PA; The pixel-focused attention mechanism PFA is a dual-channel design, which includes a sliding window attention mechanism and a pooling attention mechanism, including: Given an input feature with dimension C and size H×W Define d as the output dimension of the attention head, and define a set of pixels in the sliding window centered on pixel point (i, j) as ρ(i, j). For a sliding window of size k×k, ||ρ(i, j)||=k 2 ; The pixel point after pooling of the feature map is defined as σ(X), and the size of the feature map after pooling is defined as H p ×W p , then ||σ(X)||=H p W p ; Pixel focus attention PFA can be expressed by the following formula: A (i，j)～ρ(i，j) ,A (i，j)～σ(X) =Split(A (i，j) ,[k 2 ,||σ(X)||]) PFA(X (i，j) )=A (i，j)～ρ(i，j) V ρ(i，j) +A (i，j)～σ(X) V σ(i，j) In the formula, Q represents the query vector; K represents the key vector; B represents the relative position bias; S represents the attention score; A represents the attention weight; V represents the value vector; Concat function represents the tensor concatenation operation; Softmax function represents the Softmax activation function; Split function represents the tensor splitting operation.

5. The underwater fish identification method based on TransNeXt according to claim 4 is characterized in that: The downsampling method of the model, activation and pooling, can be expressed by the following formula: σ(X)=LayerNorm(AvgPool(GELU(Linear(X)))) In the above formula, the Linear function represents the linear transformation operation through the fully connected layer; the GELU function represents the Gaussian error linear unit activation function; the AvgPool function represents the average pooling operation; The LayerNorm function represents the layer normalization operation PFA. It adds learnable QE to all query vector tokens in the traditional QKV attention mechanism to obtain the information aggregation effect, which can be expressed by the following formula:

6. The underwater fish identification method based on TransNeXt according to claim 5, characterized in that: The model adopts a method of obtaining attention weights by using a learnable key vector to interact with a query vector from the input, namely, a QLV attention mechanism, which is specifically: Introduce a set of learnable tokens in each attention head Let the token interact with the query vector to obtain additional dynamic position bias and add it to the attention weight A (i，j)～ρ(i，j ), can be expressed by the following formula: PFA(X (i，j) )=(A (i，j)～ρ(i，j) +Q (i，j) T)V ρ(i，j) +A (i，j)～σ(X) V σ(i，j) 。 7. The underwater fish identification method based on TransNeXt according to claim 6 is characterized in that: The model adopts the length-scaled cosine attention mechanism, and the attention head dimensions of the query vector and the key vector are l2-normalized and defined as and Both have size 1; Let λ≈τlog n, where τ is a learnable variable initialized to The length ratio cosine attention mechanism formula is as follows: N(i,j)=||ρ(i,j)||+||σ(X)||-||μ(i,j)|| In the above formula, N represents the number of key vectors that can effectively interact with the query vector after removing the masked token; μ(i, j) represents the masked token; The model uses logarithmically spaced continuous position bias Log-CPB on the pooling path, according to Q (i，j) and K σ(X) The relative coordinates of space Δ (i，j)～σ(X) To calculate the relative position deviation B (i，j)～σ(X) , on the sliding window path, the TransNeXt model directly uses the learnable relative position deviation B (i，j)～ρ(X) . The aggregate attention mechanism AA can be expressed by the following formula: B (i，j) =Concat(B (i，j)～ρ(i，j) ,Log-CPB(S (i，j)～σ(X) )) A (i，j) =Softmax(τlog N*Concat(S (i，j)～ρ(i，j) ,S (i，j)～σ(X)) +B (i，j) ) A (i，j)～ρ(i，j) ,A (i，j)～σ(X) =Split(A (i，j) )with size[k 2 ,H p W p ] 8. The underwater fish identification method based on TransNeXt according to claim 1, characterized in that: The calculation formula of the multi-head attention mechanism MHA is as follows: MHA(Q,K,V)=Concat(Head1,Head2...Head n )W 0 In the above formula, W 0 is the output matrix; Head is the self-attention mechanism formula in each attention head.

9. The underwater fish identification method based on TransNeXt according to claim 1, characterized in that: The workflow of the PatchExpanding module is as follows: Before upsampling, a linear layer is applied to the input features to double the feature dimension. Then a shuffle operation is used to double the resolution of the input features and reduce the feature dimension to one quarter. In order to restore the feature resolution back to the resolution of the original image, the FinalPatch Expanding module first applies a linear layer to the input features to increase the feature dimension to 16 times the original dimension, and then directly expands the resolution of the input features by four times through a rearrangement operation.

10. The underwater fish identification method based on TransNeXt according to claim 1, characterized in that: The workflow of the CAGF module is as follows: The local features of the i-th stage are defined as The global feature is (i=1, 2, 3) To ensure the correspondence between local features and global features, sinusoidal position encoding PE is introduced to generate a unique code for each position in the sequence, which can be expressed by the following formula: In the above formula, pos represents the position in the sequence; After position encoding, local features and weight matrix W q Multiply the linear map to the query vector The global features are respectively related to the weight matrix W k and W v Multiply the linear map to the key vector Sum value vector Among them C h is the length of the hidden vector in the mapping sequence; Q f With K g The attention map obtained by cross product can represent the correlation between all spatial positions of local features and global features, and then the attention map is combined with V g The attention map gate obtained by cross product represents the semantic feature significance of each position in the local feature, which can be used to filter irrelevant noise in the local feature. The feature map obtained by dot multiplication of the local feature and the gate feature map is superimposed on the global feature and then a 1×1 convolution is performed to output the final feature.

Images