A Method and System for Detecting Underwater Biological Targets Based on Pyramid-Style Dynamic Attention

Abstract

This invention relates to the field of underwater target detection technology, specifically to a method and system for detecting marine underwater biological targets based on pyramid-shaped dynamic attention. The method includes: acquiring images of marine underwater organisms and a trained pyramid-shaped target detection network; extracting multi-scale local features in parallel and performing adaptive weighted fusion and cross-level interaction to output a multi-scale feature map; generating a query matrix, a key matrix, and a value matrix; performing pyramid-shaped multi-level downsampling on the key and value matrices and executing dynamic sparse attention calculation to output an attention feature map; outputting enhanced fusion features through refined fusion of high- and low-level features; and inputting the enhanced fusion features into a detection head to complete target classification and bounding box regression to obtain the marine underwater biological target detection result. This invention solves the problems of large target scale variations, difficulty in detecting small targets, and insufficient real-time performance in the marine underwater environment, improving detection accuracy and inference speed.

Description

Technical Field

[0001] This invention relates to the field of underwater target detection technology, specifically to a method and system for detecting marine underwater biological targets based on pyramid-shaped dynamic attention. Background Technology

[0002] Underwater biological target detection is an important research direction at the intersection of marine science and computer vision. Its task is to identify and locate marine biological targets such as fish, jellyfish, and starfish in the underwater environment in real time. With the rapid development of underwater robots, marine resource development, intelligent aquaculture, and marine ecological monitoring technologies, higher requirements are being placed on the accuracy, speed, and multi-scale adaptability of underwater target detection.

[0003] The underwater environment is characterized by complex lighting conditions, large target scale variations, and severe target occlusion. Specifically, underwater light attenuation, scattering effects, and color distortion lead to image quality degradation; the scale variation range is extremely wide, from large sharks to small starfish; mutual occlusion among marine organisms and interference from underwater suspended debris further increase the difficulty of detection. This complex scenario requires detection models to possess powerful multi-scale feature extraction capabilities, sensitivity to small targets, global context understanding capabilities, and an extreme pursuit of computational efficiency.

[0004] In recent years, deep learning-based object detection methods have made significant progress. Methods represented by convolutional neural networks (CNNs), such as the YOLO series and Faster R-CNN, have performed excellently in general object detection tasks. However, CNNs extract features by progressively stacking local convolutional kernels and repeatedly downsampling. Their local receptive field characteristics make it difficult to effectively model dependencies between distant pixels. Furthermore, repeated downsampling operations easily lead to the loss of detailed information about small targets, which is particularly problematic in underwater scenes where numerous small targets exist, severely impacting detection accuracy.

[0005] The successful application of the Transformer architecture in computer vision has provided a new approach to solving the aforementioned problems. DETR was the first to introduce the Transformer into object detection tasks, achieving object detection in an end-to-end manner. Subsequently, RT-DETR was optimized for real-time requirements, achieving real-time inference speed while maintaining high detection accuracy, becoming an important baseline model in the field of real-time object detection. However, the standard self-attention mechanism processes all tokens in the image uniformly, which is insufficient for capturing the local details that are crucial in underwater biological target detection tasks. More importantly, the computational complexity of the standard self-attention mechanism increases quadratically on high-resolution feature maps, resulting in low computational efficiency and a large number of parameters, which is significantly insufficient for the real-time requirements of edge devices such as underwater robots.

[0006] Existing feature pyramid networks and their variants employ a fixed fusion structure, fusing features from different levels through methods such as splicing, addition, or weighted summation. However, the scale distribution of targets in the underwater marine environment dynamically changes with the shooting distance, and static fusion mechanisms cannot dynamically adjust the fusion strategy according to the input scene, making it difficult to achieve optimal feature fusion.

[0007] In summary, there is an urgent need for an underwater biological target detection method and system that can adaptively process multi-scale targets, effectively reduce computational complexity, and fully mine complementary information from multi-level features. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a method and system for detecting underwater biological targets in the ocean based on pyramid-shaped dynamic attention, aiming to significantly reduce computational overhead while comprehensively improving the accuracy and real-time performance of underwater biological target detection.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] On one hand, embodiments of the present invention provide a method for detecting underwater biological targets in the ocean based on pyramid-shaped dynamic attention, the method comprising the following steps:

[0011] The method acquires images of marine underwater organisms and a trained pyramid-shaped target detection network, which includes a backbone network composed of stacked pyramid-shaped feature extraction modules, a neck network containing a pyramid-shaped dynamic sparse attention module and a pyramid-shaped grouping adaptive fusion module, and a head detection network.

[0012] The image of the marine underwater organism is input into the backbone network, and the pyramid-shaped feature extraction module extracts multi-scale local features in parallel and performs adaptive weighted fusion and cross-level feature interaction to output a multi-scale feature map.

[0013] The highest-level feature map in the multi-scale feature map is input into the pyramid-shaped dynamic sparse attention module. A query matrix, a key matrix, and a value matrix are generated through channel segmentation. The key matrix and the value matrix are subjected to pyramid-shaped multi-level spatial downsampling and dynamic sparse attention calculation is performed to output the attention feature map.

[0014] The attention feature map and the low-level feature map in the multi-scale feature map are input into the pyramid-shaped grouping adaptive fusion module. The pyramid-shaped multi-scale grouping adaptive fusion achieves fine fusion of high and low-level features and outputs enhanced fused features.

[0015] The enhanced fusion features are input into the detection head to complete the target classification and bounding box regression, thereby obtaining the detection results of marine underwater biological targets.

[0016] Optionally, the step of extracting multi-scale local features in parallel through the pyramid-shaped feature extraction module and performing adaptive weighted fusion and cross-level feature interaction includes:

[0017] The input feature map is input into the pyramid-shaped feature extraction module, and local features of different receptive fields are extracted by multiple parallel depthwise separable convolutions of different scales to obtain multi-scale local feature maps.

[0018] The input feature map is compressed in space dimension by global average pooling to obtain channel-level global descriptors. The channel-level global descriptors are then processed by continuous convolution mapping and activation functions to generate normalized pyramid-level weights.

[0019] The multi-scale local feature map is multiplied element-wise with the corresponding pyramid level weights and then summed to obtain the weighted fusion feature.

[0020] The multi-scale local feature maps are concatenated along the channel dimension and then input into a fusion network containing convolution, batch normalization, and activation functions to perform cross-channel information fusion and obtain cross-level fusion features.

[0021] The weighted fusion feature and the cross-level fusion feature are added together via residual connection to output the enhanced feature map.

[0022] Optionally, the step of extracting local features of different receptive fields through multiple parallel, depthwise separable convolutions of different scales includes:

[0023] The input feature map is fed into three parallel depthwise separable convolutional branches with kernel sizes of 3×3, 5×5, and 7×7, respectively. Each branch extracts local features of the corresponding receptive field through channel-wise convolution and pointwise convolution, outputting three local feature maps at different scales. The stride of the three parallel depthwise separable convolutional branches is 1, and the padding is half the kernel size to ensure that the output spatial size is consistent with the input feature map. Figure 1 To.

[0024] Optionally, the step of compressing the spatial dimension of the input feature map using global average pooling to obtain a channel-level global descriptor, and then processing the channel-level global descriptor through continuous convolution mapping and activation functions to generate normalized pyramid-level weights, includes:

[0025] Perform global average pooling on the input feature map to compress the spatial dimension to 1 and output a channel-level global descriptor;

[0026] The channel-level global descriptor is input into the first 1×1 convolutional layer to reduce the number of channels to one-quarter of the original number of channels;

[0027] The reduced-dimensional features are input into the second 1×1 convolutional layer, which expands the number of channels to the product of the original number of channels and the number of pyramid levels.

[0028] The expanded features are mapped to the 0-1 interval using the Sigmoid activation function, and the normalized pyramid level weights are output.

[0029] Optionally, the step of inputting the highest-level feature map from the multi-scale feature map into the pyramid-shaped dynamic sparse attention module, generating a query matrix, a key matrix, and a value matrix through channel segmentation, performing pyramid-shaped multi-level spatial downsampling on the key matrix and value matrix, and executing dynamic sparse attention calculation includes:

[0030] The highest-level feature map is divided into a first part of features and a second part of features along the channel dimension;

[0031] The first part of the features is normalized and mapped by 1×1 convolution to generate a query matrix, a key matrix, and a value matrix;

[0032] The key matrix and value matrix are subjected to three levels of average pooling spatial downsampling: original scale, 2x downsampling, and 4x downsampling, respectively, to obtain three levels of downsampled key matrix and downsampled value matrix.

[0033] The query matrix is ​​scaled and dot product attention is calculated with the downsampling key matrix of each level to obtain the attention weight matrix. The highest value of the token subset in the attention weight matrix is ​​selected according to the dynamic sparsity. The selected attention weight matrix is ​​multiplied with the downsampling value matrix of the corresponding level to obtain the attention output features of the three levels.

[0034] The attention output features of the three levels are weighted and fused by learnable hierarchical weights and cross-level interactions, then concatenated with the second part of the features and projected to output the attention feature map.

[0035] Optionally, the step of performing average pooling spatial downsampling on the key matrix and value matrix at three levels—original scale, 2x downsampling, and 4x downsampling—to obtain three levels of downsampled key matrices and downsampled value matrices includes:

[0036] Three pyramid levels are set up, each level corresponding to a scaling factor, and the scaling factors of the three pyramid levels are 1, 2 and 4 respectively;

[0037] For each level, the key matrix and value matrix are respectively input into an average pooling layer with a kernel size equal to the scale factor and a stride equal to the scale factor, and spatial downsampling operation is performed.

[0038] The downsampled key matrix and value matrix are flattened into a two-dimensional sequence to obtain three levels of downsampled key matrix and downsampled value matrix; wherein, the flattening operation compresses the spatial dimension to the token sequence dimension while keeping the channel dimension unchanged.

[0039] Optionally, the step of filtering the subset of tokens with the highest values ​​in the attention weight matrix based on the dynamic sparsity rate includes:

[0040] Using the highest-level feature map as input, a gating network consisting of two 1×1 convolutional layers and a Sigmoid activation function is used to output a layer-level gating value.

[0041] Multiply the level-based gate value by the total number of tokens at the corresponding level and round down to obtain the dynamically retained number;

[0042] For each query position, select the key positions with the highest values ​​in the attention weight matrix, which is the same number as the number of dynamically retained key positions. Set the attention scores of the unselected key positions to negative infinity to obtain a sparse attention weight matrix.

[0043] Perform Softmax normalization on the sparse attention weight matrix to output the normalized sparse attention weight matrix.

[0044] Optionally, the step of inputting the attention feature map and the low-level feature map from the multi-scale feature map into the pyramid-shaped grouping adaptive fusion module, and achieving refined fusion of high- and low-level features through pyramid-shaped multi-scale grouping adaptive fusion, includes:

[0045] Channel alignment and spatial upsampling are performed on the low-level feature map to make the number of channels and spatial size of the low-level feature map consistent with those of the attention feature map;

[0046] The low-level feature maps after channel alignment and spatial upsampling, along with the attention feature maps, are input into three levels of average pooling layers: original scale, 2x downsampling, and 4x downsampling, respectively, to obtain three levels of downsampled low-level feature maps and downsampled attention feature maps.

[0047] For each level, the downsampled low-level feature map and the downsampled attention feature map are divided into multiple independent groups in the channel dimension. Independent fusion weights are learned for each group. The low-level features and attention features in the same group are fused according to the fusion weights to obtain the level fusion features.

[0048] The three-level hierarchical fusion features are weighted and summed by learnable hierarchical weights and then input into a convolutional layer for spatial integration, outputting the enhanced fusion features.

[0049] On the other hand, embodiments of the present invention provide a marine underwater biological target detection system based on pyramid-shaped dynamic attention, comprising:

[0050] At least one processor;

[0051] At least one memory for storing at least one program;

[0052] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0053] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the above-described method.

[0054] The embodiments of the present invention have the following beneficial effects:

[0055] This invention designs a pyramid-shaped feature extraction module in the backbone network. It extracts local features from different receptive fields in parallel using pyramid-shaped multi-scale convolutional kernels and employs learnable weights for adaptive fusion. This enhances the model's feature representation capability for multi-scale targets without significantly increasing computational cost. The pyramid-shaped structure enables the model to simultaneously capture the fine textures of small targets such as starfish and small fish, as well as the overall outlines of large targets such as sharks, effectively addressing the challenge of large target scale variations in the underwater marine environment.

[0056] This invention designs a pyramid-shaped dynamic sparse attention module in the neck network to replace the original AIFI module of RT-DETR. This module first groups the feature channels and performs attention calculations on some channels to reduce computational overhead. Secondly, it captures contextual information of different granularities through pyramid-shaped multi-scale key and value downsampling, enhancing the ability to perceive global dependencies and local details. Finally, it introduces a dynamic sparse selection mechanism to adaptively select the number of tokens participating in the calculation based on the input features, significantly reducing computational complexity while preserving key information interaction, thus meeting the real-time detection needs of edge devices such as underwater robots.

[0057] This invention designs a pyramid-shaped grouping adaptive fusion module in the neck network to achieve refined fusion of multi-level features. This module first divides the input features into multiple independent groups along the channel dimension, learning independent fusion weights for each group to achieve more refined channel-level fusion control. Secondly, it constructs a pyramid-shaped multi-scale fusion hierarchy to perform feature fusion at different scales. Finally, through learnable pyramid hierarchy weights, it achieves adaptive weighted fusion of multi-scale features, enabling the model to dynamically adjust the fusion strategy according to the input scenario and fully exploit the complementary information between high- and low-level features.

[0058] This invention constructs an end-to-end pyramid-shaped dynamic attention detection framework through the cascaded optimization of three modules: a pyramid-shaped feature extraction module, a pyramid-shaped dynamic sparse attention module, and a pyramid-shaped grouping adaptive fusion module. These three modules form a synergistic enhancement mechanism: the pyramid-shaped feature extraction module extracts multi-scale pyramid features in the backbone network, providing a rich feature base for subsequent processing; the pyramid-shaped dynamic sparse attention module performs global context modeling of high-level features in the neck network, focusing on key detection regions; and the pyramid-shaped grouping adaptive fusion module achieves refined fusion of high- and low-level features during the feature fusion stage, preserving detailed information about small targets. The synergistic effect of these three modules enables the model to achieve end-to-end pyramid-shaped multi-scale enhancement from feature extraction to detection output. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating the method for detecting underwater biological targets in the ocean based on pyramid-shaped dynamic attention in an embodiment of the present invention.

[0061] Figure 2 This is a diagram showing the overall structure of the Pyramid-DETR target detection network in this embodiment of the invention.

[0062] Figure 3 This is a structural diagram of the pyramid feature extraction module (PFEM) in an embodiment of the present invention;

[0063] Figure 4 This is a structural diagram of the pyramid-shaped dynamic sparse attention module (PDSA) in an embodiment of the present invention;

[0064] Figure 5 This is a structural diagram of the pyramid-shaped grouped adaptive fusion module (PGAF) in an embodiment of the present invention. Detailed Implementation

[0065] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0067] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this invention is for descriptive purposes only and is not intended to limit the invention.

[0068] refer to Figure 1 ,like Figure 1 The image shows a method for detecting underwater biological targets in the ocean based on pyramid-shaped dynamic attention, provided by an embodiment of the present invention. The method includes the following steps:

[0069] S100, acquire images of marine underwater organisms and a trained pyramid-shaped target detection network, the pyramid-shaped target detection network including a backbone network composed of stacked pyramid-shaped feature extraction modules, a neck network including a pyramid-shaped dynamic sparse attention module and a pyramid-shaped grouping adaptive fusion module, and a head detection network.

[0070] S200, The image of the marine underwater organism is input into the backbone network, and the multi-scale local features are extracted in parallel through the pyramid-shaped feature extraction module. Adaptive weighted fusion and cross-level feature interaction are performed to output a multi-scale feature map.

[0071] S300, input the highest-level feature map of the multi-scale feature map into the pyramid-shaped dynamic sparse attention module, generate a query matrix, a key matrix and a value matrix through channel segmentation, perform pyramid-shaped multi-level spatial downsampling on the key matrix and the value matrix and perform dynamic sparse attention calculation, and output the attention feature map;

[0072] S400, the attention feature map and the low-level feature map in the multi-scale feature map are input into the pyramid-shaped grouping adaptive fusion module, and the high- and low-level features are refined by pyramid-shaped multi-scale grouping adaptive fusion, and the enhanced fusion features are output.

[0073] S500, the enhanced fusion features are input into the detection head to complete the target classification and bounding box regression, and the detection results of marine underwater biological targets are obtained.

[0074] This invention provides a method and system for detecting marine underwater biological targets based on pyramid-shaped dynamic attention. The pyramid-shaped feature extraction module performs multi-scale convolutional parallel extraction, adaptive weighted fusion, and cross-level feature interaction on input features, effectively enhancing the model's feature representation ability for multi-scale targets and solving the problem of insufficient capture of small target details by traditional convolutional neural networks. The pyramid-shaped dynamic sparse attention module generates query, key, and value matrices through channel segmentation. After multi-level spatial downsampling of the key and value matrices, dynamic sparse attention calculation is performed, significantly reducing computational complexity while preserving key global dependencies and local details, thus solving the problem of insufficient computational efficiency in traditional Transformers. The pyramid-shaped grouping adaptive fusion module achieves refined fusion of high- and low-level features through pyramid-shaped multi-scale grouping adaptive fusion, fully exploring the complementary information of high- and low-level features, and solving the problem of insufficient adaptive capability in traditional feature fusion methods. Through the synergistic enhancement of these three modules, this invention achieves end-to-end pyramid-shaped multi-scale enhancement from feature extraction to detection output, significantly improving the accuracy and real-time performance of marine underwater biological target detection.

[0075] The core of this embodiment lies in the Pyramid-DETR (Transformer Network for Target Detection) framework. It improves the RT-DETR infrastructure through three core innovative modules: a pyramid-shaped feature extraction module, a pyramid-shaped dynamic sparse attention module, and a pyramid-shaped grouping adaptive fusion module, thereby achieving efficient target detection for complex marine underwater environments.

[0076] Data Acquisition and Network Preparation: Acquire images of marine underwater organisms. Load a pre-trained pyramid-shaped object detection network.

[0077] like Figure 2 As shown, the overall structure of the pyramid-shaped target detection Transformer proposed in this invention includes:

[0078] Backbone network: It consists of multiple pyramid-shaped feature extraction modules stacked together, extracting multi-scale feature maps step by step;

[0079] Neck network: Includes a pyramid-shaped dynamic sparse attention module and a pyramid-shaped grouping adaptive fusion module to achieve global context modeling and multi-scale feature fusion;

[0080] Head detection: RT-DETR decoder is used for target classification and bounding box regression.

[0081] The input image first enters the backbone network, which consists of multiple convolutional layers and a pyramid-shaped feature extraction module, extracting multi-scale feature maps step by step. , and The neck network receives multi-scale features from the backbone network, focusing on the highest-level features. A pyramid-style dynamic sparse attention module is applied for global context modeling. Then, a feature pyramid is constructed through operations such as upsampling and a pyramid-style grouping adaptive fusion module to generate enhanced multi-scale features. The head receives the three-layer features output from the neck network, and the RT-DETR decoder is used for target classification and bounding box regression to output the final detection result.

[0082] In some embodiments, S200, the parallel extraction of multi-scale local features by the pyramid-shaped feature extraction module and the adaptive weighted fusion and cross-level feature interaction include:

[0083] S210, The input feature map is input into the pyramid-shaped feature extraction module, and local features of different receptive fields are extracted by multiple parallel depthwise separable convolutions of different scales to obtain multi-scale local feature maps.

[0084] S220, the input feature map is compressed in space dimension by global average pooling to obtain channel-level global descriptors, and the channel-level global descriptors are processed by continuous convolution mapping and activation function to generate normalized pyramid-level weights.

[0085] S230, the multi-scale local feature map is multiplied element-wise with the corresponding pyramid level weights and then summed to obtain the weighted fusion feature;

[0086] S240, the multi-scale local feature maps are concatenated along the channel dimension and then input into a fusion network containing convolution, batch normalization and activation functions to perform cross-channel information fusion and obtain cross-level fusion features;

[0087] S250, the weighted fusion feature and the cross-level fusion feature are added together via residual connection to output the enhanced feature map.

[0088] Specifically, the input feature map is fed into a pyramid-shaped feature extraction module. Multiple depthwise separable convolutional branches at different scales within this module are used to extract features from the input feature map, resulting in multi-scale local feature maps. The input feature map is then compressed using global average pooling to obtain channel-level global descriptors. These descriptors are then processed through two consecutive 1×1 convolutional layers and a sigmoid activation function to generate normalized pyramid-level weights. The multi-scale local feature maps are element-wise multiplied with the corresponding pyramid-level weights and summed to obtain weighted fusion features. The multi-scale local feature maps are concatenated along the channel dimension and then input into a fusion network containing 1×1 convolutions, batch normalization, and ReLU activation functions for cross-channel information fusion, resulting in cross-level fusion features. Finally, the weighted fusion features and the cross-level fusion features are added via residual connections to output an enhanced feature map.

[0089] The pyramid-style feature extraction module is described below:

[0090] like Figure 3 As shown, the core architecture of the backbone network in this embodiment consists of multiple stacked pyramid-shaped feature extraction modules, forming a multi-scale feature extraction backbone structure. Through stepwise downsampling and enhancement processing by the pyramid-shaped feature extraction modules, the network can effectively capture multi-scale feature information in underwater scenes, ranging from large marine organisms such as sharks and dolphins to small targets such as starfish and small fish. The specific process is as follows:

[0091] 1. Primary Feature Processing Layer: Input marine underwater organism image (B, C, H, and W represent batch size, number of channels, height, and width, respectively, and R is the set of real numbers). A 3×3 convolutional layer with a stride of 2 is used for fast downsampling and preliminary feature extraction, outputting a feature map. .

[0092] ;

[0093] Where X is the input image; This indicates a convolution operation with a kernel size of 3×3 and a stride of 2.

[0094] 2. Multi-level pyramid-style feature extraction: Features after initial processing The data is fed into a four-layer extraction network consisting of stacked pyramid-shaped feature extraction modules. Each layer uses a 3×3 convolutional layer with a stride of 2 for spatial downsampling, followed by a pyramid-shaped feature extraction module for feature extraction. The single-layer operation corresponds to:

[0095] ; ;

[0096] in, For the corresponding number of levels; Features that have undergone spatial downsampling in the corresponding stage; PFEM is a pyramid-shaped feature extraction module;

[0097] For example, in stage 1, the input is... The output is a feature map. .

[0098] ; ;

[0099] Among them, DFEB is a pyramid-shaped feature extraction module.

[0100] 3. Processing flow within the pyramid-shaped feature extraction module:

[0101] a. Pyramid-style multi-level parallel processing: Given an input feature map Where B, C, H, and W represent batch size, number of channels, feature map height, and width, respectively. The pyramid-style feature extraction module constructs a pyramid using a set of parallel depthwise separable convolutions to extract multi-scale local features. This module employs three convolutional kernels of different sizes, namely... , , The step size is 1, and the padding is... To ensure the output spatial dimensions remain constant, depthwise separable convolution decomposes standard convolution into channel-wise convolution and pointwise convolution, reducing the number of parameters. For the k-th layer, the feature extraction process can be represented as:

[0102] ;

[0103] in, Indicates the kernel size as Depth-separable convolution, This represents the local features extracted at the k-th convolutional layer, where k takes values ​​of 1, 2, and 3, corresponding to convolutional branches with three different receptive fields: 3×3, 5×5, and 7×7, respectively. This operation allows the network to focus on local details and region structure separately, with computational overhead lower than using multiple standard convolutions.

[0104] In this step, the pyramid-shaped feature extraction module constructs a pyramid using a set of parallel depthwise separable convolutions to extract multi-scale local features. Given an input feature map, the pyramid-shaped feature extraction module uses three convolutional kernels of different sizes: 3×3, 5×5, and 7×7, with a stride of 1 and padding to half the kernel size to ensure the output spatial size remains unchanged. Depthwise separable convolutions decompose standard convolutions into channel-wise and pointwise convolutions, reducing the number of parameters. For each level, the feature extraction process is implemented using depthwise separable convolutions, allowing the network to focus on local details and regional structures separately, with computational overhead lower than using multiple standard convolutions.

[0105] b. Adaptive weight generator:

[0106] This generator uses input features X as a basis and compresses the spatial dimension through global average pooling to obtain channel-level global descriptors. This descriptor aggregates global spatial information and reflects the response intensity of each channel.

[0107] ;

[0108] z goes through two consecutive 1×1 convolutional layers. The first 1×1 convolution reduces the number of channels to C / 4, lowering computational complexity, while the second convolution expands the number of channels to... Where L=3 is the number of pyramid levels. Finally, the output value is mapped to the (0,1) interval using the Sigmoid activation function to obtain the normalized attention weight w.

[0109] ;

[0110] ;

[0111] ;

[0112] Where x is the input feature, The Sigmoid activation function is used, and ReLU is the rectified linear activation function. This represents a 1×1 convolution, where w is the generated normalized attention weight, corresponding to the weight coefficient of each channel in each layer.

[0113] The weight tensor was reshaped as follows The form is such that each scale k corresponds to a weight vector. This allows for independent scale selection for each channel. Weighted fusion is achieved by multiplying the features of each level by their corresponding weights and then summing the results.

[0114] ;

[0115] in, This represents an element-wise multiplication operation. This operation enables the network to dynamically adjust the contribution ratio of features at different scales based on the current input content.

[0116] In this step, the adaptive weight generator, based on the input features, compresses the spatial dimension through global average pooling to obtain a channel-level global descriptor. This descriptor aggregates global spatial information and reflects the response intensity of each channel. The global descriptor passes through two consecutive 1×1 convolutional layers. The first 1×1 convolution reduces the number of channels to one-quarter of the original number, lowering computational complexity. The second convolution expands the number of channels to the product of the original number of channels and the number of pyramid levels. Finally, the output values ​​are mapped to the 0-1 interval using the Sigmoid activation function, resulting in normalized pyramid level weights. These weights are reshaped into a weight vector corresponding to each scale, enabling independent scale selection for each channel. Weighted fusion is achieved by multiplying the features of each level with their corresponding weights and then summing the results.

[0117] c. Cross-layer feature interaction mechanism:

[0118] This mechanism concatenates the features from the three pyramid levels along the channel dimension to obtain a single dimension. intermediate feature map : ;in, This indicates splicing along the channel dimension.

[0119] The number of channels is compressed back to C using a fusion network consisting of 1×1 convolutions, batch normalization, and ReLU activation function.

[0120] ;in, This represents a 1×1 convolution; its input channels are 3C and its output channels are C. Batch normalization is used to accelerate training and stabilize gradients; The non-linear activation function enhances the expressive power of features. This design achieves cross-channel information fusion through 1×1 convolution, enabling features at different scales to be non-linearly combined through learnable weights, compensating for the scale synergy that may be ignored by weighted summation; at the same time, the introduction of batch normalization and ReLU further improves the representational power and optimization stability of the fused features.

[0121] Output Determined by both weighted fusion features and cross-level fusion features, a residual connection approach is adopted:

[0122] ;

[0123] in, This is the fusion coefficient, used to balance the contributions of the two fusion strategies and prevent one path from dominating the output while the information of the other path is overwhelmed.

[0124] In this step, the cross-level feature interaction mechanism concatenates features from three pyramid levels along the channel dimension to obtain an intermediate feature map. This intermediate feature map is then compressed back to the original number of channels using a fusion network consisting of 1×1 convolutions, batch normalization, and ReLU activation. This design achieves cross-channel information fusion through 1×1 convolutions, allowing features at different scales to be non-linearly combined using learnable weights, compensating for the scale-to-scale synergy that might be overlooked by weighted summation. The output is jointly determined by the weighted fused features and the cross-level fused features, using residual connections, with the fusion coefficients used to balance the contributions of the two fusion strategies.

[0125] d. Main path output:

[0126] ;

[0127] in, The main path outputs features. The original input features are those of the pyramid-shaped feature extraction module. It is a depthwise separable convolution.

[0128] When the number of input channels equals the number of output channels, the module enables residual connections, directly adding the input features to the main path output to obtain the final output features of the pyramid-shaped feature extraction module. :

[0129] ;

[0130] This design helps alleviate the vanishing gradient problem in deep networks and promotes the efficient transfer of information.

[0131] 4. Output of the feature extraction backbone network: The final output of this feature extraction network is feature maps at three different scales. .

[0132] In this step, the backbone path undergoes preprocessing before entering the pyramid local attention processing, and finally, the features are fused and residually connected to output the enhanced features. When the number of input channels equals the number of output channels, the module enables residual connections, directly adding the input features to the backbone path output. This helps alleviate the gradient vanishing problem in deep networks and promotes effective information transfer. The final output of the feature extraction backbone network consists of feature maps at three different scales.

[0133] In some embodiments, S300, the step of inputting the highest-level feature map into the pyramid-shaped dynamic sparse attention module, generating a query matrix, a key matrix, and a value matrix through channel segmentation, performing pyramid-shaped multi-level spatial downsampling on the key matrix and value matrix, and performing dynamic sparse attention calculation includes:

[0134] S310, the highest layer feature map is divided into a first part of features and a second part of features along the channel dimension;

[0135] S320, the first part of features is normalized by groups and mapped by 1×1 convolution to generate a query matrix, a key matrix and a value matrix;

[0136] S330, the key matrix and value matrix are subjected to average pooling spatial downsampling at three levels: original scale, 2x downsampling and 4x downsampling, respectively, to obtain the downsampled key matrix and downsampled value matrix at three levels;

[0137] S340, the query matrix and the downsampling key matrix of each level are scaled dot product attention calculation to obtain the attention weight matrix. The highest value of the token subset in the attention weight matrix is ​​selected according to the dynamic sparsity rate. The selected attention weight matrix is ​​multiplied by the downsampling value matrix of the corresponding level to obtain the attention output features of the three levels.

[0138] S350, the attention output features of the three levels are weighted and fused by learnable hierarchical weights and cross-level interaction, then concatenated with the second part of the features and projected to output the attention feature map.

[0139] Specifically, the highest-level feature map is divided into a first part and a second part along the channel dimension, with the first part accounting for one-quarter of the original channels. The first part of the features is normalized by a group normalization layer, and a query matrix, a key matrix, and a value matrix are generated by 1×1 convolution. The dimensions of the query matrix and the key matrix are compressed to half that of the value matrix, further reducing the computational cost.

[0140] The pyramid-style dynamic sparse attention module is described below:

[0141] like Figure 4 As shown, the internal architecture of the pyramid-shaped dynamic sparse attention module proposed in this embodiment replaces the original AIFI module of RT-DETR. Through pyramid-shaped multi-scale context modeling and dynamic sparse selection mechanism, it significantly reduces computational complexity while enhancing the dual perception capability of global dependencies and local details. The specific process is as follows:

[0142] 1. Channel Segmentation and QKV (Query, Key, Value) Generation: The pyramid-style dynamic sparse attention module first segments the input feature map along the channel dimension. Divided into two parts:

[0143] , ; ;

[0144] Where Split is the channel splitting operation; hyperparameters ,Right now It occupies 1 / 4 of the original channel.

[0145] right Some applications utilize attention mechanisms. Some features are directly retained. This design preserves the remaining information in the original features, which facilitates subsequent fusion and reduces the number of parameters and computational cost of attention calculation.

[0146] Segmented After feature standardization using group normalization layers, query, key, and value feature maps are generated through 1×1 convolution:

[0147] ;

[0148] in, ; ; This design focuses on the dimension of the attention head. It compresses the dimensions of the query and key to half the value, further reducing computational cost.

[0149] 2. Pyramid-style multi-level processing: This module sets up three pyramid levels, each corresponding to a scaling factor. For the l-th layer (l=1,2,3), the scaling factor is... ,Right now .

[0150] 3. Arbitrary single-level processing structure within the pyramid:

[0151] a. Attention calculation:

[0152] The input for any single level is QKV (query, key, value) and the original features. First, spatial downsampling is performed on the keys and values ​​to reduce the resolution:

[0153] , ;

[0154] in, Indicates the kernel size as Step size is Average pooling. The downsampled size is... , .

[0155] Next, , , Flattened into a two-dimensional sequence:

[0156] ;

[0157] ;

[0158] ;

[0159] Here, Flatten is the flattening operation. The flattening operation compresses the spatial dimension into the token sequence dimension, while keeping the channel dimension unchanged. , , Let be the query matrix, the l-th level key matrix, and the l-th level value matrix obtained after flattening, respectively. Let d be the channel dimension of the query and key, p be the channel dimension of the value, and N be the total number of tokens at the original scale. This represents the total number of tokens after downsampling at level l.

[0160] The scaling dot product attention is calculated as follows:

[0161] ;

[0162] in It is the attention weight matrix; It is a scaling factor to prevent the softmax gradient from saturating due to excessively large dot product values. It is the channel dimension of query and key.

[0163] b. Dynamic sparse selection gating network:

[0164] To make the attention at each level more efficient and adaptable to the content, the pyramid-shaped dynamic sparse attention module designs an independent gating network for each pyramid level. This gating network takes the input feature X as input and outputs the sparsity control signal for that level through two 1×1 convolutional layers and a sigmoid activation function.

[0165] For level l, a lightweight gating network is used. The network takes the original feature X as input and outputs a scalar gate value. :

[0166] ;

[0167] The number of dynamically retained items at the current level is:

[0168] ;

[0169] in, It's a gating value. For each query location, only the one most similar to that query point is retained. The attention scores for the remaining positions are set to [number] key positions. The sparse attention weight matrix is ​​obtained. The attention output at this level is:

[0170] ;

[0171] Reshape it back into spatial form Downsampling scale It also needs to be upsampled back to the original resolution using bilinear interpolation: . This indicates bilinear interpolation upsampling. This dynamic sparse selection mechanism can adaptively adjust the key attention positions retained at each level according to the input content, retaining only the key tokens with the highest similarity for computation, thus significantly reducing the computational cost and memory usage of attention calculations while preserving key contextual information.

[0172] 4. Weighted fusion and cross-level feature interaction:

[0173] After calculations at three pyramid levels, three feature maps were obtained. First, learnable hierarchical weights are introduced. (Initially 1 / 3), and normalized using softmax:

[0174] ;

[0175] The weighted fusion result is:

[0176] ;

[0177] To enhance information interaction across scales, features from the three levels are concatenated along the channel dimension and compressed using a fusion network containing 1×1 convolutions, batch normalization (BatchNorm), and ReLU activation.

[0178] ;

[0179] The output of the attention path is a residual combination of the weighted fusion result and the cross-level interaction result:

[0180] ;

[0181] in, This is the balance coefficient.

[0182] 5. Feature Recovery and Projection: The output Z of the attention path is compared with the previously preserved bypass features. splicing along the channel dimension:

[0183] ;

[0184] The final output of the pyramid-shaped dynamic sparse attention module is obtained by projecting through a 1×1 convolution containing SiLU activations and learnable initialization to zero:

[0185] ; ;

[0186] Where SiLU is the Sigmoid linear activation function. This is the Sigmoid function.

[0187] In some embodiments, S400, the step of inputting the attention feature map and the low-level feature map from the multi-scale feature map into the pyramid-shaped grouping adaptive fusion module, and achieving refined fusion of high- and low-level features through pyramid-shaped multi-scale grouping adaptive fusion, includes:

[0188] S410, Perform channel alignment and spatial upsampling on the low-level feature map to make the number of channels and spatial size of the low-level feature map consistent with those of the attention feature map;

[0189] S420, input the low-level feature map after channel alignment and spatial upsampling and the attention feature map into the average pooling layer of three levels: original scale, 2x downsampling and 4x downsampling, respectively, to obtain three levels of downsampled low-level feature maps and downsampled attention feature maps;

[0190] S430, For each level, the downsampled low-level feature map and the downsampled attention feature map are divided into multiple independent groups in the channel dimension, and independent fusion weights are learned for each group. The low-level features and attention features in the same group are fused according to the fusion weights to obtain the level fusion features.

[0191] S440: The hierarchical fusion features of the three levels are weighted and summed by the learnable hierarchical weights and then input into the convolutional layer for spatial integration, and the enhanced fusion features are output.

[0192] Specifically, channel alignment is performed on the low-level feature maps. If the number of input channels is different, a 1×1 convolution is used to adjust the number of channels to match the attention features. Figure 1 To ensure complete alignment of the two feature maps, bilinear interpolation is used to upsample the low-level feature maps to the spatial dimensions of the attention feature maps.

[0193] The pyramid-style grouping adaptive fusion module is described below:

[0194] like Figure 5As shown, the internal architecture of the pyramid-shaped grouping adaptive fusion module proposed in this embodiment replaces the original simple concat operation in the RT-DETR neck network. Through channel-group independent fusion and a pyramid-shaped multi-scale fusion strategy, it achieves refined adaptive fusion of high- and low-level features. The specific process is as follows:

[0195] 1. Feature Alignment: The module first performs spatial alignment on low-resolution features. If the number of input channels is different ( Then, the number of channels is adjusted to 1×1 convolution. :

[0196] ;

[0197] If the number of channels is equal, then the original features are directly retained. The low-resolution features are upsampled to the spatial size of the high-resolution features through bilinear interpolation, ensuring that the two feature maps are fully aligned in both spatial and channel dimensions, thus laying the foundation for subsequent fusion operations.

[0198] ;

[0199] at this time, and Having the same number of channels (in ) and spatial dimensions H×W.

[0200] in, The low-level features are the input. The number of channels for low-level features. For high-level features of the input, The number of channels for high-level features. These are the low-level features after channel alignment. For low-level features with consistent dimensions after alignment, H and W represent the height and width of the aligned feature map, respectively.

[0201] 2. Pyramid-style multi-level fusion mechanism: The hierarchical scale setting defines three scale factors. .

[0202] 3. Single-level fusion processing structure within the pyramid:

[0203] a. Feature downsampling:

[0204] For the l-th level (l=1,2,3), the features are first downsampled using average pooling:

[0205] ;

[0206] in, Indicates the kernel size as Step size is Average pooling, after downsampling, the spatial size is , . This is the low-level feature map after downsampling at level l. This is the high-level attention feature map after downsampling at level l. This operation obtains feature representations at different scales through downsampling, which can capture feature context information under different receptive fields and adapt to the detection needs of marine underwater organisms of different sizes.

[0207] In this embodiment, the pyramid-style multi-level fusion mechanism uses three scale factors: 1, 2, and 4. For each level, the features are first downsampled using average pooling. The spatial size after downsampling is the original size divided by the scale factor and rounded up.

[0208] b. Adaptive grouping fusion:

[0209] At each scale, a grouping adaptive mechanism is used to... and Perform fusion. Let the number of channels be C, and divide them into G groups (G=4), with each group having C / G channels.

[0210] ;

[0211] ;

[0212] in, This is the low-level feature sub-graph of the l-th level and the G-th group. For the high-level feature subgraph of the l-th level and the G-th group:

[0213] Each group learns an independent fusion weight. And restrict it to 360° using the Sigmoid function. Interval: ;

[0214] Then the first The fusion results of the groups are as follows: ;

[0215] The fusion results from each group are concatenated along the channel dimension to obtain the fusion features at this level:

[0216] .

[0217] c. Upsampling recovery features:

[0218] For l=1, Aligned with the original resolution; for l=2,3, it needs to be upsampled back to the original resolution via bilinear interpolation: .

[0219] In this embodiment, group adaptive fusion employs a group adaptive mechanism to fuse the downsampled low-level feature map and attention feature map at each scale. The number of channels is divided into multiple groups, each with an equal number of channels. Each group learns an independent fusion weight, which is constrained to the range of 0 to 1 using a sigmoid function. The fusion result for each group is a weighted fusion of the low-level features and attention features within the same group according to the fusion weights. The fusion results of each group are concatenated along the channel dimension to obtain the fused feature at that level.

[0220] For the original scale level, the fused features are aligned with the original resolution; for other levels, they need to be upsampled back to the original resolution through bilinear interpolation.

[0221] 4. Learnable hierarchical weighted fusion:

[0222] After the pyramid processing described above, three fusion results are obtained. (Original scale) (After upsampling at a 1 / 2 scale) and (After 1 / 4 scale upsampling). To adaptively fuse these three levels, learnable hierarchical weights are introduced. (Initialized to 1 / 3), and normalized using softmax:

[0223] ;

[0224] The final multi-scale fusion feature is: .

[0225] In this embodiment, learnable hierarchical weights are used for weighted fusion: after the pyramid processing described above, three fusion results are obtained. To adaptively fuse these three levels, learnable hierarchical weights are introduced and normalized using Softmax. The final multi-scale fusion feature is the sum of the products of the fusion features of each level and the normalized hierarchical weights.

[0226] 5. Feature Output: Finally, a 3×3 convolution is used to further integrate the fused features spatially and transform the channels, resulting in the module output: ;

[0227] in, The padding size for the 3×3 convolution is 1, which is the number of output channels, to ensure that the output space size is consistent with the input.

[0228] This embodiment achieves refined fusion of high- and low-level features through a pyramid-shaped grouping adaptive fusion module. The channel grouping mechanism learns independent fusion weights for each group, enabling more precise channel-level fusion control. The pyramid-shaped multi-scale fusion hierarchy performs feature fusion at different scales, allowing the model to dynamically adjust the fusion strategy based on the input scenario. Learnable hierarchical weights enable adaptive weighted fusion of multi-scale features, fully leveraging the complementary information between high- and low-level features and enhancing the model's adaptability to complex underwater marine environments.

[0229] Experimental verification:

[0230] This invention uses a marine underwater organism dataset as the experimental benchmark. This dataset consists of images collected by Roboflow from two aquariums in the United States: the Henry Dooley Zoo in Omaha and the National Aquarium in Baltimore. After secondary processing, the dataset was divided into training, validation, and test sets in a 7:1:2 ratio, containing seven categories: fish, jellyfish, penguins, dolphins, sharks, starfish, and stingrays. Each image is 768 pixels × 1024 pixels.

[0231] Experimental results show that after introducing the pyramid-shaped feature extraction module, the model improves the detection accuracy of small targets by 5.2% (measured by AP_small) and the localization accuracy of large targets by 3.5% (measured by AP_large) on the marine underwater biological target detection dataset, achieving accurate capture of targets at all scales. The pyramid-shaped dynamic sparse attention module improves the inference speed (FPS, batch size=4) by 185.3% compared to the standard multi-head self-attention mechanism, while maintaining or even improving feature representation capabilities, resulting in an overall detection accuracy improvement of 1.8% (measured by mAP50), meeting the real-time requirements of underwater robots and other edge devices. The pyramid-shaped grouping adaptive fusion module significantly enhances the model's adaptability to complex underwater environments, improving overall detection accuracy by 2.6% (measured by mAP50) while maintaining a relatively stable computational load and parameter count.

[0232] This invention constructs an end-to-end pyramid-based dynamic attention detection framework (Pyramid-DETR) through the cascaded optimization of three modules: a pyramid-based feature extraction module, a pyramid-based dynamic sparse attention module, and a pyramid-based grouping adaptive fusion module. These three modules form a synergistic enhancement mechanism: the pyramid-based feature extraction module extracts multi-scale pyramid features in the backbone network, providing a rich feature base for subsequent processing; the pyramid-based dynamic sparse attention module performs global context modeling of high-level features in the neck network, focusing on key detection regions; and the pyramid-based grouping adaptive fusion module achieves refined fusion of high- and low-level features during the feature fusion stage, preserving detailed information about small targets. The synergistic effect of these three modules enables the model to achieve end-to-end pyramid-based multi-scale enhancement from feature extraction to detection output.

[0233] Overall performance improvement: The three-module pyramid-shaped target detection Transformer (Pyramid-DETR) improves accuracy by 5.1% (measured by mAP50) compared to the original RT-DETR model, while reducing model computation by 23.7% and increasing inference speed by 156.2%, achieving high-precision and high-efficiency detection of marine underwater biological targets.

[0234] This invention also provides a marine underwater biological target detection system based on pyramid-shaped dynamic attention, comprising:

[0235] At least one processor;

[0236] At least one memory for storing at least one program;

[0237] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.

[0238] The content of the above method embodiments is applicable to this embodiment. The specific functions implemented in this embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. Therefore, they will not be repeated here.

[0239] This invention also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0240] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0241] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0242] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0243] This invention also provides a computer program product, including a computer program or computer instructions, which are stored in a memory. A processor of a computer device reads the computer program or computer instructions from the memory and executes the computer program or computer instructions, causing the computer device to perform the above-described method.

[0244] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0245] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0246] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0247] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A method for detecting underwater biological targets in the ocean based on pyramid-shaped dynamic attention, characterized in that, The method includes the following steps: The method acquires images of marine underwater organisms and a trained pyramid-shaped target detection network, which includes a backbone network composed of stacked pyramid-shaped feature extraction modules, a neck network containing a pyramid-shaped dynamic sparse attention module and a pyramid-shaped grouping adaptive fusion module, and a head detection network. The image of the marine underwater organism is input into the backbone network, and the pyramid-shaped feature extraction module extracts multi-scale local features in parallel and performs adaptive weighted fusion and cross-level feature interaction to output a multi-scale feature map. The highest-level feature map in the multi-scale feature map is input into the pyramid-shaped dynamic sparse attention module. A query matrix, a key matrix, and a value matrix are generated through channel segmentation. The key matrix and the value matrix are subjected to pyramid-shaped multi-level spatial downsampling and dynamic sparse attention calculation is performed to output the attention feature map. The attention feature map and the low-level feature map in the multi-scale feature map are input into the pyramid-shaped grouping adaptive fusion module. The pyramid-shaped multi-scale grouping adaptive fusion achieves fine fusion of high and low-level features and outputs enhanced fused features. The enhanced fusion features are input into the detection head to complete the target classification and bounding box regression, thereby obtaining the detection results of marine underwater biological targets.

2. The method according to claim 1, characterized in that, The process of extracting multi-scale local features in parallel through the pyramid-shaped feature extraction module and performing adaptive weighted fusion and cross-level feature interaction includes: The input feature map is input into the pyramid-shaped feature extraction module, and local features of different receptive fields are extracted by multiple parallel depthwise separable convolutions of different scales to obtain multi-scale local feature maps. The input feature map is compressed in space dimension by global average pooling to obtain channel-level global descriptors. The channel-level global descriptors are then processed by continuous convolution mapping and activation functions to generate normalized pyramid-level weights. The multi-scale local feature map is multiplied element-wise with the corresponding pyramid level weights and then summed to obtain the weighted fusion feature. The multi-scale local feature maps are concatenated along the channel dimension and then input into a fusion network containing convolution, batch normalization, and activation functions to perform cross-channel information fusion and obtain cross-level fusion features. The weighted fusion feature and the cross-level fusion feature are added together via residual connection to output the enhanced feature map.

3. The method according to claim 2, characterized in that, The extraction of local features from different receptive fields through parallel, multi-scale, depthwise separable convolutions includes: The input feature map is fed into three parallel depthwise separable convolutional branches with kernel sizes of 3×3, 5×5, and 7×7, respectively. Each branch extracts local features of the corresponding receptive field through channel-wise convolution and pointwise convolution, outputting three local feature maps of different scales. The stride of the three parallel depthwise separable convolutional branches is 1, and the padding is half the kernel size to ensure that the output spatial size is consistent with the input feature map.

4. The method according to claim 2, characterized in that, The step of compressing the spatial dimension of the input feature map using global average pooling to obtain a channel-level global descriptor, and then processing the channel-level global descriptor through continuous convolution mapping and activation functions to generate normalized pyramid-level weights, includes: Perform global average pooling on the input feature map to compress the spatial dimension to 1 and output a channel-level global descriptor; The channel-level global descriptor is input into the first 1×1 convolutional layer to reduce the number of channels to one-quarter of the original number of channels; The reduced-dimensional features are input into the second 1×1 convolutional layer, which expands the number of channels to the product of the original number of channels and the number of pyramid levels. The expanded features are mapped to the 0-1 interval using the Sigmoid activation function, and the normalized pyramid level weights are output.

5. The method according to claim 1, characterized in that, The step of inputting the highest-level feature map from the multi-scale feature map into the pyramid-shaped dynamic sparse attention module, generating a query matrix, a key matrix, and a value matrix through channel segmentation, performing pyramid-shaped multi-level spatial downsampling on the key matrix and value matrix, and executing dynamic sparse attention calculation includes: The highest-level feature map is divided into a first part of features and a second part of features along the channel dimension; The first part of the features is normalized and mapped by 1×1 convolution to generate a query matrix, a key matrix, and a value matrix; The key matrix and value matrix are subjected to three levels of average pooling spatial downsampling: original scale, 2x downsampling, and 4x downsampling, respectively, to obtain three levels of downsampled key matrix and downsampled value matrix. The query matrix is ​​scaled and dot product attention is calculated with the downsampling key matrix of each level to obtain the attention weight matrix. The highest value of the token subset in the attention weight matrix is ​​selected according to the dynamic sparsity. The selected attention weight matrix is ​​multiplied with the downsampling value matrix of the corresponding level to obtain the attention output features of the three levels. The attention output features of the three levels are weighted and fused by learnable hierarchical weights and cross-level interactions, then concatenated with the second part of the features and projected to output the attention feature map.

6. The method according to claim 5, characterized in that, The key matrix and value matrix are subjected to three levels of average pooling spatial downsampling: original scale, 2x downsampling, and 4x downsampling, respectively, to obtain three levels of downsampled key matrices and downsampled value matrices, including: Three pyramid levels are set up, each level corresponding to a scaling factor, and the scaling factors of the three pyramid levels are 1, 2 and 4 respectively; For each level, the key matrix and value matrix are respectively input into an average pooling layer with a kernel size equal to the scale factor and a stride equal to the scale factor, and spatial downsampling operation is performed. The downsampled key matrix and value matrix are flattened into a two-dimensional sequence to obtain three levels of downsampled key matrix and downsampled value matrix; wherein, the flattening operation compresses the spatial dimension to the token sequence dimension while keeping the channel dimension unchanged.

7. The method according to claim 5, characterized in that, The step of filtering the subset of tokens with the highest values ​​in the attention weight matrix based on the dynamic sparsity rate includes: Using the highest-level feature map as input, a gating network consisting of two 1×1 convolutional layers and a Sigmoid activation function is used to output a layer-level gating value. Multiply the level-based gate value by the total number of tokens at the corresponding level and round down to obtain the dynamically retained number; For each query position, select the key positions with the highest values ​​in the attention weight matrix, which is the same number as the number of dynamically retained key positions. Set the attention scores of the unselected key positions to negative infinity to obtain a sparse attention weight matrix. Perform Softmax normalization on the sparse attention weight matrix to output the normalized sparse attention weight matrix.

8. The method according to claim 1, characterized in that, The step of inputting the attention feature map and the low-level feature map from the multi-scale feature map into the pyramid-shaped grouping adaptive fusion module, and achieving refined fusion of high- and low-level features through pyramid-shaped multi-scale grouping adaptive fusion, includes: Channel alignment and spatial upsampling are performed on the low-level feature map to make the number of channels and spatial size of the low-level feature map consistent with those of the attention feature map; The low-level feature maps after channel alignment and spatial upsampling, along with the attention feature maps, are input into three levels of average pooling layers: original scale, 2x downsampling, and 4x downsampling, respectively, to obtain three levels of downsampled low-level feature maps and downsampled attention feature maps. For each level, the downsampled low-level feature map and the downsampled attention feature map are divided into multiple independent groups in the channel dimension. Independent fusion weights are learned for each group. The low-level features and attention features in the same group are fused according to the fusion weights to obtain the level fusion features. The three-level hierarchical fusion features are weighted and summed by learnable hierarchical weights and then input into a convolutional layer for spatial integration, outputting the enhanced fusion features.

9. A marine underwater biological target detection system based on pyramid-shaped dynamic attention, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images