An underwater target detection method based on RT-SonarNet

Abstract

This application belongs to the field of underwater target detection technology, specifically relating to an underwater target detection method based on RT-SonarNet. A backbone network collects and preprocesses underwater sonar datasets. The last three layers of feature maps output by the backbone network are denoted as high-level feature map, mid-level feature map, and low-level feature map. The high-level feature map is input to an intra-scale feature interaction module to optimize the intra-scale interaction of image features and output an enhanced feature map. The enhanced feature map, mid-level feature map, and low-level feature map are then input to a multi-scale feature adaptive fusion module for adaptive fusion, outputting a complete image feature sequence. The detection module introduces the EIoU bounding box regression loss function as a loss evaluation index for the model. The preprocessed sonar image is input into the model for underwater target detection. This application significantly reduces computational complexity while improving detection accuracy, and the processing speed meets the requirements for real-time underwater detection.

Description

Technical Field

[0001] This application belongs to the field of underwater target detection technology, specifically relating to an underwater target detection method based on RT-SonarNet. Background Technology

[0002] Underwater target detection plays a crucial role in various fields such as deep-sea exploration, offshore oil exploration, and marine development. Due to the complex underwater environment and the influence of factors such as natural light, light waves attenuate rapidly underwater, making it difficult for traditional optical imaging systems to obtain high-quality underwater images. Compared to light waves, sound waves attenuate the least in the ocean. Utilizing this characteristic, sonar imaging systems can obtain high-quality underwater images, making them ideal for underwater target detection. Furthermore, sonar images can capture and directly reflect comprehensive underwater information, making them an indispensable tool for underwater target detection tasks.

[0003] In recent years, researchers have focused on using sonar images for object detection. These methods can be categorized into traditional detection methods and deep learning-based methods. Traditional detection methods include image processing techniques such as filtering, edge detection, feature extraction, and template matching. These methods typically rely on manually designed features and rules, making them suitable for specific scenarios but less robust in complex environments. Deep learning-based methods automatically extract representative features from images through training, and then directly locate and classify targets based on these features, exhibiting stronger generalization capabilities compared to traditional methods. Although deep learning has made significant progress in object detection in natural images, many classic object detection algorithms, such as R-CNN, YOLO, and Transformer-based methods, have been widely applied to various visual tasks due to their superior feature extraction capabilities. However, when these methods are directly applied to sonar images, the results are far less ideal than with optical images. Sonar images have complex backgrounds, blurred target boundaries, and significant noise, making it difficult for existing deep learning models developed based on optical images to cope with these problems, especially in detecting small targets and distinguishing targets from noise in complex backgrounds.

[0004] The RT-DETR model, as an advanced Transformer-based object detection framework, excels in capturing long-range dependencies and handling multi-scale features in images thanks to its multi-head self-attention mechanism. This advantage enables RT-DETR to achieve good detection performance in various complex natural image object detection tasks. However, in sonar image object detection tasks, RT-DETR still faces challenges such as high computational complexity and difficulty in effectively distinguishing targets from the background. On the one hand, multi-head self-attention is computationally expensive when handling intra-scale feature interactions, especially when real-time detection is required, increasing time consumption. On the other hand, small targets in sonar images have inconspicuous features, leading to certain limitations in bounding box localization for the RT-DETR model.

[0005] The existing deep learning-based detection methods have the following problems: high computational complexity, difficulty in effectively distinguishing between targets and background; complex backgrounds in sonar images, blurred target boundaries, and low detection accuracy for small targets; and long detection time in real-time. Summary of the Invention

[0006] To address the problems existing in the prior art, this application provides an underwater target detection method based on RT-SonarNet.

[0007] The technical solution adopted by this application to solve its technical problem is as follows: an underwater target detection method based on RT-SonarNet. The RT-SonarNet model includes an intra-scale feature interaction module based on a cascaded group attention mechanism, a multi-scale feature adaptive fusion module, and a detection module. The steps are as follows: S1. The backbone network collects underwater sonar datasets and preprocesses them. The last three feature maps output by the backbone network are denoted as high-level feature map, middle-level feature map and low-level feature map. S2. The high-level feature map with the richest semantic information output by the backbone network is input into the intra-scale feature interaction module. The intra-scale interaction of image features is optimized through the cascaded group attention mechanism, and the enhanced feature map is output. S3. Input the enhanced feature map, the middle-level feature map and the low-level feature map into the multi-scale feature adaptive fusion module for adaptive fusion. Finally, output the three feature maps and perform deep fusion to output a complete image feature sequence. S4. The detection module introduces the EIoU bounding box regression loss function as the loss evaluation index of the model to improve the detection performance of RT-SonarNet model for multi-scale and small targets. S5. Input the preprocessed sonar image into the trained RT-SonarNet model to achieve underwater target detection.

[0008] Preferably, the cascaded attention mechanism includes: first, uniformly dividing the input image features into k feature subgroups in the channel dimension; then, implementing an attention mechanism within each feature subgroup to capture fine-grained detail relationships; except for the first subgroup, the input of each subsequent subgroup integrates the output features processed by the previous subgroup, so as to realize the layer-by-layer transmission and fusion of feature information.

[0009] Preferably, the mathematical representation of the output features of each feature subgroup is as follows: ; In the formula, It is the first The first block The self-attention output values ​​calculated for each group; It is the first The first input in the block Each group segmentation feature; It is the total number of cascaded groups, satisfying ; , and They are the first Each group corresponds to a query projection matrix, a key projection matrix, and a value projection matrix; It projects the concatenated features back to the parameters of a linear layer with the same dimension as the input features.

[0010] Preferably, the intra-scale feature interaction module, i.e. the CIFI module, takes image features as input. Each encoder layer sequentially includes a cascaded group attention module, a first residual connection and layer normalization, a feedforward network, and a second residual connection and layer normalization.

[0011] Preferably, the multi-scale feature adaptive fusion module includes a FAF submodule. The processing flow of the FAF submodule includes: first, performing bilinear interpolation upsampling on the low-resolution feature map to restore its original size and dimension and keep it consistent with the high-resolution feature map; then, compressing the features and using global average pooling to convert the two-dimensional feature map into a single pixel value with the same number of channels; finally, increasing the nonlinear fitting capability between channels through two one-dimensional convolutions and the Sigmoid activation function.

[0012] Preferably, the EIoU bounding box regression loss function is calculated using the following formula: ; In the formula, The bounding box regression loss function is... To compare the loss items, This is represented as the center point distance loss term. The loss term is the length of the width and height sides. To calculate the Euclidean distance between two center points or side lengths, in pixels. These are the coordinates of the center point of the prediction box, in pixels. These are the coordinates of the center point of the actual bounding box, in pixels. It is the diagonal length of the smallest closed region covering the predicted bounding box and the ground truth bounding box, in pixels. It is the width of the prediction box, in pixels. It is the height of the predicted bounding box, in pixels. This is the width of the actual bounding box, in pixels. It is the height of the actual bounding box, in pixels. It is the width of the smallest closed region covering the predicted bounding box and the ground truth bounding box, in pixels. It is the height of the smallest closed region covering the predicted bounding box and the ground truth bounding box, in pixels.

[0013] Preferably, the multi-scale feature adaptive fusion module, i.e., the MFM module, uses one-dimensional convolution instead of fully connected layers.

[0014] Preferably, the preprocessing involves performing histogram equalization and pseudo-color processing on the sonar images in the original dataset.

[0015] Compared with the prior art, the beneficial effects of this application are as follows: 1. This application applies deep learning to the field of sonar imaging and reasonably improves the network model. By introducing the Intra-Scale Feature Interaction (CIFI) module of Cascaded Group Attention, the Multi-Scale Feature Adaptive Fusion (MFM) module, and the EIoU loss function, the shortcomings of existing models in multi-scale feature interaction, target-background differentiation, and small target localization are effectively addressed.

[0016] 2. This application uses the Intra-Scale Feature Interaction (CIFI) module to enhance the semantic information of high-level features and improve the ability to express small target features. The CIFI module replaces the traditional global multi-head attention with a cascaded group attention mechanism, which reduces redundant information computation.

[0017] 3. This application uses the multi-scale adaptive fusion module MFM to achieve deep integration of features at different levels and enhance the extraction of small target features; MFM reduces the complexity of the attention mechanism by using one-dimensional convolution instead of fully connected layers.

[0018] 4. This application uses the EIoU loss function to provide smoother gradient information, accelerate model convergence, and improve bounding box regression accuracy. While ensuring detection accuracy, the number of parameters and FLOPs are significantly reduced compared to mainstream algorithms, and the computational efficiency is improved by more than 30%. In underwater low signal-to-noise ratio environments, the detection accuracy for small targets with blurred edges is improved by 5.3% compared to YOLOv8. It also has a high processing speed far exceeding the real-time standard of 30 FPS, meeting the needs of real-time underwater detection. Visual detection results show that the detection box has the highest degree of matching with the real label and can accurately capture small targets that are easily missed by other algorithms.

[0019] In summary, the underwater target detection method using the RT-SonarNet model in this application significantly reduces computational complexity while improving detection accuracy, and its processing speed meets the requirements for real-time underwater detection. It is suitable for practical operational scenarios such as underwater AUV navigation and obstacle avoidance where reliability requirements are stringent. Attached Figure Description

[0020] Figure 1 This is an architecture diagram of CIFI, a cascaded group attention-based intra-scale feature interaction module.

[0021] Figure 2 This is an architecture diagram of Cascaded Group Attention (CGA).

[0022] Figure 3 This is an architecture diagram of the Multi-Scale Feature Fusion (MFM) module.

[0023] Figure 4 This is an architecture diagram of the Feature Adaptive Fusion (FAF) module.

[0024] Figure 5 This is a diagram of the overall structure of RT-SonarNet.

[0025] Figure 6 This is a comparison chart of the visualization detection results of each method on the UATD-2022 dataset. Detailed Implementation

[0026] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. However, this application can be implemented in many different forms and is not limited to the embodiments described in this specification. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0027] Example 1: This application proposes an underwater target detection method based on RT-SonarNet, including the following steps: Step S1: Collect sonar datasets and preprocess them.

[0028] The public underwater acoustic target detection dataset UATD-2022 was collected from lakes and shallow water environments in Maoming and Dalian, China. This dataset contains over 9000 FLS images acquired using a Tritech Gemini 1200ik sonar. It provides raw and annotated data of the high-resolution FLS images, including ten target object classes such as cubes, cylinders, and tires. UATD-2022 provides two test datasets, "Test1" and "Test2," for baseline comparison.

[0029] To achieve better visual effects, this application performs histogram equalization and pseudo-color processing on the original dataset.

[0030] Step S2: Construct the Intra-Scale Feature Interaction Module (CIFI) based on Cascaded Group Attention in RT-SonarNet to optimize the intra-scale interaction of image features through the Cascaded Group Attention mechanism; like Figure 1 As shown, in its specific structure, CIFI uses image features as input. Each encoder layer of this module sequentially includes a Cascaded Group Attention (CGA) module, a first residual connection and layer normalization (Add & normal), a feedforward network (FFN), and a second residual connection and layer normalization. Through this architectural design, RT-SonarNet can achieve lightweight feature interaction with lower computational cost, thus replacing the traditional global multi-head attention mechanism.

[0031] Because the last three stages of the backbone network output feature maps of different scales (see...) Figure 5 (Understanding S3, S4, and S5), the feature map S5, which has the richest semantic information, is selected and input into the In-Scale Feature Interaction (CIFI) module. After processing, the output feature map is F5. To reduce interference from redundant information, CIFI only uses S5 and the corresponding spatial location code as the image feature input to the CIFI module.

[0032] The backbone network is a feature extraction network used to extract multi-dimensional feature information from the preprocessed sonar images. In this application, the backbone network outputs feature maps in three stages through layer-by-layer downsampling, denoted as low-level feature map S3, mid-level feature map S4, and high-level feature map S5, respectively. Among them, S5 is located in the deepest layer of the network structure and contains the richest semantic information, and is selected as the input of the Intra-Scale Feature Interaction (CIFI) module.

[0033] Cascaded Group Attention (CGA) is the core improvement component of this method, and its detailed internal cascade logic is as follows: Figure 2As shown. CGA first divides the input image features into k feature subgroups evenly along the channel dimension, corresponding to... Figure 2 The first group (head 1) to the kth group (head k) are defined in the image. Their cascading interaction mechanism is as follows: each subgroup represents a different feature space of the image, and an attention mechanism is applied within each group to capture fine-grained details. For example... Figure 2 As shown by the cascading arrows between the heads, except for the first subgroup, the input of each subsequent subgroup integrates the output features processed by the previous subgroup. Through this channel-level cascading progression, the model achieves the layer-by-layer transmission and fusion of feature information, thereby enabling the extraction of deep contextual information from the surface to the core. Its mathematical expression is as follows: ; In the formula, It is the first The first block The self-attention output value calculated for each group (head); It is the first The first input in the block Each segmentation feature; It is the total number of cascade groups (headers) that satisfy... ; , and They are the first The query projection matrix, key projection matrix, and value projection matrix corresponding to each head; It projects the concatenated features back to the parameters of a linear layer with the same dimension as the input features.

[0034] Step S3: RT-SonarNet constructs the Intra-Scale Feature Interaction Module (CIFI) by introducing the Cascaded Group Attention (CGA) mechanism to enhance and optimize the computation of high-level features of the backbone network. It also utilizes the adaptive fusion mechanism of the Multi-Scale Feature Adaptive Fusion Module (MFM) to complete the deep integration of feature maps of different dimensions.

[0035] The improved feature fusion module is the multi-scale feature adaptive fusion module MFM-Block. The MFM-Block structure is as follows: Figure 3 As shown, S3 and S4 are feature maps at different resolutions output by the backbone network, F5 is the feature map after passing through the CIFI module, and P1, P2, and P3 are the outputs of the adaptive multi-scale feature fusion module, used to detect targets of different sizes. The adaptive feature fusion module FAF, based on a channel attention mechanism, allows the network model to selectively enhance target region features and suppress background region features, improving the nonlinear fitting ability of the attention mechanism. Furthermore, it does not perform dimensionality reduction on fully connected layers, replacing them with one-dimensional convolutions to reduce the complexity of the attention mechanism and improve performance.

[0036] like Figure 4 The diagram illustrates the specific processing flow of the Feature Adaptive Fusion (FAF) module. To describe the internal logic of this module, C1 represents the input low-resolution feature map to be processed, and C2 represents the input high-resolution feature map to be processed. These correspond to... Figure 3 The two input branches enter the FAF block. First, the feature map C1 is upsampled by bilinear interpolation to restore its original size and dimension and keep it consistent with the feature map C2. Then, the features are compressed, and global average pooling (GAP) is used to convert the two-dimensional feature map into a single pixel value with the same number of channels, so that each channel of the output feature has global receptive field information. Finally, two one-dimensional convolutions and a sigmoid activation function are used to increase the nonlinear fitting ability between channels.

[0037] like Figure 5 The diagram shows the RT-SonarNet network structure. Its overall process is as follows: The backbone network is selected to extract features from the sonar image and outputs feature maps for the last three stages, defined as S3, S4, and S5, which serve as input to the encoder. The encoder contains cascaded CIFI and MFM modules. The high-level feature map S5 first enters the intra-scale feature interaction module CIFI for semantic enhancement and outputs the enhanced feature map F5. F5, along with the mid- and low-level feature maps S3 and S4 directly output from the backbone network, are input to the multi-scale feature fusion module (MFM) for deep integration, thus forming a complete image feature sequence.

[0038] Step S4: Introduce the EIoU bounding box regression loss function as the loss evaluation metric for RT-SonarNet to improve the detection accuracy of small targets.

[0039] EIoU provides smoother gradient information and can significantly accelerate model convergence during training, reducing the optimization difficulty of the model in highly complex tasks. When dealing with non-overlapping regions, EIoU can more effectively adjust the position and size of the bounding boxes, ensuring that the bounding boxes gradually approach the true location, thereby improving detection accuracy. Its calculation formula is as follows: ; In the formula, The bounding box regression loss function is... To compare the loss items, This is represented as the center point distance loss term. The loss term is the length of the width and height sides. To calculate the Euclidean distance (in pixels) between two center points or side lengths. and These are the center coordinates of the predicted bounding box and the ground truth bounding box, respectively (unit: pixels). It is the diagonal length of the smallest closed region covering the predicted bounding box and the ground truth bounding box (unit: pixels). and These are the width and height of the prediction box (unit: pixels). and These are the width and height of the actual bounding box (unit: pixels). and These are the width and height of the smallest closed region covering the predicted bounding box and the ground truth bounding box, respectively (unit: pixels).

[0040] To test the detection performance of RT-SonarNet in this embodiment, RT-SonarNet was compared with seven existing methods: YOLOv7, YOLOv8, DETR, Deformable-DETR, DN-DETR, DINO, and RT-DETR, using the UATD-2022 dataset on an AMD Ryzen 9 5900X, NVIDIA GeForce RTX 3080 Ti, and Ubuntu 18.04 platform. The experimental results are as follows: Figure 6 As shown in Table 1. In Table 1, Ours represents RT-SonarNet of this application.

[0041] Table 1. Visual detection results of this application's RT-SonarNet compared with seven other methods. In the evaluation metrics, precision, F1 score, and mAP@0.5 reflect the model's detection accuracy, while recall reflects the model's coverage of the target; parameters and FLOPs reflect the model's computational complexity, and FPS reflects the detection speed. The results show that, except for a slightly lower recall compared to DINO, this application significantly outperforms other comparative algorithms in key metrics.

[0042] This application demonstrates significant advantages in balancing lightweight design and high accuracy: First, in terms of computational complexity, RT-SonarNet reduces the number of parameters (12.1M) and computational cost (14.3G) by 34.2% and 34.7% respectively compared to YOLOv8, exhibiting extremely high computational efficiency. Second, in terms of accuracy, RT-SonarNet achieves an mAP@0.5 of 86.7%, a significant improvement of 5.3 percentage points compared to YOLOv8. This improvement is crucial for ensuring AUV navigation safety in underwater sonar missions with extremely low signal-to-noise ratios. It is also noteworthy that this 5.3% improvement is achieved while simultaneously reducing the number of parameters and computational cost by approximately 34%, resulting in a remarkable turnaround in accuracy.

[0043] Figure 6 A comparison of visual detection algorithms is presented, with Figures (a) to (g) corresponding to the seven algorithms in Table 1, Figure (h) representing the algorithm of this application, and Figure (i) representing the ground truth. The comparison shows that the detection bounding box of the algorithm (h) of this application best matches the ground truth (i), and it can more accurately capture small, blurry targets with edges that are easily missed by other algorithms. For example, in scenarios where AUVs perform navigation tasks, accurate identification and obstacle avoidance capabilities are crucial. Although YOLOv8 leads in FPS, RT-SonarNet, with a high processing speed of 92.34 FPS (far exceeding the real-time standard of 30 FPS), achieves a significant improvement in key accuracy (5.3% improvement in mAP@0.5) with a lighter computational load, making it more suitable for underwater operating environments with extremely stringent reliability requirements compared to YOLOv8.

[0044] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. An underwater target detection method based on RT-SonarNet, characterized in that, The RT-SonarNet model includes an intra-scale feature interaction module (CIFI) based on a cascaded group attention mechanism, a multi-scale feature adaptive fusion module (MFM), and a detection module. The steps are as follows: S1. The backbone network collects underwater sonar datasets and preprocesses them. The last three feature maps output by the backbone network are denoted as high-level feature map S5, middle-level feature map S4, and low-level feature map S3. S2, the high-level feature map S5 with the richest semantic information output by the backbone network is input to the CIFI module based on the cascaded group attention mechanism. The CIFI module only uses S5 and the corresponding spatial location code as image feature input, and performs intra-scale feature interaction on the high-level feature map S5 through the cascaded group attention mechanism to output the enhanced feature map F5. S3. Input the enhanced feature map F5, the middle-layer feature map S4 and the low-layer feature map S3 into the MFM module for adaptive fusion. Finally, output the three feature maps and perform deep fusion to output a complete image feature sequence. The MFM module includes a FAF submodule. The processing flow of the FAF submodule includes: first, performing bilinear interpolation upsampling on the low-resolution feature map to restore its original size and dimension and maintain consistency with the high-resolution feature map; then, compressing the features and using global average pooling to convert the two-dimensional feature map into a single pixel value with the same number of channels; finally, increasing the non-linear fitting capability between channels through two one-dimensional convolutions and a sigmoid activation function; the MFM module uses one-dimensional convolutions instead of fully connected layers. S4. The detection module introduces the EIoU bounding box regression loss function as the loss evaluation index of the model to improve the detection performance of RT-SonarNet model for multi-scale and small targets. S5. Input the preprocessed sonar image into the trained RT-SonarNet model to achieve underwater target detection.

2. The underwater target detection method based on RT-SonarNet according to claim 1, characterized in that, The cascaded attention mechanism includes: first, uniformly dividing the input image features into k feature subgroups in the channel dimension; then, implementing an attention mechanism within each feature subgroup to capture fine-grained details; except for the first subgroup, the input of each subsequent subgroup integrates the output features processed by the previous subgroup, so as to achieve the layer-by-layer transmission and fusion of feature information.

3. The underwater target detection method based on RT-SonarNet according to claim 2, characterized in that, The mathematical representation of the output features of each feature subgroup is as follows: ； In the formula, It is the first The first block The self-attention output values ​​calculated for each group; It is the first The first input in the block Each group segmentation feature; It is the total number of cascaded groups, satisfying ; , and They are the first Each group corresponds to a query projection matrix, a key projection matrix, and a value projection matrix; It projects the concatenated features back to the parameters of a linear layer with the same dimension as the input features.

4. The underwater target detection method based on RT-SonarNet according to claim 2, characterized in that, The CIFI module uses image features as input. Each encoder layer sequentially includes a cascaded attention module, a first residual connection and layer normalization, a feedforward network, and a second residual connection and layer normalization.

5. The underwater target detection method based on RT-SonarNet according to claim 1, characterized in that, The formula for calculating the EIoU bounding box regression loss function is as follows: ； In the formula, The bounding box regression loss function is... To compare the loss items, This is represented as the center point distance loss term. The loss term is the length of the width and height sides. To calculate the Euclidean distance between two center points or side lengths; These are the coordinates of the center point of the prediction box; These are the coordinates of the center point of the actual bounding box; It is the diagonal length of the smallest closed region covering the predicted bounding box and the ground truth bounding box; It is the width of the prediction box; It is the height of the predicted bounding box; It is the width of the actual frame; It is the actual height of the bounding box; It is the width of the smallest closed region that covers the predicted bounding box and the ground truth bounding box; It is the height of the smallest closed region that covers the predicted bounding box and the ground truth bounding box.

6. The underwater target detection method based on RT-SonarNet according to claim 1, characterized in that, The preprocessing involves performing histogram equalization and pseudo-color processing on the sonar images in the original dataset.

Images