Sonar image small target detection method based on improved RT-DETR

Abstract

This invention discloses a small target detection method for sonar images based on an improved RT-DETR, belonging to the field of intelligent underwater sonar image detection technology. Addressing the problems of weak texture, blurred edges, and strong background interference in small targets in sonar images, this invention constructs a convolutional enhancement module for local-global feature interaction within the hybrid encoder of the RT-DETR framework. This module enhances the local details and spatial response of the target before global modeling. Furthermore, a cross-scale spatial-channel fusion module is constructed to jointly refine features at different scales in both spatial and channel dimensions, improving the expressive power of multi-scale fused features for small targets. This invention is validated on the UATD-2022 dataset. Compared to the original model, the mAP50 is improved to 0.8422, and the recall rate is improved to 0.8658, effectively improving the accuracy and stability of small target detection in complex underwater environments.

Description

Technical Field

[0001] This invention belongs to the field of intelligent detection and target detection technology of underwater sonar images, specifically involving a method for small target detection in sonar images based on an improved RT-DETR. Background Technology

[0002] Sonar imagery is widely used in underwater target detection, underwater robot sensing, marine resource surveying, and underwater safety monitoring. Due to significant differences between sonar imaging mechanisms and visible light imaging, sonar images typically exhibit characteristics such as weak texture information, blurred edge contours, high background noise, uneven grayscale distribution, and low contrast between the target and background. In complex underwater environments, small targets are also easily affected by reverberation, scattering, shadows, and the complex seabed background, making it difficult for detection models to accurately extract target area features.

[0003] Existing target detection methods mostly employ convolutional neural networks or Transformer structures to complete image feature extraction and target localization. Among them, RT-DETR, as a real-time end-to-end target detection model, can directly predict target categories and bounding boxes using a detection decoder, exhibiting good detection efficiency and global modeling capabilities. However, in the scenario of small target detection in sonar images, directly using this model still has the following shortcomings: First, small targets are small in size, with weak edge and texture information, while the high-level interaction modules in the model focus more on global feature interactions and lack dedicated enhancement mechanisms for local details and spatial responses, which can easily lead to small target features being submerged by background noise in deep networks; Second, in the process of cross-scale feature fusion, it mainly relies on channel concatenation and ordinary convolution, failing to fully filter and refine the spatial details and channel responses of features at different scales, resulting in insufficient expressive power of the fused features for small targets, thus affecting detection accuracy and stability.

[0004] Therefore, while maintaining the advantages of RT-DETR end-to-end detection, it is necessary to design an improved scheme adapted to the characteristics of small targets in sonar, enhance the local-global representation capability of high-level features, and improve the response capability of cross-scale fusion features to small targets. Summary of the Invention

[0005] Based on RT-DETR, this invention provides a method for small target detection in sonar images based on improved RT-DETR, aiming to enhance the local detail representation, global context modeling, and cross-scale spatial-channel fusion capability of small targets in sonar images, thereby improving the accuracy and stability of small target detection in complex sonar backgrounds.

[0006] This invention is achieved through the following technical solution: A method for small target detection in sonar images based on an improved RT-DETR includes: acquiring a sonar image dataset and dividing it into training, validation, and test sets proportionally; constructing an improved RT-DETR network, which includes a feature extraction network, a hybrid encoder, and a detection decoder; extracting multi-scale features through the feature extraction network; constructing a convolutional enhancement module for local-global feature interaction in the hybrid encoder to replace the original high-level interaction module, thereby enhancing the local details and global context modeling of high-level features; constructing a cross-scale spatial-channel fusion module in the cross-scale fusion path of the hybrid encoder to jointly enhance features at different scales spatially and through channels; inputting the enhanced multi-scale fused features into the detection decoder and outputting the detection result; training the network using the training and validation sets and updating the parameters based on the detection loss; and using the trained model to perform target detection on the test set or the image to be detected.

[0007] Preferably, the convolution-enhanced local-global feature interaction module includes ordinary convolution, dilated convolution, dynamic spatial attention, multi-head self-attention, and feedforward transform units. Through local convolution fusion and spatial attention weighting, and then global self-attention modeling, local-global feature enhancement is achieved.

[0008] Preferably, the cross-scale spatial-channel fusion module includes cross-scale convolutional fusion, two-dimensional layer normalization, spatially enhanced depthwise convolution, channel-enhanced global average pooling and fully connected layers, as well as gated residual fusion, to achieve joint refinement of spatial details and channel responses.

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: by enhancing the local-global feature interaction module through convolution, the edge, texture and spatial response of small targets are enhanced before global modeling, avoiding the loss of information of high-level features; by using the cross-scale spatial-channel fusion module, the features of each scale are refined in both spatial and channel aspects, improving the sensitivity of multi-scale fused features to small targets; at the same time, the detection decoder structure remains unchanged, which is convenient to inherit the stability and adaptability of the original model. Attached Figure Description

[0010] Figure 1 The flowchart illustrates a small target detection method for sonar images based on an improved RT-DETR, as provided in this invention.

[0011] Figure 2 This is a schematic diagram of the detection network structure in a sonar image small target detection method based on an improved RT-DETR provided by the present invention.

[0012] Figure 3 This is a schematic diagram of the structure of the convolutional enhancement local-global feature interaction module of the present invention.

[0013] Figure 4This is a schematic diagram of the cross-scale space-channel fusion module structure of the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0015] Figure 1 This is a flowchart illustrating the method of the present invention. The present invention provides a method for small target detection in sonar images based on improved RT-DETR, specifically including the following steps: Step 1: Obtain the UATD-2022 sonar image dataset and perform data partitioning. To address the need for small target detection in sonar images, the UATD-2022 sonar image dataset undergoes format conversion, annotation consistency checks, and data partitioning to construct a small target detection dataset suitable for improving the RT-DETR network training and testing.

[0016] In this embodiment, the UATD-2022 sonar image dataset is divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The training set is used for model parameter learning, the validation set is used for performance evaluation and parameter selection during training, and the test set is used to verify the detection performance of the trained sonar image small target detection model.

[0017] Step 2: Construct a small target detection network for sonar images based on an improved RT-DETR. The overall network structure is as follows: Figure 2 As shown, it mainly consists of three parts: a feature extraction network, a hybrid encoder, and a detection decoder.

[0018] First, the input sonar image undergoes multi-stage feature extraction via a feature extraction network to obtain sonar image features at different downsampling scales. These features include P3, P4, and P5 scale features. The P3 scale features have high spatial resolution, preserving more edge and location information of small targets; the P4 scale features contain mid-level semantic information; and the P5 scale features contain high-level semantic information.

[0019] Its feature extraction process is represented as follows:

[0020] in, This indicates the input sonar image. This represents a feature extraction network. , , These represent sonar image features at different downsampling scales.

[0021] Next, the P5 high-level semantic features are input into the convolutional enhancement local-global feature interaction module to obtain the enhanced high-level semantic features. Subsequently, the enhanced high-level semantic features are fused with the P4 and P3 scale features sequentially from top to bottom, and multi-scale feature fusion continues in the bottom-up path to obtain the final multi-scale fused features of the input detection decoder. , , .

[0022] Finally, the multi-scale fused features are input into the RT-DETR detection decoder. The detection decoder retains the original RT-DETR-r18 detection decoding structure and is used to output the target category information, confidence information, and bounding box location information in the sonar image.

[0023] Step 3: Construct a convolutional enhancement module for local-global feature interaction. To enhance the expressive power of high-level sonar image features for weakly textured small targets, this invention constructs a convolutional enhancement module for local-global feature interaction in the hybrid encoder to replace the AIFI module in the original RT-DETR-r18. Its structure is as follows: Figure 3 As shown.

[0024] Let the high-level sonar image features input to the convolutional enhancement local-global feature interaction module be:

[0025] in, Indicates batch size. Indicates the number of channels. and These represent the height and width of the feature map, respectively.

[0026] First, input features Inputting the ordinary convolution branch and the dilated convolution branch respectively yields local texture features. and extended receptive field features :

[0027]

[0028] in, This represents a regular convolution branch, used to extract local texture and edge detail information from small sonar targets; This indicates a dilated convolution branch, used to expand the receptive field and extract contextual information around the target.

[0029] Local texture features and extended receptive field features By adding them together, we obtain the local convolutional fusion features:

[0030] At the same time, input features Input the dynamic spatial attention branch to obtain the spatial attention weights:

[0031] in, Represents the dynamic space attention branch. This represents a spatial attention map, used to enhance the response of the target region and suppress background noise interference.

[0032] Fuse features from local convolutions Spatial attention weights Element-wise multiplication is performed, and the convolutional enhanced features are obtained through gated residual connections:

[0033] in, This indicates element-wise multiplication. These are learnable gating parameters used to control the contribution of spatial attention enhancement features.

[0034] Subsequently, the convolutional enhancement features Flattening the data into sequence features and adding two-dimensional positional encoding. Global context modeling is performed by inputting multi-head self-attention units:

[0035]

[0036] in, This indicates multi-head self-attention computation. Presentation layer normalization operation, This represents a two-dimensional positional encoding.

[0037] After global context modeling, the features are input to the feedforward feature transformation unit for nonlinear mapping and feature reshaping.

[0038] in, This represents the feedforward feature transformation unit.

[0039] Finally, the feature sequence Restored to a two-dimensional feature map, the enhanced high-level semantic features are obtained:

[0040] Through the above methods, the convolutional enhancement local-global feature interaction module of the present invention can introduce local detail enhancement and spatial attention enhancement before global context modeling, so that high-level semantic features can simultaneously possess the ability to express local details and the ability to model global associations.

[0041] Step 4: Construct a cross-scale spatial-channel fusion module. To improve the information interaction capability between sonar image features at different scales, this invention constructs a cross-scale spatial-channel fusion module in the cross-scale fusion path of the hybrid encoder, the structure of which is as follows: Figure 4 As shown.

[0042] Let the input features after cross-scale concatenation be:

[0043] in, and This represents features from sonar images at different scales. This indicates feature splicing along the channel dimension.

[0044] First, the spliced ​​cross-scale features Perform cross-scale convolutional fusion and local feature reshaping:

[0045] in, This represents a cross-scale convolutional fusion unit.

[0046] Fusion features Perform two-dimensional layer normalization:

[0047] in, This indicates a two-dimensional layer normalization operation.

[0048] Subsequently, local spatial detail features are extracted through spatial augmentation branch:

[0049] in, This represents a depthwise convolution operation. This represents spatially enhanced features.

[0050] Simultaneously, channel-enhanced branches are generated to adjust channel weights:

[0051] in, Indicates global average pooling. and Indicates channel mapping parameters, Represents the ReLU activation function. This represents the hyperbolic tangent activation function. This indicates the channel adjustment weight.

[0052] The normalized features of the two-dimensional layer are multiplied element-wise with the channel adjustment weights to obtain the channel-enhanced features:

[0053] spatial enhancement features and channel enhancement features Perform element-by-element fusion:

[0054] Perform fusion features Convolutional renormalization:

[0055] Finally, the enhanced cross-scale fusion features are output using a gated residual method:

[0056] in, These are learnable gating parameters. Through this gating residual method, the module can approximately maintain the stability of the original feature distribution in the early stages of training, and gradually learn the contribution of spatial-channel augmented features to the output results as training progresses.

[0057] Furthermore, in a specific embodiment of the present invention, the cross-scale fusion path includes a top-down fusion path and a bottom-up fusion path, and its calculation process is as follows:

[0058]

[0059]

[0060]

[0061] in, This indicates a cross-scale space-channel fusion module. Indicates an upsampling operation. This indicates a downsampling operation. This represents the enhanced high-level semantic features. Indicates intermediate fusion features, , , This represents the multi-scale fusion features of the final input detection decoder.

[0062] Step 5, Model Training and Loss Function Calculation. The training and validation sets are input into the sonar image small target detection network for training. In this embodiment, the training batch size is set to 8, and the training cycle is set to 100 rounds. During training, the multi-scale fusion features obtained in Step 4 are... , , Input the RT-DETR detection decoder to obtain the prediction result.

[0063] The prediction result output by the detector decoder is represented as follows:

[0064] The prediction results include the target category, target confidence score, and target bounding box location, which can be represented as:

[0065] in, Indicates the first Category information of each predicted target, Indicates the first Confidence information for each prediction target Indicates the first The bounding box location of each predicted target. This indicates the predicted target quantity.

[0066] During training, a detection loss function is used to guide the backpropagation update of network parameters. The detection loss function includes object category classification loss, bounding box location regression loss, and intersection-union (IoU) loss.

[0067] in, This represents the target category classification loss. This represents the regression loss for the bounding box location. This indicates the generalized intersection and comparison loss. and This is the loss weighting coefficient.

[0068] Based on the detection loss, the network parameters are updated through backpropagation:

[0069] in, Indicates the first Network parameters at the next iteration This indicates the updated network parameters. This represents the learning rate.

[0070] Step 6: Perform small target detection in sonar images using the trained model. The trained sonar image small target detection model is used to detect targets on the test set or the sonar images to be detected. The sonar images to be detected sequentially pass through a feature extraction network, a convolutional enhancement local-global feature interaction module, a cross-scale spatial-channel fusion module, and an RT-DETR detection decoder to obtain the target detection results.

[0071] For the For each detection target, the detection result is expressed as follows:

[0072] in, Indicates the target category, Indicates the target confidence level. Indicates the location of the target bounding box.

[0073] The target bounding box position is represented as:

[0074] in, and Indicates the coordinates of the center point of the target bounding box. Indicates the width of the target bounding box. This represents the height of the target bounding box. Based on the detection results, a target detection box is generated on the sonar image to be detected, and a sonar target detection image with the target detection box is output.

[0075] Furthermore, ablation experiments were conducted to verify the improved RT-DETR sonar image small target detection model of this invention. The experiment used a batch size of 8 and a training cycle of 100 rounds. To verify the effectiveness of the convolutional enhancement local-global feature interaction module and the cross-scale spatial-channel fusion module, the Precision, Recall, F1-Score, mAP50, and mAP75 metrics of different models on the UATD-2022 test set were statistically analyzed. The comparison results are shown in Table 1 below.

[0076] Table 1: Ablation Experiment Results

[0077] As shown in Table 1, compared with the original RT-DETR-r18 model, the improved method proposed in this invention achieves improvements in metrics such as Precision, Recall, F1-Score, mAP50, and mAP75. Specifically, the mAP50 of the complete improved model increases from 0.8232 to 0.8422, and the Recall increases from 0.8294 to 0.8658, indicating that the convolutional enhancement of local-global feature interaction modules and the cross-scale spatial-channel fusion module can enhance the feature representation capability of small sonar targets and improve detection performance in complex backgrounds. The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A sonar image small target detection method based on improved RT-DETR, characterized in that, Includes the following steps: S1: Obtain the sonar image dataset and divide it into training set, validation set and test set according to a preset ratio; S2: Construct a small target detection network for sonar images based on an improved RT-DETR architecture, the network including a feature extraction network, a hybrid encoder, and a detection decoder; S3: Multi-scale feature extraction is performed on the input sonar image through the feature extraction network to obtain sonar image features at different downsampling scales; S4: A convolution-enhanced local-global feature interaction module is set in the hybrid encoder to perform local detail enhancement and global context modeling on high-level sonar image features; A cross-scale spatial-channel fusion module is set in the cross-scale fusion path of the hybrid encoder to jointly enhance the spatial and channel dimensions of sonar image features at different scales, thereby obtaining enhanced multi-scale fusion features. The enhanced multi-scale fusion features are input into the detection decoder, which outputs the target category, confidence level, and bounding box location information in the sonar image. S5: Train the detection network using the training set and validation set, update the network parameters based on backpropagation of the detection loss, and obtain the trained detection model; S6: Use the trained detection model to perform target detection on the test set or the sonar image to be detected, and output the detection results.

2. The method of claim 1, wherein, The convolution-enhanced local-global feature interaction module includes a regular convolution branch, a dilated convolution branch, a dynamic spatial attention branch, a multi-head self-attention unit, and a feedforward feature transformation unit; wherein... The ordinary convolution branch is used to extract local texture and edge detail information of the input features; The dilated convolution branch is used to expand the receptive field and extract contextual information around the target; The dynamic spatial attention branch is used to generate spatial attention weights, enhance the response of the target region, and suppress background interference. The multi-head self-attention unit is used to perform global context modeling on the convolution-enhanced features.

3. The method of claim 2, wherein, The convolutional enhancement local-global feature interaction module processes data in the following order: The input high-altitude sonar image features are processed by ordinary convolution and dilated convolution respectively to obtain local texture features and extended receptive field features; The local texture features are added to the extended receptive field features to obtain the local convolutional fusion features; The input features are processed through a dynamic spatial attention branch to obtain spatial attention weights; The local convolutional fusion feature is multiplied element-wise with the spatial attention weight, and then added to the original input feature through a gated residual connection to obtain the convolutional enhancement feature; The convolutional enhanced features are flattened into a sequence, and after adding two-dimensional positional encoding, they are sequentially passed through a multi-head self-attention unit and a feedforward feature transformation unit for global modeling, and finally restored to a two-dimensional feature map output.

4. The method of claim 1, wherein, The cross-scale spatial-channel fusion module includes a cross-scale convolutional fusion unit, a two-dimensional layer normalization unit, a spatial enhancement unit, a channel enhancement unit, and a gated residual fusion unit; wherein... The cross-scale convolutional fusion unit is used to perform convolutional reshaping on the spliced ​​multi-scale features. The two-dimensional layer normalization unit is used to stabilize the feature distribution; The spatial enhancement unit uses deep convolution to extract local spatial detail features; The channel enhancement unit uses global average pooling and a fully connected layer to generate channel adjustment weights to enhance effective channels and suppress redundant background channels. The gated residual fusion unit is used to perform residual superposition of the original fused features and the enhanced features through learnable parameters.

5. The method of claim 4, wherein, The cross-scale space-channel fusion module processes data in the following order: Features from sonar images at different scales are stitched together along the channel dimension to obtain cross-scale stitched features; The cross-scale spliced ​​features are sequentially subjected to convolutional fusion and local feature reshaping to obtain fused features; The fusion features are subjected to two-dimensional layer normalization. The normalized features of the layer are input into the spatial enhancement branch and the channel enhancement branch respectively to obtain spatial enhancement features and channel enhancement features; The spatial enhancement feature and the channel enhancement feature are multiplied element-wise, and after convolutional reshaping, the enhanced cross-scale fusion feature is added to the original fusion feature through the gated residual.

6. The method of claim 1, wherein, The cross-scale fusion path includes a top-down fusion path and a bottom-up fusion path, wherein... The top-down fusion path fuses the enhanced high-level semantic features with the mid-level and shallow-level sonar image features in sequence, which is used to transfer high-level semantic information to shallow-level detail features. The bottom-up fusion path sequentially passes shallow fusion features to higher levels and fuses them with intermediate features and high-level semantic features of the corresponding level, thereby supplementing the shallow spatial details to the high-level features.

7. The method according to claim 1, characterized in that, The detection decoder maintains the original RT-DETR decoding structure and is used to receive the enhanced multi-scale fusion features and output the prediction results; The detection loss functions used during training include target category classification loss, bounding box location regression loss, and generalized intersection-union loss, which are weighted sums as the total loss.

Images