An underwater target detection system and method based on RT-DETR improvement

Abstract

The application provides an underwater target detection system and method based on RT-DETR improvement, and belongs to the technical field of underwater target detection based on computer vision. The system comprises a backbone network module, a hybrid encoder module, an InnerMPDIoU perception module and a decoder module; the backbone network module is based on a large separable kernel attention, performs large separable kernel attention calculation after feature extraction, and outputs a multi-layer feature map; the hybrid encoder module design comprises a scale inner feature interaction unit based on a deformable large kernel attention and an improved cross-scale feature fusion unit; the perception module introduces IoU calculation of the original IoU perception module into a scale factor and an auxiliary bounding box, and improves target query initialization by using query selection of InnerMPDIoU perception. The model improvement of the application is adapted to the particularity of the underwater environment, improves the underwater operation efficiency, realizes parameter lightening, and increases the accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater target detection based on computer vision, and in particular relates to an underwater target detection system and method based on improved RT-DETR. Background Art

[0002] In recent years, underwater target recognition has been a key research area in underwater deep learning. Underwater target detection can detect and monitor underwater resources, playing a crucial role in their development, utilization, and protection. However, due to the complexity of the underwater environment, the effects of attenuation, scattering, and noise, coupled with the scarcity of datasets and the varying sizes of targets, target recognition accuracy has not been effectively improved. Underwater target recognition has always been a challenging task due to its complexity and the diversity of the underwater environment. Traditional underwater target detection methods are mostly based on feature extraction and classification. Due to the unique characteristics of the underwater environment, this approach limits the accuracy and universality of target recognition, especially in complex underwater environments.

[0003] To address this issue, numerous studies are currently focusing on applying deep learning techniques to mub recognition in complex underwater environments. Current mainstream underwater object recognition networks, primarily the Yolo series, primarily utilize anchor boxes and non-maximum suppression (NMS) post-processing to remove overlapping prediction boxes. However, the NMS process relies on two key hyperparameters: the score threshold and the intersection-over-union (IoU) threshold, both of which impact detection accuracy and speed. DETR is an object detection method based on the Transformer architecture that avoids NMS and instead processes image features and location information through self-attention and cross-attention mechanisms. RT-DETR is an optimization of DETR, achieving real-time processing while maintaining high accuracy, addressing the slow performance of traditional DETR. However, RT-DETR has yet to be effectively applied in underwater object recognition. Due to the mix of large and small underwater objects, blurring, and small datasets, RT-DETR cannot achieve excellent performance in this area. Furthermore, RT-DETR has a large number of parameters and is not lightweight. Summary of the Invention

[0004] To address the above problems, the present invention is optimized based on RT-DETR, which improves underwater operation efficiency, achieves parameter lightweighting while increasing accuracy, adapts to the particularities of the underwater environment, and enhances small target detection.

[0005] The first aspect of the application provides an underwater target detection system based on RT-DETR improvement, comprising a backbone network module, a hybrid encoder module, an InnerMPDIoU perception module and a decoder module;

[0006] The backbone network module is based on large separable kernel attention, constructs multiple layers of benchmark modules, uses preprocessed underwater target images as input, performs large separable kernel attention calculation after feature extraction, and outputs multiple layers of feature maps;

[0007] The hybrid encoder module includes a scale-in feature interaction unit of deformable large kernel attention and a cross-scale feature fusion unit based on a convolutional neural network; the scale-in feature interaction unit uses deformable large kernel attention mechanism to encode the features of the uppermost layer output by the backbone network module; the cross-scale feature fusion unit performs skip connection, introduces the output of the fourth layer benchmark module from top to bottom in the backbone network for detail feature fusion on the basis of the original three-layer feature fusion network, and introduces wavelet transform to expand the receptive field without significantly increasing the parameters; the up-sampling unit is replaced by a DySample dynamic up-sampling unit, and the down-sampling unit is replaced by a lightweight down-sampling unit, and a fusion mechanism is used to combine features of different scales to output the feature representation after fusion processing;

[0008] The InnerMPDIoU perception module introduces the IoU calculation of the original IoU perception module into the scale factor and the auxiliary bounding box, and improves the target query initialization by using the query selection of InnerMPDIoU perception;

[0009] The decoder module is used to convert the output of the hybrid encoder module into the final detection result, including the bounding box coordinates and class prediction.

[0010] Preferably, the backbone network module uses a Resnet50 network, adds a large separable kernel attention module in the four-layer bottleneck layer structure, constructs four layers of benchmark modules, and is used to extract features in four stages to generate features of different levels; the first branch of each benchmark module includes a 1*1 convolution layer, a batch normalization layer, an activation function layer, a 3*3 convolution layer, a batch normalization layer, an activation function layer, a 1*1 convolution layer, a batch normalization layer and a large separable kernel attention module, so as to increase the large separable kernel attention in multiple layers of features and obtain more semantic information; the second branch includes an average pooling layer, a 1*1 convolution layer and a batch normalization layer; the first branch and the second branch are added element by element and output through an activation function layer, and the channel sizes of the four modules are 64, 128, 256 and 512 respectively, which are represented as:

[0011]

[0012] in, Indicates the The features of the layer are calculated by large separable kernel attention, Represents an ensemble of large separable kernel attentions.

[0013] Preferably, the large separable kernel attention module is specifically:

[0014] The feature map L of dimension n×c×w×h undergoes a 1×3 depth convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×1), and then a 3×1 depth convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, and the padding is 1×0), and then a 1×3 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×2, and the expansion is 2×2), and then a 3×1 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, and the padding is 2×0, and the expansion is 2×2), and then a 1×1 convolution operation (the number of input and output channels is c, and the convolution kernel size is 1×1), the result is element-wise multiplied with the input feature map L to obtain the output feature map L', with dimension n×c×w×h.

[0015] Preferably, the deformable large-core attention mechanism uses multiple convolutional layers, including standard 2D convolution and deformable convolution with offset; the deformable convolution with offset allows the network to adaptively adjust the receptive field according to the input features, and internally includes the calculation of the offset field to guide the deformable convolution layer to adjust the sampling position;

[0016] The input of the deformable large kernel attention mechanism undergoes 2D convolution, then passes through the GELU activation function, and then passes through the deformed convolution with an offset and then through the 2D convolution. It is added to the value without the deformed convolution and then passes through the 2D convolution and output after being fused with the input.

[0017] Preferably, the DySample dynamic upsampling unit is specifically:

[0018] Input each feature map or the fused feature map with dimensions C×H×W and upsampling scale factor s, and use a linear layer with input and output channels C and , the generated size is ×H×W offset is multiplied by the coefficient m to obtain the offset O, which represents the displacement of the sampling point relative to the original grid point; then, it is reshaped into 2×sH×sW dimensions by pixel shuffling, and the offset is converted to a suitable spatial resolution. The final sampling point coordinates are obtained by weighted addition of the original grid point coordinates and the offset; the sampling operation is performed by the grid_sample function, and the feature map is resampled according to the generated sampling points to obtain a feature map with an upsampled dimension of C×sH×sW. The sampling set S is the sum of the offset O and the original sampling grid G:

[0019]

[0020]

[0021] Among them, O is the offset, X is the input feature network, G is the original sampling network, S is the sampling set, and m is the weighting coefficient.

[0022] Preferably, the lightweight downsampling unit is specifically:

[0023] Input each fused feature map, divide the data into two parts after average pooling, and divide it into two branches. The first branch undergoes 3×3 convolution, and the second branch undergoes maximum pooling and 1×1 convolution. The results of the first and second branches are connected and output to the next unit, which is the downsampled feature map.

[0024] Preferably, the introduction of wavelet transformation is to replace the convolution module in the reparameterized convolution RepC3 structure with a wavelet transform module, including N dilated convolutions followed by wavelet transform convolutions, wherein the wavelet transform convolution includes a wavelet transform filter:

[0025]

[0026]

[0027]

[0028]

[0029] After the filter, the image is divided into four parts. (Low-Low) represents the filter of the low-frequency part. The part obtained by this filter contains the main information and energy of the image and is used for image compression and feature extraction;

[0030] ‌ ‌ (Low-High) represents the filter from low frequency to high frequency. The part obtained by this filter mainly contains horizontal features.

[0031] ‌ (High-Low) represents a filter of high frequency to low frequency part, the part obtained through the filter mainly contains the characteristics of the vertical direction;

[0032] ‌ (High-High) represents a filter of high frequency part, the part obtained through the filter contains the least information, which is used for noise processing;

[0033] Then after the depth convolution with a step of 2, it is divided into two branches, the second branch of which is added to the transposed convolution after the two-dimensional convolution of the first branch, the first branch takes the LL part and continues to pass through the filter and then is divided into two branches after the depth convolution with a step of 2, the first branch continues to pass through the filter and repeat the operation, and the second branch is added to the transposed convolution of the next layer after the two-dimensional convolution, and then is added to the output of the second branch of the previous layer after the transposed convolution, and finally is added to the two-dimensional convolution of the input to obtain the output, that is, the flattened feature map.

[0034] Preferably, the calculation method of the InnerMPDIoU perception module loss function is:

[0035] Suppose the real boundary box: ) , wherein: and are the center points of the edge frame, and are the width and height,

[0036] The edge frame with the introduction of the ratio scale factor is respectively:

[0037] ，

[0038] ，

[0039] , wherein, is the four vertex coordinate values of the real boundary box after the scale factor processing, is the left axis coordinate value, is the right axis coordinate value, is the upper axis coordinate value, is the lower axis coordinate value; is the boundary box after the scale factor processing Four coordinate values of the fixed point, is the left axis coordinate value, is the right axis coordinate value, is above axis coordinate value, is below axis coordinate value;

[0040] The IoU loss calculation based on the scale factor is:

[0041]

[0042]

[0043]

[0044] wherein inner represents an internal intersection, union represents a union, and innerIoU represents an IoU loss based on an auxiliary bounding box;

[0045] The IoU loss calculation based on the auxiliary bounding box and the scale factor is:

[0046]

[0047] wherein, represents an IoU loss introducing an auxiliary bounding box;

[0048] The loss function is:

[0049]

[0050] wherein, is a loss function result, used to measure the difference between the model prediction value and the true label.

[0051] The second aspect of the present application proposes an underwater target detection method based on RT-DETR improvement, which applies the underwater target detection system as described in the first aspect, and includes the following processes:

[0052] Real underwater images are captured and obtained;

[0053] The images are preprocessed, including pixel value normalization, smoothing processing, and image enhancement processing;

[0054] The preprocessed images are input into the underwater target detection system;

[0055] The underwater target detection result is output.

[0056] Compared with the prior art, the present application has the following beneficial effects:

[0057] The present invention introduces a large separable kernel attention mechanism, which effectively combines the advantages of the kernel attention mechanism, improves performance, improves computing performance and reduces memory usage, and optimizes the upsampling and downsampling mechanisms of the original model to significantly reduce the number of parameters and improve accuracy. At the same time, it adds deformable large kernel attention to adapt to complex backgrounds and different lighting conditions. Combining the technology of large convolution kernels and deformable convolution, the convolution kernel is dynamically adjusted to enhance the adaptability of underwater target detection, especially for small targets. In order to expand the receptive field of the model, the wavelet transform is introduced to achieve parameter lightweighting, solve the problem that traditional CNN is limited by the convolution size and cannot effectively capture the global context, and that the expansion of the receptive field of traditional CNN will lead to parameter expansion. At the same time, the InnerMPDIoU is introduced to improve the training accuracy and speed of the model. Through the above operations, the perception ability of the model is improved while ensuring the lightweight of parameters, thereby improving the accuracy and robustness of underwater target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, what is described below is only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0059] Figure 1 This is a block diagram of the overall structure of the underwater target detection system of the present invention.

[0060] Figure 2 This is a schematic diagram of the backbone network structure of the present invention.

[0061] Figure 3 Schematic diagram of the structure of the large separable core attention module of the present invention.

[0062] Figure 4 Schematic diagram of the network structure of the deformable large-core attention mechanism of the present invention.

[0063] Figure 5 This is a schematic diagram of the structure of the CCFM of the present invention introducing the output of the stage 2 reference module of the backbone network.

[0064] Figure 6 This is a data processing flow chart of the cross-scale feature fusion unit CCFM of the present invention.

[0065] Figure 7 This is a schematic diagram of the DySample dynamic upsampling unit module structure of the present invention.

[0066] Figure 8 Schematic diagram of the lightweight downsampling unit structure of the present invention.

[0067] Figure 9 This is a schematic diagram of the RepC3 structure that introduces wavelet transform into the present invention.

[0068] Figure 10 Schematic diagram of the network structure of the wavelet convolution module of the present invention.

[0069] Figure 11 This is the underwater target recognition effect diagram of the present invention. DETAILED DESCRIPTION

[0070] The present invention provides an underwater target detection system based on RT-DETR improvement, the overall structure of which is as follows: Figure 1 As shown in the figure, it includes a backbone network based on a large separable kernel attention module, an efficient hybrid encoder that combines deformable large kernel attention with the introduction of wavelet transform and improved up- and down-sampling mechanism, an InnerMPDIoU perception module, and a decoder module.

[0071] The backbone network, based on a large separable kernel attention module, takes preprocessed underwater target images as input, extracts features, performs large separable kernel attention calculations, and outputs multi-layer feature maps. This fully utilizes multi-semantic information to improve the performance of target recognition tasks.

[0072] The hybrid encoder module combines deformable large-kernel attention with a hybrid encoder module that introduces wavelet transforms and improves up- and down-sampling mechanisms. Its input is features at different scales of the image output by the backbone network, and its output is a feature representation of the processed image. The hybrid encoder module design consists of two main components: an intra-scale feature interaction unit based on deformable large-kernel attention and a cross-scale feature fusion unit (CCFM) based on a convolutional neural network. The intra-scale interaction is processed by the deformable large-kernel attention encoder for the features at the top-level scale of the backbone network output, enabling more efficient capture of target information. After processing the intra-scale interactions of the top-level features, the cross-scale fusion unit (CCFM) uses an efficient fusion mechanism to merge features at different scales, achieving a holistic understanding of objects of varying sizes. The final output is the fused feature representation. The present invention reduces computational redundancy and improves feature extraction efficiency by improving the encoder. The improvements include adopting a deformable large-core attention to adapt to complex backgrounds and different lighting conditions in intra-scale feature interaction, adding a p2 detection head to the cross-scale feature fusion module, and introducing a wavelet transform to expand the receptive field without significantly increasing parameters. The p2 detection head refers to the jump connection of the cross-scale feature fusion unit. On the basis of the original three-layer feature fusion network, the output of the reference module of the fourth layer from top to bottom in the backbone network is introduced to perform detail feature fusion. At the same time, upsampling is changed to dynamic upsampling, and downsampling is optimized to achieve lightweight downsampling, which greatly reduces the number of downsampling parameters.

[0073] The InnerMPDIoU-aware module optimizes the query selection process by considering the Intersection over Union (IoU) score to select features that are more likely to describe the true object. InnerMPDIoU-aware query selection provides a higher-quality initial object query to the decoder by providing an IoU constraint, further improving object detection performance. Its input is the output of the efficient hybrid encoder, which is the image feature representation processed by the efficient hybrid encoder. It then selects a fixed number of image features from the output sequence of the efficient hybrid encoder as its output, which serves as the decoder's initial object query. The InnerMPDIoU-aware module optimizes the model's detection performance by producing high classification scores for features with high InnerMPDIoU and low classification scores for features with low InnerMPDIoU during training. The InnerMPDIoU-aware module introduces a scaling factor and auxiliary bounding boxes into the IoU calculation of the original IoU-aware module, and improves object query initialization through InnerMPDIoU-aware query selection.

[0074] The decoder module is responsible for converting the output of the hybrid encoder module into the final detection results, including bounding box coordinates and category predictions. Its input is the encoder output, and after selecting good features, it is decoded and output is bounding box coordinates and category predictions.

[0075] The invention will be further described below with reference to specific embodiments.

[0076] 1. Data preprocessing

[0077] Preprocessing includes but is not limited to image denoising, image resizing, pixel value normalization, contrast enhancement, brightness adjustment, and histogram equalization, etc., aiming to improve image quality. The preprocessing process is as follows:

[0078] After acquiring real underwater images, the image size is adjusted according to the image size, and the pixel values ​​are normalized to eliminate the brightness differences between different images and improve the accuracy and stability of model training. Gaussian filtering is then used to smooth the image. The value of each pixel is obtained by weighted averaging its own value and the values ​​of other pixels in its neighborhood, improving the image quality and clarity. Contrast enhancement, brightness adjustment, and histogram equalization are then used to enhance the image, making the features and details more prominent.

[0079] 2. Backbone network module

[0080] The present invention adds a large separable kernel attention mechanism to the backbone network module, aiming to cleverly utilize the large separable kernel to enhance the attention mechanism, fully utilize multi-semantic information, greatly improve the efficiency and accuracy of image recognition and processing, and thus improve the performance of target recognition tasks. Its overall structure is as follows Figure 2As shown in the figure, a large separable kernel attention module is added to the backbone network. Taking Resnet50 as an example, the bottleneck layer structure in Resnet50 is added with a large separable kernel attention module as the four-layer benchmark module of stage2, stage3, stage4 and stage5 of the backbone network Resnet50. The backbone network is used to extract features of the four stages and generate features of different levels, which can be expressed as:

[0081]

[0082] in Indicates that the The feature information extracted by the layer, F represents A collection of .

[0083] A large separable kernel attention module is added to the above four bottleneck layers. The bottleneck layer represents the feature Through the large separable kernel attention module, the first branch of each module includes a 1*1 convolution layer, a batch normalization layer, an activation function layer, a 3*3 convolution layer, a batch normalization layer, an activation function layer, and then a 1*1 convolution layer, a batch normalization layer, and a large separable kernel attention module to add large separable kernel attention to the multi-layer features and obtain more multi-semantic information. Its second branch includes an average pooling layer, a 1*1 convolution layer, and a batch normalization layer. The first and second branches are added element by element and then output through the activation function layer. The structure is as follows Figure 1 ,After this module, the channel sizes of the four benchmark modules are 64, 128, 256, and 512, respectively, expressed as:

[0084]

[0085] in, Indicates the The features of the layer are calculated by large separable kernel attention, Represents an ensemble of large separable kernel attentions.

[0086] Among them, the large separable kernel attention module structure is as follows Figure 3 shown.

[0087] The feature map L of dimension n×c×w×h undergoes a 1×3 depth convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×1), and then a 3×1 depth convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, and the padding is 1×0), and then a 1×3 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×2, and the expansion is 2×2), and then a 3×1 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, and the padding is 2×0, and the expansion is 2×2), and then a 1×1 convolution operation (the number of input and output channels is c, and the convolution kernel size is 1×1), the result is element-wise multiplied with the input feature map L to obtain the output feature map L', with dimension n×c×w×h.

[0088] 3. Hybrid encoder module

[0089] The present invention improves the hybrid encoder module of RT-DETR, and the improvement mechanism includes:

[0090] (1) Replace the attention mechanism of the internal scale feature interaction unit with a deformable large core attention mechanism

[0091] The attention-based internal scale feature interaction module (AIFI) in the efficient hybrid encoder module is replaced by a deformable large kernel attention mechanism, such as Figure 4 As shown in the figure, it can adapt to complex backgrounds and different lighting conditions. Combining large convolution kernels with deformable convolution techniques, the convolution kernel is dynamically adjusted to enhance the adaptability of underwater target detection, especially for small targets.

[0092] The baseline module output feature map of stage 5 of Resnet with large separable kernel attention mechanism is flattened to form n×(h×w)×c after flattening. The position encoding is the same as the flattened feature. Figure 1 In the same input efficient hybrid encoder, the output feature map of dimension n×(h×w)×c is produced after the deformable large-core attention module.

[0093] The Deformable Large Kernel Attention network consists of multiple convolutional layers, including standard 2D convolution (Conv2D) and Deformed-DW Conv2d with offsets. This allows the network to adaptively adjust the receptive field and offset field calculations based on input features. The offset field is generated by the standard convolutional layer and guides the Deformed-DW Conv2d layer to adjust the sampling position. The activation function uses GELU. After the input undergoes 2D convolution, it is activated by the GELU, followed by Deformed-DW Conv2d with offsets, and then 2D convolution. The result is then added to the value without the Deformed-DW Conv2d, and finally fused with the input before output.

[0094] (2) Introducing the detection head

[0095] The detection head of the stage 2 (S2) layer of the backbone network is introduced in the CCFM of the hybrid encoder module to combine the feature maps output by the deformable attention module with the feature maps output by the baseline module of the stage 2-4 layers with the large separable kernel attention added, as shown in Figure 2. Figure 5 As shown, they are input into the CCFM unit in the efficient hybrid encoder, adding more detailed feature information. The specific data processing process of the cross-scale feature fusion unit (CCFM) is as follows Figure 6 shown.

[0096] (3) The upsampling unit is replaced by the DySample dynamic upsampling unit

[0097] The upsampling unit in the Cross-Scale Feature Fusion Module (CCFM) is replaced with the DySample dynamic upsampling unit, using a point sampling approach. Compared to other dynamic upsampling methods, this approach has fewer parameters and requires fewer operations, reducing the computational burden, making it lighter and faster. This approach bypasses dynamic convolution and describes upsampling from a point sampling perspective.

[0098] Input each feature Figure X Or the fused feature map (dimension C×H×W), and the upsampling scale factor s. Use a linear layer with input and output channels of C and , the generated size is The offset of sH×W is multiplied by the coefficient m to obtain the offset O, which represents the displacement of the sampling point relative to the original grid point. Pixel shuffling is then used to reshape the sample point into 2×sH×sW dimensions, converting the offset to the appropriate spatial resolution. The final sampling point coordinates are obtained by weighted addition of the original grid point coordinates and the offset. The grid_sample function is used to perform sampling, and the feature map is resampled based on the generated sampling points, resulting in an upsampled feature map of dimensions C×sH×sW. The sample set S is the sum of the offset O and the original sampling grid G.

[0099] Right now:

[0100]

[0101]

[0102] Among them, O is the offset, X is the input feature network, G is the original sampling network, S is the sampling set, and m is the weighting coefficient. The DySample structure is as follows Figure 7 shown.

[0103] (4) The downsampling unit is replaced by a lightweight downsampling unit

[0104] The downsampling in the cross-scale feature fusion module CCFM is replaced by a lightweight downsampling module, which greatly reduces the number of parameters and effectively improves the recognition accuracy while reducing the number of parameters.

[0105] Modified to a lightweight downsampling unit, the input of each fused feature map is average pooled and then the data is divided into two parts, divided into two branches. The first branch is subjected to 3×3 convolution, and the second branch is subjected to maximum pooling and then 1×1 convolution. The results of the first and second branches are connected and output to the next unit, which is the downsampled feature map. The downsampling structure is as follows Figure 8 shown.

[0106] (5) The reparameterized convolution (RepC3) module introduces the wavelet transform mechanism

[0107] Reparameterized convolution RepC3 is the main structure of the fusion module in CCFM. RepC3 is added with a wavelet transform mechanism and replaced with WTConv_RepC3 containing wavelet transform to achieve parameter lightweighting and solve the problem that traditional convolutional neural networks (CNNs) are limited by the convolution size and cannot effectively capture the global context. If traditional convolutional neural networks (CNNs) want to expand the receptive field, it will lead to parameter expansion. Therefore, the introduction of wavelet changes will expand the receptive field at the same time, significantly expanding the receptive field of CNN without significantly increasing the parameters. The convolution module in the RepC3 structure is replaced by a wavelet transform convolution module. The structure is as follows: Figure 9 shown.

[0108] The wavelet transform convolution module includes wavelet transform filters:

[0109]

[0110]

[0111]

[0112]

[0113] This filter can divide the image into four parts:

[0114] (Low-Low) represents the filter of the low-frequency part. The part obtained by this filter contains the main information and energy of the image and is usually used for image compression and feature extraction.

[0115] ‌ ‌ (Low-High) represents a filter from low frequency to high frequency. The part obtained by this filter mainly contains horizontal features.

[0116] ‌ ‌ (High-Low) represents a filter from high frequency to low frequency. The part obtained by this filter mainly contains vertical features.

[0117] ‌ ‌ (High-High) represents a filter for the high-frequency part. The part obtained by this filter contains the least information and is usually used for noise processing.

[0118] After the input passes through this filter, it passes through a depthwise convolution with a step size of 2 and is divided into two branches. The second branch passes through a two-dimensional convolution and waits for the transposed convolution after processing by the first branch to be added. The first branch takes the LL part and continues to pass through the filter and then passes through a depthwise convolution with a step size of 2 and is divided into two branches. The first branch continues to repeat the operation through the filter. The second branch passes through a two-dimensional convolution and adds the transposed convolution of the next layer, and then passes through the transposed convolution and adds the output of the second branch of the previous layer. Finally, the output is added to the input two-dimensional convolution to obtain the flattened feature map. The structure is as follows Figure 10 As shown in Figure 1, where + represents element-by-element addition. The wavelet convolution module includes N dilated convolutions followed by wavelet convolution to further expand the receptive field.

[0119] 4. InnerMPDIoU Perception Module

[0120] The InnerMPDIoU-aware module further improves object detection performance by providing higher-quality initial object queries to the decoder by applying an IoU constraint. Its input is the fused feature representation output from the efficient hybrid encoder, from which it selects a fixed number of image features as output, which serve as the decoder input. During training, this module optimizes the query selection process by selecting features that are more likely to describe true objects by generating high classification scores for features with high InnerMPDIoU and low scores for features with low InnerMPDIoU.

[0121] The InnerMPDIoU perception module introduces a scale factor and auxiliary bounding box based on the original IoU perception module to solve the problem of model generalization ability and improve accuracy.

[0122] The loss function is calculated as:

[0123] Assume the ground-truth bounding box: ) ),in: and is the center point of the bounding box, and for width and height,

[0124] The borders that introduce the ratio scale factor are:

[0125] ，

[0126] ，

[0127] in, are the coordinate values ​​of the four vertices of the real bounding box after being processed by the scale factor, For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For the bottom Axis coordinate values; for The coordinate values ​​of the four fixed points of the bounding box after the scale factor processing, For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For the bottom Axis coordinate values;

[0128] The IoU loss based on the scale factor is calculated as:

[0129]

[0130]

[0131]

[0132] Among them, inner represents the internal intersection, union represents the union, and innerIoU represents the IoU loss based on the auxiliary border;

[0133] The IoU loss based on the auxiliary bounding box and scale factor is calculated as:

[0134]

[0135] in, Represents the IoU loss of the auxiliary bounding box;

[0136] Then the loss function is:

[0137]

[0138] in, It is the result of the loss function, which is used to measure the difference between the model prediction value and the true label.

[0139] 5. Decoder module

[0140] The decoder module consists of an input embedding layer, a self-attention module, a criss-cross attention module, a feedforward neural network, and an output layer. The query and features of each layer are combined through self-attention and criss-cross attention mechanisms, ultimately outputting the category and location of each object. The query input to the input embedding layer is the object to be predicted in object detection, and each query corresponds to a potential object. Each query is positionally encoded and then combined with the feature map output by the encoder. The self-attention module allows for self-interaction between query information, enhancing the expressive power of queries and helping the network share information across different objects. The criss-cross attention module combines the decoder query with the encoder feature map output through a criss-cross attention mechanism to generate accurate object detections. In each decoder layer, a feedforward neural network processes the output from the attention mechanism, applying a nonlinear transformation to each query to further enhance its representational power. In the output layer, the output of each decoder query contains the object's location (bounding box coordinates) and category information.

[0141] 6. Experimental Results

[0142] Experimental results of underwater target detection dataset. In Table 1, Precision is accuracy and Recall is recall. The experiment compares SSD, Faster R-CNN, Yolo, RT-DETR and the average accuracy of all categories and the average accuracy of all categories at an IOU threshold of 0.5. It is verified that the performance of this model is excellent. The evaluation indicators of target detection, the average accuracy of all categories (mAP) and the average accuracy of all categories at an IOU threshold of 0.5 (mAP50), are better than other model structures.

[0143] Table 1 Experimental results

[0144]

[0145] Figure 11 This is an underwater target recognition effect diagram of the present invention.

[0146] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

[0147] Although the above describes the specific implementation methods of the present invention, it does not limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. An underwater target detection system based on RT-DETR improvement, characterized by: It includes backbone network module, hybrid encoder module, InnerMPDIoU perception module and decoder module; The backbone network module builds a multi-layer benchmark module based on large separable kernel attention, takes the pre-processed underwater target image as input, extracts features, performs large separable kernel attention calculation, and outputs a multi-layer feature map; The hybrid encoder module includes a scale-intra feature interaction unit with deformable large kernel attention and a cross-scale feature fusion unit based on convolutional neural network; The intra-scale feature interaction unit uses a deformable large-core attention mechanism to encode the features of the top-level scale output by the backbone network module; the cross-scale feature fusion unit uses a skip connection. On the basis of the original three-layer feature fusion network, it introduces the output of the fourth-layer reference module from the top to the bottom of the backbone network to fuse detailed features. At the same time, it introduces a wavelet transform to expand the receptive field without significantly increasing the parameters; the upsampling unit is replaced by a DySample dynamic upsampling unit, and the downsampling unit is replaced by a lightweight downsampling unit. A fusion mechanism is used to merge features of different scales and output the fused feature representation; The InnerMPDIoU perception module introduces a scale factor and auxiliary bounding box into the IoU calculation of the original IoU perception module, and improves the target query initialization using the InnerMPDIoU-aware query selection; The decoder module is used to convert the output of the hybrid encoder module into the final detection results, including bounding box coordinates and category predictions.

2. The underwater target detection system based on RT-DETR improvement according to claim 1, characterized in that: The backbone network module adopts the Resnet50 network, adds a large separable kernel attention to the four-layer bottleneck layer structure, and constructs a four-layer benchmark module for extracting features of four stages and generating features of different levels. The first branch of each benchmark module includes a 1*1 convolution layer, a batch normalization layer, an activation function layer, a 3*3 convolution layer, a batch normalization layer, an activation function layer, and then a 1*1 convolution layer, a batch normalization layer, and a large separable kernel attention module to add large separable kernel attention to the multi-layer features and obtain more semantic information. The second branch includes an average pooling layer, a 1*1 convolution layer, and a batch normalization layer; the first branch and the second branch are element-by-element added and output through the activation function layer. The channel sizes of the four modules are 64, 128, 256, and 512, respectively, expressed as: in, Indicates the The features of the layer are computed by large separable kernel attention, Represents an ensemble of large separable kernel attentions.

3. The underwater target detection system based on RT-DETR improvement according to claim 2, characterized in that: The large separable kernel attention module is specifically: The feature map L with a dimension of n×c×w×h undergoes a 1×3 depth convolution, a 3×1 depth convolution, a 1×3 depth expansion convolution, a 3×1 depth expansion convolution, and a 1×1 convolution operation. The result is multiplied element-by-element with the input feature map L to obtain the output feature map L' with a dimension of n×c×w×h.

4. The underwater target detection system based on RT-DETR improvement according to claim 1, characterized in that: The deformable large-kernel attention mechanism uses multiple convolutional layers, including standard 2D convolution and deformable convolution with offset. The deformable convolution with offset allows the network to adaptively adjust the receptive field based on input features. It internally includes the calculation of the offset field to guide the deformable convolution layer to adjust the sampling position. The input of the deformable large kernel attention mechanism undergoes 2D convolution, then passes through the GELU activation function, and then passes through the deformed convolution with an offset and then through the 2D convolution. It is added to the value without the deformed convolution, and then passes through the 2D convolution and fused with the input before output.

5. The underwater target detection system based on RT-DETR improvement according to claim 1, characterized in that: The DySample dynamic upsampling unit is specifically: Input each feature map or the fused feature map with dimensions C×H×W and upsampling scale factor s, and use a linear layer with input and output channels C and , the generated size is ×H×W offset is multiplied by the coefficient m to obtain the offset O, which represents the displacement of the sampling point relative to the original grid point; then, it is reshaped into 2×sH×sW dimensions by pixel shuffling, and the offset is converted to a suitable spatial resolution. The final sampling point coordinates are obtained by weighted addition of the original grid point coordinates and the offset; the sampling operation is performed by the grid_sample function, and the feature map is resampled according to the generated sampling points to obtain a feature map with an upsampled dimension of C×sH×sW. The sampling set S is the sum of the offset O and the original sampling grid G: Among them, O is the offset, X is the input feature network, G is the original sampling network, S is the sampling set, and m is the weighting coefficient.

6. The underwater target detection system based on RT-DETR improvement according to claim 1, characterized in that: The lightweight downsampling unit is specifically: Input each fused feature map, divide the data into two parts after average pooling, and divide it into two branches. The first branch undergoes 3×3 convolution, and the second branch undergoes maximum pooling and 1×1 convolution. The results of the first and second branches are connected and output to the next unit, which is the downsampled feature map.

7. The underwater target detection system based on RT-DETR improvement according to claim 1, characterized in that: The introduction of wavelet transformation is to replace the convolution module in the reparameterized convolution RepC3 structure with a wavelet transform module, including N dilated convolutions followed by wavelet transform convolutions, and the wavelet transform convolution includes a wavelet transform filter: After the filter, the image is divided into four parts. (Low-Low) represents the filter of the low-frequency part. The part obtained by this filter contains the main information and energy of the image and is used for image compression and feature extraction; ‌ ‌ (Low-High) represents the filter from low frequency to high frequency. The part obtained by this filter mainly contains horizontal features. ‌ ‌ (High-Low) represents the filter from high frequency to low frequency. The part obtained by this filter mainly contains the features in the vertical direction. ‌ ‌ (High-High) represents the filter of the high-frequency part. The part obtained by this filter contains the least information and is used for noise processing; Then after the depth convolution with a stride of 2, it is divided into two branches. The second branch is added with the transposed convolution after the two-dimensional convolution and the first branch. The first branch takes the LL part and continues to pass through the filter and then passes through the depth convolution with a stride of 2 to divide into two branches. The first branch continues to repeat the operation through the filter. The second branch is added with the transposed convolution of the next layer through the two-dimensional convolution and then added with the output of the second branch of the previous layer through the transposed convolution. Finally, the output is added with the input two-dimensional convolution to obtain the feature map after flattening.

8. The underwater target detection system based on RT-DETR improvement according to claim 1, characterized in that: The calculation method of the InnerMPDIoU perception module loss function is: Assume the ground-truth bounding box: ) ),in: and is the center point of the bounding box, and for width and height, The borders that introduce the ratio scale factor are: ， ， in, are the coordinate values ​​of the four vertices of the real bounding box after being processed by the scale factor, For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For the bottom Axis coordinate values; for The coordinate values ​​of the four fixed points of the bounding box after the scale factor processing, For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For the bottom Axis coordinate values; The IoU loss based on the scale factor is calculated as: Among them, inner represents the internal intersection, union represents the union, and innerIoU represents the IoU loss based on the auxiliary border; The IoU loss based on the auxiliary bounding box and the scale factor is calculated as: in, Represents the IoU loss of the auxiliary bounding box; Then the loss function is: in, It is the result of the loss function, which is used to measure the difference between the model prediction value and the true label.

9. An underwater target detection method based on RT-DETR improvement, characterized by: The underwater target detection system according to any one of claims 1 to 8 is applied, and includes the following process: Shoot and obtain real underwater images; Perform image preprocessing, including pixel value normalization, smoothing, and image enhancement; Input the preprocessed image into the underwater target detection system; Output underwater target detection results.

Images