Underwater target detection system and method based on RT-DETR improvement

Abstract

The invention provides an improved underwater target detection system and method based on RT-DETR, and belongs to the technical field of underwater target detection based on computer vision. The system comprises a backbone network module, a hybrid encoder module, an InerMPDIOU sensing module and a decoder module, the backbone network module performs large separable kernel attention calculation after extracting features based on the large separable kernel attention, and outputs a multi-layer feature map; the hybrid encoder module design comprises a deformable large kernel attention-based in-scale feature interaction unit and an improved cross-scale feature fusion unit; the sensing module introduces a scale factor and an auxiliary bounding box into IoU calculation of an original IoU sensing module, and target query initialization is improved by using query selection sensed by InerMPDIOU. According to the model improvement, adaptation is carried out aiming at the particularity of the underwater environment, the underwater operation efficiency is improved, and the accuracy is improved while the parameter lightweight is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater target detection based on computer vision, and in particular relates to an underwater target detection system and method based on RT-DETR improvement. Background Art

[0002] In recent years, underwater target recognition has been an important research direction of underwater deep learning. Underwater target detection can detect and detect underwater resources, and plays an important role in the development, utilization and protection of underwater resources. However, due to the complexity of the underwater environment, there will be attenuation scattering, noise and other effects, and the data set is scarce and the target size varies, resulting in the accuracy of target recognition has not been effectively improved. Underwater target recognition has always been a difficult recognition task due to its recognition complexity and the diversity of the underwater environment. Traditional underwater target detection methods are mostly based on feature extraction and classification. Due to the particularity of the underwater environment, this method limits the accuracy and universality of target recognition, especially in more complex underwater environments.

[0003] To solve this problem, many studies are currently focusing on the application of deep learning technology in mub recognition in complex underwater environments. For the current mainstream underwater target recognition network, the Yolo series is the main one. Yolo mainly uses the anchor box mechanism and removes overlapping prediction boxes through non-maximum suppression (NMS) post-processing technology. However, the NMS process depends on two key hyperparameters: the score threshold and the IoU (intersection over union) threshold. The settings of these two parameters have an impact on the accuracy and speed of detection. DETR is a target detection method based on the Transformer architecture. It avoids the use of NMS and processes image features and location information through self-attention and cross-attention mechanisms. RT-DETR is an optimization based on DETR. It can achieve real-time processing while maintaining high accuracy, solving the problem of slow running speed of traditional DETR. However, RT-DETR has not yet been effectively applied in the field of underwater target recognition. In addition, due to the problems of large and small underwater targets, ambiguity and small data sets, RT-DETR cannot achieve excellent performance in the field of underwater target recognition, and RT-DETR has many parameters and is not lightweight enough. Summary of the invention

[0004] In view of the above problems, the present invention is optimized on the basis of RT-DETR, which improves the underwater operation efficiency, realizes the lightweight of parameters while increasing the accuracy, adapts to the particularity of the underwater environment, and enhances the detection of small targets.

[0005] The first aspect of the present invention proposes an underwater target detection system based on RT-DETR improvement, including a backbone network module, a hybrid encoder module, an InnerMPDIoU perception module and a decoder module; The backbone network module builds a multi-layer benchmark module based on large separable kernel attention, takes the preprocessed underwater target image as input, performs large separable kernel attention calculation after feature extraction, and outputs a multi-layer feature map; The hybrid encoder module includes an intra-scale feature interaction unit of a deformable large core attention and a cross-scale feature fusion unit based on a convolutional neural network; the intra-scale feature interaction unit uses a deformable large core attention mechanism to encode the features of the top-level scale output by the backbone network module; the cross-scale feature fusion unit performs jump connection, and on the basis of the original three-layer feature fusion network, introduces the output of the fourth-layer benchmark module from top to bottom in the backbone network for detail feature fusion, and introduces wavelet transform to expand the receptive field without significantly increasing parameters; the upsampling unit is replaced by a DySample dynamic upsampling unit, and the downsampling unit is replaced by a lightweight downsampling unit, and a fusion mechanism is used to merge features of different scales, and the feature representation after fusion processing is output; The InnerMPDIoU perception module introduces the scale factor and auxiliary bounding box into the IoU calculation of the original IoU perception module, and improves the target query initialization by using the InnerMPDIoU-aware query selection; The decoder module is used to convert the output of the hybrid encoder module into the final detection result, including bounding box coordinates and category prediction.

[0006] Preferably, the backbone network module adopts the Resnet50 network, adds a large separable kernel attention module to the four-layer bottleneck layer structure, and constructs a four-layer benchmark module for extracting features of four stages and generating features of different levels. The first branch of each benchmark module includes a 1*1 convolution layer, a batch normalization layer, an activation function layer, a 3*3 convolution layer, a batch normalization layer, an activation function layer, and then a 1*1 convolution layer, a batch normalization layer, and a large separable kernel attention module to increase the large separable kernel attention in the multi-layer features and obtain more multi-semantic information. The second branch includes an average pooling layer, a 1*1 convolution layer, and a batch normalization layer; the first branch and the second branch are added element by element and then output through the activation function layer. The channel sizes of the four modules are 64, 128, 256, and 512, respectively, expressed as:

[0007] in, Indicates The features of the layer are computed with large separable kernel attention, Represents a collection of large separable kernel attentions.

[0008] Preferably, the large separable core attention module is specifically: The feature map L with dimension n×c×w×h undergoes a 1×3 depth convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×1), and then a 3×1 depth convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, and the padding is 1×0), and then a 1×3 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×2, and the expansion is 2×2), and then a 3×1 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, the padding is 2×0, and the expansion is 2×2), and then a 1×1 convolution operation (the number of input and output channels is c, and the convolution kernel size is 1×1), the result is multiplied element-by-element with the input feature map L to obtain the output feature map L', with dimension n×c×w×h.

[0009] Preferably, the deformable large-core attention mechanism uses multiple convolutional layers, including standard 2D convolution and deformable convolution with offset; the deformable convolution with offset allows the network to adaptively adjust the receptive field according to the input features, and contains the calculation of the offset field to guide the deformable convolution layer to adjust the sampling position; The input of the deformable large kernel attention mechanism passes through a 2D convolution, then a GELU activation function, then a deformable convolution with an offset, and then a 2D convolution. It is added to the value without the deformed convolution, and then passed through a 2D convolution and output after being fused with the input.

[0010] Preferably, the DySample dynamic upsampling unit is specifically: Input each feature map or the fused feature map with dimensions C×H×W and upsampling scale factor s, and use a linear layer with input and output channels C and , the generated size is ×H×W offset is multiplied by the coefficient m to obtain the offset O, which represents the displacement of the sampling point relative to the original grid point; then, it is reshaped into 2×sH×sW dimensions by pixel shuffling, and the offset is converted to a suitable spatial resolution. The final sampling point coordinates are obtained by weighted addition of the original grid point coordinates and the offset; the sampling operation is performed through the grid_sample function, and the feature map is resampled according to the generated sampling points to obtain a feature map with an upsampled dimension of C×sH×sW. The sampling set S is the sum of the offset O and the original sampling grid G:

[0011]

[0012] Among them, O is the offset, X is the input feature network, G is the original sampling network, S is the sampling set, and m is the weighting coefficient.

[0013] Preferably, the lightweight downsampling unit is specifically: Input each fused feature map, divide the data into two parts after average pooling, and divide it into two branches. The first branch undergoes 3×3 convolution, and the second branch undergoes 1×1 convolution after maximum pooling. The results of the first and second branches are connected and output to the next unit, which is the downsampled feature map.

[0014] Preferably, the introduction of wavelet transformation is to replace the convolution module in the reparameterized convolution RepC3 structure with a wavelet transform module, including N dilated convolutions followed by wavelet transform convolutions, wherein the wavelet transform convolution includes a wavelet transform filter:

[0015]

[0016]

[0017]

[0018] After the filter, the image is divided into four parts. (Low-Low) represents the filter of the low-frequency part. The part obtained by this filter contains the main information and energy of the image and is used for image compression and feature extraction; ‌ ‌ (Low-High) represents the filter from low frequency to high frequency. The part obtained by this filter mainly contains horizontal features. ‌ ‌ (High-Low) represents the filter from high frequency to low frequency. The part obtained by this filter mainly contains the features in the vertical direction. ‌ ‌ (High-High) represents the filter of the high-frequency part. The part obtained by this filter contains the least information and is used for noise processing; Then after a depthwise convolution with a step size of 2, it is divided into two branches. The second branch undergoes a two-dimensional convolution and waits for the transposed convolution after processing by the first branch to be added. The first branch takes the LL part and continues to pass through the filter and then undergoes a depthwise convolution with a step size of 2 to be divided into two branches. The first branch continues to repeat the operation through the filter. The second branch is added to the transposed convolution of the next layer through a two-dimensional convolution, and then added to the output of the second branch of the previous layer through a transposed convolution. Finally, the output is added to the input two-dimensional convolution to obtain the flattened feature map.

[0019] Preferably, the calculation method of the InnerMPDIoU perception module loss function is: Assume the ground-truth bounding box: ) ),in: and is the center point of the border, and for width and height, The borders that introduce the ratio scale factor are: ，

[0020] ，

[0021] in, are the coordinate values ​​of the four vertices of the real bounding box after being processed by the scale factor, For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For below Axis coordinate values; for The coordinate values ​​of the four fixed points of the bounding box after the scale factor is processed. For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For below Axis coordinate values; The IoU loss based on the scale factor is calculated as:

[0022]

[0023]

[0024] Among them, inner represents the internal intersection, union represents the union, and innerIoU represents the IoU loss based on the auxiliary border; The IoU loss based on the auxiliary bounding box and the scale factor is calculated as:

[0025] in, Represents the IoU loss of the auxiliary bounding box; Then the loss function is:

[0026] in, It is the result of the loss function, which is used to measure the difference between the model prediction value and the true label.

[0027] The second aspect of the present invention proposes an underwater target detection method based on RT-DETR improvement, which uses the underwater target detection system as described in the first aspect and includes the following process: Shoot and obtain real underwater images; Preprocess the image, including pixel value normalization, smoothing and image enhancement; Input the preprocessed image into the underwater target detection system; Output underwater target detection results.

[0028] Compared with the prior art, the present invention has the following beneficial effects: The present invention introduces a large separable kernel attention mechanism, effectively combines the advantages of the kernel attention mechanism, improves performance, improves computing performance and reduces memory usage, and optimizes the upsampling and downsampling mechanisms of the original model to greatly reduce the number of parameters and improve accuracy, while increasing deformable large kernel attention to adapt to complex backgrounds and different lighting conditions. Combining the technology of large convolution kernels and deformable convolutions, the convolution kernel is dynamically adjusted to enhance the adaptability of underwater target detection, especially for small targets. In order to expand the receptive field of the model, the wavelet transform is introduced to achieve parameter lightweight, solve the problem that the traditional CNN is limited by the convolution size and cannot effectively capture the global context, and the expansion of the receptive field of the traditional CNN will lead to parameter expansion, and introduce InnerMPDIoU to improve the training accuracy and speed of the model. Through the above operation, the perception ability of the model is improved while ensuring the lightweight of parameters, thereby improving the accuracy and robustness of underwater target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, what is described below is only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 It is the overall structural block diagram of the underwater target detection system of the present invention.

[0031] Figure 2 It is a schematic diagram of the backbone network structure of the present invention.

[0032] Figure 3 Schematic diagram of the structure of the large separable core attention module of the present invention.

[0033] Figure 4 Schematic diagram of the network structure of the deformable large core attention mechanism of the present invention.

[0034] Figure 5 This is a schematic diagram of the structure of the output of the stage 2 reference module of the backbone network introduced into the CCFM of the present invention.

[0035] Figure 6 This is a data processing flow chart of the cross-scale feature fusion unit CCFM of the present invention.

[0036] Figure 7 It is a schematic diagram of the structure of the DySample dynamic upsampling unit module of the present invention.

[0037] Figure 8 It is a schematic diagram of the structure of the lightweight downsampling unit of the present invention.

[0038] Fig. 9 This is a schematic diagram of the RepC3 structure that introduces wavelet transform into the present invention.

[0039] Fig.10 Schematic diagram of the network structure of the wavelet convolution module of the present invention.

[0040] Fig.11 This is an underwater target recognition effect diagram of the present invention. DETAILED DESCRIPTION

[0041] The present invention provides an underwater target detection system based on RT-DETR improvement, the overall structure is as follows Figure 1 As shown in the figure, it includes a backbone network based on a large separable kernel attention module, an efficient hybrid encoder that combines deformable large kernel attention and introduces wavelet transform and improves the up and down sampling mechanism, an InnerMPDIoU perception module, and a decoder module.

[0042] The backbone network based on the large separable kernel attention module uses the preprocessed underwater target image as input, extracts features, performs large separable kernel attention calculations, and outputs multi-layer feature maps. It makes full use of multi-semantic information to improve the performance of target recognition tasks.

[0043] The hybrid encoder module combines deformable large kernel attention with a hybrid encoder module that introduces wavelet transform and improves the up-down sampling mechanism. The input is the features of different scales of the image output by the backbone network, and the output is the feature representation of the processed image. The hybrid encoder module design includes two main parts: the intra-scale feature interaction unit based on deformable large kernel attention and the cross-scale feature fusion unit (CCFM) based on convolutional neural network. The intra-scale interaction is for the features of the top-level scale output by the backbone network, and is processed by the encoder of deformable large kernel attention, which can capture the target information more efficiently. After processing the intra-scale interaction of the top-level features, the cross-scale fusion (CCFM) uses an efficient fusion mechanism to merge features of different scales, thereby achieving an overall understanding of objects of different sizes, and finally outputs the feature representation after fusion processing. The present invention reduces the redundancy of calculation and improves the efficiency of feature extraction by improving the encoder. The improvements include that the intra-scale feature interaction adopts a deformable large-core attention to adapt to complex backgrounds and different lighting conditions, the cross-scale feature fusion module adds a p2 detection head, and introduces wavelet transform to expand the receptive field without significantly increasing parameters, wherein the p2 detection head refers to the cross-scale feature fusion unit for jump connection, and on the basis of the original three-layer feature fusion network, the output of the fourth layer of the reference module from top to bottom in the backbone network is introduced to perform detail feature fusion; at the same time, the upsampling is changed to dynamic upsampling, and the downsampling is optimized at the same time to achieve lightweight downsampling, which greatly reduces the number of downsampling parameters.

[0044] The InnerMPDIoU-aware module optimizes the query selection process by considering the IoU score to select features that are more likely to describe the true object. The InnerMPDIoU-aware query selection provides a higher quality initial object query to the decoder by providing an IoU constraint, further improving the performance of object detection. Its input is the output of the efficient hybrid encoder, that is, the image feature representation after processing by the efficient hybrid encoder, and then a fixed number of image features are selected from the output sequence of the efficient hybrid encoder as the output, that is, the initial object query of the decoder. The InnerMPDIoU-aware module optimizes the detection performance of the model by producing high classification scores for features with high InnerMPDIoU and low classification scores for features with low InnerMPDIoU during training. The InnerMPDIoU-aware module introduces the scale factor and auxiliary bounding box into the IoU calculation of the original IoU-aware module, and improves the object query initialization using the InnerMPDIoU-aware query selection.

[0045] The decoder module is mainly responsible for converting the output of the hybrid encoder module into the final detection results, including bounding box coordinates and category predictions. Its input is the output of the encoder, and it is decoded after selecting good features, and the output is bounding box coordinates and category predictions.

[0046] The invention will be further described below in conjunction with specific embodiments.

[0047] 1. Data preprocessing Preprocessing includes but is not limited to image denoising, image size unification, pixel value normalization, contrast enhancement, brightness adjustment, and histogram equalization, etc., aiming to improve image quality. The preprocessing process is: After obtaining the real underwater image, the image size is adjusted according to the image size, and the pixel value is normalized to eliminate the brightness difference between different images and improve the accuracy and stability of model training. Then, Gaussian filtering is used to smooth the image. The value of each pixel is obtained by weighted averaging the value of itself and other pixels in the neighborhood to improve the quality and clarity of the image. After that, contrast enhancement, brightness adjustment and histogram equalization are used to enhance the image to make the features and details more prominent.

[0048] 2. Backbone network module The present invention adds a large separable kernel attention mechanism to the backbone network module, aiming to skillfully utilize the large separable kernel to enhance the attention mechanism, make full use of multi-semantic information, greatly improve the efficiency and accuracy of image recognition and processing, and thus improve the performance of target recognition tasks. Its overall structure is as follows Figure 2 As shown. It adds a large separable kernel attention module to the backbone network. Taking Resnet50 as an example, the bottleneck layer structure in Resnet50 adds a large separable kernel attention module as the four-layer benchmark module of stage2, stage3, stage4 and stage5 of the backbone network Resnet50. The backbone network is used to extract features of the four stages and generate features of different levels, which are expressed as:

[0049] in Indicates that the The feature information extracted by the layer, F represents A collection of .

[0050] A large separable kernel attention module is added to the above four bottleneck layers. The bottleneck layer represents the feature Through the large separable kernel attention module, the first branch of each module includes a 1*1 convolution layer, a batch normalization layer, an activation function layer, a 3*3 convolution layer, a batch normalization layer, an activation function layer, and then a 1*1 convolution layer, a batch normalization layer, and a large separable kernel attention module to add large separable kernel attention to the multi-layer features and obtain more multi-semantic information. Its second branch includes an average pooling layer, a 1*1 convolution layer, and a batch normalization layer. The first branch and the second branch are added element by element and then output through the activation function layer. The structure is as follows: Figure 1 , after this module, the channel sizes of the four benchmark modules are 64, 128, 256 and 512 respectively, expressed as:

[0051] in, Indicates The features of the layer are computed with large separable kernel attention, Represents a collection of large separable kernel attentions.

[0052] Among them, the large separable kernel attention module structure is as follows Figure 3 shown.

[0053] The feature map L with dimension n×c×w×h undergoes a 1×3 depth convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×1), and then a 3×1 depth convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, and the padding is 1×0), and then a 1×3 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×2, and the expansion is 2×2), and then a 3×1 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, the padding is 2×0, and the expansion is 2×2), and then a 1×1 convolution operation (the number of input and output channels is c, and the convolution kernel size is 1×1), the result is multiplied element-by-element with the input feature map L to obtain the output feature map L', with dimension n×c×w×h.

[0054] 3. Hybrid encoder module The present invention improves the hybrid encoder module of RT-DETR, and the improvement mechanism includes: (1) Replace the attention mechanism of the internal scale feature interaction unit with a deformable large core attention mechanism The attention-based internal scale feature interaction module (AIFI) in the efficient hybrid encoder module is replaced by a deformable large kernel attention mechanism, such as Figure 4As shown in the figure, it can adapt to complex backgrounds and different lighting conditions. Combining the technology of large convolution kernel and deformable convolution, the convolution kernel is dynamically adjusted to enhance the adaptability of underwater target detection, especially for small targets.

[0055] The baseline module output feature map of stage 5 of Resnet with large separable kernel attention mechanism is flattened to form n×(h×w)×c feature maps with the same position encoding as the flattened feature map. Figure 1 In the same input efficient hybrid encoder, the output dimension is n×(h×w)×c feature map after the deformable large core attention module.

[0056] The deformable large core attention consists of multiple convolutional layers, including standard 2D convolution (Conv2D) and deformable convolution with offset (Deform-DW Conv2d), which allows the network to adaptively adjust the receptive field and offset field calculation according to the input features. It is generated by the standard convolution layer and guides the deformable convolution layer to adjust the sampling position. The activation function uses GELU. After the input passes through the 2D convolution, it passes through the GELU activation function, and then passes through the deformable convolution with offset, and then passes through the 2D convolution, and then the value is added to the value without the deformed convolution, and then passes through the 2D convolution, and then the output is fused with the input.

[0057] (2) Introducing the detection head The detection head of the stage2 (S2) layer of the backbone network is introduced in the CCFM of the hybrid encoder module to combine the feature maps output by the deformable attention module with the feature maps output by the baseline module of the stage2-4 layers with the large separable kernel attention added, as shown in Figure 5 As shown in the figure, they are input into the CCFM unit in the efficient hybrid encoder, adding more detailed feature information. The specific data processing process of the cross-scale feature fusion unit (CCFM) is as follows: Figure 6 shown.

[0058] (3) The upsampling unit is replaced by the DySample dynamic upsampling unit The upsampling unit in the cross-scale feature fusion module (CCFM) is replaced by the DySample dynamic upsampling unit, and the point sampling method is used. Compared with other dynamic upsampling methods, it has fewer parameters and operations, reduces the computational burden, is lighter and runs faster. It bypasses dynamic convolution and describes upsampling from the perspective of point sampling.

[0059] Enter each feature Figure X Or the fused feature map (dimension C×H×W), and the upsampling scale factor s. Use a linear layer with input and output channels of C and , the generated size is ×H×W offset is multiplied by the coefficient m to obtain the offset O, which represents the displacement of the sampling point relative to the original grid point. Then, it is reshaped into 2×sH×sW dimensions through pixel shuffling, and the offset is converted to a suitable spatial resolution. The final sampling point coordinates are obtained by weighted addition of the original grid point coordinates and the offset. The sampling operation is performed through the grid_sample function, and the feature map is resampled according to the generated sampling points to obtain a feature map with an upsampled dimension of C×sH×sW. The sampling set S is the sum of the offset O and the original sampling grid G.

[0060] Right now:

[0061]

[0062] Among them, O is the offset, X is the input feature network, G is the original sampling network, S is the sampling set, and m is the weighting coefficient. The DySample structure is as follows Figure 7 shown.

[0063] (4) The downsampling unit is replaced by a lightweight downsampling unit The downsampling in the cross-scale feature fusion module CCFM is replaced by a lightweight downsampling module, which greatly reduces the number of parameters and effectively improves the recognition accuracy while reducing the number of parameters.

[0064] Modified to a light downsampling unit, the input of each fused feature map is divided into two parts after average pooling, divided into two branches, the first branch is subjected to 3×3 convolution, the second branch is subjected to maximum pooling and then 1×1 convolution, the first and second branch results are connected and output to the next unit, which is the downsampled feature map. The downsampling structure is as follows Figure 8 shown.

[0065] (5) The reparameterized convolution (RepC3) module introduces the wavelet transform mechanism Reparameterized convolution RepC3 is the main structure of the fusion module in CCFM. RepC3 is added with a wavelet transform mechanism and replaced with WTConv_RepC3 containing wavelet transform to achieve parameter lightweighting and solve the problem that traditional convolutional neural networks (CNNs) are limited by the size of convolutions and cannot effectively capture global contexts. If traditional convolutional neural networks (CNNs) want to expand the receptive field, it will lead to parameter expansion. Therefore, the introduction of wavelet changes will expand the receptive field at the same time, and significantly expand the receptive field of CNN without significantly increasing the parameters. The convolution module in the RepC3 structure is replaced by a wavelet transform convolution module. The structure is as follows: Fig. 9 shown.

[0066] The wavelet transform convolution module includes wavelet transform filters:

[0067]

[0068]

[0069]

[0070] This filter can divide the image into four parts: (Low-Low) represents the filter of the low-frequency part. The part obtained by this filter contains the main information and energy of the image and is usually used for image compression and feature extraction.

[0071] ‌ ‌ (Low-High) represents the filter from low frequency to high frequency. The part obtained by this filter mainly contains horizontal features.

[0072] ‌ ‌ (High-Low) represents the filter from high frequency to low frequency. The part obtained by this filter mainly contains the features in the vertical direction.

[0073] ‌ ‌ (High-High) represents the filter of the high-frequency part. The part obtained by this filter contains the least information and is usually used for noise processing. After the input passes through this filter, it passes through a depth convolution with a step size of 2 and is divided into two branches. The second branch passes through a two-dimensional convolution and waits for the transposed convolution after processing by the first branch to be added. The first branch takes the LL part and continues to pass through the filter and then passes through a depth convolution with a step size of 2 to be divided into two branches. The first branch continues to repeat the operation through the filter. The second branch passes through a two-dimensional convolution and adds the transposed convolution of the next layer, then passes through the transposed convolution and adds the output of the second branch of the previous layer. Finally, the output is added to the input two-dimensional convolution to obtain the flattened feature map. The structure is as follows: Fig.10 As shown in the figure, + represents element-by-element addition. The wavelet convolution module includes N dilated convolutions followed by wavelet convolutions to further expand the receptive field.

[0074] 4. InnerMPDIoU perception module The InnerMPDIoU-aware module further improves the performance of object detection by providing an IoU constraint to provide a higher quality initial object query to the decoder. Its input is the output of the efficient hybrid encoder, i.e. the fused feature representation, from which a fixed number of image features are selected as the output, i.e. the input to the decoder. The query selection process is optimized by producing high classification scores for features with high InnerMPDIoU and low classification scores for features with low InnerMPDIoU during training, thereby selecting features that are more likely to describe the true object.

[0075] The InnerMPDIoU perception module introduces a scale factor and auxiliary bounding box based on the original IoU perception module to solve the problem of model generalization ability and improve accuracy.

[0076] The loss function is calculated as: Assume the ground-truth bounding box: ) ),in: and is the center point of the border, and for width and height, The borders that introduce the ratio scale factor are: ，

[0077] ，

[0078] in, are the coordinate values ​​of the four vertices of the real bounding box after being processed by the scale factor, For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For below Axis coordinate values; for The coordinate values ​​of the four fixed points of the bounding box after the scale factor is processed. For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For below Axis coordinate values; The IoU loss based on the scale factor is calculated as:

[0079]

[0080]

[0081] Among them, inner represents the internal intersection, union represents the union, and innerIoU represents the IoU loss based on the auxiliary border; The IoU loss based on the auxiliary bounding box and the scale factor is calculated as:

[0082] in, Represents the IoU loss of the auxiliary bounding box; Then the loss function is:

[0083] in, It is the result of the loss function, which is used to measure the difference between the model prediction value and the true label.

[0084] 5. Decoder module The decoder module includes an input embedding layer, a self-attention module, a cross-attention module, a feedforward neural network, and an output layer. The query and features of each layer are combined through self-attention and cross-attention mechanisms, and the category and position of each target are finally output. Among them: the query input to the input embedding layer is the object to be predicted in target detection, and each query corresponds to a potential target. After position encoding, each query is combined with the feature map output by the encoder; the self-attention module is a self-interaction between query information, which enhances the expressiveness of the query and helps the network share information between different targets; the cross-attention module: the query of the decoder is combined with the feature map output by the encoder through the cross-attention mechanism to generate accurate detection of the target. Feedforward neural network layer: In each decoder layer, the feedforward neural network processes the output from the attention mechanism and provides a nonlinear transformation for each query to further improve its representation ability. Output layer: The output of each decoder query contains the location (bounding box coordinates) and category information of the target.

[0085] 6. Experimental results Experimental results of underwater target detection dataset. In Table 1, Precision is accuracy and Recall is recall. The experiment compares SSD, Faster R-CNN, Yolo and RT-DETR as well as the average accuracy of all categories and the average accuracy of all categories at an IOU threshold of 0.5 of the present invention. It is verified that the performance of this model is excellent. The evaluation indicators of target detection, the average accuracy of all categories (mAP) and the average accuracy of all categories at an IOU threshold of 0.5 (mAP50), are better than other model structures.

[0086] Table 1 Experimental results

[0087] Fig.11 This is an underwater target recognition effect diagram of the present invention.

[0088] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0089] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. An underwater target detection system based on RT-DETR improvement, characterized by: It includes backbone network module, hybrid encoder module, InnerMPDIoU perception module and decoder module; The backbone network module builds a multi-layer benchmark module based on large separable kernel attention, takes the preprocessed underwater target image as input, performs large separable kernel attention calculation after feature extraction, and outputs a multi-layer feature map; The hybrid encoder module includes an intra-scale feature interaction unit of a deformable large kernel attention and a cross-scale feature fusion unit based on a convolutional neural network; The intra-scale feature interaction unit uses a deformable large-core attention mechanism to encode the features of the top-level scale output by the backbone network module; the cross-scale feature fusion unit performs jump connection, and on the basis of the original three-layer feature fusion network, introduces the output of the fourth-layer benchmark module from top to bottom in the backbone network to perform detail feature fusion, and introduces wavelet transform to expand the receptive field without significantly increasing parameters; the upsampling unit is replaced by a DySample dynamic upsampling unit, and the downsampling unit is replaced by a lightweight downsampling unit, and a fusion mechanism is used to merge features of different scales, and outputs the feature representation after fusion processing; The InnerMPDIoU perception module introduces the scale factor and auxiliary bounding box into the IoU calculation of the original IoU perception module, and improves the target query initialization by using the InnerMPDIoU-aware query selection; The decoder module is used to convert the output of the hybrid encoder module into the final detection result, including bounding box coordinates and category prediction.

2. The underwater target detection system based on RT-DETR improvement as claimed in claim 1, characterized in that: The backbone network module adopts the Resnet50 network, adds a large separable kernel attention to the four-layer bottleneck layer structure, and constructs a four-layer benchmark module for extracting features of four stages and generating features of different levels. The first branch of each benchmark module includes a 1*1 convolution layer, a batch normalization layer, an activation function layer, a 3*3 convolution layer, a batch normalization layer, an activation function layer, and then a 1*1 convolution layer, a batch normalization layer, and a large separable kernel attention module to add a large separable kernel attention to the multi-layer features and obtain more multi-semantic information. The second branch includes an average pooling layer, a 1*1 convolution layer, and a batch normalization layer; the first branch and the second branch are added element by element and then output through the activation function layer. The channel sizes of the four modules are 64, 128, 256, and 512, respectively, expressed as: in, Indicates The features of the layer are computed with large separable kernel attention, Represents a collection of large separable kernel attentions.

3. The underwater target detection system based on RT-DETR improvement as claimed in claim 1, characterized in that: The large separable kernel attention module is specifically: The feature map L with dimension n×c×w×h undergoes a 1×3 depth convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×1), and then a 3×1 depth convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, and the padding is 1×0), and then a 1×3 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 1×3, the step size is 1×1, and the padding is 0×2, and the expansion is 2×2), and then a 3×1 depth expansion convolution (the number of input and output channels is c, the convolution kernel size is 3×1, the step size is 1×1, the padding is 2×0, and the expansion is 2×2), and then a 1×1 convolution operation (the number of input and output channels is c, and the convolution kernel size is 1×1), the result is multiplied element-by-element with the input feature map L to obtain the output feature map L', with dimension n×c×w×h.

4. The underwater target detection system based on RT-DETR improvement as claimed in claim 1, characterized in that: The deformable large-core attention mechanism uses multiple convolutional layers, including standard 2D convolution and deformable convolution with offset; the deformable convolution with offset allows the network to adaptively adjust the receptive field according to the input features, and contains the calculation of the offset field to guide the deformable convolution layer to adjust the sampling position; The input of the deformable large kernel attention mechanism passes through a 2D convolution, then a GELU activation function, then a deformable convolution with an offset, and then a 2D convolution. It is added to the value without the deformed convolution, and then passed through a 2D convolution and output after being fused with the input.

5. The underwater target detection system based on RT-DETR improvement as claimed in claim 1, characterized in that: The DySample dynamic upsampling unit is specifically: Input each feature map or the fused feature map with dimensions C×H×W and upsampling scale factor s, and use a linear layer with input and output channels C and , the generated size is ×H×W offset is multiplied by the coefficient m to obtain the offset O, which represents the displacement of the sampling point relative to the original grid point; then, it is reshaped into 2×sH×sW dimensions by pixel shuffling, and the offset is converted to a suitable spatial resolution. The final sampling point coordinates are obtained by weighted addition of the original grid point coordinates and the offset; the sampling operation is performed through the grid_sample function, and the feature map is resampled according to the generated sampling points to obtain a feature map with an upsampled dimension of C×sH×sW. The sampling set S is the sum of the offset O and the original sampling grid G: Among them, O is the offset, X is the input feature network, G is the original sampling network, S is the sampling set, and m is the weighting coefficient.

6. The underwater target detection system based on RT-DETR improvement as claimed in claim 1, characterized in that: The lightweight downsampling unit is specifically: Input each fused feature map, divide the data into two parts after average pooling, and divide it into two branches. The first branch undergoes 3×3 convolution, and the second branch undergoes 1×1 convolution after maximum pooling. The results of the first and second branches are connected and output to the next unit, which is the downsampled feature map.

7. The underwater target detection system based on RT-DETR improvement as claimed in claim 1, characterized in that: The introduction of wavelet transformation is to replace the convolution module in the reparameterized convolution RepC3 structure with a wavelet transform module, including N dilated convolutions followed by wavelet transform convolutions, and the wavelet transform convolution includes a wavelet transform filter: After the filter, the image is divided into four parts. (Low-Low) represents the filter of the low-frequency part. The part obtained by this filter contains the main information and energy of the image and is used for image compression and feature extraction; ‌ ‌ (Low-High) represents the filter from low frequency to high frequency. The part obtained by this filter mainly contains horizontal features. ‌ ‌ (High-Low) represents the filter from high frequency to low frequency. The part obtained by this filter mainly contains the features in the vertical direction. ‌ ‌ (High-High) represents the filter of the high-frequency part. The part obtained by this filter contains the least information and is used for noise processing; Then after a depthwise convolution with a step size of 2, it is divided into two branches. The second branch undergoes a two-dimensional convolution and waits for the transposed convolution after processing by the first branch to be added. The first branch takes the LL part and continues to pass through the filter and then undergoes a depthwise convolution with a step size of 2 to be divided into two branches. The first branch continues to repeat the operation through the filter. The second branch is added to the transposed convolution of the next layer through a two-dimensional convolution, and then added to the output of the second branch of the previous layer through a transposed convolution. Finally, the output is added to the input two-dimensional convolution to obtain the flattened feature map.

8. The underwater target detection system based on RT-DETR improvement as claimed in claim 1, characterized in that: The calculation method of the InnerMPDIoU perception module loss function is: Assume the ground-truth bounding box: ) ),in: and is the center point of the border, and for width and height, The borders that introduce the ratio scale factor are: ， ， in, are the coordinate values ​​of the four vertices of the real bounding box after being processed by the scale factor, For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For below Axis coordinate values; for The coordinate values ​​of the four fixed points of the bounding box after the scale factor is processed. For the left Axis coordinate values, For the right side Axis coordinate values, For the top Axis coordinate values, For below Axis coordinate values; The IoU loss based on the scale factor is calculated as: Among them, inner represents the internal intersection, union represents the union, and innerIoU represents the IoU loss based on the auxiliary border; The IoU loss based on the auxiliary bounding box and the scale factor is calculated as: in, Represents the IoU loss of the auxiliary bounding box; Then the loss function is: in, It is the result of the loss function, which is used to measure the difference between the model prediction value and the true label.

9. An underwater target detection method based on RT-DETR improvement, characterized in that: The underwater target detection system according to any one of claims 1 to 8 is applied, and includes the following process: Shoot and obtain real underwater images; Preprocess the image, including pixel value normalization, smoothing and image enhancement; Input the preprocessed image into the underwater target detection system; Output underwater target detection results.

Images