Marine organism detection method of lightweight WCSPOmni-DETR based on enhanced fine-grained feature expression branch

Abstract

The invention discloses a marine organism detection method based on lightweight WCSPOmni-DETR of an enhanced fine-grained feature expression branch. The method comprises the steps of obtaining a data set of marine organisms and performing preprocessing; improving the backbone network of the RT-DETR by using wavelet convolution, and carrying out lightweight processing; constructing an enhanced fine-grained feature expression branch for the RT-DETR; the AIFI structure of the improved RT-DETR (reverse transcription-DETR) is an Officient Advanced attention AIFI, and the AIFI structure of the improved RT-DETR is a reverse transcription AIFI; training hyper-parameters such as a training round, a batch processing size, a learning rate and an input picture size are set; inputting data of the training set and the verification set into the model for training, and obtaining a group of optimal weight parameters; and inputting test data by using the optimal weight parameter to obtain a predicted visualization result.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater target detection, and in particular relates to a marine life detection method based on a lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branch. Background Art

[0002] As land resources become increasingly scarce, the development of marine resources has become a strategic need for many countries and regions. Underwater images are an important carrier and form of expression of marine information. However, affected by the specific physical and chemical properties of the underwater environment, underwater images usually suffer from degradation phenomena such as uneven illumination, reduced contrast, and color distortion, making it difficult to fully extract the features of marine organisms in the image, especially some small marine organisms that gather in groups. The spatial information between them is diluted during the process of deep convolutional neural network feature extraction. Therefore, conventional target detection models cannot use efficient features to locate and classify targets. Various feature enhancement and feature fusion modules have been proposed and applied to the neck network of the model to address this situation. Zhuoyan Liu et al. proposed a plug-and-play underwater joint image enhancement module: UnitModule. Hui Zhou et al. proposed an underwater light detection network (UODN) based on the you only look once version 8 (YOLOv8) framework. The network solves the underwater detection problem through the cross-stage multi-branch (CSMB) module and the large kernel space pyramid (LKSP) module. Xia Hua et al. designed a feature enhancement gating module, an adjacent feature fusion mechanism, and a dynamic fusion module to prevent the loss of small target information. These modules either still use several layers of mediocre feature maps extracted by the original backbone network, but on this basis, they have added some operations of interaction of features at different levels, or they bring a large increase in the number of parameters and cannot adapt to the limited hardware resources of the underwater target detection platform.

[0003] Since its introduction, Real-Time Detection with Transformer (RT-DETR) has been known for its ability to beat the You Only Look Once (YOLO) series in terms of detection accuracy and speed. The model efficiently processes multi-scale features through intra-scale interaction and cross-scale fusion. However, it still faces the challenges of large computational complexity and insufficient performance in processing fine targets. Qianfeng Ge et al. implemented Zoom-cat scale splicing and SSF in the CCFM module, enhanced the network's multi-scale feature fusion capability, and replaced the original Multi-Head Self-Attention (MHSA) with Deformable attention. Xiaoxin Li et al. proposed a space-to-depth transformer module based on the RT-DETR-L scale to replace the deep convolution in the original backbone network, thereby promoting deep interaction between local and global features, and used contextual transformer as an encoder to enhance the model's ability to handle overlapping targets. Hailin Feng et al. used RT-DETR as the main structure, combined with soft thresholding and Cascaded-Group-Attention (CGA) modules, called SC-RTDETR, and achieved significant performance improvements on the pine wilt remote sensing image dataset. Although these improved RT-DETRs have made some progress in their respective fields, they either come at the expense of replacing modules with larger parameters, or the extracted features fail to effectively enhance the spatial information in the image, and cannot cope with the situation where underwater images are distorted and the hardware resources of the detection platform are limited. Summary of the invention

[0004] The purpose of the present invention is to address the problems of large algorithm scale and lack of accuracy in existing underwater target detection methods. A WCSPOmni-DETR lightweight marine life detection method is proposed, which uses wavelet convolution and improved Efficient Additive AIFI for lightweight processing, and then uses enhanced fine-grained feature expression branches to enhance detail information extraction.

[0005] The present invention provides a method for detecting marine organisms based on a lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branches, comprising the following steps:

[0006] Step S1: obtaining a marine biological target detection data set and performing preprocessing; the marine biological target detection data set includes image data of marine organisms;

[0007] Step S2: Construct a WCSPOmni-DETR model, the input of which is the image data of marine organisms, and the output is the predicted value of the type of marine organisms in the image: obtain the RT-DETR model and the wavelet convolution module, use the wavelet convolution module to replace the second convolution module of the BasicBlock module in the backbone network of the RT-DETR model, and obtain an improved backbone extraction network; obtain the Omni-Kernel module, which is a full-kernel module consisting of three branches: global branch, large branch and local branch; use the Cross Stage Partial The Omni-Kernel module is improved by lightweight network architecture, and the convolution operation of the Omni-Kernel module is performed on the local part of the input feature map. The rest of the part is directly spliced ​​with the convolution result to obtain the CSPOMNIKERN module; the CSPOMNIKERN module and the SPDConv module are used to build the enhanced fine-grained feature expression branch of the WCSPOmni-DETR model, including: the s2 shallow feature map in the backbone network of the RT-DETR model is extracted, and after SPDConv convolution, it is combined with the deep features in the CSPOMNIKERN module to perform global to local feature fusion to enhance the expression of fine-grained features; the Efficient Additive Attention module is obtained, and it is used to improve the AIFI module structure of the RT-DETR model to obtain the WCSPOmni-DETR model;

[0008] Step S3: Use the marine biological target detection dataset to train the WCSPOmni-DETR model to obtain a trained WCSPOmni-DETR model;

[0009] Step S4: Get the image to be tested, input the trained WCSPOmni-DETR model, and get the prediction result.

[0010] Preferably, the obtaining of the Efficient Additive Attention module and utilizing it to improve the AIFI module structure of RT-DETR specifically includes: replacing the original MHSA module of the AIFI module with the Efficient AdditiveAttention module.

[0011] Preferably, the scale of the RT-DETR model used in step S2 is RT-DETR-Resnet18.

[0012] Preferably, in step S2, when the Omni-Kernel module is light-weighted using the idea of ​​Cross Stage Partial Network, the ratio of convolution operations on the feature map is 1 / 4.

[0013] Preferably, step S3 also includes the following hyperparameter settings for the WCSPOmni-DETR model: the training rounds are set to 160, the batch size is set to 16, the optimizer adopts AdamW, and the learning rate is set to 0.0001.

[0014] Preferably, the marine organisms in step S1 include starfish, corals, sea urchins, sea cucumbers, and scallops.

[0015] Beneficial effects of the present invention:

[0016] Aiming at the special needs of underwater target detection scenarios, the present invention takes the RT-DETR-Resnet18 model as a benchmark and makes a series of improvement measures, taking advantage of its advantages in real-time, accuracy, multi-scale detection and multi-category detection, and improving its problems in computational complexity and large number of parameters, and proposes an underwater real-time target detection algorithm based on wavelet convolution and CSPOMniKernel module improvement, called WCSPOmni-DETR. In order to strengthen the feature expression of fine-grained information in marine biological images, an additional shallow feature processing branch is introduced, and the processed S2 feature layer and deep information are comprehensively represented in the proposed CSPOMniKernel module. In addition, in order to make the scale of the algorithm better adapt to the hardware resources of the underwater platform, wavelet convolution is introduced into the BasicBlock of the backbone network to reduce the number of parameters while expanding the receptive field. Finally, the encoder of WCSPOmni-DETR adopts an efficient additive attention structure to produce faster underwater reasoning speed and more robust context representation. Experiments have shown that the proposed detection algorithm reduces the number of parameters by 25.5% compared to the baseline model, and improves the map50 indicator by 0.82% compared to the baseline on the URPC2021 dataset. The scale and accuracy of the model are comparable to the current most advanced real-time target detection model YOLO series. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the network structure diagram of the model WCSPOmni-DETR proposed in the present invention;

[0018] Figure 2 It is a wavelet convolution structure diagram;

[0019] Figure 3 It is the module structure diagram of the trunk improvement;

[0020] Figure 4 It is the network structure diagram of the RT-DETR model;

[0021] Figure 5 This is the structural diagram of the SPDConv convolution module;

[0022] Figure 6 It is the structure diagram of the CSPOMniKernel module proposed in the present invention;

[0023] Figure 7 This is the structure diagram of the attention module;

[0024] Figure 8 It is a curve chart of the training process;

[0025] Fig. 9 It is the comparison of confusion matrix before and after improvement;

[0026] Fig.10 It is the comparison of heat maps before and after the model improvement;

[0027] Fig.11 It is compared with the detection results of the YOLO model; DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings.

[0029] like Figure 1 As shown in the figure, a method for detecting marine life based on lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branch is presented. This method is an underwater real-time target detection method based on wavelet convolution and the proposed CSPOMNIKERN module. The backbone network of RT-DETR is improved by cascaded wavelet convolution for lightweight processing and the encoder adopts Efficient Additive Attention to enhance the real-time reasoning performance. In addition, the additional S2 feature layer in the backbone network is introduced and processed by the SPDConv unit to enhance the expression of fine-grained information, and then the global to local feature representation is performed in the CSPOMNIKERN module together with the deep features containing rich semantic information.

[0030] The present invention comprises the following steps:

[0031] Step 1: Prepare the dataset and perform preprocessing

[0032] The URPC2021 dataset was released in the 2021 National Underwater Robot Competition. The dataset contains four categories: sea cucumbers, sticklebacks, scallops, and starfish, with a total of 7,600 images. We randomly divided it into 6,080 images for training and 1,520 images for testing model performance. The sizes of the images in the dataset are mostly 3840×2160 and 720×405. The image size input to the model is 640×640.

[0033] Step 2: Use wavelet convolution to improve the backbone network of RT-DETR for lightweight processing

[0034] (2.1) Wavelet convolution

[0035] Shahaf E. Finder used wavelet transform (WT) to effectively increase the receptive field of convolution without being affected by over-parameterization, and thus proposed wavelet convolution (WTConv), and through cascading, it can also guide CNN to better respond to low frequencies. WTConv can effectively replace deep convolution (DwConv) to build a lightweight backbone network, while expanding the receptive field and improving the ability to capture fine-grained information in the image. Multi-level wavelet convolution such as Figure 2 shown.

[0036] The two-dimensional Haar wavelet convolution operates in combination in two dimensions, using a low-pass filter (f LL ) and 3 high-pass filters (f LH , f HL , f HH ) as filters for depthwise convolution with stride 2.

[0037]

[0038] The wavelet transform and inverse wavelet transform in the two-dimensional Haar wavelet convolution are shown in the following two formulas.

[0039] [X LL , X LH , X HL , X HH ]=Conv([f LL , f LH , f HL , f HH ],X)

[0040] X=Conv-transposed([f LL , f LH , f HL , f HH ],[X LL , X LH , X HL , XHH ])

[0041] All convolutions in wavelet convolution are depth-wise convolutions, and the convolution kernel size is 5×5.

[0042] (2.2) Lightweight backbone

[0043] Considering the real-time requirements of underwater target detection scenarios, this method uses the smallest RT-DETR-resnet18 as the benchmark model. Its backbone network resnet18 is composed of many standard convolutional units and residual edges stacked together. Each BasicBlock is as follows Figure 3 As shown in , it still has a large number of parameters (11.17M) and computational consumption. We replace the second standard convolution unit of BasicBlock with a cascaded wavelet convolution, as Figure 3 As shown in the figure, the size of the model backbone is effectively reduced (5.41M). The first convolution unit still uses standard convolution, which increases the number of channels while reducing the feature map.

[0044] Each level of cascaded wavelet convolution increases the size of the layer's receptive field, while the number of trainable parameters increases only slightly. Each convolution focuses on 4 different frequency bands of the input in an increasingly larger receptive field, and places more emphasis on low frequencies in the input, thereby making up for the lack of fine-grained information in deep convolutional networks and forming a feature extraction network that balances speed and performance. Experiments have shown that the model performance achieved by using a cascaded wavelet convolution of 2 levels is optimal, and the calculation formula for the second-level wavelet convolution is as follows.

[0045]

[0046]

[0047]

[0048] Where i = 1, 2, is the original input, represents the three high-frequency components in the i-th level wavelet convolution, represents a low-frequency component, and DwConv is a deep convolution. (i) is the output of the i-th level after DwConv and inverse wavelet transform. The final output needs to be added to the result of the previous layer after DwConv. The final efficient and lightweight backbone network is shown in the following table.

[0049] Lightweight backbone network structure table

[0050]

[0051]

[0052] Step 3: Constructing an enhanced fine-grained feature expression branch for RT-DETR

[0053] As the standard convolution process proceeds, the fine-grained spatial information in the feature map of the conventional deep convolution feature extraction backbone is gradually diluted. Although the various types of up- and down-sampling feature fusion neck networks that follow are often based on three-level multi-scale features and have made significant remedies, there is still room for improvement in the face of clustered small targets and blurred boundaries in the detection of marine life. Our approach is not to optimize the backbone in this regard, but to additionally introduce the S2 feature layer after SPDConv and perform global-to-local feature fusion with deep features containing rich semantic information in the proposed CSPOMniKernel module.

[0054] (3.1) RT-DETR model

[0055] The main feature of the RT-DETR model is that it uses an efficient hybrid encoder based on attention-based intra-scale feature interaction (AIFI) and CNN-based cross-scale feature fusion (CCFF). It fully combines the ability of convolutional computing to capture local information and the ability of Transformer to build complex long-distance dependencies. The structure of the RT-DETR model is as follows: Figure 4 shown.

[0056] (3.2)SPDConv

[0057] Marine biological target detection often encounters small organisms that gather in groups. They block each other and have blurred boundaries, which is a major difficulty in the field of marine biological target detection. Standard convolution seeks to expand the receptive field of features and dilutes fine-grained spatial information during execution. Sunkara and Luo specifically designed spatial-to-depth convolution (SPDConv) to solve this problem. Figure 5 As shown in Figure 1, the spatial information is converted to the channel dimension and then non-strided convolution is performed, thereby fully retaining these feature information.

[0058] (3.3) Fine-grained feature expression branch

[0059] As the standard convolution process proceeds, the fine-grained spatial information in the feature map of the conventional deep convolution feature extraction backbone is gradually diluted. Although the various types of up- and down-sampling feature fusion neck networks that follow are often based on three-level multi-scale features and have made great remedies, there is still room for improvement in the face of clustered small targets and blurred boundaries in the detection of marine life. Our approach is not to optimize the backbone in this regard, but to additionally introduce the S2 feature layer after SPDConv and perform global-to-local feature fusion with the deep features containing rich semantic information in the proposed CSPOMniKernel module. The shallow fine-grained spatial information is fully retained in the SPDConv unit, so that the features fused by the CSPOMniKernel module can more fully express the features of the clustered organisms and blurred boundaries in the image, and can effectively detect details.

[0060] The Omni-Kernel Module is a full-kernel module consisting of three branches: the global branch, the large branch, and the local branch. It effectively learns the global to local feature representation. The module's pre- and post-processing uses 1×1 convolution. In the scenario of this experiment, the complete Omni-Kernel Module has certain redundant calculations, and the model scale increased by the introduction of the module exceeds the requirements of the real-time target detection model. Therefore, we combine the idea of ​​Cross Stage Partial Network (CSP) to construct global to local features for part of the input feature map, and the rest still retains its original characteristics and is processed by the downsampling feature fusion process of the CCFF structure in RT-DETR. The CSP OmniKernel module structure is as follows: Figure 6 shown.

[0061] The large branch mainly uses cheap deep convolution with a kernel size of K×K to pursue a large receptive field. In this paper, K=31. The local branch is a 1×1 deep convolution. The global branch increases the global modeling function through dual-domain processing, including the dual-domain channel attention module (DCAM) and the frequency-based spatial attention module (FSAM). The calculation formulas are as follows.

[0062]

[0063]

[0064]

[0065] Where x is the feature map before input to the branch, F and IF are the fast Fourier transform and its inverse operation respectively, and AvgPool is the global average pooling layer. The derivation process of the proposed CSPOMniKernel module is shown below.

[0066] Branch, Identity = Split channel (e, x)

[0067] x Large =DwConv 31×1 (Branch)+DwConv 31×31 (Branch)+DwConv 1×31 (Branch)

[0068] X out =Concat(DwConv 1×1 (Branch)+X Large +X FSAM (Branch, Identity)

[0069] Where e is the ratio of the original feature to be split along the channel. In this paper, e = 0.25. At this time, the number of parameters and the amount of calculation reduced by the original OmniKernel module are 1.14M and 9.9G respectively. Branch is the split feature for complex processing, accounting for 1 / 4 of the original feature. DwConv is the depth convolution, X FSAM (Branch) is the result of replacing the input x in the continuous derivation process composed of the above equations (1), (2), and (3) with Branch.

[0070] Step 4: Improve the AIFI structure of RT-DETR

[0071] The AIFI module of the benchmark RT-DETR uses the MHSA mechanism, which performs self-attention calculation on the last layer of the multi-scale feature map extracted by the backbone. Although the smallest high-semantic information feature map is used, the attention calculation here is still a time-consuming operation due to its quadratic complexity with the number of tokens. The hardware resources of the underwater detection platform are limited, so an efficient calculation method is required. Here we replace MHSA with Efficient Additive attention to form Efficient Additive Intra-Scale Feature Interaction, which effectively reduces the inference time and generates attention feature maps with more robust context representation.

[0072] (4.1)Efficient Additive attention

[0073] The VisionTransformer model is based on the self-attention mechanism, such as Figure 7As shown. The input is projected into query (Q), key (K) and value (V) matrices through linear transformation, and then the attention scores between tokens of Q and K are calculated and interacted with V to construct the long-distance dependency relationship of features in the input feature graph. N is the number of tokens, d is the hidden dimension, and as the number of tokens increases, the calculation and memory requirements of Q·K grow quadratically, resulting in slow inference speed and high memory usage. The computational complexity is O(N 2 d). The calculation formula is:

[0074]

[0075] To address the above problems, Abdelrahman Shaker et al. introduced Efficient Additive Attention, such as Figure 7 As shown. Experiments have shown that key-value interactions can be replaced by linear layers without sacrificing any accuracy. This mechanism effectively replaces quadratic matrix multiplication operations with linear element multiplications, has a linear complexity relationship with the number of tokens, and produces a more robust contextual representation. The calculation process is shown below:

[0076]

[0077] Query matrix Q∈R N×d Multiply by the learnable parameter vector w∈R d Get the query attention weight vector α∈R N .

[0078]

[0079] Then each value of the attention weight vector is multiplied and summed with each column in the query matrix to obtain a single global query vector q∈R d .

[0080]

[0081] Finally, the normalized query matrix is ​​added to the key matrix multiplied element-wise with the q broadcast to obtain the output attention feature map.

[0082] Step 5: Set training hyperparameters

[0083] Our experiments were all completed on a server with a Linux operating system, the operating system version is Ununtu20.04, the CPU configuration is 16vCPU Intel(R)Xeon(R)Platinum 8481C, the GPU configuration is GeForceRTX 4090D GPU, and the Pytorch version is 2.0.0. The specific hyperparameter settings of the experiment are shown in the following table.

[0084] Experimental hyperparameter settings

[0085]

[0086] Step 5: Input the dataset into the model for training

[0087] According to step 2, step 3, and step 4, the WCSPOmni-DETR marine biological detection model proposed by the present invention is obtained. In step 1, the training data set is processed, and the training hyperparameters are set in step 5. Next, the training set and the validation set need to be input into the model for training. After 160 rounds of forward derivation, loss value calculation, back propagation, gradient update, and indicator calculation on the validation set, a set of optimal weight parameters is obtained, which is saved for obtaining prediction results.

[0088] Figure 8 The changing trend of each single indicator during the training process is shown. After 160 epochs of training, the positive indicators of the model (precision, recall, mAP50, mAP50:95) have converged on both datasets, and the regression box loss (giou loss), classification loss (cls loss) and 11loss have also converged on the validation set.

[0089] Step 6: Enter test data to obtain prediction results

[0090] Use the optimal set of weight parameters obtained in step 5 to forward deduce new marine life images in the test set, obtain the predicted results and test the final performance of the model. Fig. 9 This is a comparison of the confusion matrices of the models before and after the improvement. There are 5 categories in total, namely holothurian, echinus, scallops, starfish, and a background category background. The values ​​on the main diagonal represent the ratio of the model's correct classification. Among them, the detection accuracy of holothurian and scallops has been improved. The values ​​on the non-main diagonal represent the ratio of the model misclassifying one category as another. We can see that the misclassification is mainly identified as background, and the misclassification between classes is less, and the misclassification is also about 0.01. This shows that the possibility of confusing categories in the optimized model is low, and the target is mainly identified as background.

[0091] In order to verify the enhancement effect of the proposed improved module on the model, we combined different improvement points to establish 7 groups of burning experiments. The experimental results are shown in the following table.

[0092] Ablation experiment at URPC2021

[0093]

[0094] (1) Precision: The proportion of correct positive samples among all instances identified as positive samples.

[0095]

[0096] (2) Recall (Recal1): The proportion of all positive samples that are correctly identified.

[0097]

[0098] (3) Mean Average Precision (mAP): a comprehensive measure of the detection performance of the target detection algorithm.

[0099]

[0100] Among them, TP (True Positives) represents the number of positive samples correctly identified as positive samples, FP (False Positives) represents the number of negative samples incorrectly identified as positive samples, FN (False Negatives) represents the number of positive samples incorrectly identified as negative samples, TN (True Negative) represents the number of negative samples correctly identified as negative samples, Q R is the number of categories. Among them, the model that only uses wavelet convolution to improve the backbone has the smallest parameters and computational complexity, but the performance is relatively poor and still needs to be improved. The model that adds a global to local feature fusion module branch to the baseline model and improves AIFI has relatively the best performance, but the computational complexity increases by 14.5% compared to the baseline model. Finally, the proposed WCSPOmni-DETR model achieved a balance, with the number of parameters and computational complexity reduced by 25.5% and 10.2% respectively compared with the baseline model, and the mAP0.5 index increased by 0.82 percentage points on the URPC2021 dataset, respectively. In this group of burning experiments, a balance between model scale and accuracy was achieved. Fig.10 The comparison of the detection heatmaps of the baseline model and WCSPOmni-DETR for 5 test images is shown. From the distribution of the significant regions of the heatmap, WCSPOmni-DETR is more delicate in detecting clustered targets. The higher heat values ​​more accurately focus on the key parts of the organisms, and there are obvious heat value valleys between clustered organisms. In the last row of the detection of a single sea cucumber, it can be seen intuitively that the proposed model is affected by the additional fine-grained feature branches, focusing more on the detailed feature parts of the organisms, and judging the entire target through the multiple thorny structures of the sea cucumber.

[0101] In order to verify the superiority of the performance of the proposed WCSPOmni-DETR model, I compared it with some of the most advanced general target detection models on different datasets. These models are mainly divided into three categories: 1) Two-stage models: mainly the R-CNN series, among which the relatively new and powerful Faster R-CNN is selected. 2) Single-stage models: Taking the YOLO series as a typical example, Yolov5, Yolov8, Yolov9 and Yolov10 are selected, as well as other single-stage models including CenterNet and TOOD. 3) Transformer-based models: Some models developed based on DETR include Deformable DETR, DAB-DETR, DINO, Lite-DETR and RT-DETR. If there are multiple scales of comparison models, the scale of similar or smaller scale is selected. All training does not use pre-trained weights, and all training starts from scratch until the model converges. The comparison results on the URPC2021 dataset are shown in the following table. Among them, black bold and gray bold represent the best and second best indicators among all comparison models.

[0102] Comparison results of different models on the URPC2021 dataset

[0103]

[0104]

[0105] The proposed WCSPOmni-DETR detection method has achieved optimal or suboptimal performance in all indicators, proving that our model has high detection accuracy for marine organisms while effectively controlling the scale of the model, achieving a balance between model accuracy and scale, and can be applied to scenarios where the hardware resources of underwater target detection platforms are limited. Compared with other models, the detection performance of DAB-DETR, which has a smaller computational workload, on the URPC2021 dataset is quite different from our WCSPOmni-DETR model, while Lite-DETR is the Transformer-based model with the closest performance to WCSPOmni-DETR, but its parameter volume and computational workload are both more than twice that of WCSPOmni-DETR. In general, the models of the YOLO series are close to the scale and performance of our model. The latest YOLOv10m has the best parameter volume and computational workload among YOLOs of the same level, but its performance is relatively poor. The performance of YOLOv9m is closest to our model. Its Recall and mAP0.5:0.95 indicators are 0.19 points and 0.56 points higher than WCSPOmni-DETR, while Precision and mAP0.5 lag behind WCSPOmni-DETR by 0.29 and 0.23 percentage points, respectively. Although the detection performance is comparable to that of YOLOv9m, the scale of our model has a greater advantage over YOLOv9m, with 5.17M fewer parameters and 25.3GFloats less computation than YOLOv9m, respectively. In summary, the proposed WCSPOmni-DETR model achieves the best performance among the above-mentioned comparison models and is more suitable for marine biological detection.

[0106] The two best performing models among the above comparison models, YOLOv8m and YOLOv9m, were compared with WCSPOmni-DETR for actual marine life detection. The test results are as follows Fig.11As shown in the figure. The black boxes in the leftmost column of the original image represent the target instance, and the model name next to it is the model that did not detect the instance. The white box and the model name next to it represent that the model detected the background as the target instance. Several images of marine organisms with typical characteristics were selected, including the following types: 1) The boundary between the organism and the seabed environment is blurred and difficult to distinguish, 2) The same or different organisms are densely grouped and block each other, 3) The camera is far away and the organism is small, 4) A single large organism. The only three sea cucumbers with blurred boundaries in Figure (a) are marked with black boxes. YOLOv8 has the error of detecting the background as the target. YOLOv9 and WCSPOmni-DETR have correctly detected the three sea cucumbers, but our model has a higher confidence than YOLOv9 for these three sea cucumbers. In Figure (b), there are clusters of different types of organisms, which poses a certain challenge to these detectors. Both YOLOv8 and YOLOv9 failed to detect the sea cucumber in the lower left corner, which gradually fades in color. Our model also missed the sea cucumber in the middle. YOLOv8 and YOLOv9 also failed to completely distinguish the two partially overlapping targets. Figure (c) is mainly a cluster of similar organisms. The target sizes are generally moderate. Except for YOLOv8, which mistakenly detects the background in the upper left corner as a target instance, the performance of other detectors is generally good. In Figure (d), there are individual small targets. Our model shows good detection performance in this case, and correctly detects all the target instances in the image, especially the small sea urchin on the right, which is missed by YOLOv8 and YOLOv9. Figure (e) is a single large target. YOLOv8 failed to completely mark this single large target, and YOLOv9 had a slight positioning deviation. Only our model accurately identified it. From the detection of the above representative marine life images, it can be seen that the proposed WCSPOmni-DETR has a relatively strong control over details, thanks to the global to local feature fusion branch based on the S2-level feature map. It effectively retains the shallow spatial information, and after the down-sampling feature fusion, the target detection head of RT-DETR can use more efficient features to locate the target regression frame.

[0107] This paper proposes a lightweight RT-DETR marine life detection model based on enhanced fine-grained feature expression branches, which can have high marine life detection performance on the basis of adapting to the hardware conditions of the underwater target detection platform. The model uses a lightweight backbone improved by secondary wavelet convolution as the feature extraction backbone network, reducing the number of parameters of the original backbone by 51.7%. Secondly, the S2 shallow feature is extracted and converted to the channel dimension after SPDConv, and then the shallow and deep features are fused from global to local in the proposed CSPOMniKernel module, which enhances the fine-grained features in the fused features and effectively strengthens the model to handle small marine organisms in groups. At the same time, it can also effectively deal with the blurred boundary between organisms and the environment caused by reduced contrast and color distortion in the seabed scene. Finally, the improved Efficient Additive AIFI can further increase the reasoning efficiency of the model and the generated attention feature map has a more robust context representation, which improves the ability of MHSA to construct global feature representation to a certain extent. Through experiments, the effectiveness of the proposed model is proved by various indicators and instance detection. Compared with the baseline model, the number of parameters and the amount of calculation of the model have been effectively reduced (25.5%, 10.2%), while also improving the detection accuracy of the model. Although WCSPOmni-DETR has been improved, when facing small organisms that gather in groups and have blurred boundaries, accurate detection is still a direction that needs to be pursued.

Claims

1. A method for detecting marine organisms based on a lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branches, characterized in that: The following steps are involved: Step S1: obtaining a marine biological target detection data set and performing preprocessing; the marine biological target detection data set includes image data of marine organisms; Step S2: Construct a WCSPOmni-DETR model, the input of which is the image data of marine organisms, and the output is the predicted value of the type of marine organisms in the image: Obtain the RT-DETR model and the wavelet convolution module, and use the wavelet convolution module to replace the second convolution module of the BasicBlock module in the backbone network of the RT-DETR model to obtain the improved backbone extraction network; Obtain an Omni-Kernel module, wherein the Omni-Kernel module is a full-kernel module consisting of three branches: a global branch, a large branch, and a local branch; The Cross Stage Partial Network architecture is used to lightweight improve the Omni-Kernel module. The Omni-Kernel module performs convolution operations on parts of the input feature map, and the rest is directly concatenated with the convolution result to obtain the CSP OmniKernel module. The enhanced fine-grained feature expression branch of the WCSPOmni-DETR model is constructed using the CSPOMNIKernel module and the SPDConv module, including: extracting the s2 shallow feature map in the RT-DETR model backbone network, convolving it with the deep features through SPDConv, and performing global-to-local feature fusion in the CSPOMNIKernel module to enhance the expression of fine-grained features; Get the Efficient Additive Attention module and use it to improve the AIFI module structure of the RT-DETR model to obtain the WCSPOmni-DETR model. Step S3: Use the marine biological target detection dataset to train the WCSPOmni-DETR model to obtain a trained WCSPOmni-DETR model; Step S4: Get the image to be tested, input the trained WCSPOmni-DETR model, and get the prediction result.

2. The method for detecting marine organisms based on a lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branch as claimed in claim 1, characterized in that: The obtaining of the Efficient Additive Attention module and using it to improve the AIFI module structure of RT-DETR specifically includes: replacing the original MHSA module of the AIFI module with the Efficient Additive Attention module.

3. The method for detecting marine organisms based on a lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branch as claimed in claim 1, characterized in that: The scale of the RT-DETR model used in step S2 is RT-DETR-Resnet18.

4. The method for detecting marine organisms based on a lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branch as claimed in claim 1, characterized in that: In the step S2, when the Omni-Kernel module is light-weighted using the idea of ​​Cross Stage Partial Network, the ratio of convolution operation on the feature map is 1 / 4.

5. The method for detecting marine organisms based on a lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branch as claimed in claim 1, characterized in that: The step S3 also includes the following hyperparameter settings for the WCSPOmni-DETR model: the training rounds are set to 160, the batch size is set to 16, the optimizer is AdamW, and the learning rate is set to 0.0001.

6. A method for detecting marine organisms based on a lightweight WCSPOmni-DETR with enhanced fine-grained feature expression branches as claimed in claim 1, Features: The marine organisms in step S1 include starfish, corals, sea urchins, sea cucumbers, and scallops.

Images