Underwater feature degraded marine organism detection method based on dynamic convolution

Abstract

The invention discloses a dynamic convolution-based underwater feature degradation marine organism detection method, which comprises the following steps of: establishing a marine organism image data set with an underwater feature degradation problem, remodeling a cross-channel residual block by using dynamic convolution, defining the cross-channel residual block as a dynamic residual block, and establishing a backbone network and a feature fusion network based on the dynamic residual block; designing a dynamic sparse detection head, and taking the dynamic sparse detection head as an output network; integrating the network framework, and constructing an underwater feature degraded marine organism detection network based on dynamic convolution; and training the underwater feature degradation marine organism detection network by using the training set to obtain a marine organism detection model, deploying the marine organism detection model, and adjusting hyper-parameters so as to be used for marine organism target detection. According to the method, a Haar wavelet transform mode is introduced to pre-process the image, high-frequency noise of the image is effectively removed, and the problem of low contrast of the underwater image caused by uneven illumination and color degradation is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular, especially relates to a kind of underwater feature degradation marine organism detection method based on dynamic convolution. BACKGROUND

[0002] Marine organisms are an important part of marine ecosystems, and the rational development of marine resources is of great significance. Although existing resource development techniques have achieved certain results, the fishing link of marine products still needs to be improved, and a precise and efficient marine organism detection method is urgently needed to assist underwater operations of robots and improve work efficiency.

[0003] Marine organism detection is one of the most advanced research directions in the field of computer vision, and has attracted the attention of many scholars in recent years. With the rise of deep learning technology, many mainstream detection models have been applied to marine organism detection tasks. For example, single-stage anchor-based detection methods represented by YOLOv5 and SSD, and attention-based detection methods represented by DETR.

[0004] In actual underwater environments, the selective absorption of water to light and the scattering effect of light cause image degradation characteristics such as color deviation, low contrast, and detail blur. At the same time, some marine organisms are attached to habitats with similar shapes, making it difficult to distinguish between targets and backgrounds, further weakening feature expression. Therefore, it is urgent to design a detection method that is highly adaptable to underwater feature degradation environments to achieve efficient detection of marine organisms. SUMMARY

[0005] To solve the above technical problems, a marine organism detection method based on dynamic convolution is provided. The present application establishes a marine organism image dataset with underwater feature degradation problems, and pre-processes and divides the dataset. Dynamic convolution is used to reshape the cross-channel residual block into a dynamic residual block, and a backbone network and a feature fusion network using dynamic residual blocks are built. A dynamic sparse detection head is designed as an output network. The network framework is integrated to build a marine organism detection network based on dynamic convolution. Finally, the detection network is trained on the dataset to obtain a marine organism detection model, and the trained model is deployed and the hyperparameters are adjusted for marine organism detection tasks.

[0006] The technical means adopted by the present application are as follows: A marine organism detection method based on dynamic convolution includes: S1, a marine organism image dataset with underwater feature degradation problems is established, and the dataset is pre-processed and divided; S2, dynamic convolution is used to reshape the cross-channel residual block into a dynamic residual block, and a backbone network and a feature fusion network using dynamic residual blocks are built; S3, design a dynamic sparse detection head as an output network; S4, integrate the network framework to build an underwater feature degradation marine organism detection network based on dynamic convolution; S5, train the detection network on the data set to obtain a marine organism detection model, deploy the trained model and adjust the hyperparameters for marine organism detection tasks.

[0007] Further, in S1, the steps of establishing a marine organism image data set with underwater feature degradation problems and preprocessing and dividing the data set include: Use the local sensitive hashing algorithm to evaluate the image redundancy of the public data set Detecting Underwater Objects (DUO), and remove repeated and invalid labeled images; Read the images in the marine organism image data set with underwater feature degradation problems, use the Haar wavelet transform method to enhance the image details, and generate image set A; Read the images in image set A, scale the image size to a uniform size, generate a training image set B, and randomly divide the training set and test set according to a 7:3 ratio.

[0008] Further, in S2, the step of using dynamic convolution to reshape the cross-channel residual block includes: Replace the convolution module in the feature extraction block with a dynamic convolution module; replace the ordinary convolution in the cross-channel residual block with a dynamic convolution module; apply the improved feature extraction block to the cross-channel residual block to obtain a dynamic residual block.

[0009] Further, in S3, the step of designing a dynamic sparse detection head as an output network includes: Build a scale-aware module to enhance the model's ability to adjust the importance of different levels of features; Build a spatial-aware module to enhance the model's spatial modeling ability for different scale and shape targets; Build a channel-aware module to enhance the model's adaptability to different tasks; Combine the modules constructed in the above steps to obtain a dynamic sparse detection head, which accepts input feature maps as an output network and outputs prediction results.

[0010] Further, in S4, the scale-aware module calculation process includes: Perform 2x2 average pooling on the input feature map; Perform 3x3 convolution on the input feature map; Perform linear rectification processing on the input feature map; Perform hard Sigmoid normalization processing on the input feature map; The attention matrix obtained in the previous step is multiplied element by element with the input feature map to obtain an output feature map; Further, in S4, the spatial perception module calculation process includes: Encoding the standard 3x3 convolution sampling points; Divide the input underwater image into G branches along the channel dimension, and the number of channels in each branch is G times the input channel; 3x3 convolution is performed on each branch feature map to obtain modulation and offset; Add and normalize the output modulation and offset of each branch to obtain global modulation and global offset; Apply the global modulation and global offset to the input feature map to obtain a new feature map.

[0011] Further, in S4, the channel perception module calculation process includes: 2x2 average pooling is performed on the input feature map; Full connection is performed on the input feature map; Linear rectification processing is performed on the input feature map; Full connection is performed on the input feature map; Regularization processing is performed on the input feature map to obtain an activation threshold, and a dynamic correction linear unit is input; The feature map is input into the dynamic correction linear unit to obtain an output feature map; Further, in S4, the integration network framework, the step of building an underwater feature degradation marine organism detection network based on dynamic convolution includes: Build an input layer to receive input underwater images; Build a backbone network applying dynamic residual blocks and convolution modules to extract input image features; Build a neck network applying dynamic residual blocks and spatial pyramid pooling layers to fuse multi-scale features; And the fused features are input into a dynamic sparse detection head to generate detection results; Combine the network constructed by the above steps to obtain an underwater feature degradation marine organism detection method based on dynamic convolution.

[0012] Further, in S5, the process of deploying the trained model and adjusting the hyperparameters for the marine organism detection task includes: Deploy the marine organism detection model on the detection device; Read the parameter configuration file and load the pre-trained model weight; Adaptively adjust the hyperparameters and configure the model processing rate; Read the input image and pre-process the input image; The preprocessed image is sent into the marine organism detection model to perform target prediction; The position and category information in the detection result are visualized. The underwater feature degradation marine organism detection method based on dynamic convolution disclosed in the present application introduces a Haar wavelet transform method for image preprocessing, effectively removes image high-frequency noise, and improves the low-contrast problem of underwater images caused by uneven illumination and color degradation. In addition, the method replaces the ordinary convolution with dynamic convolution to improve the traditional cross-channel residual block and enhance the adaptive feature extraction capability of the backbone network, improving the model parameter efficiency and generalization capability to solve the marine organism image feature degradation problem, and improving the detection accuracy and stability of the algorithm for feature degradation marine organism images. At the same time, the method proposes a dynamic sparse detection head as an output network, which can be optimized for different levels, and continuously combines multiple self-attention mechanisms among feature levels, spatial positions and output channels, significantly improving the representation ability of the target detection head, and without additional computational overhead, improving the expression ability and task adaptability of the marine organism detection model. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0014] Figure 1 is a flowchart of the underwater feature degradation marine organism detection method based on dynamic convolution provided by the present application.

[0015] Figure 2 is a network structure schematic diagram of the single-stage anchor-free box detection algorithm provided by the present application.

[0016] Figure 3 is a flowchart of the feature extraction block and the dynamic residual block.

[0017] Figure 4 is a flowchart of the dynamic sparse head.

[0018] Figure 5 is the comparative experiment result of the detection method compared with other mainstream detection methods; Figure 6 is a visualization schematic diagram of the detection result of the underwater feature degradation marine organism detection method based on dynamic convolution DETAILED DESCRIPTION In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] As Figure 1 shown, the present application provides a method for detecting marine organisms with underwater feature degradation based on dynamic convolution, comprising: S1, establishing a marine organism image dataset with underwater feature degradation problem, and preprocessing and dividing the dataset; In specific implementation, as a preferred embodiment of the present application, the public dataset Detecting Underwater Objects (DUO) is processed, the local sensitive hashing algorithm is used to evaluate the image redundancy of the dataset, and the images with repeated and invalid labels are removed; the images in the marine organism image dataset with underwater feature degradation problem are read in, the Haar wavelet transform method is used to enhance the image details, and an image set A is generated; the images of the image set A are read in, the image size is scaled to a unified size, a training image set B is generated, and the training set and the test set are randomly divided according to a 7:3 ratio.

[0021] S2, using dynamic convolution to reshape the cross-channel residual block into a dynamic residual block, and building a backbone network and a feature fusion network applying the dynamic residual block; In a specific implementation, as a preferred embodiment of the present application, the convolution module in the feature extraction block is replaced by a dynamic convolution module; the ordinary convolution in the cross-channel residual block is replaced by a dynamic convolution module; the improved feature extraction block is applied to the cross-channel residual block to obtain a dynamic residual block; and finally, the dynamic residual block, the down-sampling convolution block, the spatial pyramid pooling block and the up-sampling convolution block are organically combined to build the backbone network and the neck network for the image feature extraction and fusion stage.

[0022] S3, design a dynamic sparse detection head as an output network; In a specific implementation, as a preferred embodiment of the present application, the scale perception module is built to enhance the model's ability to adjust the importance of different level features, and the calculation process includes: performing 2x2 average pooling on the input feature map; performing 3x3 convolution on the input feature map; performing linear rectification processing on the input feature map; performing Sigmoid normalization processing on the input feature map; and multiplying the attention matrix obtained in the previous step with the input feature map element by element to obtain an output feature map.

[0023] The spatial perception module is built to enhance the model's spatial modeling ability for different scale and shape targets, and the calculation process includes: encoding the standard 3x3 convolution sampling points; dividing the input underwater image into G branches along the channel dimension, and the number of channels of each branch is G times of the input channel; performing 3x3 convolution on the feature maps of each branch to obtain modulation and offset; adding and normalizing the output modulation and offset of each branch to obtain global modulation and global offset; and applying the global modulation and global offset to the input feature map to obtain a new feature map.

[0024] The channel perception module is built to enhance the model's adaptability to different tasks, and the calculation process includes: performing 2x2 average pooling on the input feature map; performing full connection on the input feature map; performing linear rectification processing on the input feature map; performing full connection on the input feature map; performing regularization processing on the input feature map to obtain an activation threshold, and inputting the dynamic correction linear unit; and inputting the feature map into the dynamic correction linear unit to obtain an output feature map.

[0025] The modules built in the above steps are combined to obtain a dynamic sparse detection head, which is used as an output network to accept an input feature map and output a prediction result.

[0026] S4, integrate the network framework to build an underwater feature degradation marine organism detection network based on dynamic convolution; In specific implementation, as a preferred embodiment of the present application, the input layer is built to receive the input underwater image; the backbone network applying the dynamic residual block and the convolution module is built to extract the input image features; the neck network applying the dynamic residual block, the up-sampling block and the spatial pyramid pooling layer is built to fuse the multi-scale features; and the fused features are input into the dynamic sparse detection head to generate the detection results; and the network constructed through the above steps is combined to obtain the underwater feature degradation marine organism detection network based on dynamic convolution.

[0027] S5, deploying the trained model and adjusting the hyperparameters for the marine organism detection task In specific implementation, as a preferred embodiment of the present application, the marine organism detection model is deployed on a detection device; a parameter configuration file is read and the pre-trained model weight is loaded; the hyperparameters are adaptively adjusted and the model processing rate is configured; an input image is read and preprocessed; the preprocessed image is input into the marine organism detection model to perform target detection; and the position and category information in the detection result are visualized.

[0028] Embodiment As shown in Figure 1 , the present application provides a dynamic convolution-based underwater feature degradation marine organism detection method The data set is obtained by processing the public data set Detecting Underwater Objects (DUO), and the local sensitive hashing algorithm is used to evaluate the image redundancy of the data set, and the images with repeated and invalid labels are removed. Finally, 4973 images are retained, accounting for 62% of the original data set. The data set includes four types of objects: sea cucumber, sea urchin, starfish and scallop, and is randomly divided into training set and test set in a ratio of 7:3.

[0029] Training process: all experiments are carried out on an Ubuntu 22.04 workstation equipped with an Intel Core i7 processor, three NVIDIA RTX A6000 gpus and 32 GB of memory. The experiment is implemented using PyTorch 1.12.1, CUDA 11.6 and cuDNN 8.3.2 versions, and the number of training rounds is set to 200. The experimental results show that the multi-scale marine benthic organism detection method based on global-local attention improves the AP, AP 50 and AP 75 indices by 3.25%, 1.20% and 4.07% respectively compared with the benchmark model.

[0030] As shown in Figure 2 , the present embodiment provides a single-stage anchor-free detection algorithm.

[0031] As shown in Figure 3As shown, this embodiment provides a dynamic residual block for reshaping cross-channel residual blocks using dynamic convolution, used to build the backbone network and neck network. The convolutional modules in the feature extraction block are replaced with dynamic convolutional modules; the ordinary convolutions in the cross-channel residual block are replaced with dynamic convolutional modules; and the improved feature extraction block is applied to the cross-channel residual block to obtain the dynamic residual block.

[0032] like Figure 4 As shown, this embodiment provides a dynamic sparse detection head as the output network, which is used to build a scale-aware module: 2×2 average pooling is performed on the input feature map; 3×3 convolution is performed on the input feature map; linear rectification is performed on the input feature map; sigmoid normalization is performed on the input feature map; and the attention matrix obtained in the previous step is multiplied element-wise with the input feature map to obtain the output feature map. A spatial awareness module is built: standard 3×3 convolution sampling points are encoded; the input underwater image is divided into G branches along the channel dimension, with each branch having 1 / G of the number of input channels; 3×3 convolution is performed on the feature map of each branch to obtain the modulation and offset; the output modulation and offset of each branch are added and normalized to obtain the global modulation and global offset; the global modulation and global offset are applied to the input feature map to obtain a new feature map. The channel sensing module is constructed as follows: 2×2 average pooling is performed on the input feature map; a fully connected layer is formed on the input feature map; linear rectification is performed on the input feature map; a fully connected layer is formed on the input feature map; regularization is performed on the input feature map to obtain the activation threshold, which is then input into the dynamic correction linear unit; the feature map is then processed by the dynamic correction linear unit to obtain the output feature map.

[0033] The modules constructed in the above steps are combined to obtain a dynamic sparse detection head, which serves as the output network to receive input feature maps and output detection results.

[0034] like Figure 5 As shown, this embodiment provides comparative experimental results of the underwater feature degradation marine organism detection method based on dynamic convolution and other mainstream detection methods in marine organism detection tasks.

[0035] like Figure 6 As shown in the figure, this embodiment provides a visualization diagram of the detection results of the underwater feature degradation marine organism detection method based on dynamic convolution. The position of the detection box represents the spatial position of the target, and the number above the detection box represents the confidence level of the detection result.

[0036] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting underwater feature degradation of marine organisms based on dynamic convolution, characterized in that, include: A dataset of marine biological images with underwater feature degradation was established, and the dataset was preprocessed and divided into training and testing sets. Dynamic convolution is used to reshape cross-channel residual blocks and define them as dynamic residual blocks. A backbone network and feature fusion network based on dynamic residual blocks are then established. Design a dynamic sparse detection head and use it as the output network; Integrate network frameworks to build a marine organism detection network based on dynamic convolution for underwater feature degradation; A marine life detection model was obtained by training an underwater feature degradation marine life detection network using a training set. The marine life detection model was then deployed and its hyperparameters were adjusted for use in marine life target detection.

2. The method for detecting marine organisms with underwater feature degradation based on dynamic convolution according to claim 1, characterized in that: when preprocessing the dataset: the locality-sensitive hashing algorithm is used to evaluate the duplication of the acquired marine organism image dataset, and duplicate and invalid labeled images are removed; images in the marine organism image dataset with underwater feature degradation problems are read; the Haar wavelet transform method is used to enhance image details and generate image set A; images in image set A are read; the image size is scaled to a uniform size to generate image set B to be trained; and the training set and test set are randomly divided according to a 7:3 ratio.

3. The method for detecting underwater feature degradation marine organisms based on dynamic convolution according to claim 1, characterized in that: when reshaping the cross-channel residual block using dynamic convolution: the convolution module in the feature extraction block is replaced with a dynamic convolution module, the ordinary convolution in the cross-channel residual block is replaced with a dynamic convolution module, and the improved feature extraction block is applied to the cross-channel residual block to obtain the dynamic residual block.

4. The method for detecting underwater feature degradation marine organisms based on dynamic convolution according to claim 1, characterized in that: when designing the dynamic sparse detection head: a scale-aware module is built to enhance the model's ability to adjust the importance of features at different levels; a spatial-aware module is built to enhance the model's ability to spatially model targets of different scales and shapes; a channel-aware module is built to enhance the model's adaptability to different tasks; the scale-aware module, spatial-aware module, and channel-aware module are combined to obtain the dynamic sparse detection head, and the dynamic sparse detection head is used as the output network.

5. The method for detecting underwater feature degradation marine organisms based on dynamic convolution according to claim 4, characterized in that: the scale-aware module performs average pooling, convolution, linear rectification and sigmoid normalization on the input feature map to obtain an attention matrix, and multiplies the attention matrix with the input feature map element by element to obtain an output feature map.

6. A method for detecting marine organisms with degraded underwater features based on dynamic convolution according to claim 4, characterized in that: the spatial perception module encodes standard convolution sampling points; the input underwater image is divided into G branches along the channel dimension, with each branch having one-Gth of the number of channels as the input channels; convolution operations are performed on the feature maps of each branch to obtain modulation and offset; the output modulation and offset of each branch are added together and normalized to obtain global modulation and global offset; the global modulation and global offset are applied to the input feature map to obtain a new feature map.

7. The method for detecting underwater feature degradation marine organisms based on dynamic convolution according to claim 4, characterized in that: the channel sensing module performs average pooling, fully connected, linear rectification and regularization processing on the input feature map to obtain an activation threshold, and inputs the activation threshold into the dynamic correction linear unit to obtain the output feature map.

8. The method for detecting underwater feature degradation marine organisms based on dynamic convolution according to claim 4, characterized in that: when constructing the underwater feature degradation marine organism detection network based on dynamic convolution: Build an input layer to receive underwater images; A backbone network using dynamic residual blocks and convolutional modules is constructed to extract features from the input image; A neck network with dynamic residual blocks and spatial pyramid pooling layers is constructed to fuse multi-scale features; The fused features are then fed into a dynamic sparse detection head to generate detection results. The input layer, backbone network, neck network, and dynamic sparse detection head are combined to obtain an underwater feature degradation marine organism detection network based on dynamic convolution.

9. The method for detecting underwater feature degradation marine organisms based on dynamic convolution according to claim 4, characterized in that: when deploying and adjusting the hyperparameters of the marine organism detection model: deploying the marine organism detection model on the detection device, reading the parameter configuration file and loading the pre-trained model weights, adaptively adjusting the hyperparameters and configuring the model processing rate; reading the input image and preprocessing the input image; The preprocessed image is fed into a marine organism detection model to perform target prediction. Visualize the location and category information in the detection results.

Images