An underwater target detection method based on an improved YOLOv6 algorithm

Abstract

This invention discloses an underwater target detection method based on an improved YOLOv6 algorithm. By improving the backbone and neck layers, a lightweight structure is designed as the backbone network, combining lightweight convolutional ODConv and the lightweight network EfficientNetv2. Then, the neck layer is improved using lightweight convolutional GSConv and VoVGSCSP methods to enhance the network's ability to detect small targets. Simultaneously, an SPD-Conv module is introduced to improve the detection accuracy of underwater targets, and the Adan optimizer is incorporated to improve network performance. This invention addresses the problem of interfering targets in underwater target detection datasets by performing data cleaning, followed by training, validation, and testing experiments on the dataset. This invention is more lightweight, achieving high detection accuracy for various specific targets in complex underwater environments. It reduces the number of model parameters and computational cost, making it more suitable for mobile or embedded devices, while also increasing the model's inference speed, resulting in a more lightweight and real-time algorithm.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and in particular to an underwater target detection method based on an improved YOLOv6 algorithm. Background Technology

[0002] With the rapid development of computer vision, underwater target detection technology based on optical images has been widely applied, playing a significant role in marine fisheries, aquaculture, and marine pollution control. In marine fisheries, traditional seafood harvesting methods primarily rely on manual diving. However, this method is not only inefficient, but also requires workers to have extensive diving experience, and long-term work can cause significant health problems, even irreversible diseases. Furthermore, the cost of manual fishing operations is gradually increasing. The use of underwater robots for underwater fishing operations is becoming increasingly popular, but due to time constraints and the influence of the marine environment, these operations face enormous challenges.

[0003] Compared to terrestrial datasets, underwater images typically exhibit color casts and low contrast, and sediment can also cause blurring. Furthermore, the small size and clustered nature of underwater organisms, due to their habits, makes underwater target identification challenging. Deep learning-based target detection systems are a popular area in computer vision and are increasingly being extended to underwater scenarios. These systems can be categorized into one-stage and two-stage detection frameworks. Two-stage methods first locate the target object, generating a series of candidate bounding boxes, and then classify the samples using convolutional neural networks, such as Faster R-CNN and Cascade R-CNN. In contrast, single-stage target detection algorithms are simpler in structure and faster, directly identifying the target's category and location using extracted network features, such as SSD and the YOLO series. Among these, algorithms based on improvements to the YOLO series are more widely used than other target detection frameworks.

[0004] For example, patent document No. 201910879419.1 discloses an underwater target detection method based on an improved YOLO algorithm. The detection method improves the YOLO algorithm by replacing the original 2*2*1 pooling layer with a 1*1*4 recombination layer, using a loss function to process the aspect ratio information of the target to be detected, using the improved YOLO algorithm to train a deep network on the labeled file to obtain the detection model weights, using the detection model weights to test the underwater image of the target to be detected, and outputting the location information and classification information of the target to be detected.

[0005] The aforementioned patented solution uses a reconstructed layer to improve detection speed, preserves the detailed features of underwater images, and reduces image dimensionality. Its loss function has a good detection effect for recognizing objects that frequently rotate or roll over in underwater scenes. In addition, its loss function incorporates the aspect ratio information of the object, reducing the number of images fed into the training and enhancing the robustness of the model.

[0006] However, the aforementioned patented solutions utilize large networks to improve detection accuracy, resulting in complex network structures, large model sizes, slow detection speeds, high computational requirements for devices, and difficulty in deploying the models on embedded and mobile devices.

[0007] Therefore, it is necessary to improve such an algorithm to overcome the above-mentioned defects. Summary of the Invention

[0008] The purpose of this invention is to provide an underwater target detection method based on the improved YOLOv6 algorithm, which aims to solve the problem that existing related algorithms have high detection accuracy, but due to their complex network structure, large amount of computation, and slow inference speed, they are not suitable for use in lightweight algorithms.

[0009] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0010] An underwater target detection method based on an improved YOLOv6 algorithm includes the following steps:

[0011] Step 1: Obtain the initial dataset and perform data cleaning to obtain the experimental dataset;

[0012] Step 2: Divide the experimental dataset into a training set and a test set. Perform data augmentation on the training set to increase the amount of data and improve the performance of the model, thus obtaining an expanded training set.

[0013] Step 3: Combine lightweight convolutional network ODConv and lightweight network EfficientNetv2 to redesign a lightweight structure as the backbone network of the YOLOv6 model, thereby reducing the number of network parameters and computational cost.

[0014] Step 4: Introduce the SPD-Conv module to improve the detection accuracy of underwater targets;

[0015] Step 5: Improve the neck network using lightweight convolution GSConv and VoVGSCSP methods to enhance the network's ability to detect small targets.

[0016] Step 6: Introduce the Adan optimizer to improve the network's detection performance, ultimately obtaining the improved model YOLOv6-ESG;

[0017] Step 7: Input the expanded training set into the improved model YOLOv6-ESG for deep learning training until the loss function of the improved model YOLOv6-ESG converges and tends to stabilize. At this point, the improved model YOLOv6-ESG is considered to have reached the optimal training and the corresponding weight file is obtained.

[0018] Step 8: Use the trained improved model YOLOv6-ESG to detect images in the test set or actual underwater videos or images, obtain classification and location information of various specific targets in the images, and annotate them in the videos or images, and output the corresponding files.

[0019] Furthermore, the method for step 2 is as follows:

[0020] The experimental dataset was randomly divided into 7102 training images and 887 test images as the initial dataset. The dataset contains underwater images and annotation information files. The annotation information is the location and category information of the target to be detected in the underwater images. The file format was modified to YOLO format to obtain the underwater optical image dataset, which was then used in the training of the YOLOv6-ESG model.

[0021] The data augmentation employs two methods: mosaic and mixup.

[0022] The mosaic method randomly crops four images and then stitches them together into a single image as training data; the mixup method randomly selects two images, overlays them, and mixes them proportionally to obtain a new image as training data.

[0023] Furthermore, in step 3, the lightweight convolutional layer ODConv and the lightweight network EfficientNetv2 are combined in such a way that the ordinary convolutional layers in the lightweight network EfficientNetv2 are replaced with the lightweight convolutional layer ODConv to reduce the number of model parameters and computational cost, thereby achieving the goal of lightweighting.

[0024] Furthermore, the SPD-Conv module in step 4 is introduced by adding corresponding SPD-Conv modules between some modules of the backbone network and the neck network to improve the detection accuracy of the model.

[0025] Furthermore, the improvement to the neck network in step 5 is as follows: the GSConv module is used to replace the SimConv module of the original model, and the VoVGSCSP module is used to replace the BepC3 module of the original model, so as to reduce the number of model parameters and computational load, and improve the detection accuracy and speed of the model.

[0026] Furthermore, the Adan optimizer introduced in step 6 is used to replace the original SGD optimizer to accelerate model convergence and improve the model's detection performance.

[0027] The specific calculation formula is as follows:

[0028]

[0029] Where k represents the number of updates (steps), α represents the learning rate (stepsize), and λ... k θ represents the weight decay coefficient, θ represents the parameter to be updated, and g k β1 represents the gradient of the loss function f(θ) with respect to θ, β2 represents the first-order moment decay coefficient, β3 represents the third-order moment decay coefficient, ε represents the constant added to the denominator to increase numerical stability, and m represents the gradient of the loss function f(θ) with respect to θ. k Describing the gradient g k The first moment, v k Describing the gradient g k The second moment, n k Describing the gradient g k The third moment;

[0030] The method for updating the parameters is as follows:

[0031] First, initialize θ0 with a learning rate of α, and momentum (β1, β2, β3) ∈ [0, 1]. 3 Stability parameter ε > 0, weight decay coefficient λ k >0, set the initial parameters m0=g0, v0=0, v1=g1-g0 and n0=g0 2 Then, update the parameter θ using the above formula.

[0032] Furthermore, the training process of the improved YOLOv6-ESG model in step 7 is as follows:

[0033] Set the number of training passes to 300.

[0034] The expanded training set images were adjusted to a resolution of 640x640 and input into the backbone feature extraction network of the improved model YOLOv6-ESG. The backbone feature extraction network was used to extract features from the original images to obtain three feature layers: 80x80, 40x40, and 20x20.

[0035] The GSConv+VoVGSCSP module transmits three effective feature layers to the neck layer and performs upsampling, downsampling, and multi-scale feature fusion on the three feature layers at different scales to better extract fused features and improve performance.

[0036] Finally, the three effective feature layers of large, medium and small after fusion are transmitted to the prediction layer of the network for prediction. The three feature layers of different scales are used to detect targets of different sizes. The non-maximum suppression method is used and the prior boxes are adjusted to determine the final target location and classification result.

[0037] During the training phase, GIOU is used as the loss function, which is defined as follows:

[0038]

[0039] L GIoU =1-GIoU

[0040] Where IoU represents the intersection-union ratio of the ground truth bounding box and the predicted bounding box, A c It is the area of ​​the smallest bounding rectangle of the two rectangles, and U represents the sum of the areas of the two rectangles minus the area where the two rectangles intersect.

[0041] In summary, the present invention has the following beneficial effects:

[0042] This invention proposes a lightweight underwater target detection method based on YOLOv6, which can be mounted on embedded or mobile devices with limited computing power. When detecting multiple targets in different underwater scenarios, it can ensure high accuracy while improving the speed of underwater target detection, and detect the target category and location information in underwater optical images in real time.

[0043] This invention combines lightweight convolutional ODConv and lightweight network EfficientNetv2 to design a lighter structure as the backbone network of the YOLOv6 model, effectively reducing the number of parameters and computational cost. Lightweight convolutional GSConv and VoVGSCSP modules replace the original SimConv and BepC3 modules in the Neck layer, further reducing the number of parameters and computational cost, improving the network's ability to detect small targets, and increasing the detection speed. Addressing the challenges of detecting underwater targets due to poor image quality, low resolution, and small size, SPD-Conv modules are used in both the backbone network and the Neck layer, effectively improving the detection accuracy of underwater targets. To accelerate model convergence and improve detection accuracy, the Adan optimizer is introduced to further enhance network performance. Attached Figure Description

[0044] Figure 1 This is a network structure diagram of the improved YOLOv6-ESG model described in this invention.

[0045] Figure 2 (a)~ Figure 2(c) are schematic diagrams of the FusedMBConv module, ODConv module, and OD-FusedMBConv module structure in the improved backbone network described in this invention.

[0046] Figure 3 This is a schematic diagram of the SPD-Conv module structure described in this invention.

[0047] Figure 4 This is a schematic diagram of the GSConv module structure in the improved neck network described in this invention;

[0048] Figure 5 This is a schematic diagram of the detection results of the seven target detection models described in this invention on an underwater target detection dataset.

[0049] Figure 5 (a)~ Figure 5 (d) are the prediction results images under randomly selected color cast, blurry, small target and clustered states, respectively. Ground Truth refers to the target present in the original image (with only label), and the other rows refer to the detection results (with label and probability) of different models.

[0050] Figure 6 This is a schematic diagram of the video detection results described in this invention. Detailed Implementation

[0051] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to the figures and specific embodiments.

[0052] like Figures 1 to 6 As shown, this invention proposes an underwater target detection method based on an improved YOLOv6 algorithm. It uses a redesigned lightweight backbone network OD-E2 as the feature extraction network and employs a lightweight neck network to fuse the extracted large, medium, and small feature maps at multiple scales, outputting three new feature maps of different sizes to the detection head to detect targets of varying sizes. These introduced lightweight structures significantly reduce the model's parameter count and computational cost, while maintaining high detection accuracy and improving detection speed. This allows the model to achieve higher accuracy and speed in embedded or mobile devices with limited computing power for detecting multiple underwater targets. This lightweight underwater target detection method is easier to integrate into small underwater robots, enabling high-precision, real-time detection and harvesting of underwater targets.

[0053] An underwater target detection method based on an improved YOLOv6 algorithm includes the following steps:

[0054] Step 1: Obtain the initial dataset and perform data cleaning: Download data from the URPC2022 website to obtain the initial dataset under different underwater environments, multiple orientations, and different lighting conditions. Remove seaweed tags (interference factors) from the dataset and delete images without targets to obtain the experimental dataset.

[0055] Step 2: Divide the experimental dataset into a training set and a test set. Perform data augmentation on the training set to increase the amount of data and improve the performance of the model, thus obtaining an expanded dataset.

[0056] Step 3: Use lightweight convolutional ODConv and lightweight network EfficientNetv2 to redesign a lighter structure as the backbone network of YOLOv6 model, reducing the number of network parameters and computational cost.

[0057] Step 4: To address the challenges of poor underwater image quality, low resolution, and small underwater targets, the SPD-Conv module is introduced to improve the detection accuracy of underwater targets.

[0058] Step 5: Improve the neck network using lightweight convolution GSConv and VoVGSCSP methods to enhance the network's ability to detect small targets.

[0059] Step 6: To accelerate model convergence and improve detection accuracy, the Adan optimizer was introduced to further enhance the network's detection performance; the resulting improved model is called YOLOv6-ESG.

[0060] Step 7: Input the expanded training set into the improved YOLOv6-ESG model for deep learning training, set the corresponding number of training iterations. As the number of training iterations increases, the loss function curve of the model gradually converges. When the loss function curve converges and tends to stabilize, the YOLOv6-ESG model training reaches its optimal state, and the corresponding weight file is obtained.

[0061] Step 8: Use the trained YOLOv6-ESG model to detect images in the test set or actual underwater videos or images, obtain classification and location information of various specific targets in the images, and annotate them in the videos or images, and output the corresponding files.

[0062] Step 2 above includes the following steps:

[0063] The images were randomly divided proportionally into a training set of 7102 images and a test set of 887 images as the initial dataset for this experiment. The dataset contains underwater images and annotation information files. The annotation information includes the location and category information of the targets to be detected in the underwater images. The file format was modified to YOLO format to obtain the underwater optical image dataset, which was then used to train the YOLOv6-ESG model. The network structure diagram of the YOLOv6-ESG model is shown below. Figure 1 As shown. Data augmentation mainly uses two methods at the network input: mosaic and mixup. The main idea of ​​the mosaic method is to randomly crop four images and then stitch them together into a single image as training data. The mixup method randomly selects two images and overlays them proportionally.

[0064] The redesigned backbone feature extraction network in step 3 above uses ODConv (structural diagram as shown in Figure 3). Figure 2 (b) and EfficientNetv2 (where the FusedMBConv module structure diagram is shown in Figure 1) Figure 2 (a)) The combination method is as follows: the ordinary convolutional layer of the FusedMBConv module in the original EfficientNetv2 network is replaced with a lightweight convolutional layer ODConv. The schematic diagram of the combined structure is shown below. Figure 2 As shown in (c), it further reduces the number of model parameters and computational cost, achieving the goal of lightweight design.

[0065] Step 4 above references the SPD-Conv module, which addresses the issues of low resolution and small targets in underwater images. The module's structure is shown in the diagram below. Figure 3 As shown, the improvement mainly involves adding corresponding SPD-Conv modules between some modules of the backbone network and the neck network, which significantly improves the detection accuracy of the model.

[0066] The improvement to the neck network in step 5 above is as follows: The SimConv module of the original model is replaced with the GSConv module, and the BepC3 module of the original model is replaced with the VoVGSCSP module. The GSConv module is as follows: Figure 4 As shown, this improvement not only further reduces the number of model parameters and computational load, but also further enhances detection accuracy and speed.

[0067] In step 6 above, the Adan optimizer was introduced during the training phase. It replaces the original SGD optimizer and, with the same computational resources, can accelerate model convergence and improve the model's detection performance. The specific calculation formula is as follows:

[0068]

[0069] Where k represents the number of updates (steps), α represents the learning rate (stepsize), and λ... k θ represents the weight decay coefficient, θ represents the parameter to be updated, and g k β1 represents the gradient of the loss function f(θ) with respect to θ, β2 represents the first-order moment decay coefficient, β3 represents the third-order moment decay coefficient, ε represents the constant added to the denominator to increase numerical stability, and m represents the gradient of the loss function f(θ) with respect to θ. k Describing the gradient g k The first moment, v k Describing the gradient g k The second moment, n k Describing the gradient g k The third moment;

[0070] The method for updating the parameters is as follows:

[0071] First, initialize θ0 with a learning rate of α, and momentum (β1, β2, β3) ∈ [0, 1]. 3 Stability parameter ε > 0, weight decay coefficient λ k >0, set the initial parameters m0=g0, v0=0, v1=g1-g0 and n0=g0 2 Then, update the parameter θ using the above formula.

[0072] Furthermore, the training process of the improved YOLOv6-ESG model in step 7 is as follows:

[0073] Set the number of training passes to 300.

[0074] The expanded training set images were adjusted to a resolution of 640x640 and input into the backbone feature extraction network of the improved model YOLOv6-ESG. The backbone feature extraction network was used to extract features from the original images to obtain three feature layers: 80x80, 40x40, and 20x20.

[0075] The GSConv+VoVGSCSP module transmits three effective feature layers to the neck layer and performs upsampling, downsampling, and multi-scale feature fusion on the three feature layers at different scales to better extract fused features and improve performance.

[0076] Finally, the three effective feature layers of large, medium and small after fusion are transmitted to the prediction layer of the network for prediction. The three feature layers of different scales are used to detect targets of different sizes. The non-maximum suppression method is used and the prior boxes are adjusted to determine the final target location and classification result.

[0077] During the training phase, GIOU is used as the loss function, which is defined as follows:

[0078]

[0079] L GIoU =1-GIoU

[0080] Where IoU represents the intersection-union ratio of the ground truth bounding box and the predicted bounding box, A c It is the area of ​​the smallest bounding rectangle of the two rectangles, and U represents the sum of the areas of the two rectangles minus the area where the two rectangles intersect.

[0081] This embodiment presents a lightweight underwater target detection method based on YOLOv6, which can effectively detect various targets in different waters and scenarios. This method significantly reduces the number of model parameters and computational load while maintaining high detection accuracy, thus improving detection speed. It is also easier to deploy in small embedded or mobile devices with limited computing power, enabling real-time detection of underwater targets of different sizes.

[0082] In this document, the terms "upper," "lower," "front," "back," "left," "right," "top," "bottom," "inner," "outer," "vertical," and "horizontal," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only used for the clarity of expressing the technical solution and for the convenience of description, and therefore should not be construed as limiting the present invention.

[0083] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0084] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. An underwater target detection method based on an improved YOLOv6 algorithm, characterized in that, Includes the following steps: Step 1: Obtain the initial dataset and perform data cleaning to obtain the experimental dataset; Step 2: Divide the experimental dataset into a training set and a test set. Perform data augmentation on the training set to increase the amount of data and improve the performance of the model, thus obtaining an expanded training set. Step 3: Combine lightweight convolutional network ODConv and lightweight network EfficientNetv2 to redesign a lightweight structure as the backbone network of the YOLOv6 model, thereby reducing the number of network parameters and computational cost. The combination of lightweight convolution ODConv and lightweight network EfficientNetv2 is as follows: lightweight convolution ODConv is used to replace the ordinary convolutional layers in lightweight network EfficientNetv2 to reduce the number of model parameters and computational cost, thereby achieving the goal of lightweighting. Step 4: Introduce the SPD-Conv module to improve the detection accuracy of underwater targets; Step 5: Improve the neck network using lightweight convolution GSConv and VoVGSCSP methods to enhance the network's ability to detect small targets. The improvement to the neck network is as follows: the GSConv module is used to replace the SimConv module of the original model, and the VoVGSCSP module is used to replace the BepC3 module of the original model, so as to reduce the number of model parameters and computational cost, and improve the detection accuracy and speed of the model. Step 6: Introduce the Adan optimizer to improve the network's detection performance, ultimately obtaining the improved model YOLOv6-ESG; Step 7: Input the expanded training set into the improved model YOLOv6-ESG for deep learning training until the loss function of the improved model YOLOv6-ESG converges and tends to stabilize. At this point, the improved model YOLOv6-ESG is considered to have reached the optimal training and the corresponding weight file is obtained. Step 8: Use the trained improved model YOLOv6-ESG to detect images in the test set or actual underwater videos or images, obtain classification and location information of various specific targets in the images, and annotate them in the videos or images, and output the corresponding files.

2. The underwater target detection method based on the improved YOLOv6 algorithm according to claim 1, characterized in that, The method for step 2 is as follows: The experimental dataset was randomly divided into 7102 training images and 887 test images as the initial dataset. The dataset contains underwater images and annotation information files. The annotation information is the location and category information of the target to be detected in the underwater images. The file format was modified to YOLO format to obtain the underwater optical image dataset, which was then used in the training of the YOLOv6-ESG model. The data augmentation employs two methods: mosaic and mixup. The mosaic method randomly crops four images and then stitches them together into a single image as training data; the mixup method randomly selects two images, overlays them, and mixes them proportionally to obtain a new image as training data.

3. The underwater target detection method based on the improved YOLOv6 algorithm according to claim 1, characterized in that, The SPD-Conv module in step 4 is introduced by adding corresponding SPD-Conv modules between some modules of the backbone network and the neck network to improve the detection accuracy of the model.

4. The underwater target detection method based on the improved YOLOv6 algorithm according to claim 1, characterized in that, The Adan optimizer introduced in step 6 is used to replace the original SGD optimizer to accelerate model convergence and improve the model's detection performance. The specific calculation formula is as follows: ； in, This indicates the number of steps in the update. The learning rate is used to control the step size. This represents the weight decay coefficient. This indicates the parameter to be updated. Represents the loss function right The gradient obtained by taking the derivative, This represents the first-order moment attenuation coefficient. This represents the second-order moment attenuation coefficient. This represents the third-order moment attenuation coefficient. This represents a constant added to the denominator to increase numerical stability. Gradient The first moment, Gradient The second moment, Gradient The third moment; The method for updating the parameters is as follows: First initialize The learning rate is ,momentum Stable parameters Weight decay coefficient Set initial parameters , , as well as Update parameters using the above formula .

5. The underwater target detection method based on the improved YOLOv6 algorithm according to claim 1, characterized in that, The training process of the improved YOLOv6-ESG model in step 7 is as follows: Set the number of training passes to 300. The expanded training set images were adjusted to a resolution of 640x640 and input into the backbone feature extraction network of the improved model YOLOv6-ESG. The backbone feature extraction network extracted features from the original images to obtain three feature layers: 80x80, 40x40, and 20x20. By transmitting three effective feature layers to the neck layer, the GSConv + VoVGSCSP module performs upsampling, downsampling, and multi-scale feature fusion on the three different scale feature layers, thereby better extracting fused features and improving performance. Finally, the three effective feature layers of large, medium and small after fusion are transmitted to the prediction layer of the network for prediction. The three feature layers of different scales are used to detect targets of different sizes. The non-maximum suppression method is used and the prior boxes are adjusted to determine the final target location and classification result. During the training phase, GIOU is used as the loss function, which is defined as follows: ； in, This represents the intersection-union ratio (IoU) between the ground truth bounding box and the predicted bounding box. It is the area of ​​the smallest bounding rectangle of the two rectangles. This represents the sum of the areas of the two rectangles minus the area where the two rectangles intersect.

Images