An improved YOLOv8n-seg-based marine garbage sonar image instance segmentation method

Abstract

This application discloses a method for instance segmentation of marine debris sonar images based on an improved YOLOv8n-seg model, relating to the field of image segmentation. The method involves constructing an improved YOLOv8n-seg model, replacing the C2f module in the backbone with a C2PC-Cone module, replacing the C2f module in the neck with a VoV-GSCSP module, and replacing the Segment module in the head with an SA-segment segmentation head. The improved YOLOv8n-seg model is then trained and validated to obtain a marine debris sonar image instance segmentation model. The marine debris sonar image to be segmented is input into the trained model, and the instance segmentation result is output. This method achieves low-complexity, high-precision instance segmentation of marine debris sonar images, improving the model's deployment capability and segmentation accuracy on resource-constrained devices.

Description

Technical Field

[0001] This application relates to the field of marine debris sonar image segmentation technology, and in particular to a method for segmenting marine debris sonar images based on an improved YOLOv8n-seg. Background Technology

[0002] Marine debris refers to persistent man-made or processed solid waste in the marine and coastal environment. It not only affects the marine landscape but also threatens navigational safety and biodiversity, causing degradation of marine biological resources and negatively impacting marine ecological health and the marine economy. To efficiently clean up marine debris, its type and location must be accurately determined. Instance segmentation using optical images can simultaneously obtain information on debris type and location; however, optical imaging is only applicable to clear seawater and limited to close-range scenarios. Imaging sonar, on the other hand, is applicable to both clear and turbid seawater, and to both close-range and long-range scenarios, thus making instance segmentation using sonar images more widely applicable. Currently, deep learning-based instance segmentation methods include two-stage and single-stage methods. Two-stage methods offer high segmentation accuracy but poor real-time performance, while single-stage methods offer high real-time performance but relatively low segmentation accuracy. Underwater robots equipped with imaging sonar have become a typical method for marine debris detection. This application scenario requires high real-time performance for instance segmentation methods, making two-stage methods unsuitable.

[0003] YOLOv8n-seg is a mainstream single-stage instance segmentation model based on the YOLO model. It consists of three parts: backbone, neck, and head. It has been used by researchers for instance segmentation of underwater debris optical images. However, directly applying it to instance segmentation of marine debris sonar images has the following problems: (1) high computational complexity and poor real-time performance; (2) large number of model parameters, which is not suitable for deployment on sonar; (3) the model's feature pyramid fails to adapt to the multi-scale characteristics of sonar images, resulting in low segmentation accuracy; (4) speckle noise reduces the contrast, resolution, and edges of marine debris sonar images, thus affecting segmentation accuracy. Summary of the Invention

[0004] The purpose of this application is to provide a method for segmenting marine debris sonar images based on an improved YOLOv8n-seg, thereby solving the aforementioned problems in the existing technology.

[0005] To achieve the above objectives, this application provides a method for instance segmentation of marine debris sonar images based on an improved YOLOv8n-seg, comprising the following steps: S1: Obtain the sonar image dataset of marine debris and divide it into the original training set, validation set and test set; perform data augmentation on the original training set to obtain the augmented training set, and label the augmented training set and validation set; S2: Construct an improved YOLOv8n-seg model. The C2PC-Cone module replaces the C2f module in the YOLOv8n-seg model backbone. The C2PC-Cone module includes convolutional layers with Cone activation functions, channel segmentation layers, a PConv bottleneck layer, and a feature concatenation and fusion layer. The VoV-GSCSP module replaces the C2f module in the YOLOv8n-seg model neck. The SA-segment segmentation head replaces the Segment module in the YOLOv8n-seg model head. The SA-segment segmentation head includes Proto, Box, Cls, and Mask Coef branches, each embedding a self-attention layer. S3: The improved YOLOv8n-seg model is trained and validated using an annotated enhanced training and validation set to obtain a marine debris sonar image instance segmentation model; S4: Input the marine debris sonar image to be segmented into the trained marine debris sonar image instance segmentation model, and output the instance segmentation result.

[0006] Preferably, data augmentation includes: adding speckle noise, horizontal flipping, vertical flipping, translation, and rotation operations; annotation is performed using Labelme software.

[0007] Preferably, the specific structure of the C2PC-Cone module includes: firstly, channel expansion and basic feature extraction of the input feature map are performed through a 1×1 convolutional layer with the Cone activation function; then, the feature map is evenly divided into a main path feature flow and residual branches through a channel segmentation layer; in the main path feature flow, the main features are processed through... n Each cascaded PConv bottleneck layer progressively enhances the data. Each PConv bottleneck layer performs 3×3 convolution on 1 / 4 of the channels while retaining 3 / 4 of the channel information for direct transmission. The original shallow features of the residual branches, the initial segmentation features of the main path, and the deep processing features of each PConv bottleneck layer are concatenated to form multi-scale fusion features. The multi-scale fusion features are compressed and spatially optimized through 1×1 convolutional layers with Cone activation functions to generate high-resolution feature maps.

[0008] Preferably, in the SA-segment segmentation head: Set a self-attention layer after the first convolutional layer in the Proto branch and before the upsampling operation; Set a self-attention layer between the two convolutional layers in the Box branch and the Cls branch; Set a self-attention layer between the two convolutional layers in the Mask Coef branch.

[0009] Preferably, when training the improved YOLOv8n-seg model, the following training conditions are adopted: input image resolution is 640×640; batch size is 8; training epochs are 200; the stochastic gradient descent optimization algorithm is adopted, the momentum is 0.937, and the weight decay is 0.0005; a dynamic learning rate adjustment strategy is adopted, the initial learning rate is 0.01, a linear preheating mechanism is introduced, and a cosine annealing strategy is adopted in the later stage.

[0010] Preferably, when training the improved YOLOv8n-seg model, a composite loss function is used for end-to-end training. The total loss is equal to the sum of the detection branch loss and the segmentation branch loss. The detection branch loss is the sum of the classification loss and the regression loss. The classification loss is the binary cross-entropy loss. The regression loss is the sum of the distribution focus loss and the complete intersection-union ratio loss. The segmentation branch loss is the weighted sum of the binary cross-entropy loss and the Dice coefficient loss.

[0011] Therefore, the above-mentioned method for segmenting marine debris sonar images based on the improved YOLOv8n-seg has the following beneficial effects: (1) This application has lower computational complexity and number of parameters. Through the design of the C2PC-Cone module and the introduction of the VoV-GSCSP module, the computational complexity and number of parameters of the method in this application are lower than those of the original method YOLOv8n-seg, which enables it to be deployed on imaging sonar with limited resources.

[0012] (2) This application solves the problems of incorrect segmentation and inaccurate segmentation: The multi-scale adaptive fusion mechanism of the VoV-GSCSP module is combined with the global context modeling capability of the segmentation head SA-segment, which enables the method of this application to distinguish and segment targets of different scales, thereby solving the problems of incorrect segmentation and inaccurate segmentation of YOLOv8n-seg.

[0013] (3) This application has better instance segmentation performance: Experimental results show that compared with YOLOv8n-seg, the computational complexity of the method in this application is lower, but the instance segmentation accuracy P, the average accuracy mAP@50 when the cross-union ratio is 0.5, and the average accuracy mAP@50-95 when the cross-union ratio is in the range of 0.5 to 0.95 are all improved.

[0014] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0015] Figure 1 This is a structural diagram of the improved YOLOv8n-seg model in this application; Figure 2 This is a structural diagram of the YOLOv8n-seg model in this application; Figure 3 This is a structural diagram of the C2PC-Cone module in this application; Figure 4 This is a diagram of the SA-segment structure in this application; Figure 5 These are the sonar image labels for marine debris in this application; wherein, (a) is an image label containing a propeller, a beverage bottle and a tire, (b) is an image label containing an iron chain, (c) is an image label containing a shampoo bottle and the wall of an experimental water tank, (d) is an image label containing a valve, (e) is an image label containing an aluminum can and a beverage box, and (f) is an image label containing a tire, a propeller and an iron hook; Figure 6 The images show a comparison of the segmentation results before and after the improvement. In the images, (a) is the original image, (b) is the labeled image, (c) is the image after YOLOv8n-seg instance segmentation, and (d) is the image after instance segmentation using the method of this application. Detailed Implementation

[0016] The following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0017] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning as understood by a person of ordinary skill in the art to which this application pertains.

[0018] The terms "comprising" or "including," as used in this application, mean that the element preceding the term encompasses the element listed after the term, and do not exclude the possibility of encompassing other elements as well. The terms "inner," "outer," "upper," and "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In this application, unless otherwise expressly specified and limited, the term "attached," etc., should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can refer to a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication of two elements or the interaction relationship between two elements. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0019] Example 1: A method for segmenting marine debris sonar images based on an improved YOLOv8n-seg includes the following steps: S1: Obtain the sonar image dataset of marine debris and divide it into the original training set, validation set and test set; perform data augmentation on the original training set to obtain the augmented training set, and label the augmented training set and validation set; Data augmentation included adding speckle noise, horizontal flipping, vertical flipping, translation, and rotation operations; annotation was performed using Labelme software.

[0020] S2: Construct an improved YOLOv8n-seg model; wherein, the C2f module in the backbone of the YOLOv8n-seg model is replaced by the C2PC-Cone module, and the C2PC-Cone module includes a convolutional layer with the Cone activation function, a channel segmentation layer, a PConv bottleneck layer, and a feature concatenation and fusion layer; the C2f module in the neck of the YOLOv8n-seg model is replaced by the VoV-GSCSP module; the Segment module in the head of the YOLOv8n-seg model is replaced by the SA-segment segmentation head, which includes: a Proto branch, a Box branch, a Cls branch, and a Mask Coef branch, with a self-attention layer embedded in each branch; Specifically, such as Figure 2As shown, the C2f module in the backbone of the YOLOv8n-seg model enhances gradient flow and feature reuse through multi-branch cross-layer connections. However, the complex topology leads to low computational efficiency. Frequent cross-layer connections and channel splitting significantly increase inference latency and memory usage, resulting in high computational complexity, poor real-time performance, and a large number of parameters, making it unsuitable for deployment on sonar. The C2f module in the neck of the YOLOv8n-seg model also suffers from similar drawbacks. The multi-branch stacking and bidirectional feature pyramid architecture used in the neck of the YOLOv8n-seg model are insufficient in information transmission efficiency and shallow detail preservation, making it difficult to effectively coordinate spatial details and high-level semantics in multi-scale target processing, thus leading to target segmentation errors and decreased accuracy. Furthermore, the segmentation head in the head of the YOLOv8n-seg model relies on local convolutional receptive fields and lacks the ability to capture global contextual information. Static convolutional kernels struggle to dynamically focus on key regions, resulting in blurred target edges and limited segmentation accuracy.

[0021] Therefore, constructing such Figure 1 The improved YOLOv8n-seg model shown replaces the C2f module in the YOLOv8n-seg model backbone with the C2PC-Cone (CrossStage Partial Bottleneck with 2 Partial Convolutions and Cone activation function) module, as follows: Figure 3As shown, the specific structure of the C2PC-Cone module includes: First, channel expansion and basic feature extraction are performed on the input feature map through a 1×1 convolutional layer with the Cone activation function; then, the feature map is evenly divided into a main path feature flow and residual branches through a channel segmentation layer; in the main path feature flow, the main features are progressively enhanced through n cascaded PConv bottleneck layers, each PConv bottleneck layer performs 3×3 convolution processing on 1 / 4 of the channels, while retaining 3 / 4 of the channel information for direct transmission; this main path feature flow design reduces computational overhead and enhances feature representation capability. In the feature fusion stage, the original shallow features of the residual branches, the initial segmentation features of the main path, and the deep processing features of each PConv bottleneck layer are concatenated to form a multi-scale fusion feature containing (n+2)×C channels (where C represents the hidden channel dimension); the multi-scale fusion feature is compressed and its spatial dimension optimized through a 1×1 convolutional layer with the Cone activation function to generate a high-resolution feature map. Replacing the C2f module in the backbone of YOLOv8n-seg with the C2PC-Cone module reduces the computational complexity and number of parameters in this application's method. Furthermore, the C2PC-Cone module balances learning ability and computational efficiency, generating discriminative feature representations that combine fine-grained texture details with high-level semantic information; therefore, this design approach also contributes to improving instance segmentation accuracy. Figure 3 middle, H Indicates the height of the feature map. W Indicates the width of the feature map. C_in Represents the number of input feature channels. C_out Represents the number of output feature channels. n The number of repetitions for the bottleneck layer is indicated, and Conv-Cone indicates a 1×1 convolution using the Cone activation function.

[0022] The C2f module in the YOLOv8n-seg model neck is replaced with the VoV-GSCSP (Variety of View Gather-Scatter cross-stage partial) module, reducing the computational complexity and number of parameters of the proposed method. The VoV-GSCSP module, through a single-step aggregation mechanism and the collaborative design of a lightweight feature enhancement unit, achieves effective fusion of multi-scale features, contributing to improved instance segmentation accuracy.

[0023] Replace the Segment module in the YOLOv8n-seg model head with an SA-segment segmentation head, such as... Figure 4 As shown, the SA-segment segmentation head includes: Proto branch, Box branch, Cls branch and Mask Coef branch, with self-attention layers embedded in each branch; In the SA-segment segmentation header: Setting a self-attention layer after the first convolutional layer in the Proto branch and before the upsampling operation can enhance global semantic consistency and suppress noise interference in sonar images.

[0024] Setting a self-attention layer between the two convolutional layers in the Box and Cls branches can improve target localization accuracy and relevance recognition capabilities.

[0025] A self-attention layer is placed between the two convolutional layers in the Mask Coef branch. The first convolutional layer is responsible for extracting local features to capture basic texture and edge information, while the self-attention module dynamically adjusts the weights of each spatial location by modeling the global relationships between these local features. This enables dynamic focusing on the target region. These design features of SA-segment all contribute to improving instance segmentation accuracy.

[0026] S3: The improved YOLOv8n-seg model is trained and validated using an annotated enhanced training and validation set to obtain a marine debris sonar image instance segmentation model; When training the improved YOLOv8n-seg model, the following training conditions were adopted: input image resolution of 640×640; batch size of 8; training epochs of 200; stochastic gradient descent optimization algorithm with momentum of 0.937 and weight decay of 0.0005; dynamic learning rate adjustment strategy with an initial learning rate of 0.01, a linear warm-up mechanism to stabilize the initial training, and a cosine annealing strategy to gradually decay the learning rate in the later stages.

[0027] When training the improved YOLOv8n-seg model, a composite loss function is used for end-to-end training. The total loss is equal to the sum of the detection branch loss and the segmentation branch loss. The detection branch loss is the sum of the classification loss and the regression loss. The classification loss is the binary cross-entropy loss. The regression loss is the sum of the distribution focus loss and the complete intersection-union ratio loss. The segmentation branch loss is the weighted sum of the binary cross-entropy loss and the Dice coefficient loss.

[0028] S4: Input the marine debris sonar image to be segmented into the trained marine debris sonar image instance segmentation model, and output the instance segmentation result.

[0029] Example 2: To further verify the effectiveness of the method in this application, this embodiment divides a publicly available dataset of marine debris sonar images containing 1868 images of beverage bottles, tires, valves, chains, cans, beverage boxes, propellers, shampoo bottles, and hooks into an original training set of 1308 images, a validation set of 374 images, and a test set of 186 images. Images from the original training set are selected and subjected to speckle noise, horizontal / vertical flipping, translation, and rotation to obtain 2607 new images. These 2607 new images are combined with the images from the original training set to obtain an enhanced training set containing 3915 images. The enhanced training set, validation set, and test set of marine debris sonar images are labeled using Labelme software. Figure 5 Six labeled images are provided, with different colored outlines used to distinguish each category.

[0030] Segmentation results as follows Figure 6 As shown, where, Figure 6 Counting from top to bottom, the original image in the first row contains a propeller, a can, and a tire; the original image in the second row contains a shampoo bottle; the original image in the third row contains a drink carton; and the original image in the fourth row contains two bottles and a tire.

[0031] Depend on Figure 6 As can be seen, compared with YOLOv8n-seg, the method in this application achieves more accurate instance segmentation. Furthermore, by... Figure 6 As can be seen from the first and second rows (counting from top to bottom), YOLOv8n-seg performs incorrect instance segmentation, while the method in this application performs correct instance segmentation. Therefore, the instance segmentation performance of the method in this application is superior to that of YOLOv8n-seg.

[0032] Segmentation performance can also be evaluated using objective metrics. In this embodiment, the objective metrics used are precision P (Precision), mean average precision mAP@50 when the intersection-union ratio (IU) is 0.5, mean average precision mAP@50-95 when the IU is between 0.5 and 0.95, floating-point operations (FLOPs), and the number of parameters (Params) to evaluate the instance segmentation performance of the method in this application. The evaluation results are shown in Table 1 below: Table 1 Evaluation Results

[0033] As shown in Table 1, the method of this application is superior to YOLOv8n-seg in terms of accuracy P, average accuracy mAP@50 when the cross-union ratio is 0.5, average accuracy mAP@50-95 when the cross-union ratio is in the range of 0.5 to 0.95, floating-point operations (FLOPs), and number of parameters (Params).

[0034] Therefore, this application adopts the aforementioned instance segmentation method for marine debris sonar images based on an improved YOLOv8n-seg. By improving YOLOv8n-seg, the improved YOLOv8n-seg is made suitable for instance segmentation of marine debris sonar images. By combining the multi-scale adaptive fusion mechanism of the VoV-GSCSP module with the global context modeling capability of the segmentation head SA-segment, it has the ability to distinguish and segment targets at different scales, thereby solving the problems of incorrect and inaccurate segmentation of YOLOv8n-seg and improving the instance segmentation accuracy of marine debris sonar images.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and not to limit them. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of this application, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of this application.

Claims

1. A method for instance segmentation of marine debris sonar images based on an improved YOLOv8n-seg, characterized in that, Includes the following steps: S1: Obtain the sonar image dataset of marine debris and divide it into the original training set, validation set and test set; perform data augmentation on the original training set to obtain the augmented training set, and label the augmented training set and validation set; S2: Construct an improved YOLOv8n-seg model. The C2PC-Cone module replaces the C2f module in the YOLOv8n-seg model backbone. The C2PC-Cone module includes convolutional layers with Cone activation functions, channel segmentation layers, a PConv bottleneck layer, and a feature concatenation and fusion layer. The VoV-GSCSP module replaces the C2f module in the YOLOv8n-seg model neck. The SA-segment segmentation head replaces the Segment module in the YOLOv8n-seg model head. The SA-segment segmentation head includes Proto, Box, Cls, and Mask Coef branches, each embedding a self-attention layer. S3: The improved YOLOv8n-seg model is trained and validated using an annotated enhanced training and validation set to obtain a marine debris sonar image instance segmentation model; S4: Input the marine debris sonar image to be segmented into the trained marine debris sonar image instance segmentation model, and output the instance segmentation result.

2. The method for instance segmentation of marine debris sonar images based on improved YOLOv8n-seg as described in claim 1, characterized in that, Data augmentation included adding speckle noise, horizontal flipping, vertical flipping, translation, and rotation operations; annotation was performed using Labelme software.

3. The method for instance segmentation of marine debris sonar images based on improved YOLOv8n-seg as described in claim 1, characterized in that, The specific structure of the C2PC-Cone module includes: first, channel expansion and basic feature extraction of the input feature map are performed through a 1×1 convolutional layer with the Cone activation function; then, the feature map is evenly divided into a main path feature flow and residual branches through a channel segmentation layer; in the main path feature flow, the main features are processed through... n Each cascaded PConv bottleneck layer progressively enhances the data. Each PConv bottleneck layer performs 3×3 convolution on 1 / 4 of the channels while retaining 3 / 4 of the channel information for direct transmission. The original shallow features of the residual branches, the initial segmentation features of the main path, and the deep processing features of each PConv bottleneck layer are concatenated to form multi-scale fusion features. The multi-scale fusion features are compressed and spatially optimized through 1×1 convolutional layers with Cone activation functions to generate high-resolution feature maps.

4. The method for instance segmentation of marine debris sonar images based on improved YOLOv8n-seg as described in claim 1, characterized in that, In the SA-segment segmentation header: Set a self-attention layer after the first convolutional layer in the Proto branch and before the upsampling operation; Set a self-attention layer between the two convolutional layers in the Box branch and the Cls branch; Set a self-attention layer between the two convolutional layers in the Mask Coef branch.

5. The method for instance segmentation of marine debris sonar images based on improved YOLOv8n-seg as described in claim 1, characterized in that, When training the improved YOLOv8n-seg model, the following training conditions were adopted: input image resolution of 640×640; batch size of 8; training epochs of 200; stochastic gradient descent optimization algorithm with momentum of 0.937 and weight decay of 0.0005; dynamic learning rate adjustment strategy with an initial learning rate of 0.01, a linear warm-up mechanism, and a cosine annealing strategy in the later stage.

6. The method for instance segmentation of marine debris sonar images based on improved YOLOv8n-seg as described in claim 1, characterized in that, When training the improved YOLOv8n-seg model, a composite loss function is used for end-to-end training. The total loss is equal to the sum of the detection branch loss and the segmentation branch loss. The detection branch loss is the sum of the classification loss and the regression loss. The classification loss is the binary cross-entropy loss. The regression loss is the sum of the distribution focus loss and the complete intersection-union ratio loss. The segmentation branch loss is the weighted sum of the binary cross-entropy loss and the Dice coefficient loss.

Images