An underwater debris detection method based on an improved YOLOv5 model
By introducing the Clo module and AttnConv operation into the YOLOv5 model, and combining data augmentation techniques, the problems of unstable light and limited resolution in underwater debris detection were solved, achieving high-precision underwater debris detection.
Patent Information
- Application Number
- CN202410076486.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-18
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2044-01-18
AI Technical Summary
Existing underwater debris detection technologies face problems such as unstable underwater light and severe color distortion, resulting in a decline in the performance of traditional visual detection. Sonar imaging has limited resolution, making it difficult to effectively detect small-sized debris or mixed debris piles. The YOLOv5 model needs to be adjusted to meet the high-precision requirements of underwater debris detection.
The Clo module is introduced into the backbone of the YOLOv5 model. Combined with the AttnConv operation in the Clo module, the model's ability to detect underwater debris is improved through data augmentation and feature fusion. This includes data preprocessing, training set partitioning, the Mosaic method, and feature fusion techniques, and the optimization of model parameters to adapt to the underwater environment.
It improves the accuracy and robustness of underwater debris detection, effectively overcoming the effects of complex underwater lighting and suspended particulate matter, and enabling rapid and accurate detection and identification of underwater debris.
Smart Images
Figure CN118038255B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of underwater debris detection technology, specifically to an underwater debris detection method based on an improved YOLOv5 model. Background Technology
[0002] Currently, my country possesses abundant water resources, but its water environment also faces numerous challenges and pressures. Although various measures have been implemented over the past few decades to improve water quality and manage the water environment, water pollution problems persist due to rapid industrialization and urbanization, as well as the impact of agricultural activities and domestic sewage discharge. Major issues include severe water pollution and damage to aquatic ecosystems. In particular, water pollution is relatively prominent in some areas, with large amounts of garbage in the water, affecting people's lives, ecological health, and sustainable development. Therefore, efforts are being intensified to implement stricter water environmental protection policies and measures, aiming to improve water quality, protect aquatic ecosystems, and promote sustainable water resource utilization and protection.
[0003] Currently, traditional underwater debris detection work can be divided into two categories: traditional visual detection techniques and sonar technology. Traditional visual detection techniques face significant challenges in underwater environments due to unstable lighting and severe color distortion, leading to performance degradation of traditional image processing and object detection algorithms. Overcoming these problems is crucial for underwater visual detection, requiring the use of specialized optical equipment or color correction techniques to improve image quality and algorithm stability. Sonar technology is a widely used detection method in underwater environments. Sonar imaging can penetrate water, using sound waves to acquire information about the shape, size, and location of targets. However, sonar imaging resolution is limited, and it may perform poorly with small debris or mixed debris piles. The YOLOv5 deep learning model has significant advantages in handling nonlinear uncertainties in areas such as data trend prediction, pattern recognition, and classification. For underwater debris detection, YOLOv5 offers efficient real-time target detection capabilities, good recognition of small targets, and high accuracy. Despite YOLOv5's excellent performance, it still needs to be adjusted and optimized for the specific characteristics of underwater image data to adapt to underwater debris detection tasks and achieve the high precision requirements of underwater debris detection. Summary of the Invention
[0004] To address the problems existing in the prior art, this invention provides an underwater debris detection method based on an improved YOLOv5 model. By utilizing data obtained from a publicly available underwater debris detection dataset and employing an improved YOLOv5 model, the method detects underwater debris, thus resolving the problems mentioned in the background section.
[0005] To achieve the above objectives, the present invention provides the following technical solution: an underwater debris detection method based on an improved YOLOv5 model, comprising the following steps:
[0006] S1. Select several images from the public dataset as the original dataset, and preprocess the dataset and divide it into training set, validation set and test set;
[0007] S2. Data augmentation of the training set using the Mosaic method;
[0008] S3. Construct an underwater debris detection model based on the improved YOLOv5 model;
[0009] S4. Feed the training set and validation set into the improved YOLOv5 model for training and validation, and then use the best-performing improved YOLOv5 model as the final underwater debris detection model.
[0010] S5. Input the test set into the improved YOLOv5 model after training for testing;
[0011] S6. Input the image to be detected into the final underwater debris detection model. The model outputs a detection box and marks the location of the debris in the image.
[0012] Preferably, in step S1, the preprocessing includes converting the annotation file format and filtering out some image files and annotation files with too few categories; the division into training set, validation set and test set is to randomly divide the preprocessed dataset into training set, validation set and test set in a ratio of 8:1:1.
[0013] Preferably, the converted annotation file format is converted into a YOLO format annotation file that can be recognized by the improved YOLOv5 model.
[0014] Preferably, the improved YOLOv5 model specifically introduces the Clo module into the Backbone part of the YOLOv5 object detection algorithm, allowing the model to pay more attention to important underwater debris features in the input image. Each Clo module consists of a local branch and a global branch.
[0015] Preferably, while using a traditional attention mechanism in the global branch, the key information (K) and value (V) are downsampled to reduce computation, thereby capturing low-frequency global information. The expression for the global branch is as follows:
[0016] X global =Attntion(Q,Pool(K),Pool(V))
[0017] Among them, X globalThe output of the global branch is represented by , Attention represents the traditional attention mechanism, Q is the vector used to acquire information, represents the current context of the given task, K is the vector used to measure the correlation between different inputs and queries, V is the vector corresponding to the actual numerical representation of each key, and Pool represents the downsampling operation.
[0018] A designed convolution operator, AttnConv, is introduced in the local branch, where a linear transformation is applied to obtain Q, K, and V. The specific formula is as follows:
[0019] Q,K,V=FC(X in )
[0020] Among them, X in This is the input to AttnConv, where FC represents a fully connected layer. AttnConv combines shared weights and context-aware weights to aggregate high-frequency local information.
[0021] Finally, the global and local features are fused together, and a fully connected layer is used to obtain the final output. The expression is as follows:
[0022] X t =Concat(X) local ,X global )
[0023] X out =FC(X) t )
[0024] Among them, X t This represents the result of fusing global and local features. Concat represents the feature fusion operation. X local X represents a local feature. global X represents all features. out This indicates the final output.
[0025] Preferably, the AttnConv integrates shared weights and context-aware weights, specifically including the following:
[0026] AttnConv first uses depthwise convolution (DWconv) to extract local representations, where DWconv has shared weights:
[0027] V s =DWconv(V)
[0028] Where DWconv represents the depthwise convolution operation, V s This represents the output result of extracting V information using depthwise convolution.
[0029] Then, AttnConv uses a gating mechanism to generate context-aware weights, introducing stronger nonlinearity than commonly used attention mechanisms. Context-aware weights are used to enhance local features. AttnConv applies convolution operators to Q and K to aggregate local information, then calculates the Hadamard product of Q and K, and performs a series of linear or nonlinear transformations on the result to generate context-aware weights in the range [-1,1].
[0030] Preferably, in step S4, during model training, the input image size is set to 640*640, the batch size is set to 8, the number of training iterations is 300, the initial learning rate is 0.01, the learning rate momentum is 0.937, and the weight decay coefficient is 0.0005. After training, the model is validated on the validation set, and the best-performing improved YOLOv5 model is selected as the final underwater debris detection model and saved as the best.pt file.
[0031] Preferably, in step S6, the best.pt weight file obtained after training is loaded, the image to be detected is input into the final underwater debris detection model, the model outputs a detection box and marks the location of the debris in the image, thus completing the underwater debris detection.
[0032] The beneficial effects of this invention are:
[0033] 1) This invention introduces the Clo module into the Backbone part of the original YOLOv5 model, which enables the model to have a global receptive field, while reducing the number of parameters, improving parameter efficiency, and better capturing multi-scale features, effectively alleviating the problems of missed detections and false detections.
[0034] 2) The AttnConv introduced by the Clo module in this invention can effectively integrate shared weights and context-aware weights to aggregate high-frequency local information, which can more effectively extract the features of underwater debris, improve the accuracy of image data processing, and thus increase the reliability of underwater debris detection methods.
[0035] 3) The proposed algorithm for detecting underwater debris in real underwater scenarios can overcome the adverse factors of complex underwater lighting conditions, water flow and suspended particles that may cause blurry or distorted images, and can quickly and accurately detect and identify underwater debris. Attached Figure Description
[0036] Figure 1 This is a schematic diagram of the underwater debris detection algorithm proposed in this invention;
[0037] Figure 2 This is a schematic diagram of underwater debris data collected by the present invention;
[0038] Figure 3This is a schematic diagram of the Clo module used in this invention;
[0039] Figure 4 This is a schematic diagram of the detection results of the collected underwater debris images. Detailed Implementation
[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0041] Please see Figures 1-4 This invention provides a technical solution: an underwater debris detection method based on an improved YOLOv5 model, such as... Figure 1 As shown, it includes the following steps:
[0042] The first step was to download the publicly available underwater debris detection dataset. This dataset contains images and annotation files. Several images and their corresponding annotation files were selected, and a Python script was used to convert the annotation files into a YOLO format that can be recognized by the improved YOLOv5. Additionally, some image and annotation files with too few categories were removed. A total of 691 images were collected, and the filtered dataset was randomly divided into training, validation, and test sets in an 8:1:1 ratio. Figure 2 As shown, Figure 2 This is underwater debris image data from the dataset.
[0043] The second step involves data augmentation using the Mosaic method. This process first involves selecting multiple images or image fragments from the training dataset. Subsequently, these images are cropped and stitched together according to certain rules or randomness to form a larger, more complex image, typically grouped together at a quarter ratio. This process provides more diverse training samples, enabling the model to learn multiple features and patterns in different scenarios, thus enhancing its generalization ability and robustness. The large image generated by stitching needs to have its label information adjusted accordingly to maintain consistency between the label and the image. Finally, these synthesized images are used as part of the training set for the model to train. This data augmentation technique helps improve the model's ability to recognize complex scenes, thereby improving its performance.
[0044] The third step involves constructing an underwater debris detection model based on an improved YOLOv5 model. A Clo module is added after the first convolutional layer in the YOLOv5 algorithm's backbone, allowing the model to focus more on important underwater debris features in the input image, thus improving performance. Each Clo module consists of a local branch and a global branch. The global branch uses a traditional attention mechanism, but downsamples the key information (K) and values (V) to reduce computation, thereby capturing low-frequency global information. The specific formula is as follows:
[0045] X global =Attntion(Q,Pool(K),Pool(V))
[0046] Among them, X global The output of the global branch is represented by , Attention represents the traditional attention mechanism, Q is the vector used to acquire information, represents the current context of the given task, K is the vector used to measure the correlation between different inputs and queries, V is the vector corresponding to the actual numerical representation of each key, and Pool represents the downsampling operation.
[0047] In the local branch, a carefully designed AttnConv is introduced, a simple yet efficient convolution operator that adopts the style of an attention mechanism. Specifically, in AttnConv, we first apply a linear transformation to obtain Q, K, and V, which is the same as standard attention:
[0048] Q,K,V=FC(X in )
[0049] Where X in It is the input of AttnConv, and FC indicates a fully connected layer.
[0050] AttnConv effectively fuses shared weights and context-aware weights to aggregate high-frequency local information. Specifically, AttnConv first uses depthwise convolution (DWconv) to extract local representations, where DWconv has shared weights:
[0051] V s =DWconv(V)
[0052] Where DWconv represents the depthwise convolution operation, V s This represents the output result of extracting V information using depthwise convolution.
[0053] Then, it uses context-aware weights to enhance local features. AttnConv uses a gating mechanism to generate context-aware weights, introducing stronger non-linearity than commonly used attention mechanisms. Furthermore, AttnConv...
[0054] Convolution operators are applied to Q and K to aggregate local information. Then, the Hadamard product of Q and K is computed, and the result undergoes a series of linear or nonlinear transformations to generate context-aware weights ranging from [-1, 1]. Notably, AttnConv inherits the translational equivariance of convolution, as all its operations are based on convolution. Finally, global and local features are combined, and a fully connected layer is used to obtain the final output, as shown in the following formula:
[0055] X t =Concat(X) local ,X global )
[0056] X out =FC(X) t )
[0057] Among them, X t This represents the result of fusing global and local features. Concat represents the feature fusion operation. X local X represents a local feature. global X represents all features. out This indicates the final output; FC represents a fully connected layer.
[0058] The fourth step involves feeding the training and validation sets into the improved YOLOv5 for training and validation. The network model parameters are updated via backpropagation, and the best-performing improved YOLOv5 model is used as the final underwater debris detection model. The input image size is set to 640*640, the batch size to 8, the number of training iterations to 300, the initial learning rate to 0.01, the learning rate momentum to 0.937, and the weight decay coefficient to 0.0005. After training, the best-performing improved YOLOv5 model validated on the validation set is used as the final underwater debris detection model and saved as the best.pt file.
[0059] The fifth step involves inputting the test set into the trained improved YOLOv5 model for testing to evaluate the model's metrics. To evaluate the underwater debris detection method based on the improved YOLOv5 model proposed in this invention, comparative experiments were conducted between the improved network and the original network to verify the superiority of the method used in the invention, aiming to verify its effectiveness and superiority. In all experiments, the same dataset and experimental environment were used to ensure the comparability and accuracy of the results. The purpose of these experiments was to verify the performance improvement of the improved model in underwater debris detection and to verify its feasibility and effectiveness in practical applications. The comparative experimental results are shown in Table 1. It can be seen that the improvement strategy for YOLOv5 in this invention helps to improve the model's performance in detecting underwater debris, and its various metrics are significantly better than the baseline network.
[0060] Table 1 Comparison of experimental results
[0061] method Parameters computational load Precision YOLOv5 165.01M 203.8G 90.2％ Improved YOLOv5 128.56M 170.1G 95.7％
[0062] The sixth step is to load the best.pt weight file obtained after training and input the image to be detected into the network to complete underwater debris detection. Figure 4 As shown, underwater debris in the image can be effectively located and the location of plastic waste can be marked in the image.
[0063] This invention overcomes the adverse effects of complex underwater lighting conditions, water flow, and suspended particulate matter that can lead to blurred or distorted images. It utilizes publicly available underwater debris datasets and trains the YOLOv5 network on these datasets, extracting information between data points to obtain stable weight parameters, resulting in improved detection performance.
[0064] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for underwater debris detection based on an improved YOLOv5 model, characterized in that, Includes the following steps: S1. Select several images from the public dataset as the original dataset, and preprocess the dataset and divide it into training set, validation set and test set; S2. Data augmentation of the training set using the Mosaic method; S3. Construct an underwater debris detection model based on the improved YOLOv5 model; S4. Feed the training set and validation set into the improved YOLOv5 model for training and validation, and then use the best-performing improved YOLOv5 model as the final underwater debris detection model. The improved YOLOv5 model specifically introduces Clo modules into the Backbone part of the YOLOv5 object detection algorithm. Each Clo module consists of a local branch and a global branch. The formula expression for the global branch is as follows: X global =Attntion(Q,Pool(K),Pool(V)) Among them, X global The output of the global branch is represented by , Attention represents the traditional attention mechanism, Q is the vector used to acquire information, represents the representation of the current context of the given task, K is the vector used to measure the correlation between different inputs and queries, V is the vector corresponding to the actual numerical representation of each key, and Pool represents the downsampling operation. The formula expression for the local branch is as follows: Q, K, V=FC(X in ) Among them, X in It is the input of AttnConv. FC represents a fully connected layer. AttnConv combines shared weights and context-aware weights to aggregate high-frequency local information. X t =Concat(X local ,X global ) X out =FC(X t ) Among them, X t This represents the result of fusing global and local features. Concat represents the feature fusion operation. X local X represents a local feature. global X represents all features. out This indicates the final output; AttnConv first uses depthwise convolution (DWconv) to extract local representations, where DWconv has shared weights: V s =DWconv(V) Where DWconv represents the depthwise convolution operation, V s This represents the output result of extracting V information using depthwise convolution; Then, AttnConv uses a gating mechanism to generate context-aware weights, introducing stronger non-linearity than commonly used attention mechanisms. It uses context-aware weights to enhance local features. AttnConv applies convolution operators to Q and K to aggregate local information, then calculates the Hadamard product of Q and K, and performs a series of linear or non-linear transformations on the result to generate context-aware weights in the range [-1,1]. S5. Input the test set into the improved YOLOv5 model after training for testing; S6. Input the image to be detected into the final underwater debris detection model. The model outputs a detection box and marks the location of the debris in the image.
2. The underwater debris detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S1, the preprocessing includes converting the annotation file format and filtering out some image files and annotation files with too few categories; the division into training set, validation set and test set is to randomly divide the preprocessed dataset into training set, validation set and test set in a ratio of 8:1:
1.
3. The underwater debris detection method based on the improved YOLOv5 model according to claim 2, characterized in that: The conversion of the annotation file format is to convert it into a YOLO format annotation file that can be recognized by the improved YOLOv5 model.
4. The underwater debris detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S4, during model training, the input image size is set to 640*640, the batch size is set to 8, the number of training iterations is 300, the initial learning rate is 0.01, the learning rate momentum is 0.937, and the weight decay coefficient is 0.0005. After training, the model is validated on the validation set, and the best-performing improved YOLOv5 model is selected as the final underwater debris detection model and saved as the best.pt file.
5. The underwater debris detection method based on the improved YOLOv5 model according to claim 1, characterized in that: In step S6, the best.pt weight file obtained after training is loaded, and the image to be detected is input into the final underwater debris detection model. The model outputs a detection box and marks the location of the debris in the image, thus completing the underwater debris detection.