A method and system for detecting submarine biological targets based on yolov5 optimization

Abstract

The present invention discloses a method for detecting seabed biological targets based on yolov5 optimization, and the method comprises the following steps: S1: collecting images containing various seabed biological targets in a public database, and processing to obtain an original data set; S2: performing light processing and ocean background object occlusion data enhancement processing on the original data set; S3: optimizing the yolov5 model to obtain a detection model for seabed biological target detection; S4: using the data set after the data enhancement processing in step S2 to train and learn the detection model in step S3 to obtain a trained detection model; S5: using the trained detection model to detect seabed biological targets. The present invention realizes rapid and accurate identification of various seabed biological targets, is applied to automatic fishing of seabed biological targets, improves automatic fishing of seabed biological targets, improves fishing efficiency, reduces manpower and reduces fishing operation risks.

Description

Technical Field

[0001] The present invention relates to the field of underwater intelligent image recognition, and more specifically, to a method and system for detecting seabed biological targets based on YOLOv5 optimization. Background Art

[0002] Seabed organisms, such as sea cucumbers, sea urchins, starfish and scallops, have high edible and medicinal value. Traditional fishing often relies on manual fishing, which requires a lot of manpower, is inefficient and poses certain risks to operators. In recent years, image intelligent recognition and artificial intelligence technologies have developed rapidly. Applying image detection technology to underwater fishing robots allows them to automatically identify the location and type of fishing objects, thereby completing intelligent automatic fishing, greatly improving fishing efficiency, reducing labor costs and reducing operational risks.

[0003] The prior art discloses a method for identifying and detecting submarine organisms based on a single-stage deep learning network, which includes the following steps: extracting submarine organism features based on a convolutional neural network; calculating the regression error between the true value box and the prediction box using the GIoU strategy; and dimensional clustering based on the K-means algorithm. This solution uses a convolutional neural network-based method to extract effective features of submarine organisms, effectively suppressing the interference of background noise on submarine organisms, while highlighting the features of submarine organisms, which is beneficial to improving the accuracy of submarine organism detection and identification. With the help of the GIoU method, the regression error between the prediction box and the true value box can be accurately calculated, so that the convolution weight of the entire detection system can be fully optimized. Another prior art discloses a method and system for ocean target detection based on self-supervised representation learning. In the self-supervised ship feature learning stage, a momentum comparison method is used to train a feature extraction model on unlabeled ocean target data. The momentum comparison process maintains the dictionary as a sample queue and updates the key encoder by using a momentum update method. In the supervised ocean target detection stage, a Faster RCNN model is used for ocean target detection. The Faster RCNN model includes a backbone network for feature extraction, a region candidate network for generating regions of interest, and a RoI Head network. The backbone network is initialized using the feature extraction network parameters obtained in the self-supervised ship feature learning stage to provide the target detection model with prior knowledge related to the ocean environment and ships, and the backbone network parameters are fine-tuned while training the model.

[0004] None of the above existing solutions deal with the diversity and complexity of the marine environment, and the model generalization ability is insufficient. Or when there are few image samples and the categories are unbalanced, the model training effect is not good. At the same time, the performance of the existing model network structure is not high, and the effect of seabed biological target detection can be improved by improving it. The detection efficiency of the model used is low, and there are certain limitations in practical applications. Summary of the invention

[0005] The primary purpose of the present invention is to provide a method for detecting submarine biological targets based on yolov5 optimization, to deal with the diversity and complexity of the marine environment accordingly, and to optimize the model to improve the accuracy of identifying submarine small organisms.

[0006] A further object of the present invention is to provide a seabed biological target detection system based on yolov5 optimization.

[0007] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0008] A method for detecting submarine biological targets based on yolov5 optimization includes the following steps:

[0009] S1: Collect images containing various seafloor organisms from public databases and process them to obtain the original dataset;

[0010] S2: performing data enhancement processing such as light processing and ocean background object occlusion on the original data set;

[0011] S3: Optimize the yolov5 model to obtain a detection model for submarine biological target detection

[0012] S4: Using the data set processed by the data enhancement in step S2 to train the detection model in step S3, to obtain a trained detection model;

[0013] S5: Use the trained detection model to detect submarine biological targets.

[0014] Preferably, the step S1 specifically includes the following steps:

[0015] S1.1: Obtain images containing a variety of seafloor organisms through public databases and crawlers;

[0016] S1.2: Screen and annotate the images to obtain an original data set.

[0017] Preferably, in step S1.2, labelimg tool is used for labeling to obtain labeling information, wherein the labeling information includes the object category in the image, the object center coordinates x and y, and the length and width information of the object frame, wherein the length and width information of the object frame is: the ratio of the length and width of the object frame to the length and width of the image.

[0018] Preferably, the step S2 specifically includes the following steps:

[0019] The diversity of the data set is increased by switching the background color of the image, adjusting the light brightness, adding different seabed environment backgrounds for the target object, and increasing the occlusion of seabed organisms. The formula for adjusting the light brightness is:

[0020] g(x,yj=a*f(x,y)+b

[0021] Where f(x,y) is the original value of the pixel (x,y), a and b are adjustment factors, and g(x,y) is the adjusted pixel value.

[0022] The occlusion scheme is: take common ocean scenes, such as stones, seaweed and reefs as occlusion objects, randomly place them at the top, bottom, left, right or center of the target object, and set the occlusion coverage by adjusting the length and width of the occlusion image.

[0023] Preferably, the yolov5 model is optimized in step S3, specifically:

[0024] 1) Optimize the yolov5 network structure, add branches and upsampling network layers in multiple places in the original yolov5 network, connect them separately, add detection heads, and add 3 corresponding anchor frames for the added detection layers. The optimized yolov5 structure has 5 detection heads, which can enhance the detection ability of small and large objects. Specifically: the 3rd layer in the original yolov5 network leads to branches, which are connected to the upsampling network layer added later; add a set of convolutional layers after the 10th layer in the original yolov5 network, and lead to branches to connect with the upsampling layer added later.

[0025] 2) Optimize the original Conv module of yolov5, and modify the convolution layer activation function in the Conv module to the Half_Hardswish activation function. The Half_Hardswish activation function is shown in the formula:

[0026]

[0027] 3) Add SE-HHslayer module to the last layer of yolov5 backbone;

[0028] 4) Add the CBAM attention mechanism module to the Conv module optimized in 2);

[0029] 5) Optimize the BottleneckCSP module of yolov5, replace the convolutional layers in the module with the Conv module optimized by 2), and change the original two-branch structure to a three-branch structure. The outputs of the three branches are combined as the input of the module. The feature map of the optimized BottleneckCSP module is mapped as follows:

[0030] y1=ConvHHs(x)

[0031] y2=Bottleneck(ConvHHs(x))

[0032] y2=Bottleneck(ConvHHs(x))

[0033] BottleneckCSP_Cat(x)=Concat(y1, y2, y3)

[0034] Where BottleneckCSP_Cat() is the output of the optimized BottleneckCSP module.

[0035] Preferably, before the detection model of step S3 is trained using the data set processed by the data enhancement process of step S2 in step S4, the following steps are further included:

[0036] Extract the Backbone network part of the detection model in step S3, establish a classification model, perform pre-training on a large-scale marine biological classification dataset, and obtain pre-training weights;

[0037] The pre-trained weights are loaded into the detection model of step S3.

[0038] Preferably, the step S4 specifically includes the following steps:

[0039] S4.1: Build the environment required for detection model training;

[0040] S4.2: Divide the data set after the data enhancement process in step S2 into a training set, a validation set, and a test set;

[0041] S4.3: Set hyperparameters for network model training;

[0042] S4.4: Loading the divided data set into the detection model for training to obtain a trained detection model.

[0043] Preferably, in step S4.2, 60% of the data set after data enhancement processing in step S2 is used as a training set, 20% is used as a validation set, and 20% is used as a test set.

[0044] Preferably, stratified sampling is performed during the process of dividing the data set in step S4.1.

[0045] A submarine biological target detection system based on yolov5 optimization, including:

[0046] A raw data module, which collects images containing various seafloor organisms in a public database and processes them to obtain a raw data set;

[0047] An enhancement module performs light processing and ocean background object occlusion data enhancement processing on the original data set;

[0048] A detection model module, wherein the detection model module optimizes the yolov5 model to obtain a detection model for detecting submarine biological targets;

[0049] A training module, wherein the training module uses the data set after the data enhancement processing in step S2 to train and learn the detection model in step S3 to obtain a trained detection model;

[0050] A detection module, wherein the detection module uses a trained detection model to detect seabed biological targets.

[0051] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0052] (1) The present invention can effectively balance the samples of the data set through the data enhancement method, reducing the impact of sample imbalance on model training. At the same time, by switching the background color and light of the picture and adding different seabed environment backgrounds to the target object, the diversity of the data set can be increased, thereby improving the generalization ability of the model.

[0053] (2) The present invention optimizes the yolov5 model by mainly modifying the structural logic of the underlying convolutional layer to improve its image feature learning ability, and introduces SE-HHslayer and CBAM attention mechanisms in multiple places in the network structure to improve the channel learning ability and spatial learning ability of the model, thereby further improving the accuracy of seabed biological target detection.

[0054] (3) The present invention connects the shallow network of Yolov5 with the subsequent upsampling network layer, adds a detection head, and adds three corresponding groups of smaller anchor frames to the detection layer, thereby improving the model's detection ability for small objects.

[0055] (4) The present invention uses a large-scale marine biological category data set to pre-train the classification model of the backbone model of the yolov5 optimization model, which is beneficial to the model's learning ability to extract marine biological image features.

[0056] (5) The present invention can improve the training effect of the model by adjusting the hyperparameters of the yolov5 optimization model, such as the learning rate, the number of learning rounds, and the number of pictures learned each time. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1It is a schematic diagram of the method flow of the present invention.

[0058] Figure 2 This is a schematic diagram of the optimized yolov5 network structure.

[0059] Figure 3 Schematic diagram of the ConvHHs module structure provided in the embodiment.

[0060] Figure 4 A schematic diagram of the SE-HHslayer module structure provided in an embodiment.

[0061] Figure 5 This is a schematic diagram of the Bottleneck CSP_Cat module structure.

[0062] Figure 6 It is a schematic diagram of the system module of the present invention. DETAILED DESCRIPTION

[0063] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0064] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0065] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0066] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0067] Example 1

[0068] This embodiment provides a method for detecting submarine biological targets based on yolov5 optimization. Figure 1 As shown, the following steps are included:

[0069] S1: Collect images containing various seafloor organisms from public databases and process them to obtain the original dataset;

[0070] S2: performing light processing and ocean background object occlusion data enhancement processing on the original data set;

[0071] S3: Optimize the yolov5 model to obtain a detection model for seabed biological target detection;

[0072] S4: Using the data set processed by the data enhancement in step S2 to train the detection model in step S3, to obtain a trained detection model;

[0073] S5: Use the trained detection model to detect submarine biological targets

[0074] The step S1 collects images containing various seafloor organisms in a public database and processes them to obtain an original data set, specifically comprising the following steps:

[0075] S1.1: Obtain images containing a variety of seafloor organisms through public databases and crawlers;

[0076] S1.2: Screen and annotate the images to obtain an original data set.

[0077] In step S1.2, labelimg tool is used to perform labeling to obtain labeling information, which includes the object category in the image, the object center coordinates x and y, and the length and width information of the object frame, wherein the length and width information of the object frame is: the ratio of the length and width of the object frame to the length and width of the image.

[0078] The step S2 performs data enhancement processing on the original data set, specifically comprising the following steps:

[0079] The diversity of the data set is increased by switching the background color of the image, adjusting the light brightness, adding different seabed environment backgrounds for the target object, and increasing the occlusion of seabed organisms. The formula for adjusting the light brightness is:

[0080] =(x, y) = a*f(x, y)+b

[0081] Where f(x,y) is the original value of the pixel (x,y), a and b are adjustment factors, and g(x,y) is the adjusted pixel value.

[0082] The occlusion scheme is: take common ocean scenes, including stones, seaweed and reefs as occlusion objects, randomly place them at the top, bottom, left, right corners or center of the target object, and set the occlusion coverage by adjusting the length and width of the occlusion image.

[0083] After the above data enhancement processing, the diversity of the data set can be increased, thereby improving the generalization ability of the model.

[0084] Before the detection model of step S3 is trained and learned using the data set processed by the data enhancement process of step S2 in step S4, the following steps are also included:

[0085] Extract the Backbone network part of the detection model in step S3, establish a classification model, perform pre-training on a large-scale marine biological classification dataset, and obtain pre-training weights;

[0086] The pre-trained weights are loaded into the detection model of step S3.

[0087] The pre-trained weights obtained above serve as the prior knowledge of the detection model, which is conducive to the training and learning of the target detection model.

[0088] The step S4 uses the data set processed by the data enhancement process in step S2 to train the detection model in step S3, and specifically includes the following steps:

[0089] S4.1: Build the environment required for detection model training;

[0090] S4.2: Divide the data set after the data enhancement process in step S2 into a training set, a validation set, and a test set;

[0091] S4.3: Set hyperparameters for network model training;

[0092] S4.4: Loading the divided data set into the detection model for training to obtain a trained detection model.

[0093] In terms of the training method of the model in this embodiment, a large-scale marine biological category data set is first used to pre-train the classification model of the backbone model of the yolov5 optimization model. The pre-training parameters serve as prior knowledge of the marine biological target detection model, which is conducive to the model's learning ability to extract marine biological image features.

[0094] In step S4.2, 60% of the data set after data augmentation in step S2 is used as a training set, 20% as a validation set, and 20% as a test set.

[0095] In step S4.2, stratified sampling is performed during the data set division process, that is, the allocation of each category follows the above ratio to avoid excessive differences in the distribution of the data set.

[0096] Some hyperparameters are set as follows: batch-size: 10; maximum number of epochs: 300; model learning rate lr0: 0.002; cycle learning rate lrf: 0.15; weight decay coefficient weight_decay: 0.0003; warmup learning momentum warmup_momentum: 0.8; warmup initial learning rate warmup_bias_lr: 0.08.

[0097] Example 2

[0098] Based on Example 1, this example provides specific content of optimizing yolov5, including:

[0099] In step S3, the yolov5 model is optimized, specifically:

[0100] 1) Due to the size of the target of underwater organisms or the distance of the camera, the size of the images presented is diverse, some are small and some are large. The model can have better recognition ability for small objects in the shallow layer and stronger recognition ability for large targets in the deep layer. Therefore, branches and subsequent upsampling network layers are added in multiple positions of the yolov5 network, and they are connected accordingly. The detection head is added, and 3 corresponding groups of anchor boxes are added to the detection layer. The length and width of the anchor box added in the shallow layer are set to [5,6], [7,12], [16,10], respectively, and the length and width of the anchor box added in the deep layer are set to [200,120], [520,110], [510,300], respectively, to improve the detection ability of small and large objects; specifically: the third layer in the original yolov5 network leads to a branch, which is connected to the subsequently added upsampling network layer; a group of convolutional layers is added after the 10th layer in the original yolov5 network, and the branch is connected to the subsequently added upsampling layer; the optimized yolov5 network architecture is as follows Figure 2 As shown;

[0101] 2) Optimize the original Conv module of yolov5, and modify the convolution layer activation function in the Conv module to the Half_Hardswish activation function to avoid the problem that the original activation function cannot perform gradient descent optimization iterative training when the input value is small. The Half_Hardswish activation function is shown in the formula:

[0102]

[0103] The optimized module is named ConvHHs module. The structure diagram of ConvHHs module is as follows: Figure 3 As shown;

[0104] 3) In the last layer of the backbone of yolov5, add the SE-HHslayer module. SE-HHslayer is an attention module, which is mainly used for learning the weight size of layer channels, improving the sensitivity of the model to channel features, so that pixels in different channels have different importance. The SE-HHslayer module mainly passes the input feature map through a fully connected neural network, and passes through the Half_Hardswish and sigmoid activation functions to learn the weight values ​​of each position of the feature map, and then performs a dot multiplication operation with the weight value and the input feature map to obtain the output feature map. The structure is as follows: Figure 4 As shown;

[0105] 4) Add the CBAM attention mechanism module to the shallow ConvHHs module. CBAM combines the spatial and channel attention mechanisms, which not only measures the importance of pixels in different channels, but also measures the importance of pixels in different spatial positions in the same channel, which can improve the feature extraction ability of the convolutional network.

[0106] The SE-HHslayer and CBAM attention mechanisms are introduced in multiple places in the network structure to improve the channel learning and spatial learning capabilities of the model, and further improve the accuracy of seabed biological target detection.

[0107] 5) Optimize the BottleneckCSP module of yolov5, replace the convolutional layers in the module with ConvHHs modules, and change the original two-branch structure to a three-branch structure. The outputs of the three branches are merged as the input of the module, and the optimized module is named BottleneckCSP_Cat module. The feature map mapping of the BottleneckCSP_Cat module is as follows:

[0108] y1=ConvHHs(x)

[0109] y2=Bottleneck(ConvHHs(x))

[0110] y2=Bottleneck(ConvHHs(x))

[0111] BottleneckCSP_Cat(x)=Concat(y1, y2, y3)

[0112] Where BottleneckCSP_Cat() is the output of the optimized BottleneckCSP module.

[0113] BottleneckCSP_Cat module structure is as follows Figure 5 As shown in Figure 1, BottleneckCSP in yolov5 adds superposition branches and convolution modules to improve its ability to extract image features by convolution.

[0114] Example 3

[0115] This embodiment provides a submarine biological target detection system based on yolov5 optimization. Figure 6 As shown, including:

[0116] A raw data module, which collects images containing various seafloor organisms in a public database and processes them to obtain a raw data set;

[0117] An enhancement module performs light processing and ocean background object occlusion data enhancement processing on the original data set;

[0118] A detection model module, wherein the detection model module optimizes the yolov5 model to obtain a detection model for detecting submarine biological targets;

[0119] A training module, wherein the training module uses the data set after the data enhancement processing in step S2 to train and learn the detection model in step S3 to obtain a trained detection model;

[0120] A detection module, wherein the detection module uses a trained detection model to detect seabed biological targets.

[0121] The same or similar reference numerals correspond to the same or similar components;

[0122] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0123] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for detecting submarine biological targets based on yolov5 optimization, characterized in that: The following steps are involved: S1: Collect images containing various seafloor organisms from public databases and process them to obtain the original dataset; S2: performing light processing and ocean background object occlusion data enhancement processing on the original data set; S3: Optimize the yolov5 model to obtain a detection model for seabed biological target detection; S4: Using the data set processed by the data enhancement in step S2 to train the detection model in step S3, to obtain a trained detection model; S5: Use the trained detection model to detect submarine biological targets; In step S3, the yolov5 model is optimized, specifically: 1) Optimize the yolov5 network structure, add branches and upsampling network layers in multiple places in the original yolov5 network, connect them separately, add detection heads, and add corresponding 3 groups of anchor frames for the added detection layers. The optimized yolov5 structure has 5 groups of detection heads, which can enhance the detection ability of small and large objects. Specifically: the 3rd layer in the original yolov5 network leads to branches, which are connected to the subsequently added upsampling network layer; add a group of convolutional layers after the 10th layer in the original yolov5 network, and lead to branches to connect with the subsequently added upsampling layer; 2) Optimize the original Conv module of yolov5, and modify the convolution layer activation function in the Conv module to the Half_Hardswish activation function. The Half_Hardswish activation function is specifically shown in the following formula: 3) Add SE-HHslayer module to the last layer of yolov5 backbone; 4) Add the CBAM attention mechanism module to the Conv module optimized in 2); 5) Optimize the BottleneckCSP module of yolov5, replace the convolutional layers in the module with the Conv module optimized by 2), and change the original two-branch structure to a three-branch structure. The outputs of the three branches are combined as the input of the module. The feature map of the optimized BottleneckCSP module is mapped as follows: y1=ConvHHS(x1) y2=Bottleneck(ConvHHS(x1)) y3=Bottleneck(ConvHHS(x1)) BottleneckCSP_Cat(x1)=Concat(y1, y2, y3) Where BottleneckCSP_Cat() is the output of the optimized BottleneckCSP module.

2. The method for detecting submarine biological targets based on yolov5 optimization according to claim 1 is characterized in that: The step S1 specifically includes the following steps: S1.1: Obtain images containing a variety of seafloor organisms through public databases and crawlers; S1.2: Screen and annotate the images to obtain an original data set.

3. The method for detecting submarine biological targets based on yolov5 optimization according to claim 2 is characterized in that: In step S1.2, labelimg tool is used to perform labeling to obtain labeling information, which includes the object category in the image, the object center coordinates and the length and width information of the object frame, wherein the length and width information of the object frame is: the ratio of the length and width of the object frame to the length and width of the image.

4. The method for detecting submarine biological targets based on yolov5 optimization according to claim 1 is characterized in that: Before the detection model of step S3 is trained and learned using the data set processed by the data enhancement process of step S2 in step S4, the following steps are also included: Extract the Backbone network part of the detection model in step S3, establish a classification model, perform pre-training on a large-scale marine biological classification dataset, and obtain pre-training weights; The pre-trained weights are loaded into the detection model of step S3.

5. The method for detecting submarine biological targets based on yolov5 optimization according to claim 4 is characterized in that: The step S4 specifically comprises the following steps: S4.1: Build the environment required for detection model training; S4.2: Divide the data set after the data enhancement process in step S2 into a training set, a validation set, and a test set; S4.3: Set hyperparameters for network model training; S4.4: Loading the divided data set into the detection model for training to obtain a trained detection model.

6. The method for detecting submarine biological targets based on yolov5 optimization according to claim 5 is characterized in that: In step S4.2, 60% of the data set after data augmentation in step S2 is used as a training set, 20% as a validation set, and 20% as a test set.

7. The method for detecting submarine biological targets based on yolov5 optimization according to claim 6 is characterized in that: In step S4.1, stratified sampling is performed during the partitioning of the data set.

Images