A YOLOv5 marine fish target detection method based on attention mechanism and DropBlock

Abstract

A YOLOv5 marine fish target detection method based on attention mechanism and DropBlock, comprising the following steps: step 1: acquiring a target fish data set; step 2: labeling and dividing the acquired data set; step 3: constructing a YOLOv5 marine fish target detection model with attention mechanism and DropBlock; step 4: using the constructed marine fish target detection model to detect fish. The purpose of the present invention is to solve the technical problem that the existing fish detection method has a large range of marine fish activities and a variety of living environments, and the fish photographed underwater are deformed, and a clearer fish target cannot be captured, which affects the accuracy of fish detection, and the single convolutional neural network is easily disturbed by image noise when extracting fish features, and the fish cannot be quickly detected, thereby affecting the speed and accuracy of the target detection method.

Description

Technical Field

[0001] The present invention belongs to the technical field of fish research, and in particular relates to a YOLOv5 marine fish target detection method based on attention mechanism and DropBlock. Background Art

[0002] Marine fish species are numerous and their range of activities is wide, so manual counting is inefficient and difficult. In addition, most fish images are taken underwater, which results in turbid water, fluctuations, and light disturbances, which results in low clarity of fish images.

[0003] In the prior art, a patent document with application publication number CN112926652A discloses a method for fine-grained fish image recognition based on deep learning. It selects a public marine fish video dataset and cannot adapt to and extract the fish characteristics of the country's waters. When using a convolutional neural network that fuses candidate regions to detect fish images, the detection speed is not fast enough, and the detection effect and generalization are not good enough. Summary of the invention

[0004] The purpose of the present invention is to solve the technical problems that the existing fish detection methods have large activity ranges and diverse living environments of marine fish, and the fish photographed underwater are deformed, and clearer fish targets cannot be captured, which affects the accuracy of fish detection. The single convolutional neural network for extracting fish features is easily interfered by image noise and cannot quickly detect fish, thereby affecting the speed and accuracy of the target detection method.

[0005] A YOLOv5 marine fish target detection method based on attention mechanism and DropBlock, which includes the following steps:

[0006] Step 1: Acquire the target fish data set;

[0007] Step 2: Label and divide the obtained data set;

[0008] Step 3: Construct a YOLOv5 marine fish target detection model with attention mechanism and DropBlock;

[0009] Step 4: Use the constructed marine fish target detection model to detect fish.

[0010] In step 3, the model structure is: the fish data set obtained in step 2 is sequentially passed through the first CBS module---the second CBS module---the first C3 module---the third CBS module---the second C3 module---the fourth CBS module---the third C3 module---the fifth CBS module---the fourth C3 module---the SPPF module---the sixth CBS module---the first Unsample module---the first Concat module---the fifth C3 module---the seventh CBS module---the second Upsample module---the second Concat module---the sixth C3 module---the eighth CBS module---the third Concat module---the seventh C3 module---the ninth CBS module---the fourth Concat module---the eighth C3 module;

[0011] The output of the first Unsample module is concatenated with the output of the third C3 module and input into the first Concat module;

[0012] The output of the second Unsampale module is concatenated with the output of the second C3 module and input into the second Concat module;

[0013] The output of the sixth C3 module is input into the input end of the first convolution module Conv, and the output of the first convolution module Conv is used as the first output result;

[0014] The output of the seventh C3 module is input into the input end of the second convolution module Conv, and the output of the second convolution module Conv is used as the second output result;

[0015] The output of the eighth C3 module is input into the input end of the third convolution module Conv, and the output of the third convolution module Conv is used as the third output result;

[0016] The first output result, the second output result, and the third output result are combined to obtain a combined detection result.

[0017] The first C3 module, the second C3 module, the third C3 module, the fourth C3 module and the fifth C3 module are improved C3 modules, which include C3 modules and ECA modules connected to the outputs of the C3 modules;

[0018] The specific structure is as follows: the C3 module includes two branches, the first branch includes a CBS module and a bottleneck connected in series; the other branch includes a CBS module; the outputs of the two branches are spliced ​​and then connected to the ECA module.

[0019] In the ECA module, the input passes through the pooling layer, then through the convolution, and finally through the Sigmoid activation function to obtain the output, which is multiplied by the input feature map, and the result is used as the output of the ECA module;

[0020] The sixth C3 module, the seventh C3 module and the eighth C3 module are improved C3 modules, which include a DropBlock module, a C3 module connected to the output of the DropBlock module, an ECA connected to the output of the C3 module, and an SSA connected to the output of the ECA;

[0021] The specific structure is as follows: the C3 module includes two branches, the first branch includes a CBS module and a bottleneck connected in series; the other branch includes a CBS module; the outputs of the two branches are spliced ​​and connected in series to the CBS module, then connected to the ECA module, and then connected to the SSA module.

[0022] In the DropBlock module, the input passes through the DropBlock layer to discard the local area information and obtain the output. In the ECA module, the input passes through the pooling layer, then through the convolution, and finally through the Sigmoid activation function to obtain the output, which is multiplied by the input feature map, and the result is used as the output of the ECA module. In the SSA module, the input passes through the global maximum pooling layer and the global average pooling layer respectively to obtain the maximum value of the feature map in the W direction and the H direction, passes through the BN layer, then through the convolution, and finally through the Sigmoid activation function to obtain the output, which is multiplied by the input feature map, and the result is used as the output of the SSA module.

[0023] In step 2, the data set obtained in step 1 is manually proofread and labeled using LabelImg labeling software to obtain corresponding label data.

[0024] In step 4, the following steps are included:

[0025] 4-1: Input the marine fish training set into the network in step 3 for training, adjust the learning rate, momentum, and number of iterations of the model, and use the cosine annealing algorithm to dynamically adjust the learning rate;

[0026] 4-2: Put the marine fish test set into the model trained in step 4-1, and calculate relevant evaluation indicators based on the model output results to evaluate the performance of the model;

[0027] If the model performance does not meet expectations, adjust the network parameters and repeat steps 4-1 and 4-2; measure the performance of the method by calculating accuracy, precision, recall and parameters;

[0028] The calculation formula of accuracy is shown in formula (3), the calculation formula of precision is shown in formula (4), and the calculation formula of recall is shown in formula (5);

[0029]

[0030]

[0031]

[0032] Among them: TP represents true positive examples, FP represents false positive examples, FN represents false negative examples, and TN represents true negative examples.

[0033] Compared with the prior art, the present invention has the following technical effects:

[0034] 1) The present invention uses a public dataset for small target detection in a fusion complex environment and a self-built dataset suitable for large target detection to adapt to the diverse living environments and detection scenarios of fish; the feature extraction network combining the C3 structure and ECA can effectively increase the backbone network's ability to understand channel information; the SSA structure based on feature enhancement and spatial attention mechanism can improve the clarity of feature information, allowing the network to more efficiently process features that need to be paid attention to in the feature map; the target detection network based on the DropBlock regularization method can effectively reduce the redundant features of the YOLOv5 neck during training, increase the network's perception of local features, and effectively alleviate the overfitting phenomenon in the training process;

[0035] 2) The present invention combines the channel attention mechanism, spatial attention mechanism and DropBlock's YOLOv5 target detection model, which can improve the accuracy of marine fish target detection without increasing additional time consumption, and has a wide range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0037] Figure 1 is a process flow chart of the present invention;

[0038] Figure 2 This is a schematic diagram of the YOLOv5 target detection network structure in the present invention;

[0039] Figure 3 It is a schematic diagram of the CBS structure in the YOLOv5 target detection network in the present invention;

[0040] Figure 4 A schematic diagram of the bottleneck structure in the YOLOv5 target detection network in the present invention;

[0041] Figure 5 It is a schematic diagram of the C3 structure in the YOLOv5 target detection network of the present invention;

[0042] Figure 6It is a schematic diagram of the SPPF structure in the YOLOv5 target detection network of the present invention;

[0043] Figure 7 Schematic diagram of the channel attention ECA structure in the present invention;

[0044] Figure 8 Schematic diagram of the placement of the channel attention ECA in the present invention;

[0045] Fig. 9 Schematic diagram of the spatial attention SSA structure in the present invention;

[0046] Fig.10 Schematic diagram of the placement of spatial attention SSA in the present invention. DETAILED DESCRIPTION

[0047] A YOLOv5 marine fish target detection method based on attention mechanism and DropBlock, including:

[0048] 1. Select and collect data sets;

[0049] 2. Data set annotation and division;

[0050] 3. Construct a YOLOv5 marine fish target detection model with attention mechanism and DropBlock;

[0051] 4. Use the constructed marine fish target detection model to detect fish.

[0052] Step 1 specifically includes:

[0053] Select appropriate fish sample images from public data sets, collect fish from Chinese waters, and manually take marine fish images.

[0054] Step 2 specifically includes:

[0055] 2.1 Use LabelImg annotation software to manually proofread and annotate the public dataset obtained in step 1 to obtain the corresponding correct label data.

[0056] 2.2 Manually label the self-built fish dataset obtained in step 1 using LabelImg labeling software to obtain the corresponding label data.

[0057] 2.3 After the above steps, 10,325 fish images in the public data set and 2,675 fish images in the self-built data set were obtained. The merged data set contained 13,000 fish images and 13,000 label data, which were randomly divided into 9,000 training sets, 2,000 validation sets, and 2,000 test sets.

[0058] Step 3 specifically includes:

[0059] 3.1 Input the fish dataset images obtained in step 2 into the YOLOv5 target detection network with added attention mechanism and DropBlock.

[0060] 3.2 Take the lightweight version of the YOLOv5 target detection network YOLOv5s in step 3.1 as an example. The network input is an image of size 640×640 and 3 channels. It passes through the first layer of the network, that is, the first standard convolution block CBS. CBS is a basic module in convolutional neural networks, consisting of a convolution layer (Conv), a batch normalization layer (BN), and a SiLU activation function. That is, the first layer of the network consists of a Conv with a convolution kernel size of 6×6, a convolution kernel number of 32, and a stride of 2, a BN, and a SiLU activation function. Finally, the output is a feature map of 320×320×32.

[0061] 3.3 Input the output feature map obtained in step 3.2 into the second layer of the network, that is, the second standard convolution block CBS, which consists of a convolution layer with a convolution kernel size of 3×3 and a stride of 2, a BN layer, and a SiLU layer. A feature map of 160×160×64 is obtained.

[0062] 3.4 Input the output feature map obtained in step 3.3 into the third layer of the network, that is, add channel attention as Figure 4 As shown, the C3 module is the feature extraction module proposed by the original YOLOv5, Figure 4 As shown in the dotted box in , the present invention inserts an ECA attention mechanism after C3, which effectively increases the feature extraction network's ability to understand channel information. The feature map input in step 3.3 passes through two branches. One branch passes through a standard convolution block CBS with a convolution kernel size of 1×1 and a step size of 1 and a standard bottleneck in turn. The bottleneck passes through a standard convolution block CBS with a convolution kernel size of 1×1 and a step size of 1 and a standard convolution block CBS with a convolution kernel size of 3×3 and a step size of 1, and its output is added to the input of the bottleneck. The other branch passes through a standard convolution block CBS with a convolution kernel size of 1×1 and a step size of 1. The outputs of the two branches are spliced ​​and then pass through a channel attention module (ECA) to obtain a 160×160×64 feature map.

[0063] 3.5 The channel attention module (ECA) in step 3.4 is as follows Figure 3As shown in the figure, a 1×1×C feature map is first obtained through a global average pooling, and then a one-dimensional convolution of size k is performed, where k is adaptively determined by the function of the channel dimension C, and finally a weight is obtained through a Sigmoid activation function and multiplied with the input feature map. The calculation formula of k is shown in formula (1). Where γ = 2, b = 1.

[0064]

[0065] 3.6 Input the output feature map obtained in step 3.4 into the 4th layer of the network, that is, the standard convolution block CBS, and obtain a feature map of 80×80×128 as in step 3.3.

[0066] 3.7 Input the output feature map obtained in step 3.6 into the 5th layer of the network, that is, the C3 module with channel attention added. Repeat step 3.4 twice to obtain a feature map of 80×80×128.

[0067] 3.8 Input the feature map obtained in step 3.7 into the 6th layer of the network, that is, the standard convolution block CBS, and obtain a 40×40×256 feature map as in step 3.3.

[0068] 3.9 Input the feature map obtained in step 3.8 into the 7th layer of the network, that is, the C3 module that adds channel attention. Repeat step 3.4 three times to obtain a 40×40×256 feature map.

[0069] 3.10 Input the feature map obtained in step 3.9 into the 8th layer of the network, that is, the standard convolution block CBS, and obtain a 20×20×512 feature map as in step 3.3.

[0070] 3.11 Input the feature map obtained in step 3.10 into the 9th layer of the network, that is, the C3 module with channel attention added, and obtain a 20×20×512 feature map as in step 3.4.

[0071] 3.12 Input the feature map obtained in step 3.11 into the 10th layer of the network, namely the spatial pyramid pooling layer (SPPF), to reduce information redundancy and increase the receptive field. First, pass through the standard convolution block CBS with a convolution kernel size of 1×1 and a stride of 1, then pass through three maximum pooling layers SPPF with a convolution kernel size of 5×5, a stride of 1, and a padding of 2, and then perform a Concat splicing operation on the four outputs of the standard convolution block CBS, the first maximum pooling layer SPPF, the second maximum pooling layer SPPF, and the third maximum pooling layer SPPF, and finally pass through the standard convolution block CBS with a convolution kernel size of 1×1 and a stride of 1 to obtain a 20×20×512 feature map.

[0072] 3.13 Input the feature map obtained in step 3.12 into the 11th layer of the network, that is, the standard convolution block CBS. Same as step 3.3, set the convolution kernel size to 1×1 and the step size to 1 to obtain a 20×20×256 feature map.

[0073] 3.14 Input the feature map obtained in step 3.13 into the 12th layer of the network, that is, the upsampling layer Unsample. The number of channels of the feature map is not changed, but the width and height of the feature map are increased to obtain a 40×40×256 feature map.

[0074] 3.15 Input the feature map obtained in step 3.14 into the 13th layer of the network, that is, the concatenation layer Concat, and concatenate it with the output of the 7th layer of the network. The size of the feature map is not changed, but the number of channels of the feature map is increased to obtain a 40×40×512 feature map.

[0075] 3.16 Input the feature map obtained in step 3.15 into the 14th layer of the network, that is, the C3 module with channel attention added, and obtain a 40×40×256 feature map as in step 3.4.

[0076] 3.17 Input the feature map obtained in step 3.16 into the 15th layer of the network, that is, the standard convolution block CBS. Same as step 3.3, set the convolution kernel size to 1×1 and the step size to 1 to obtain a 40×40×128 feature map.

[0077] 3.18 Input the feature map obtained in step 3.17 into the 16th layer of the network, that is, the upsampling layer Unsample. The number of channels of the feature map is not changed, but the width and height of the feature map are increased to obtain a feature map of 80×80×128.

[0078] 3.19 Input the feature map obtained in step 3.18 into the 17th layer of the network, i.e. the concatenation layer Concat, and concatenate it with the output of the 5th layer of the network. The size of the feature map is not changed, but the number of channels of the feature map is increased to obtain a feature map of 80×80×256.

[0079] 3.20 Input the feature map obtained in step 3.19 into the 18th layer of the network, that is, the C3 module that integrates channel attention, spatial attention, and DropBlock. First, perform the DropBlock operation on the input feature map to discard the local area information of the feature map without changing the size of the feature map. The calculation of the discard rate γ is related to the feature map size feat_size, the threshold probability keep_prob, and the discard block size block_size. The specific calculation formula is shown in formula (2). The discard block size of DropBlock is set to 5, and the discard rate is set to 0.1. After adding the C3 module of channel attention, the same as step 3.4. Then pass through the spatial attention module to strengthen the feature information of the fish target. The addition method is as follows Figure 6 The present invention adds strong W-direction information and H-direction information to the traditional spatial attention mechanism, and enhances the neural network's ability to understand spatial position relationships. The structure of the spatial attention module (SSA) is shown in Figure 5 As shown in the figure, the feature map is subjected to global average pooling and global maximum pooling respectively, and then the feature map obtained by pooling is taken to have the maximum value in the W direction and the H direction respectively, and then a vector multiplication is performed according to the rule in formula (3), where A represents the maximum value in the H direction and B represents the maximum value in the W direction, and finally the results obtained after feature enhancement are concatenated. Since matrix multiplication changes the distribution of features, it is also necessary to use a BN layer to normalize the features, and then send them to a 1×1 convolution for parameter learning. After the Sigmoid activation function, the weight obtained is multiplied with the input feature map. Finally, a feature map of 80×80×128 is obtained.

[0080]

[0081] OUT (H×W×C) =A (1×W×C) *B (H×1×C) (3)

[0082] 3.21 Input the feature map obtained in step 3.20 into the 19th layer of the network, that is, the standard convolution block CBS, and obtain a 40×40×128 feature map as in step 3.3.

[0083] 3.22 Input the feature map obtained in step 3.21 into the 20th layer of the network, that is, the concatenation layer Concat, and concatenate it with the output of the 15th layer of the network. The size of the feature map is not changed, but the number of channels of the feature map is increased to obtain a 40×40×256 feature map.

[0084] 3.23 Input the feature map obtained in step 3.22 into the 21st layer of the network, that is, the C3 module of the fusion channel attention and spatial attention and DropBlock. Same as step 3.20, the drop block size of DropBlock is set to 5, the drop rate is set to 0.1, and a 40×40×256 feature map is obtained.

[0085] 3.24 Input the feature map obtained in step 3.23 into the 22nd layer of the network, that is, the standard convolution block CBS, and obtain a 20×20×256 feature map as in step 3.3.

[0086] 3.25 Input the feature map obtained in step 3.24 into the 23rd layer of the network, i.e. the concatenation layer Concat, and concatenate it with the output of the 11th layer of the network. The size of the feature map is not changed, but the number of channels of the feature map is increased to obtain a 20×20×512 feature map.

[0087] 3.26 Input the feature map obtained in step 3.25 into the 24th layer of the network, that is, the C3 module of the fusion channel attention and spatial attention and DropBlock. Same as step 3.20, the drop block size of DropBlock is set to 3, the drop rate is set to 0.1, and a 20×20×512 feature map is obtained.

[0088] 3.27 The outputs of step 3.20, step 3.23, and step 3.26 are convolved to obtain the detection results and fused, and the detection frames with confidence levels lower than 0.25 are filtered out. Non-Maximum Suppression (NMS) is used for post-processing to extract target detection frames with high confidence levels and suppress false detection frames with low confidence levels to obtain the final detection results.

[0089] Step 4 specifically includes:

[0090] 4.1 Input the marine fish training set into the network in step 3 for training, adjust the model's learning rate, momentum, number of iterations, etc., and use the cosine annealing algorithm to dynamically adjust the learning rate, that is, the learning rate is continuously adjusted according to the learning rate and the callback function. Finally, the optimal model is selected from the model with the best performance in each epoch as the final training result.

[0091] 4.2 Put the marine fish test set into the model trained in step 4.1, and calculate the relevant evaluation indicators based on the model output results to evaluate the performance of the model. If the model performance does not meet expectations, you can adjust the network parameters and repeat steps 4.1 and 4.2. The performance of the method is measured by calculating the accuracy, precision, recall and parameters. The accuracy calculation formula is shown in formula (3), the precision calculation formula is shown in formula (4), and the recall calculation formula is shown in formula (5).

[0092]

[0093]

[0094]

[0095] Among them: TP represents true positive examples, FP represents false positive examples, FN represents false negative examples, and TN represents true negative examples.

[0096] The present invention uses a public dataset for small target detection in a fusion complex environment and a self-built dataset suitable for large target detection to adapt to the diverse living environments and detection scenarios of fish; the feature extraction network combining the C3 structure and ECA can effectively increase the backbone network's ability to understand channel information; the SSA structure based on feature enhancement and spatial attention mechanism can improve the clarity of feature information, allowing the network to more efficiently process features that need to be paid attention to in the feature map; the target detection network based on the DropBlock regularization method can effectively reduce the redundant features of the YOLOv5 neck during training, increase the network's perception of local features, and effectively alleviate the overfitting phenomenon in the training process.

Claims

1. A YOLOv5 marine fish target detection method based on attention mechanism and DropBlock, It is characterized in that It includes the following steps: Step 1: Acquire the target fish data set; Step 2: Label and divide the obtained data set; Step 3: Construct a YOLOv5 marine fish target detection model with attention mechanism and DropBlock; Step 4: Use the constructed marine fish target detection model to detect fish; In step 3, the structure of the model is: the fish data set obtained in step 2 is sequentially passed through the first CBS module---the second CBS module---the first C3 module---the third CBS module---the second C3 module---the fourth CBS module---the third C3 module---the fifth CBS module---the fourth C3 module---the SPPF module---the sixth CBS module---the first Unsample module---the first Concat module---the fifth C3 module---the seventh CBS module---the second Upsample module---the second Concat module---the sixth C3 module---the eighth CBS module---the third Concat module---the seventh C3 module---the ninth CBS module---the fourth Concat module---the eighth C3 module; The output of the first Unsample module is concatenated with the output of the third C3 module and input into the first Concat module; The output of the second Unsampale module is concatenated with the output of the second C3 module and input into the second Concat module; The output of the sixth C3 module is input into the input end of the first convolution module Conv, and the output of the first convolution module Conv is used as the first output result; The output of the seventh C3 module is input into the input end of the second convolution module Conv, and the output of the second convolution module Conv is used as the second output result; The output of the eighth C3 module is input into the input end of the third convolution module Conv, and the output of the third convolution module Conv is used as the third output result; The first output result, the second output result, and the third output result are combined to obtain a combined detection result.

2. The method according to claim 1, It is characterized in that The first C3 module is an improved C3 module, which includes a C3 module and an ECA module connected to the output of the C3 module; In the first C3 module, the second C3 module, the third C3 module, the fourth C3 module and the fifth C3 module, the specific structure is: the C3 module includes two branches, the first branch includes a CBS module and a bottleneck connected in series; the other branch includes a CBS module; the outputs of the two branches are spliced ​​and connected in series to the CBS module, and then connected to the ECA module.

3. The method according to claim 2, It is characterized in that In the ECA module, the input passes through the pooling layer, then through the convolution, and finally through the Sigmoid activation function to obtain the output, which is multiplied by the input feature map, and the result is used as the output of the ECA module.

4. The method according to claim 2, It is characterized in that The sixth C3 module is an improved C3 module, which includes a DropBlock module, a C3 module connected to the output of the DropBlock module, an ECA connected to the output of the C3 module, and an SSA connected to the output of the ECA; In the sixth C3 module, the seventh C3 module and the eighth C3 module, the specific structure is as follows: the C3 module includes two branches, the first branch includes a CBS module and a bottleneck connected in series; the other branch includes a CBS module; the outputs of the two branches are spliced ​​and then connected in series to the CBS module, which is then connected to the ECA module and then to the SSA module; In the DropBlock module, the input is passed through DropBlock to discard the local area information and obtain the output; In the ECA module, the input passes through the pooling layer, then through the convolution, and finally through the Sigmoid activation function to obtain the output, which is multiplied by the input feature map, and the result is used as the output of the ECA module; In the SSA module, the input passes through the global maximum pooling layer and the global average pooling layer respectively to obtain the maximum value of the feature map in the W direction and the H direction, passes through the BN layer, then passes through the convolution, and finally passes through the Sigmoid activation function to get the output, which is multiplied by the input feature map, and the result is used as the output of the SSA module.

5. The method according to claim 1, It is characterized in that In step 2, the data set obtained in step 1 is manually proofread and labeled using LabelImg labeling software to obtain corresponding label data.

6. The method according to claim 1, It is characterized in that In step 4, the following steps are included: 4-1: Input the marine fish training set into the network in step 3 for training, adjust the learning rate, momentum, and number of iterations of the model, and use the cosine annealing algorithm to dynamically adjust the learning rate; 4-2: Put the marine fish test set into the model trained in step 4-1, and calculate relevant evaluation indicators based on the model output results to evaluate the performance of the model; If the model performance does not meet expectations, adjust the network parameters and repeat steps 4-1 and 4-2; measure the performance of the method by calculating the accuracy, precision, recall and parameters; The calculation formula of accuracy is shown in formula (3), the calculation formula of precision is shown in formula (4), and the calculation formula of recall is shown in formula (5); Among them: TP represents true positive examples, FP represents false positive examples, FN represents false negative examples, and TN represents true negative examples.

Images