Netting damage detection model establishment and detection method

Abstract

The invention relates to the field of underwater target detection, in particular to a netting damage detection model establishment and detection method, which comprises the following steps of: establishing a YOLOv8-EAC model which is an improved YOLOv8 model; comprising a backbone network Backbone for carrying out feature extraction on an input image, a neck network Neck for carrying out feature extraction and fusion on a feature map, and a head network Head for outputting fusion features provided by the neck network Neck; replacing the first two Conv layers in the original backbone network Backbone with an EES module, replacing other Conv modules in the YOLOv8 model with an ASAD module, and replacing a C2F module in the YOLOv8 model with a C2F-FEC module; the netting damage is detected by adopting the YOLOv8-EAC model, so that the non-contact real-time detection of the netting damage can be realized, and the netting damage detection precision and speed are improved.

Description

Technical Field

[0001] The present invention relates to the field of underwater target detection, and particularly to a method for establishing a netting damage detection model for detection. Background Art

[0002] The global ocean area is approximately 360 million square kilometers, providing a vast open sea area for the development of the mariculture industry, with huge potential. Compared with inshore aquaculture, offshore aquaculture can accelerate the growth rate of fish, reduce mortality, and decrease the visceral fat content. To achieve the sustainable development of fisheries, seawater cage culture has become the main direction of seawater fish culture in China. In the structure of a marine cage, the most critical component is the condition of the netting system. Under harsh environmental loads and attacks by large predators, the netting is easily damaged. If the damaged netting is not detected in time, fish may escape, resulting in economic losses to the aquaculture industry and potential ecological damage. Previously, netting inspections usually involved regularly replacing the netting or hiring professional divers to inspect the underwater netting. However, these traditional methods are not only inefficient and costly but also pose certain safety hazards. Therefore, developing an efficient, accurate, and economical intelligent net loss detection method has become an important research topic. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for establishing a netting damage detection model that can achieve real-time monitoring of the netting state.

[0004] To achieve the above purpose, the present invention adopts the following technical solution: A method for establishing a netting damage detection model, which establishes a YOLOv8-EAC model. The YOLOv8-EAC model is an improved YOLOv8 model, including a backbone network Backbone for feature extraction of the input image, a neck network Neck for feature extraction and fusion of the feature map, and a head network Head for outputting the fused features provided by the neck network Neck; replacing the first two Conv layers in the original backbone network Backbone with EES modules, replacing other Conv modules in the YOLOv8 model with ASAD modules, and replacing the C2F module in the YOLOv8 model with a C2F-FEC module.

[0005] Preferably, the EES module processes the input image through a first convolutional module to obtain a preliminary feature map, divides the preliminary feature map into three paths for processing. One path uses a SobelConv module for edge detection, one path uses a Pool-branch module for spatial information extraction, and one path uses a second convolutional module for feature extraction. The outputs of the three paths are concatenated, and the concatenated feature map is processed by a convolutional layer with a stride of 2. A 1x1 convolution is applied to the convolved feature map to output the image dimension.

[0006] Preferably, the ASAD module processes the input feature map in two paths. One path sequentially performs average pooling operation, convolution operation, rearrangement, and Softmax operation on the feature map; The other path evenly divides the channels of the feature map into g subgroups, performs convolution operations on each group of feature maps through independent convolutional kernels, concatenates the convolved feature maps in the channel dimension, and performs convolution and rearrangement operations on the concatenated feature map; The outputs of the two paths are summed to output the first feature image.

[0007] Preferably, the C2F-FEC module is obtained by replacing the original bottleneck module in the C2F module with the FasterNet_EMA_CBAM module; The FasterNet_EMA_CBAM module includes a Pconv module, a first PWConv module, a second PWConv module, an EMA module, a CBAM module, and a Concat module connected in sequence; The Pconv module is used to perform a convolution operation on the input image to obtain a first feature map; The first PWConv module is used to perform a point convolution operation on the first feature map, perform batch normalization on the feature map after the point convolution operation, and output a second feature map through a Relu activation function; The second PWConv module is used to perform a point convolution operation on the second feature map to obtain a third feature map; The EMA module reshapes the channel dimension of the third feature map, remaps some channels to the batch dimension, and divides the batch dimension into multiple groups of sub-feature maps , and three branches are used to process the multiple groups of sub-feature maps respectively Among them, two branches perform global average pooling operations on the multiple groups of sub-feature maps along the height direction and the width direction respectively, perform concatenation operations and convolution operations on the outputs of the two average pooling operations respectively, output a fourth feature map, use two activation functions to process the fourth feature map respectively, and then concatenate the processed feature maps to output a fifth feature map. Perform weighted operation, normalization operation, average pooling operation, and Softmax operation on the fifth feature map in sequence to output a sixth feature map; Another branch uses a 3×3 convolution to process the multiple groups of sub-feature maps Perform a convolution operation to generate the seventh feature map, perform an average pooling operation and a Softmax operation on the seventh feature map in sequence, and perform a Matmul operation on the feature map after the other two branches perform a weighting operation and a normalization operation on the fifth feature map and the feature map after the average pooling operation and the Softmax operation on the seventh feature map of this branch, and output the eighth feature map; Perform a Matmul operation on the sixth feature map and the seventh feature map to output the ninth feature map, splice the eighth feature map and the ninth feature map, and perform an activation function process and a weighting operation on the spliced feature map to output the aggregated feature map; The CBAM module performs global average pooling and global max pooling on the aggregated feature map respectively, and outputs a channel attention map through a shared fully connected layer and an activation function for the feature map after global average pooling and the feature map after global max pooling; Finally, splice and output the channel attention map and the first feature map through the Concat module.

[0008] A method for detecting damaged fishing nets includes the following steps executed in sequence: S1: Obtain videos and pictures of damaged fishing nets, make a data set, and divide the data set into a training set, a validation set and a test set according to a preset ratio; S2: Input the training set into the YOLOv8-EAC model established by the method for establishing a fishing net damage detection model as described above, train the YOLOv8-EAC model, and use the test set to verify the detection effect.

[0009] Preferably, the production of the data set specifically includes the following steps: S1-1: Shoot videos and pictures of damaged fishing nets through an underwater robot; S1-2: Perform frame extraction on the video, convert the video into multiple images, and the multiple images and the obtained pictures form the original data set; S1-3: Use online image annotation software to annotate the original data set to obtain the labeled label data, and the label data constitutes the data set.

[0010] By adopting the foregoing design scheme, the beneficial effects of the present invention are as follows: The present application constructs a YOLOv8-EAC model based on the YOLOv8 model. The first two Conv layers in the original backbone network Backbone of the YOLOv8 model are replaced with EES modules to improve the model detection accuracy; other Conv modules in the YOLOv8 model are replaced with ASAD modules to reduce the model volume and improve the detection accuracy; the C2F module in the YOLOv8 model is replaced with a C2F-FEC module to further reduce the model volume and improve the detection accuracy; the YOLOv8-EAC model of the present application can achieve non-contact real-time detection of netting damage, improving the detection accuracy and speed of netting damage. Description of the Drawings

[0011] Figure 1 It is a structural diagram of the YOLOv8-EAC model of the present invention; Figure 2 It is a structural diagram of the EES module of the present invention; Figure 3 It is a structural diagram of the ASAD module of the present invention; Figure 4 It is a structural diagram of the C2F-FEC module of the present invention. Detailed Embodiments

[0012] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0013] The terms "first", "second", "third", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include unlisted steps or units, or may optionally further include other steps or units inherent to these processes, methods, products, or devices.

[0014] A method for establishing a netting damage detection model, establishing as Figure 1The YOLOv8-EAC model shown, which is an improved YOLOv8 model, includes a Backbone network for feature extraction of the input image, a Neck network for feature extraction and fusion of the feature maps, and a Head network for outputting the fused features provided by the Neck network; the first two Conv layers in the original Backbone network are replaced with EES modules, other Conv modules in the YOLOv8 model are replaced with ASAD modules, and the C2F module in the YOLOv8 model is replaced with a C2F-FEC module.

[0015] In this embodiment, as Figure 2 shown, the EES module processes the input image through a first convolution module with a 3×3 convolution kernel to obtain a preliminary feature map, and divides the preliminary feature map into three paths for processing. One path uses a SobelConv module for edge detection. The SobelConv module extracts the edge information of the image through two parallel 3D convolution layers; specifically, the Sobel filter is used to detect the gradients of the image in the horizontal and vertical directions and identify the edge information in the image. In the implementation process, the Sobel operators are defined as a 3x3 convolution kernel and then extended to a 3D convolution kernel to adapt to multi-channel input images. In this way, the model can extract the edge information in the horizontal and vertical directions of the image respectively to obtain the edge features in two directions. These features will be added in the subsequent stage to generate the complete edge detection result.

[0016] One path uses a Pool-branch module for spatial information extraction. The Pool-branch module performs zero-padding operation and max-pooling operation on the preliminary feature map in sequence. The zero-padding operation ensures that the size of the image will not be lost during the processing, and the max-pooling operation is used to reduce the spatial resolution of the image while retaining the most significant spatial features. The result after max-pooling contains the high-level spatial information of the image, such as structure and position, etc.

[0017] One path uses a second convolution module with a 3×3 convolution kernel for feature extraction, and performs a concatenation operation on the outputs of the three paths, that is, combines the feature maps of the three into an overall feature representation. The concatenated feature map is processed by a convolution layer with a stride of 2 to further reduce the spatial resolution, enabling the network to focus on more abstract features. Apply a 1x1 convolution to the convolved feature map to map the combined features to the final output shape, outputting the image dimension (C, W, H).

[0018] The EES module is an efficient image front-end that integrates edge and spatial information extraction, combining edge feature detection and spatial context capture through multiple convolution layers to enhance the overall feature representation.

[0019] In this embodiment, as Figure 3 shown, the ASAD module processes the input feature map in two paths. Assume that the size of the input feature map is (1, ch, h, w), where 1 represents the batch size, ch represents the number of channels of the image, and h and w represent the height and width of the image respectively. One path sequentially performs average pooling operation, convolution operation, rearrangement, and Softmax operation on the feature map; In this embodiment, the average pooling (AvgPool) operation with a stride of 3 and a padding of 1 is adopted for the feature map. The purpose of this operation is to reduce the spatial resolution to reduce the computational amount while retaining key information, and the size of the pooled feature map remains unchanged; the Softmax operation maps the feature at each position to a probability distribution, generating the class probability at each position in the image. At this time, the size of Branch 1 is (1,ch,h / 2,w / 2,4).

[0020] The other path evenly divides the channels of the feature map into g subgroups, and the number of channels in each group is ch / g. Assume that the size of a certain feature map is (1, ch, h, w), and ch is the number of channels. After dividing into g groups, the size of the feature map in each group becomes (1,ch / g, h, w); independent convolutional kernels are used to perform convolution operations on the feature maps of each group to extract different features, and the convolved feature maps of each group are concatenated in the channel dimension, and the number of channels is restored to ch, and the spatial resolution is halved, obtaining a feature map with a size of (1,ch, h / 2, w / 2). The concatenated feature map is subjected to convolution and rearrangement operations; the outputs of the two paths are summed. The purpose of this step is to integrate the feature maps of all groups together, and finally output a first feature image with a size of (1, ch, h / 2,w / 2).

[0021] Through this way of combining pooling and grouped convolution, the ASAD module can effectively extract various features in the image, reduce the computational complexity, and classify the features through the Softmax operation, while keeping the size of the feature map gradually decreasing, so that the finally output feature map can fully express the key information of the image.

[0022] In this embodiment, as Figure 4 shown, the C2F-FEC module is obtained by replacing the original bottleneck module in the C2F module with the FasterNet_EMA_CBAM module. The FasterNet_EMA_CBAM module integrates innovative PConv and PWConv (pointwise convolution) operators into the FasterNetBlock model, aiming to effectively solve the redundant load problem in the model, further reduce the model volume, and improve the detection accuracy.

[0023] The FasterNet_EMA_CBAM module consists of a Pconv module, a first PWConv module, a second PWConv module, an EMA module, a CBAM module, and a Concat module connected in sequence.

[0024] The Pconv (3*3) module applies convolution only to a part of the input channels and does not process other channels to utilize the redundancy in the feature map. Specifically, during consecutive or regular memory access, the Pconv module only uses the first or the last consecutive cp channels as representatives of the entire feature map for calculation. Therefore, integrating Pconv into the FasterNetBlock model not only significantly improves the model's computational speed but also plays a crucial role in feature extraction. The formula for calculating the floating-point operations (FLOP) of the Pconv module is as follows: ; where h and w represent the height and width of the feature map, k is the size of the convolution kernel, is the number of channels of the convolution. During model training, r = cp / c = 1 / 4, where r represents a proportional relationship of the convolution kernel parameters, specifically the ratio of the number of channels of the convolution kernel, and c represents the number of channels of the convolution kernel. Therefore, the FLOP of Pconv is only 1 / 16 of the FLOP of the standard convolution. This greatly reduces the computational load and makes the model more lightweight. In addition, the Pconv module has a smaller memory access volume, that is:

[0025] The memory access volume of the Pconv module is only 1 / 4 of that of the standard convolution, and the remaining channels do not participate in the calculation and thus do not require memory access.

[0026] The convolution kernel size of the Pconv module is 3×3, which is used to perform convolution operations on the input image to obtain the first feature map; this convolution is used to retain the depth information of the features while reducing the computational amount.

[0027] The first PWConv module is used to perform point convolution operations on the first feature map, perform batch normalization on the feature map after the point convolution operation, and output the second feature map through the Relu activation function; the point convolution operation of the first PWConv module is used to further process the features of the first feature map, which helps to reduce the computational complexity while maintaining the spatial information; the batch normalization process is used to normalize the input of each layer and reduce the internal covariate shift to improve the model performance. It standardizes the output of the previous layer to ensure that each feature map has zero mean and unit variance; the Relu activation function is used to introduce non-linearity into the model so that it can learn more complex patterns. The output of the ReLU function is the larger of the input value and 0, that is, when the input value is greater than 0, the input value is output, otherwise 0 is output. This function can effectively set all negative values to zero.

[0028] The second PWConv module is used to perform point convolution operations on the second feature map to obtain the third feature map; the second PWConv module is used to refine the features learned in the previous steps.

[0029] The EMA module reshapes the channel dimension of the third feature map, remaps some channels to the batch dimension (Batch Dimension), and divides the batch dimension into multiple groups of sub-feature maps , this grouping method ensures that the spatial semantic information is evenly distributed in each feature group, thereby reducing the computational cost while retaining the information of each channel.

[0030] To aggregate multi-scale spatial structure information, three parallel branches are used to process the multiple groups of sub-feature maps respectively where two branches perform global average pooling operations on the multiple groups of sub-feature maps respectively along the height direction and the width direction , perform concatenation operations and convolution operations on the outputs of the two average pooling operations respectively, output the fourth feature map, use two activation functions to process the fourth feature map respectively, then concatenate the processed feature maps to output the fifth feature map, and perform weighted operations, normalization operations, average pooling operations and Softmax (normalization function) operations on the fifth feature map in sequence to output the sixth feature map; Another branch uses a 3×3 convolution to perform convolution operations on the multiple groups of sub-feature maps to generate the seventh feature map to capture multi-scale feature representations. Perform average pooling operations and Softmax (normalization function) operations on the seventh feature map in sequence, and perform Matmul (matrix multiplication) operations on the feature map obtained by performing weighted operations and normalization operations on the fifth feature map by the other two branches and the feature map obtained by performing average pooling operations and Softmax operations on the seventh feature map by this branch to output the eighth feature map; Perform a Matmul operation on the sixth feature map and the seventh feature map, output the ninth feature map, concatenate the eighth feature map and the ninth feature map, and perform Sigmoid activation function processing and weighted operation on the concatenated feature map to output the aggregated feature map.

[0031] To further dynamically adjust the attention degree of features, effectively suppress noise, and avoid overfitting, the CBAM module is introduced. The CBAM module performs global average pooling and global max pooling on the aggregated feature map respectively, and outputs the channel attention map through a shared fully connected layer and activation function for the feature map after global average pooling and the feature map after global max pooling; the CBAM module can improve performance without significantly increasing the model complexity with less computational and parameter overhead. CBAM improves the performance of the convolutional neural network through channel attention and spatial attention mechanisms.

[0032] Finally, output by concatenating the channel attention map and the first feature map through the Concat module.

[0033] This application integrates the EMA and CBAM attention modules into the C2F-Faster module to form the C2F-FEC module, realizing the aggregation of multi-scale spatial information, thereby improving the adaptability of the model in complex scenarios.

[0034] This embodiment also provides a detection method for detecting net clothing damage using the above YOLOv8-EAC model.

[0035] A detection method for net clothing damage includes the following steps executed in sequence: S1: Obtain videos and pictures of damaged net clothing, make a data set, and divide the data set into a training set, a validation set, and a test set according to a preset ratio; The production of the data set specifically includes the following steps: S1-1: Shoot videos and pictures of damaged net clothing through an underwater robot; S1-2: Perform frame extraction on the video, convert the video into multiple images, and the multiple images and the obtained pictures form the original data set; S1-3: Use online image annotation software to annotate the original data set to obtain the labeled label data, and the label data constitutes the data set.

[0036] S2: Input the training set into the YOLOv8-EAC model established by the net clothing damage detection model establishment method as described above, train the YOLOv8-EAC model, and use the test set to verify the detection effect.

[0037] In summary, by adopting the method for detecting the damage of the fishing net provided by the present application, non-contact real-time detection of the damage of the fishing net can be achieved, and the detection accuracy and speed of the damage of the fishing net are improved.

[0038] The specific embodiments described above further elaborate on the purpose, technical solution and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for establishing a net damage detection model, characterized in that: Build the YOLOv8-EAC model, The YOLOv8-EAC model is an improved YOLOv8 model, including a backbone network Backbone for feature extraction of input images, a neck network Neck for feature extraction and fusion of feature maps, and a head network Head for outputting the fusion features provided by the neck network Neck; the first two Conv layers in the original backbone network Backbone are replaced by EES modules, the other Conv modules in the YOLOv8 model are replaced by ASAD modules, and the C2F module in the YOLOv8 model is replaced by a C2F-FEC module.

2. The method for establishing a net damage detection model according to claim 1, characterized in that: The EES module processes the input image through the first convolution module to obtain a preliminary feature map, and divides the preliminary feature map into three paths for processing. One path uses the SobelConv module for edge detection, one path uses the Pool-branch module for spatial information extraction, and one path uses the second convolution module for feature extraction. The three outputs are spliced, and the spliced ​​feature map is processed with a convolution layer with a step size of 2. 1x1 convolution is applied to the convolved feature map to output the image dimension.

3. The method for establishing a net damage detection model according to claim 1 or 2, characterized in that: The ASAD module performs two-way processing on the input feature map, one of which performs average pooling operation, convolution operation, rearrangement and Softmax operation on the feature map in sequence; The other path divides the channels of the feature map into g subgroups on average, performs convolution operation on each group of feature maps through independent convolution kernels, splices the convolved feature maps in the channel dimension, and performs convolution and rearrangement operations on the spliced ​​feature maps; the two outputs are summed to output the first feature image.

4. The method for establishing a net damage detection model according to claim 3, characterized in that: The C2F-FEC module is obtained by replacing the original bottleneck module in the C2F module with the FasterNet_EMA_CBAM module; The FasterNet_EMA_CBAM module includes a Pconv module, a first PWConv module, a second PWConv module, an EMA module, a CBAM module, and a Concat module connected in sequence; The Pconv module is used to perform a convolution operation on the input image to obtain a first feature map; The first PWConv module is used to perform a point convolution operation on the first feature map, perform batch normalization processing on the feature map after the point convolution operation, and output a second feature map through a Relu activation function; The second PWConv module is used to perform a point convolution operation on the second feature map to obtain a third feature map; The EMA module reshapes the channel dimension of the third feature map, remaps part of the channels to the batch dimension, and divides the batch dimension into multiple groups of sub-feature maps , three branches are used to respectively deal with the multiple groups of sub-feature maps Processing is performed, wherein the two branches perform global average pooling operations on the multiple groups of sub-feature maps along the height direction and the width direction respectively, and the outputs of the two average pooling operations are spliced ​​and convolved respectively to output a fourth feature map, and the fourth feature map is processed by two activation functions respectively, and then the processed feature maps are spliced ​​to output a fifth feature map, and the fifth feature map is sequentially subjected to weighting operations, normalization operations, average pooling operations, and Softmax operations, and a sixth feature map is output; Another branch uses 3×3 convolution to perform multi-group feature map Perform a convolution operation to generate a seventh feature map, perform an average pooling operation and a Softmax operation on the seventh feature map in sequence, perform a Matmul operation on the feature map after the other two branches perform weighted operations and normalization operations on the fifth feature map and the feature map after the branch performs average pooling operations and Softmax operations on the seventh feature map, and output an eighth feature map; Performing a Matmul operation on the sixth feature map and the seventh feature map to output a ninth feature map, splicing the eighth feature map and the ninth feature map, performing activation function processing and weighting operation on the spliced ​​feature map, and outputting an aggregated feature map; The CBAM module performs global average pooling and global maximum pooling on the aggregated feature map, respectively, and outputs a channel attention map through a shared fully connected layer and an activation function for the feature map after global average pooling and the feature map after global maximum pooling; Finally, the channel attention map and the first feature map are concatenated and outputted through the Concat module.

5. A method for detecting damaged nets, characterized in that: The process includes the following steps: S1: Obtain videos and pictures of damaged nets, create a data set, and divide the data set into a training set, a validation set, and a test set according to a preset ratio; S2: Input the training set into the YOLOv8-EAC model established by the method for establishing a net damage detection model as described in any one of claims 1 to 4, train the YOLOv8-EAC model, and use the test set to verify the detection effect.

6. The method for detecting damaged nets according to claim 5, characterized in that: The preparation of the data set specifically includes the following steps: S1-1: Video and pictures of damaged nets taken by underwater robots; S1-2: performing frame extraction processing on the video to convert the video into multiple images, wherein the multiple images and the acquired pictures constitute an original data set; S1-3: Use online image annotation software to annotate the original data set to obtain annotated label data, which constitutes the data set.

Images