A winter fishing classification and recognition method based on MG-ShuffleNet network structure

Abstract

The present invention discloses a winter fishing classification and recognition method based on the MG-ShuffleNet network structure, which belongs to the field of image recognition and deep learning technology. According to the winter fishing image features, the number of basic unit stacking in the models Stage2, Stage3 and Stage4 is reduced by half (rounded down); the overall architecture of ShuffleNet V2 is redesigned, and the proposed multi-scale hole fusion module is used to replace the convolution in the basic unit and downsampling unit of ShuffleNet V2, and the unimportant convolution in the right branch is deleted, aiming to enhance the feature expression by utilizing a variety of hole rates and reduce the number of parameters and the amount of calculation of the network. A global channel-spatial attention module is designed, which aims to capture the global dependency in the feature map and improve the model performance. In order to further improve the recognition accuracy of the model, the Mish activation function is used to replace the traditional ReLU activation function. In addition, the maximum pooling is replaced by the average pooling to enhance the robustness of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a winter fishing classification and recognition method based on an MG-ShuffleNet network structure. Background Art

[0002] In the field of automated classification of winter fish, it is difficult to distinguish and sort directly because different types of winter fish have similar characteristics such as size, color and texture. At present, there is no classification and identification method for winter fish based on image recognition. This brings great challenges to the classification and identification of winter fish, and also limits the development of automated sorting systems for winter fish. Misidentification and missed identification are prone to occur during the detection process, which further affects the accuracy and efficiency of classification and identification. Therefore, in the field of automated classification of winter fish, it is necessary to use advanced deep learning technology and effective classification and identification algorithms to improve the accuracy and efficiency of classification and identification, and promote the development of classification and identification and sorting systems for winter fish.

[0003] The winter fish classification and recognition method based on deep learning is an emerging visual inspection task, which aims to accurately and efficiently classify and recognize different types of winter fish. Especially in the field of fishery production, different types of winter fish have similar morphological characteristics and are difficult to distinguish directly. In order to improve the efficiency and accuracy of the fishing process, it is also necessary to classify and recognize different types of winter fish. Therefore, a fast, lightweight and high recognition accuracy deep learning recognition method is more suitable for winter fish classification and recognition tasks, which can improve the performance and efficiency of classification and recognition. This method can accurately classify and recognize different types of winter fish, with high recognition accuracy and fast recognition speed. Therefore, this deep learning recognition method based on fast, lightweight and high recognition accuracy has broad application and development prospects in the field of fishery production automation.

[0004] However, there are also some challenges in applying deep learning technology to the field of automatic classification of winter fish. The main challenges are as follows: (1) The environmental background of winter fishing activities is usually very complex, and there may be many interfering objects or ice blood stains, etc., and these objects are similar to the texture of the target winter fish, making it difficult for the model to accurately identify the target. (2) The size, color, texture and other characteristics of different types of winter fish are very similar, and they may also undergo morphological changes at different times and in different environments. This poses a challenge to their automatic classification and recognition, resulting in the model being unable to accurately distinguish different winter fish targets. (3) Existing winter fish classification and recognition models are usually very large because they are built based on deep learning models and require a large amount of training sets and computing resources for training, resulting in a high number of parameters and computational complexity, which makes it difficult to meet the real-time requirements of actual applications and the deployment of actual edge devices. Summary of the invention

[0005] The technical solution of the present invention to solve the above technical problem is to provide a winter fishing classification and identification method based on the MG-ShuffleNet network structure, comprising the following steps:

[0006] Collect image data sets, process the image data sets, and divide them into training sets and validation sets;

[0007] Build a network model including MDFA module and GCSA attention mechanism module:

[0008] Pass the training set through a 3x3 convolutional layer to capture the local features of the image;

[0009] Use the average pooling layer to downsample the convolutional feature map;

[0010] In module stage2, the downsampling unit and the basic unit are stacked once each;

[0011] In module stage3, the downsampling unit is stacked once and the basic unit is stacked three times to enhance the learning ability of features;

[0012] In module stage4, the downsampling unit and the basic unit are stacked once each;

[0013] A 1x1 convolutional layer is used for cross-channel feature fusion and dimensionality reduction;

[0014] The global pooling layer reduces the spatial dimension of the feature map to a single value to capture the global information of the entire feature map;

[0015] The fully connected layer receives the feature vector from the global pooling layer and outputs a predicted value;

[0016] Among them, the 3x3 convolution layer and the 1x1 convolution layer respectively introduced the Mish activation function.

[0017] Further, the downsampling unit and the basic unit both include an MDFA module, and the MDFA module includes:

[0018] The first branch uses a 1x1 convolution kernel to directly extract features;

[0019] The second branch uses a 3x3 convolution kernel with a dilation rate of 6 to expand the receptive field;

[0020] The third branch uses a 3x3 convolution kernel with a dilation rate of 12 to further expand the receptive field to capture a wider range of contextual information;

[0021] The fourth branch uses a 3x3 convolution kernel with a dilation rate of 18 to provide the widest receptive field;

[0022] The fifth branch uses global average pooling to extract global context features and enhance the model's ability to understand the overall layout;

[0023] The first branch, the second branch, the third branch, the fourth branch and the fifth branch are spliced ​​in the channel dimension to form a comprehensive feature map;

[0024] The comprehensive feature map is calibrated in parallel through the channel attention mechanism and the spatial attention mechanism;

[0025] The output feature maps of the channel attention and spatial attention mechanisms are fused through element-wise addition operations to obtain enhanced feature maps.

[0026] The enhanced feature map is reduced in dimension and integrated through a 1x1 convolutional layer to generate the output feature map.

[0027] Further, the downsampling unit and the basic unit both include a GCSA module, and the GCSA module includes:

[0028] The input feature map is first dimensionally permuted from C×H×W to W×H×C;

[0029] The dependency between channels is captured through two layers of multi-layer perceptrons (MLPs). The first layer of MLP reduces the number of channels to 1 / 4 of the original number, then introduces nonlinearity through the ReLU activation function, and then restores the number of channels to the original dimension through the second layer of MLP. The inverse permutation is performed to restore it to C×H×W, and the channel attention map is generated through the Sigmoid activation function. The input feature map and the channel attention map are multiplied element by element to obtain the enhanced feature map.

[0030] The enhanced feature maps are divided into 4 groups, each containing C / 4 channels; the grouped feature maps are transposed to disrupt the channel order within each group; the disrupted feature maps are restored to their original shape (C×H×W);

[0031] In the spatial attention submodule, the input feature map passes through a 7x7 convolution layer, and the number of channels is reduced to 1 / 4 of the original; it undergoes nonlinear transformation through batch normalization and ReLU activation function; the number of channels is restored to the original dimension C through the second 7x7 convolution layer, and then passes through the batch normalization layer; the spatial attention map is generated through the Sigmoid activation function; the shuffled feature map and the spatial attention map are multiplied element by element to obtain the output feature map.

[0032] Compared with the prior art, the advantages of the present invention are as follows:

[0033] (1) MG-ShuffleNet achieves a perfect balance between high accuracy and light weight by integrating the advantages of MDFA and GCSA and the simplified ShuffleNet V2 architecture. MDFA improves the model's adaptability to various multi-scale features in the image, while the GCSA module maintains the richness and accuracy of features through the combination of channel attention and spatial attention mechanisms. Compared with traditional ShuffleNet and other networks, MG-ShuffleNet significantly improves efficiency in image classification and object detection tasks, and is suitable for edge device deployment and real-time requirements.

[0034] (2) The temperature in Northeast China is low in winter, and the winter fish caught are often covered with ice and blood stains, which makes it difficult to identify winter fish and the recognition accuracy is low. In order to overcome this problem, the Mish activation function is used to replace the traditional ReLU activation function to further improve the recognition accuracy of the model. Compared with the ReLU activation function, the Mish activation function is smoother in a larger input range, which can avoid the problems of gradient explosion and gradient disappearance during training, and improve the stability and robustness of the model. In addition, the nonlinear characteristics of the Mish activation function can better process complex data and provide more accurate activation responses, thereby improving the recognition accuracy of the model. The use of the Mish activation function can help the model more accurately identify winter fish caught in the complex environment of Northeast China, including fish covered with blood stains and ice, which can further improve the recognition accuracy of the model and help improve the benefits of the winter fish farming industry in Northeast China.

[0035] (3) In order to ensure the robustness of the model in the practical application of the winter fishing sorting environment, the MG-ShuffleNet model has been improved for the pooling layer and overfitting problems. In terms of the pooling layer, the average pooling (AvgPooling) is used to replace the maximum pooling (MaxPooling), which can reduce the loss and imbalance of information, and at the same time improve the robustness and generalization ability of the model. In addition, in order to prevent the overfitting problem, L2 regularization is introduced to avoid overfitting of the model during training. L2 regularization can limit the value of the parameter by increasing the model penalty term, thereby reducing the complexity of the model and improving the robustness and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0037] Figure 1 The MG-ShuffleNet network structure diagram of the winter fishing classification and identification method based on the MG-ShuffleNet network structure of the present invention;

[0038] Figure 2 This is a structural diagram of the MDFA module of the present invention;

[0039] Figure 3 This is the structural diagram of the GCSA attention mechanism described in the present invention;

[0040] Figure 4 This is the winter fishing image data enhancement diagram described in the present invention. DETAILED DESCRIPTION

[0041] The present invention proposes a winter fishing classification and recognition method based on the MG-ShuffleNet network structure, aiming to effectively improve the recognition accuracy of the model.

[0042] The winter fishing classification and identification method based on the MG-ShuffleNet network structure proposed by the present invention will be described in the following specific embodiments:

[0043] In the technical solution of this embodiment, Figure 1 As shown, a winter fishing classification and recognition method based on the MG-ShuffleNet network structure includes the following steps:

[0044] Collect image data sets, process the image data sets, and divide them into training sets and validation sets;

[0045] It can be understood that the data set of this application includes original images and enhanced images. Figure 4 As shown, the original image is the original collected image, the salt and pepper noise is the image after the original image is enhanced by adding salt and pepper, the random rectangular occlusion is the image after the original image is enhanced by adding salt and pepper, and the brightness adjustment is the image after the original image is adjusted by brightness. The present invention uses a self-made Chagan Lake winter fishing actual collection image data set for experimental verification and analysis. The self-made data set is based on the Chagan Lake winter fishing photographed by a digital camera. The collected images include 3329 field-collected images, which are bighead carp (599), carp (600), grass carp (605), bream (322), silver carp (601) and whitefish (602).

[0046] In the winter fishing classification and recognition task, in order to avoid the problem of overfitting of the training model due to insufficient data, the winter fishing data taken on site was augmented using data augmentation methods to expand the initial 3329 images to 13919. In order to ensure the training effect of the model, the selected winter fishing pictures were divided into training set and validation set in a ratio of 8:2 to ensure the generalization performance of the model.

[0047] Build a network model including MDFA module and GCSA attention mechanism module:

[0048] Pass the training set through a 3x3 convolutional layer to capture the local features of the image;

[0049] Use the average pooling layer to downsample the convolutional feature map;

[0050] In module stage2, the downsampling unit and the basic unit are stacked once each;

[0051] In module stage3, the downsampling unit is stacked once and the basic unit is stacked three times to enhance the learning ability of features;

[0052] In module stage4, the downsampling unit and the basic unit are stacked once each;

[0053] A 1x1 convolutional layer is used for cross-channel feature fusion and dimensionality reduction;

[0054] The global pooling layer reduces the spatial dimension of the feature map to a single value to capture the global information of the entire feature map;

[0055] The fully connected layer receives the feature vector from the global pooling layer and outputs a predicted value;

[0056] Among them, the 3x3 convolution layer and the 1x1 convolution layer respectively introduced the Mish activation function.

[0057] It can be understood that the present application provides a MG-ShuffleNet network structure that combines the MDFA module and the GCSA attention mechanism module on the basis of a simplified ShuffleNet network structure. The specific network structure is as follows: Figure 1 As shown in the figure. First, the network structure is simplified by halving the number of basic unit stacking times in model Stage2, model Stage3, and model Stage4 (rounded down); at the same time, the number of output channels of the first convolutional layer is increased from 24 to 32, and the number of output channels of the terminal convolutional layer is reduced from 2048 to 1024. This simplified approach can reduce computational complexity and the number of parameters, improve operating efficiency, and reduce storage overhead, while maintaining a high accuracy rate, which is suitable for edge devices and real-time scenario deployment.

[0058] Usually, convolution is used for two purposes before or after DWConv. One is to fuse the information between channels and make up for the lack of DWConv's inter-channel information fusion function. The other is to reduce and increase the dimension, such as the reverse residual module in MobileNetV2. However, the right branch of ShuffleNetV2 uses two convolutions. In the present invention, there is no need for dimensionality increase and dimensionality reduction. It is only necessary to meet the fusion of inter-channel information of DWConv, so only one convolution needs to be retained. Therefore, the convolution after DWConv in the right branch of ShuffleNetV2 is deleted. Secondly, in order to further solve the problem of excessive memory occupation and large amount of calculation (FLOPs) and parameter (Params) of the remaining convolution in the model, and in order to further effectively capture the information of cross-channel interaction, the MDFA module replaces the retained convolution to achieve the performance gain of feature model extraction while maintaining lightweight. In order to enhance the expressiveness of the output features of MDFA, it is further corrected by the GCSA attention mechanism to obtain the final feature map. In addition, in order to further avoid the problem of overfitting of the model, L2 regularization is added during network training. Finally, the Mish activation function is used to replace the ReLU activation function in convolutional layer 1 and convolutional layer 5, and the maximum pooling after convolutional layer 1 is replaced by average pooling to further improve the recognition accuracy of the model.

[0059] Specifically, the recognition accuracy of the traditional ShufflenetV2 network structure is 95.37%, the precision is 95.03%, the recall rate is 95.50%, the f1 score is 95.14%, the parameter amount is 2278604, and the model size is 8.69MB; the recognition accuracy of the MG-ShuffleNet network model improved by the present application is 96.35%, the precision is 96.65%, the recall rate is 96.26%, the F1 score is 96.38%, the parameter amount is 1063730, and the model size is 4.06MB; compared with the traditional network structure, the technical solution of the present application effectively improves the recognition accuracy of the model.

[0060] Furthermore, the downsampling unit and the basic unit both include an MDFA module, such as Figure 2 As shown, the MDFA module includes:

[0061] The first branch uses a 1x1 convolution kernel to directly extract features;

[0062] The second branch uses a 3x3 convolution kernel with a dilation rate of 6 to expand the receptive field;

[0063] The third branch uses a 3x3 convolution kernel with a dilation rate of 12 to further expand the receptive field to capture a wider range of contextual information;

[0064] The fourth branch uses a 3x3 convolution kernel with a dilation rate of 18 to provide the widest receptive field;

[0065] The fifth branch uses global average pooling to extract global context features and enhance the model's ability to understand the overall layout;

[0066] The first branch, the second branch, the third branch, the fourth branch and the fifth branch are spliced ​​in the channel dimension to form a comprehensive feature map;

[0067] The comprehensive feature map is calibrated in parallel through the channel attention mechanism and the spatial attention mechanism;

[0068] The output feature maps of the channel attention and spatial attention mechanisms are fused through element-wise addition operations to obtain enhanced feature maps.

[0069] The enhanced feature map is reduced in dimension and integrated through a 1x1 convolutional layer to generate the output feature map.

[0070] Understandably, the MDFA module extracts features at different spatial scales by introducing multi-scale dilated convolutions. The MDFA module is divided into two parts, the first part is the multi-scale dilated convolution part, and the second part is the fusion of channel and spatial attention. Traditional convolutional networks increase the receptive field by stacking multiple layers of convolutional layers, but this method will lead to a large increase in computational complexity and may cause information dilution. MDFA uses dilated convolutions, especially different dilation rates, to significantly expand the receptive field without losing resolution, which is crucial for capturing a wider range of contextual information in the image. The MDFA architecture achieves feature extraction at different scales through five parallel convolution branches, each of which is configured with a different dilation rate to expand the receptive field and capture spatial information in different ranges.

[0071] Further, the downsampling unit and the basic unit both include a GCSA module, such as Figure 3 As shown, the GCSA module includes:

[0072] The input feature map is first dimensionally permuted from C×H×W to W×H×C;

[0073] The dependency between channels is captured through two layers of multi-layer perceptrons (MLPs). The first layer of MLP reduces the number of channels to 1 / 4 of the original number, then introduces nonlinearity through the ReLU activation function, and then restores the number of channels to the original dimension through the second layer of MLP. The inverse permutation is performed to restore it to C×H×W, and the channel attention map is generated through the Sigmoid activation function. The input feature map and the channel attention map are multiplied element by element to obtain the enhanced feature map.

[0074] The enhanced feature maps are divided into 4 groups, each containing C / 4 channels; the grouped feature maps are transposed to disrupt the order of channels in each group; the disrupted feature maps are restored to their original shape (C×H×W) (this method can better mix feature information and enhance feature expression capabilities).

[0075] In the spatial attention submodule, the input feature map passes through a 7x7 convolution layer, and the number of channels is reduced to 1 / 4 of the original; it undergoes nonlinear transformation through batch normalization and ReLU activation function; the number of channels is restored to the original dimension C through the second 7x7 convolution layer, and then passes through the batch normalization layer; the spatial attention map is generated through the Sigmoid activation function; the shuffled feature map and the spatial attention map are multiplied element by element to obtain the output feature map.

[0076] Understandably, the GCSA attention mechanism combines channel attention, channel shuffling, and spatial attention mechanisms. In the channel attention submodule, the input feature map is first dimensionally permuted from C×H×W to W×H×C. The dependencies between channels are captured by two layers of multi-layer perceptrons (MLPs). The first layer of MLP reduces the number of channels to 1 / 4 of the original, then introduces nonlinearity through the ReLU activation function, and then restores the number of channels to the original dimension through the second layer of MLP. Finally, the inverse permutation is restored to C×H×W, and the channel attention map is generated through the Sigmoid activation function. The input feature map and the channel attention map are multiplied element by element to obtain an enhanced feature map.

[0077] To further mix and share information, a channel shuffle operation is applied. The enhanced feature maps are divided into (4) groups, each containing (C / 4) channels. The grouped feature maps are transposed to shuffle the order of channels within each group. Subsequently, the shuffled feature maps are restored to their original shape (C×H×W). This approach can better mix feature information and enhance feature expression capabilities.

[0078] In the spatial attention submodule, the input feature map passes through a 7x7 convolutional layer, and the number of channels is reduced to (1 / 4) times the original. Then, it is transformed nonlinearly through batch normalization and ReLU activation function. Next, the number of channels is restored to the original dimension C through a second 7x7 convolutional layer, and then passed through a batch normalization layer. Finally, the spatial attention map is generated through the Sigmoid activation function. The shuffled feature map and the spatial attention map are multiplied element by element to obtain the final output feature map.

[0079] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A winter fishing classification and recognition method based on the MG-ShuffleNet network structure, characterized in that: The following steps are involved: Collect image data sets, process the image data sets, and divide them into training sets and validation sets; Construct a network model including the MDFA module and the GCSA attention mechanism module. The processing flow of the network model includes: Pass the training set through a 3x3 convolutional layer to capture the local features of the image; Use the average pooling layer to downsample the convolutional feature map; In module stage2, the downsampling unit and the basic unit are stacked once each; In module stage3, the downsampling unit is stacked once and the basic unit is stacked three times to enhance the learning ability of features; In module stage4, the downsampling unit and the basic unit are stacked once each; A 1x1 convolutional layer is used for cross-channel feature fusion and dimensionality reduction; The global pooling layer reduces the spatial dimension of the feature map to a single value to capture the global information of the entire feature map; The fully connected layer receives the feature vector from the global pooling layer and outputs a predicted value; Among them, the 3x3 convolution layer and the 1x1 convolution layer respectively introduce the Mish activation function; The down-sampling unit and the basic unit both include an MDFA module, The MDFA module includes: The first branch uses a 1x1 convolution kernel to directly extract features; The second branch uses a 3x3 convolution kernel with a dilation rate of 6 to expand the receptive field; The third branch uses a 3x3 convolution kernel with a dilation rate of 12 to further expand the receptive field to capture a wider range of contextual information; The fourth branch uses a 3x3 convolution kernel with a dilation rate of 18 to provide the widest receptive field; The fifth branch uses global average pooling to extract global context features and enhance the model's ability to understand the overall layout; The first branch, the second branch, the third branch, the fourth branch and the fifth branch are spliced ​​in the channel dimension to form a comprehensive feature map; The comprehensive feature map is calibrated in parallel through the channel attention mechanism and the spatial attention mechanism; The output feature maps of the channel attention and spatial attention mechanisms are fused through element-wise addition operations to obtain enhanced feature maps. The enhanced feature map is reduced in dimension and integrated through a 1x1 convolution layer to generate the output feature map; The down-sampling unit and the basic unit both include a GCSA module, and the GCSA module includes: The input feature map is first dimensionally permuted from C×H×W to W×H×C; The dependency between channels is captured through two layers of multi-layer perceptrons (MLPs). The first layer of MLP reduces the number of channels to 1 / 4 of the original number, then introduces nonlinearity through the ReLU activation function, and then restores the number of channels to the original dimension through the second layer of MLP. The inverse permutation is performed to restore it to C×H×W, and the channel attention map is generated through the Sigmoid activation function. The input feature map and the channel attention map are multiplied element by element to obtain the enhanced feature map. The enhanced feature maps are divided into 4 groups, each containing C / 4 channels; the grouped feature maps are transposed to disrupt the channel order in each group; the disrupted feature maps are restored to the original shape C×H×W; In the spatial attention submodule, the input feature map passes through a 7x7 convolution layer, and the number of channels is reduced to 1 / 4 of the original; it undergoes nonlinear transformation through batch normalization and ReLU activation function; the number of channels is restored to the original dimension C through the second 7x7 convolution layer, and then passes through the batch normalization layer; the spatial attention map is generated through the Sigmoid activation function; the shuffled feature map and the spatial attention map are multiplied element by element to obtain the output feature map.

Images