A deep learning-based eriocheir sinensis strain identification method

Abstract

The present application relates to the technical field of computer vision, and in particular to a Chinese mitten crab strain identification method based on deep learning, wherein a plurality of crab pictures of different strains are used to generate a training data set, a backbone network containing a CSPNet is trained, the extracted feature vectors are fused in different scales, a prediction operation is performed through a prediction layer, the prediction result is compared with the labeled data, a loss value is calculated according to a loss function, iterative calculation, back propagation and loss value optimization are performed, a trained model is obtained, and finally the trained model is used to predict the crab strain, so that crab strain detection automation is realized, and further, if the detection fails, special treatment can be performed in combination with manual operation, and the detection model can be optimized again.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a Chinese mitten crab strain identification method based on deep learning. BACKGROUND

[0002] Chinese mitten crab (Eriocheir sinensis), also known as river crab, is an important aquaculture species in China. Due to its unique flavor, rich in various nutrients such as mineral elements, fatty acids, amino acids and other nutrients required by human body, it is deeply loved by people, thus having important economic significance. In recent years, the cultivation scale of river crab has gradually expanded, among which the output of Jiangsu, Anhui and Hubei provinces in the Yangtze River Basin accounts for about 80% of the country, which is a high-quality aquatic species in China.

[0003] Since the breakthrough of artificial breeding technology of river crab in the early 1980s, the artificial breeding of river crab in China has developed rapidly. While developing rapidly, the river crab seed industry has problems such as chaotic seed source, degeneration of germplasm, unstable quality of seed, small individual of adult crab, increasing proportion of precociousness, and low coverage of high-quality breeds. In addition, the shell characteristics are highly similar, and it is difficult for non-professionals to distinguish the strain of the crab. This has brought difficulties to the breeding of high-quality breeds of river crab, and seriously affected the development of river crab industry. However, the traditional DNA identification method is too complex, the detection environment is harsh, and the time cost is too high. SUMMARY

[0004] The purpose of the present application is to provide a Chinese mitten crab strain identification method based on deep learning, to solve the problem that river crab breeders cannot quickly identify the strain of river crab, and to introduce deep learning and computer vision technology for identification, which is convenient for breeders to select the required strain.

[0005] To achieve the above purpose, the present application provides a Chinese mitten crab strain identification method based on deep learning, comprising the following steps:

[0006] Step 1: taking pictures of river crabs of each strain;

[0007] Step 2: manually labeling the taken pictures using labelme to generate a training data set;

[0008] Step 3: performing Mosaic data enhancement on the training set to obtain an enhanced data set, and calculating adaptive anchor frames;

[0009] Step 4: sending the enhanced data set into an improved backbone network for feature extraction;

[0010] Step 5: sending the feature vectors extracted in step 4 into an improved neck network for feature fusion to obtain feature vectors of large, medium and small scales;

[0011] Step 6: The three feature vectors obtained in step 5 are respectively sent into the prediction layer for prediction to obtain a prediction result;

[0012] Step 7: The prediction result is compared with the labeled data, and a loss value is calculated according to a loss function;

[0013] Step 8: After iterative optimization, a trained model is obtained;

[0014] Step 9: The model obtained in step 8 is used for prediction of river crab strains;

[0015] Step 10: If the river crab strain is successfully detected in step 9 and the confidence is greater than 0.8, it is considered that the detection is successful, otherwise the picture is marked as detection failure and put into a specific folder for manual processing.

[0016] Optionally, in the process of manually labeling the photographed pictures using labelme, the position of the crab shell in the picture is framed and marked as the corresponding category.

[0017] Optionally, the improved backbone network comprises a transform module, a CSPNet network and a Mish activation function, the transform module focuses on the global and models the pixel relationship between the global, the CSPNet network splits the input vector into two parts and then fuses them, and the Mish activation function is used to alleviate the situation of model divergence.

[0018] Optionally, the improved neck network adopts the structure of FPN+PAN, FPN is top-down, which transmits strong semantic features of high layers down to enhance the entire pyramid, and PAN is a bottom-up pyramid which supplements FPN by transmitting strong positioning features of low layers up.

[0019] Optionally, the prediction layer performs ordinary convolution operation to finally obtain a one-dimensional vector, which includes the prediction box center point, the prediction box height and the confidence information.

[0020] Optionally, the loss function comprises three parts of classification loss, positioning loss and confidence loss, and the expression of the loss function is

[0021]

[0022] wherein N is the number of detection layers, B is the number of target center points falling in the prior box, D i is the number of grids corresponding to the current scale, is the bounding box regression loss, is the target loss, For the classification loss, lambda1, lambda2 and lambda3 are three weight parameters respectively.

[0023] Optionally, the positioning loss is used to measure the error condition of the predicted frame and the real frame, and CIoU Loss is used in the positioning loss process, which considers the width-height ratio information of the predicted frame and the real frame, the IoU value of the real frame and the predicted frame, and the distance between the two center points of the real frame and the predicted frame.

[0024] Optionally, the binary cross-entropy loss function is used for the classification loss and the confidence loss, and is calculated according to the category score of the predicted frame and the category score of the real frame.

[0025] Optionally, in the process of obtaining the trained model after iterative optimization, the number of iterations is 300 rounds, the batch-size is set to 6, cache images are used, and the optimizer uses SGD.

[0026] The application provides a Chinese mitten crab strain identification method based on deep learning, which generates a training data set by using a plurality of strain river crab pictures, trains a backbone network containing a CSPNet, performs feature fusion on the extracted feature vectors in different scales, performs a prediction operation through a prediction layer, compares the prediction result with labeled data, calculates a loss value according to a loss function, repeatedly iterates, reversely propagates, optimizes the loss value, obtains a trained model, finally uses the trained model to predict the river crab strain, realizes the automation of river crab strain detection, and further, if the detection fails, special processing can be performed in combination with manual operation, or the detection model can be optimized again. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0028] Figure 1 is a whole process schematic diagram of a Chinese mitten crab strain identification method based on deep learning of the present application.

[0029] Figure 2 is a training process schematic diagram of a Chinese mitten crab strain identification method based on deep learning of the present application.

[0030] Figure 3 is a detection process schematic diagram of a Chinese mitten crab strain identification method based on deep learning of the present application.

[0031] Figure 4 is a schematic diagram of the overall network structure of a deep learning-based eriocheir sinensis strain identification method of the present application. DETAILED DESCRIPTION

[0032] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numbers represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0033] Referring to Figure 1 , the present application provides a deep learning-based eriocheir sinensis strain identification method, comprising the following steps:

[0034] S1: taking pictures of river crab of each strain;

[0035] S2: manually labeling the taken pictures using labelme to generate a training data set;

[0036] S3: performing Mosaic data enhancement on the training set to obtain an enhanced data set, and calculating adaptive anchor frames;

[0037] S4: feeding the enhanced data set into the improved backbone network for feature extraction;

[0038] S5: feeding the feature vectors extracted in step S4 into the improved neck network for feature fusion to obtain feature vectors of large, medium and small scales;

[0039] S6: feeding the three feature vectors obtained in step S5 into the prediction layer for prediction to obtain a prediction result;

[0040] S7: comparing the prediction result with the labeled data, and calculating the loss value according to the loss function;

[0041] S8: obtaining a trained model after iterative optimization;

[0042] S9: using the model obtained in step S8 to predict the eriocheir sinensis strain;

[0043] S10: if the eriocheir sinensis strain is successfully detected in step S9 and the confidence is greater than 0.8, it is considered that the detection is successful, otherwise the picture is marked as detection failure and put into a specific folder for manual processing.

[0044] The deep learning-based eriocheir sinensis strain identification method of the present application specifically includes two processes of training and detection, wherein steps S1 to S8 are the training process, as specifically shown in Figure 2 , and step S9 is the detection process, please refer toFigure 3 .

[0045] Further, the application is further described below through specific examples and implementation steps:

[0046] Step S1, take pictures of each strain of river crab.

[0047] Among them, 3276 pictures of "Yangtze River No. 1", 3800 pictures of "Yangtze River No. 2", 3826 pictures of "Photosynthesis No. 1", and 3711 pictures of "Noah No. 1", a total of 14613 river crab pictures.

[0048] Step S2, manually label the pictures taken using labelme to generate a training data set.

[0049] Using labelme to frame the position of the crab shell in the picture and marking the corresponding category.

[0050] Step S3, data augmentation is performed on the training set to obtain a more abundant data set, and adaptive anchor boxes are calculated.

[0051] Data augmentation method 1 is to randomly select 4 pictures, and the four pictures are spliced by random scaling, random cropping, and random arrangement to obtain new pictures. Method 2 is to flip, rotate, adjust contrast and brightness, etc. Operation to obtain new pictures, greatly enriching the detection data set, making the robustness of the deep learning network better.

[0052] Step S4, the enhanced data set is sent into the improved backbone network for feature extraction;

[0053] As shown in Figure 4 , the backbone network of the present method is improved based on the CSPDarknet53 network.

[0054] The CSPDarknet53 network mainly consists of CBS, CSPNet and other modules. CBS is a short form of convolution (conv), normalization (Batch Normalization) and SiLU activation function. CSPNet integrates the change of gradient from head to tail into the feature map, which reduces the amount of calculation while ensuring the accuracy. The core idea of CSPNet is to divide the input vector into two parts, one part is convolution operation, and the other part is fused with the result of the previous part convolution operation to finally obtain a feature vector. Through block operation to reduce the amount of calculation and maintain performance. SiLU activation function is used to alleviate the situation of model not converging.

[0055] However, the CSPDarknet53 network mainly uses traditional convolution operations, but convolution operations usually only model the pixel relationship between local pixels, and the Swin Transformer module focuses on the global, which can model the pixel relationship between the global. In order to give the original backbone network this global modeling ability and improve the feature extraction ability of the model backbone network, the method modifies the last two CSP modules in the CSPDarknet53 network into Swin Transformer modules. The core idea of the module is to use a multi-head self-attention mechanism based on a moving window to calculate the self-attention weight in a small window. The improved backbone network greatly improves the feature extraction ability of the river crab shell.

[0056] Step S5, the feature vector extracted in step S4 is sent to the improved neck network for feature fusion, and feature vectors of large, medium and small scales are obtained;

[0057] The original neck network mainly uses the structure of FPN (Feature Pyramid Networks) + PAN (Path Aggregation Network).

[0058] Firstly, through FPN from top to bottom, the high-dimensional semantic features are fused with the low-dimensional position features through upsampling, so as to improve the detection ability of the model on the target; secondly, through PAN from bottom to top, the low-dimensional position features are fused with the high-dimensional semantic features through downsampling, so as to improve the positioning ability of the model. However, this structure lacks the division information of the importance of the image region.

[0059] In order to further improve the detection ability of the model, improve the weight of the position of the crab shell in the whole image, make the model pay more attention to the crab shell, and suppress the background features irrelevant to the crab shell, the present application adds a ShuffleAttention module before each of the large, medium and small detection heads of YOLOv5s. At present, the common attention mechanism can be divided into spatial attention mechanism and channel attention mechanism, and the use of both kinds of attention mechanisms can effectively improve the detection ability of the model, but undoubtedly increases the calculation amount. The SA attention mechanism cleverly reduces the calculation amount of the model while ensuring the effect of the model through the grouping calculation method. The overall process of SA is as follows: firstly, the channels are grouped, secondly, the spatial and channel attention weight of each group is calculated in parallel, and finally, the Channel Shuffle operation is used to fuse and intersect the feature information of each group, efficiently combining the spatial and channel attention mechanisms and improving the detection ability of the model.

[0060] Step S6, the three feature vectors obtained in step S5 are respectively sent into the prediction layer for prediction to obtain a prediction result.

[0061] The prediction layer is a common convolution operation, and finally a one-dimensional vector is obtained, which includes the center point of the prediction frame, the frame height of the prediction frame and the confidence information.

[0062] Step S7, the prediction result is compared with the labeled data, and the loss value is calculated according to the loss function.

[0063] The loss function is composed of three parts, namely classification loss (cls_loss), positioning loss (box_loss) and confidence loss (obj_loss). The positioning loss is used to measure the error between the prediction frame and the real frame. The CIoU Loss used in the method considers the width-height ratio information of the prediction frame and the real frame, the IoU value of the real frame and the prediction frame, and the distance between the center points of the real frame and the prediction frame. The confidence loss is obtained by calculating the binary cross entropy loss (Binary Cross Entropy Loss) according to the detection confidence score in the prediction frame and the IoU value of the prediction frame and the target frame. The classification loss is the same as the confidence loss, and the binary cross entropy loss function is also used, which is calculated according to the class score of the prediction frame and the class score of the real frame.

[0064] The total loss function is shown in formula 1.

[0065]

[0066] Where N is the number of detection layers, B is the number of target center points falling in the prior frame, D i is the number of grids corresponding to the current scale, is the bounding box regression loss, is the target loss, is the classification loss. λ1, λ2, λ3 are three weight parameters.

[0067] The specific formula of the bounding box regression loss is shown in formulas 2, 3 and 4.

[0068]

[0069]

[0070]

[0071] Wherein, b, bgt are the prediction frame and the real frame respectively, wgt, hgt, w, h are the width and height of the real frame and the width and height of the prediction frame respectively, p is the distance between the center points of the prediction frame and the real frame, and a is a weight coefficient.

[0072] The target loss and the classification loss both use a binary cross-entropy loss function, and the calculation formulas thereof are shown in (5) and (6).

[0073]

[0074]

[0075] S8, repeatedly iterate, back propagation, and optimize the loss value to finally obtain a trained model.

[0076] The specific iteration rounds are 300 rounds, the batch-size is set to 6, cache images are used, and the optimizer uses SGD.

[0077] S9, using the model obtained in step S8 to predict the river crab strain.

[0078] S10, if the river crab strain is successfully detected in step S9 and the confidence is greater than 0.8, it is considered that the detection is successful, otherwise the picture is marked as detection failure and put into a specific folder for manual processing.

[0079] Experiments show that the accuracy of the model in predicting the river crab strain is as high as 93.6%, which can well identify the strain of river crab. Compared with the traditional biochemical method, the present method uses deep learning technology, effectively enables the model network to learn the characteristics of the river crab shell, avoids the redundant and complex steps of the traditional method, greatly reduces the identification cost and time of the river crab breeders, and has certain application value.

[0080] The above only discloses one preferred embodiment of the present application, and of course cannot limit the scope of the rights of the present application, and those skilled in the art can understand that all or part of the above-mentioned embodiments can be implemented, and equivalent changes made according to the claims of the present application still belong to the scope covered by the present application.

Claims

1. A method for identifying a Chinese mitten crab (Eriocheir sinensis) strain based on deep learning, characterized in that, It comprises the following steps: Step 1: Take pictures of each strain of river crab; Step 2: Manually label the pictures taken using labelme to generate a training dataset; Step 3: Perform Mosaic data augmentation on the training set to obtain an augmented dataset and calculate adaptive anchor boxes; Step 4: Send the augmented dataset into the improved backbone network for feature extraction; The improved backbone network includes a transform module, a CSPNet network, and a Mish activation function. The transform module focuses on the global and models the pixel relationship between the global. The CSPNet network splits the input vector into two parts and then fuses them, reducing the computational complexity through block operations. The Mish activation function is used to alleviate the model divergence; Specifically, the CSPDarknet53 network is improved by modifying the last two CSP modules in the CSPDarknet53 network to Swin Transformer modules; Step 5: Send the feature vectors extracted in step 4 into the improved neck network for feature fusion to obtain large, medium, and small scale feature vectors; The improved neck network adopts the structure of FPN+PAN. FPN is a top-down structure that transmits high-level semantic features down to enhance the entire pyramid. PAN is a bottom-up pyramid that complements FPN by transmitting low-level positioning features up; Before the large, medium, and small detection heads, a ShuffleAttention module is added to each; Step 6: Send the three feature vectors obtained in step 5 into the prediction layer for prediction to obtain the prediction results; Step 7: Compare the prediction results with the labeled data and calculate the loss value according to the loss function; Step 8: Obtain the trained model after iterative optimization; Step 9: Use the model obtained in step 8 to predict the river crab strain; Step 10: If the river crab strain is successfully detected in step 9 and the confidence is greater than 0.8, it is considered as a successful detection. Otherwise, mark this picture as a failed detection and put it in a specific folder for manual processing.

2. The deep learning-based Chinese mitten crab strain identification method of claim 1, wherein During the manual labeling of the taken pictures using labelme, the labelme is used to select the position of the crab shell in the picture and mark it as the corresponding category.

3. The deep learning-based Chinese mitten crab strain identification method of claim 2, wherein The prediction layer performs ordinary convolution operations to obtain a one-dimensional vector, which includes the prediction box center point, the prediction box height, and the confidence information.

4. The deep learning-based Chinese mitten crab strain identification method of claim 3, wherein The loss function includes three parts: classification loss, positioning loss, and confidence loss. The expression of the loss function is where N is the number of detection layers, B is the number of target center points falling in the prior box, D i is the number of grids corresponding to the current scale, is the bounding box regression loss, is the target loss, is the classification loss, and λ1, λ2, and λ3 are three weight parameters.

5. The deep learning-based Chinese mitten crab strain identification method of claim 4, wherein The positioning loss is used to measure the error of the predicted frame and the real frame. In the positioning loss process, CIoU loss is adopted. CIoU loss considers the width-height ratio information of the predicted frame and the real frame, the IoU value of the real frame and the predicted frame, and the distance between the centers of the real frame and the predicted frame. 6.The deep learning-based Eriocheir sinensis strain identification method according to claim 5, wherein, The classification loss and the confidence loss adopt a binary cross-entropy loss function, which is calculated according to the category score of the predicted frame and the category score of the real frame. 7.The deep learning-based Eriocheir sinensis strain identification method according to claim 6, wherein, In the process of obtaining the trained model after iterative optimization, the number of iterations is 300 rounds, the batch-size is set to 6, cache images are used, and the optimizer uses SGD.

Images