A method for crop disease recognition based on convolutional neural network
The GrabCut algorithm is used to process image background and channel pruning technology, combined with the New-CA attention module, the robustness and lightweight characteristics of the convolutional neural network are improved, and the problems of low crop disease recognition accuracy and large model size are solved. It realizes efficient identification on agricultural platforms.
Patent Information
- Application Number
- CN202210295446.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-03-24
AI Technical Summary
The existing convolutional neural network model has low accuracy and large model size and calculation volume when identifying crop diseases in real environments, resulting in high hardware costs and cannot be effectively applied to agricultural production.
The GrabCut algorithm is used to process the image background, extract the channel pruning method in the convolutional neural network, compress the model, and combine the New-CA attention module to improve the robustness and lightweight characteristics of the model.
It improves the accuracy of crop disease recognition, reduces the size and calculation amount of the model, makes it suitable for agricultural platforms with low memory and computing capabilities, solves the problem of high error rate of manual identification, and improves crop yield and quality.
Smart Images

Figure CN114445715B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of plant classification, and in particular relates to a crop disease identification method based on convolutional neural network. Background Art
[0002] The reduction of plant yield is affected by many factors, among which the reduction of plant yield caused by pests and diseases is between 20-30%. In order to prevent the spread of diseases and protect yields, the traditional method is to judge the type of plant disease through manual observation and then spray the corresponding pesticides. However, due to the diversity of plant diseases, manual observation is likely to cause misdiagnosis, resulting in a decrease in plant yield. Therefore, it is very necessary to study a model for plant disease classification that can meet the needs of agricultural production.
[0003] Because the symptoms of plant diseases usually appear on leaves, people have begun to study how to use computer vision to assist in the diagnosis of plant disease types. Traditional image classification techniques rely on manually designed features, such as color-based features, texture-based features, and shape-based features. Since manually designed features rely on engineering experience and cannot adapt to the huge changes in imaging conditions, their accuracy cannot meet people's actual needs. The emergence of convolutional neural networks solves this problem. Convolutional neural networks can extract image features by directly convolving with image pixels, which is very suitable for extracting complex features. Some scholars have trained a deep convolutional neural network to identify 38 types of states of 14 crops, and the trained model has achieved an accuracy of 99.35% on the test set. Other scholars have improved the convolutional neural network by multi-scale fusion of the feature data of the convolutional layer, which has increased the accuracy of the model on the test set to 91.7%. Some scholars have used an improved vgg16 network to classify apple leaf diseases. By using a global average pooling layer instead of a fully connected layer to reduce parameters and adding a batch normalization layer to increase the convergence speed, an accuracy of 99.01% has been achieved on the test set. However, the high accuracy of their model is obtained through a test set similar to the training set, and the accuracy will be greatly reduced when their trained model is applied to the real environment. This application found that the background of plant leaves in the real environment is diverse, while the background of the data set used for training the model in the experiment is relatively single, which leads to the fact that the trained model in the experiment is not robust. Some scholars use convolutional neural networks for plant disease classification and achieve an accuracy of 99.35% on the test set. However, when using a test set with different shooting conditions from the training images, the model has an accuracy of only 34.4. Another problem is that these models are large in size and computationally expensive, and it costs huge hardware costs to deploy them in practice. Agricultural research aims to increase food production and quality at a lower cost and higher profit to meet consumer demand. Therefore, it is very necessary to study a high-accuracy and lightweight convolutional neural network model that can be applied to actual agricultural production for crop disease identification. Summary of the invention
[0004] In view of the above technical problems, the present invention provides a crop disease identification method based on convolutional neural network.
[0005] The technical solution adopted by the present invention to solve the technical problem is:
[0006] A method for identifying crop diseases based on convolutional neural networks, the method comprising the following steps:
[0007] Step S100: obtaining a training data set, and preprocessing the training data set to obtain a preprocessed data set;
[0008] Step S200: inputting the preprocessed data set into the initial convolutional neural network model for training to obtain a trained convolutional neural network model;
[0009] Step S300: compressing the trained convolutional neural network model using a channel pruning method to obtain a compressed convolutional neural network model;
[0010] Step S400: Acquire crop images, pre-process the crop images and input them into a compressed convolutional neural network model to obtain crop disease recognition results.
[0011] Preferably, step S100 includes: using a GrabCut algorithm to segment the foreground and background of an image in the data set, and processing the background of the image to black.
[0012] Preferably, the convolutional neural network includes a first convolution module, a plurality of Bottleneck modules, a second convolution module, an average pooling module, a third convolution module and a fully connected layer connected in sequence.
[0013] The first convolution module is used to receive the preprocessed image and perform a 3*3 convolution operation to extract features in the image;
[0014] Multiple Bottleneck modules are used to extract deep features of images after convolution operations;
[0015] The second convolution module is used to use a 1*1 convolution operation to fuse the deep features for the first time;
[0016] The average pooling module is used to reduce the size of the features after the first fusion using a 7*7 pooling operation;
[0017] The third convolution module is used to use a 1*1 convolution operation to fuse the pooled features for the second time;
[0018] The fully connected layer is used to obtain the classification result based on the features after the second fusion.
[0019] Preferably, the Bottleneck module includes a first feature extraction module, a New-CA attention module, a second feature extraction module and a shortcut connection module connected in sequence.
[0020] The first feature extraction module is used to extract the features of the image after the convolution operation;
[0021] The New-CA attention module is used to learn the features extracted by the first feature extraction module;
[0022] The second feature extraction module is used to extract the features learned by the New-CA attention module again;
[0023] The shortcut connection module is used to perform a shortcut connection between the feature map output by the second feature extraction module and the feature map input by the first feature extraction module to obtain deep-level features.
[0024] Preferably, a fourth convolution module is also included, which is arranged between the New-CA attention module and the second feature extraction module. When the stride is greater than 1, the fourth convolution module is used to change the size of the feature map after learning the New-CA attention module; the second feature extraction module is used to extract features from the feature map after the size is changed.
[0025] Preferably, the New-CA attention module includes an average pooling module in the C direction, an average pooling module in the H direction, an average pooling module in the W direction, a fifth convolution module, a splicing convolution module, a first normalization module, a second normalization module, a sixth convolution module, a seventh convolution module, an eighth convolution module, a first sigmoid function module, a second sigmoid function module, a third sigmoid function module and a combination module, the average pooling module in the C direction, the average pooling module in the H direction and the average pooling module in the W direction are connected to the first feature extraction module, and the average pooling module in the C direction The fifth convolution module, the first normalization module, the sixth convolution module and the first sigmoid function module are connected in sequence, the average pooling module in the H direction and the average pooling module in the W direction are connected to the splicing convolution module, the splicing convolution module is connected to the second normalization module, the second normalization module is connected to the seventh convolution module and the eighth convolution module, the seventh convolution module is connected to the second sigmoid function module, the eighth convolution module is connected to the third sigmoid function module, and the first sigmoid function module, the second sigmoid function module and the third sigmoid function module are all connected to the combination module.
[0026] Preferably, the first feature extraction module has the same structure as the second feature extraction module, which is one of the Vgg16, ResNet50, EfficientNet, ShuffleNetV2, MobileNetV3 and GhostNet models.
[0027] Preferably, step S300 includes:
[0028] Get the scale factors of all channels in the trained convolutional neural network model;
[0029] All channels whose scale factors are less than a preset threshold and all input and output connections to the channel are deleted to obtain a compressed convolutional neural network model.
[0030] The above-mentioned crop disease recognition method based on convolutional neural network effectively improves the classification and recognition accuracy of the convolutional neural network model for crop diseases, and when the classification and recognition accuracy of the model does not change much, the lightweight convolutional neural network model is more suitable for platforms with low memory and computing power. The model proposed in this application can be well applied to agriculture to identify the category of crop diseases, solves the problem of high error rate of manual recognition, and thus improves the yield and quality of crops. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 This is a flow chart of a method for identifying crop diseases based on a convolutional neural network in one embodiment of the present invention;
[0032] Figure 2 This is a schematic diagram of 63 types of plant disease states in one embodiment of the present invention;
[0033] Figure 3 A schematic diagram of the segmentation principle of the GrabCut algorithm in one embodiment of the present invention;
[0034] Figure 4 Schematic diagram of image segmentation effects with different K values in one embodiment of the present invention;
[0035] Figure 5 is a data set after image segmentation in one embodiment of the present invention;
[0036] Figure 6 A schematic diagram of the structure of a convolutional neural network model in one embodiment of the present invention;
[0037] Figure 7 This is a schematic diagram of the structure of a Bottleneck module in one embodiment of the present invention;
[0038] Figure 8 This is a schematic diagram of the structure of a Bottleneck module in another embodiment of the present invention;
[0039] Fig. 9 A schematic diagram of the structure of the New-CA attention module in one embodiment of the present invention;
[0040] Fig.10 Schematic diagram of a compressed convolutional neural network model in one embodiment of the present invention;
[0041] Fig.11 is an error curve diagram during CIFAR-10 training in one embodiment of the present invention;
[0042] Fig.12 1 is a diagram showing the compression ratio of the model size and the amount of computation in one embodiment of the present invention. DETAILED DESCRIPTION
[0043] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings.
[0044] In one embodiment, Figure 1 As shown, a method for identifying crop diseases based on a convolutional neural network comprises the following steps:
[0045] Step S100: Acquire a training data set, and preprocess the training data set to obtain a preprocessed data set.
[0046] Specifically, many disease types in the data set used in the present invention are divided into two levels: general and severe. Classifying the severity of the same disease is conducive to early detection of plant diseases, which is particularly important in protecting food production and can provide guidance on the amount of pesticides used.
[0047] The plant disease dataset used in the experiment of the present invention has 37,302 images, including 63 types of disease states of 11 plants. The 11 plants include apple, cherry, corn, grape, citrus, peach, pepper, potato, strawberry, tomato, and tea. Figure 2 Examples of 63 types of disease states can be seen in . Among them, 10 plants come from the public dataset AI Challenger2018, which contains 59 types of disease states. In addition, because tea has a positive effect on human health, tea beverages have a huge market all over the world. For example, black tea can reduce complications in patients with diabetes, tea polyphenols in tea can be used to reduce the production of sebum in the skin and treat common acne and have anti-inflammatory effects, and black tea and green tea can be used to prevent and treat high cholesterol. Therefore, this application also adds tea to the data set of this application. The tea data set comes from the shooting of this application from the tea base and contains 4 types of disease states, including tea health, tea white star disease, tea ring spot disease and tea anthracnose. This application uses Adobe Photoshop (an image processing software) to process the photographed tea pictures into pictures similar to the AI Challenger 2018 plant disease dataset, and then divides the entire dataset into a training set and a validation set, which account for 80% and 20% respectively.
[0048] The background of an image is an important part of the image and has a great impact on the accuracy of image classification. The data set used in the experiment was taken under specific conditions, so the types of backgrounds in the data set are limited. However, the backgrounds of pictures in real environments are diverse and no data set can include all backgrounds in real environments. Therefore, the accuracy of the model trained directly using the data set in the experimental environment will be much worse when deployed in practice. In order to enable the trained model to achieve the same high accuracy when deployed in practice, the present invention proposes to process the background of the data set picture into black through image segmentation technology. When the trained model is used for prediction, the background of the real picture with a variety of backgrounds is first processed into black. This ensures that the trained model can have the same high accuracy as during training when deployed in practice, thereby improving the robustness of the model.
[0049] Image segmentation algorithms mainly include those based on graph theory and those based on semantics. Semantic segmentation is based on deep learning and can be regarded as pixel-level classification. Therefore, semantic segmentation requires a high computational cost, and the main purpose of this paper is to study a classification model that can be applied to agricultural practice. Therefore, this application uses the traditional image segmentation technology based on graph theory, which can complete the segmentation task well in the experiment.
[0050] In one embodiment, step S100 includes: using the GrabCut algorithm to segment the foreground and background of the image in the data set, and processing the background of the image to black.
[0051] Specifically, the GrabCut algorithm relates the image segmentation problem to the minimum cut problem of a graph. Figure 3 Represented by an undirected graph G =<V,E> to represent the image to be segmented. V and E represent vertices and edges respectively. S and T are two additional points, and the other points in the figure correspond to the pixels of the image. The edges in the figure are weighted to represent the loss. The weight of the dotted line between the pixel and point S represents the loss of classifying the pixel as the background, and the weight of the dotted line between the pixel and point T represents the loss of classifying the pixel as the foreground. The Grabcut algorithm establishes a Gaussian mixture model (GMM) for the foreground and background respectively. Each GMM, one for the background and one for the foreground, is considered to be a full-variance Gaussian mixture with K components (K is usually 5). If K is too small, it means that the sample library model of the GMM is small, which will increase the probability of misjudgment when dividing the pixel attribution. Of course, the complexity of the calculation will increase with the increase of K. Figure 4It is the image segmentation effect of different K values. Therefore, the present invention adopts a Gaussian mixture model with 5 Gaussian components. The foreground Gaussian mixture model is used to calculate the probability that a pixel belongs to the foreground, and the background Gaussian mixture model is used to calculate the probability that a pixel belongs to the background. The weight of the solid line in the figure represents the similarity of two pixels, which is measured by the Euclidean distance. The more similar two pixels are, the greater the weight of the solid line between them. Based on this, the weights of each edge in the figure have been obtained, and then the minimum cut of the figure is used to segment the figure into the foreground and the background by minimizing the sum of the weights of the cut edges.
[0052] The present invention uses the GrabCut algorithm to segment the foreground and background of the image, and processes the background of the image to black. In order to achieve better image segmentation, a part of the pixels of the image are set as the background to help the GrabCut algorithm to better achieve segmentation. This application found that a feature of the AI Challenger 2018 plant disease dataset and the tea dataset shot by this application is that there is a large range of background around the leaves. Therefore, all pixels within five pixels of the edges of the image are set as the background. The processed dataset is as follows Figure 5 .
[0053] Step S200: input the preprocessed data set into the initial convolutional neural network model for training to obtain a trained convolutional neural network model.
[0054] In one embodiment, Figure 6 As shown, the convolutional neural network includes a first convolution module, multiple Bottleneck modules, a second convolution module, an average pooling module, a third convolution module and a fully connected layer, which are connected in sequence.
[0055] The first convolution module is used to receive the preprocessed image and perform a 3*3 convolution operation to extract features in the image;
[0056] Multiple Bottleneck modules are used to extract deep features of images after convolution operations;
[0057] The second convolution module is used to use a 1*1 convolution operation to fuse the deep features for the first time;
[0058] The average pooling module is used to reduce the size of the features after the first fusion using a 7*7 pooling operation;
[0059] The third convolution module is used to use a 1*1 convolution operation to fuse the pooled features for the second time;
[0060] The fully connected layer is used to obtain the classification result based on the features after the second fusion.
[0061] Specifically, in this embodiment, the number of Bottleneck modules is 9, which can be set according to actual conditions.
[0062] In one embodiment, Figure 7 As shown, the Bottleneck module includes a first feature extraction module, a New-CA attention module, a second feature extraction module and a shortcut connection module connected in sequence.
[0063] The first feature extraction module is used to extract the features of the image after the convolution operation;
[0064] The New-CA attention module is used to learn the features extracted by the first feature extraction module;
[0065] The second feature extraction module is used to extract the features learned by the New-CA attention module again;
[0066] The shortcut connection module is used to perform a shortcut connection between the feature map output by the second feature extraction module and the feature map input by the first feature extraction module to obtain deep-level features.
[0067] In one embodiment, Figure 8 As shown, it also includes a fourth convolution module, which is arranged between the New-CA attention module and the second feature extraction module. When the stride is greater than 1, the fourth convolution module is used to change the size of the feature map after learning the New-CA attention module; the second feature extraction module is used to extract features from the feature map after the size is changed.
[0068] Specifically, the features of the image are first extracted through the first feature extraction module; then the New-CA attention module proposed in this application is used to learn the obtained features so that the features can better express the image. If stride>1, it means that the size of the feature map has changed, so the fourth convolution module, namely DWConv convolution, is used to change the size of the feature map, and then the second feature extraction module is used to extract the features of the image; finally, the obtained feature map is shortcut-connected with the input feature map of Bottleneck to prevent the network from being too deep and causing the network's expression ability to decrease.
[0069] In one embodiment, preferably, the New-CA attention module includes an average pooling module in the C direction, an average pooling module in the H direction, an average pooling module in the W direction, a fifth convolution module, a splicing convolution module, a first normalization module, a second normalization module, a sixth convolution module, a seventh convolution module, an eighth convolution module, a first sigmoid function module, a second sigmoid function module, a third sigmoid function module and a combination module, the average pooling module in the C direction, the average pooling module in the H direction and the average pooling module in the W direction are connected to the first feature extraction module, and the average pooling module in the C direction is The pooling module, the fifth convolution module, the first normalization module, the sixth convolution module and the first sigmoid function module are connected in sequence, the average pooling module in the H direction and the average pooling module in the W direction are connected to the splicing convolution module, the splicing convolution module is connected to the second normalization module, the second normalization module is connected to the seventh convolution module and the eighth convolution module, the seventh convolution module is connected to the second sigmoid function module, the eighth convolution module is connected to the third sigmoid function module, and the first sigmoid function module, the second sigmoid function module and the third sigmoid function module are all connected to the combination module.
[0070] Specifically, the depth, width, and resolution of the network have a great impact on network performance. Increasing the depth of the network can improve classification accuracy. By increasing the width of the network, a 16-layer network can perform better than a deeper network. In addition, while keeping other conditions unchanged, increasing the resolution can improve classification accuracy. Moreover, increasing the resolution is more effective than increasing the depth and width of the network.
[0071] In addition to these methods that can be used to improve accuracy, recent studies have shown that attention mechanisms can be used to improve the representation ability of existing networks by focusing on important features and suppressing unnecessary features. The attention module can be regarded as a computational unit that can be easily inserted into existing network models to achieve feature recalibration. The attention mechanism has been shown to improve performance in many computer vision tasks, such as image classification, image segmentation, and object detection. The SE attention module improves the representation quality of the network by explicitly modeling the interdependence between feature channels. Since standard convolutions are difficult to encode the relationship between channels, the attention module obtains global information through two-dimensional global average pooling, and then uses the global information to selectively emphasize important features and suppress relatively unimportant features. However, SE only considers the relationship between encoding feature channels, while ignoring the location information that is crucial for visual tasks that need to capture the target structure. The CBAM attention module (Convolutional Block Attention Module) adds spatial attention on the basis of channel attention. Channel attention focuses on "what" is meaningful in the image, while spatial attention focuses on "where" is an information part. CBAM reduces the number of feature channels by performing average pooling and maximum pooling on the input features, and then uses a 7*7 convolution to generate a spatial attention map that tells the application where to focus. However, CBAM uses 7*7 convolution to achieve spatial attention, which can only capture local relationships, but cannot obtain long-distance dependencies that are very important for visual tasks. The emergence of the Coordinate attention module solves the problem of not being able to obtain long-distance dependencies. In order to obtain long-distance dependencies with accurate location information, Coordinate decomposes the two-dimensional global pooling operation used in the SE module into two parallel one-dimensional feature encoding operations. Specifically, given an input X∈R H×W×C ,Using two different one-dimensional average pooling operations, one along the H direction and the other along the W direction, two direction-aware feature maps are produced. These two direction-aware feature maps are then encoded into two attention maps, each of which captures the long-range dependencies of the input feature map along a spatial direction with precise location information. Finally, the two attention maps are applied to the input features by multiplication to emphasize the features of interest.
[0072] The Coordinate Attention Module only obtains two long-range dependencies along two spatial directions and ignores the important information that the input features are three-dimensional. Therefore, the long-range dependency information along the channel direction is also important for visual tasks. Based on this, this application proposes a new Coordinate Attention Mechanism (New-CA). The New-CA attention mechanism is as follows Fig. 9 Given an input X∈RH×W×C , this application uses three different one-dimensional global pooling operations to aggregate the input features into three independent direction-aware feature maps along the c direction, h direction, and w direction, respectively, which contain a large amount of position information. Then, a 1*1 convolution operation is used to encode the spatial information of the three different feature maps respectively, and r is used to control the size of the block and then a 1*1 convolution is used to restore the empty size. Therefore, this application obtains three attention maps, each of which captures the long-range dependency of the input feature map along a spatial direction with precise position information. Finally, the three attention maps obtained are applied complementary to the input features. This encoding process enables the New-CA attention module of this application to more accurately locate the exact position of the object of interest, thereby improving the performance of the entire model by inserting it into the appropriate position in the model.
[0073] In one embodiment, the first feature extraction module has the same structure as the second feature extraction module, which is one of the Vgg16, ResNet50, EfficientNet, ShuffleNetV2, MobileNetV3 and GhostNet models.
[0074] Specifically, this application uses some of the most advanced CNN models to demonstrate the effectiveness of the method of this application on public datasets, including Vgg16, ResNet50, EfficientNet, ShuffleNetV2, MobileNetV3 and GhostNet.
[0075] Step S300: Use the channel pruning method to compress the trained convolutional neural network model to obtain a compressed convolutional neural network model.
[0076] In one embodiment, step S300 includes:
[0077] Get the scale factors of all channels in the trained convolutional neural network model;
[0078] All channels whose scale factors are less than a preset threshold and all input and output connections to the channel are deleted to obtain a compressed convolutional neural network model.
[0079] Specifically, with the continuous development of GPUs, people continue to explore deeper and wider network models. Larger network models generally perform better in visual tasks, but require very high hardware support. For example, a 50-layer ResNet network has more than 26 million parameters and requires 70Giga floating point operations (FLOPs) when inferring an image with a resolution of 650×650. Deploying such a large network model to actual agricultural production requires a lot of hardware costs, which is contrary to the purpose of agricultural research. Therefore, it is very necessary for agricultural development to study how to compress large network models without changing the accuracy so that they can be deployed on low-cost and resource-limited platforms.
[0080] The main reasons why convolutional neural networks cannot be used well on resource-limited platforms are that the model size is too large, the memory usage is too high, and the computation is too large. Model pruning algorithms can better solve the above problems. Sparsity can be achieved at different levels, such as weight level, kernel level, channel level, or layer level.
[0081] Some scholars have proposed to prune unimportant connections with small weights in trained neural networks. However, this method requires dedicated sparse matrix operation libraries and hardware to achieve acceleration. Other scholars have proposed a new method for filter pruning. Considering the computational overhead and ease of implementation of pruning, this application chooses to use channel pruning for model compression.
[0082] Considering the computational overhead and ease of implementation of pruning, channel pruning is chosen for model compression. Below are some implementation details of this algorithm. In order to make the network converge quickly and have better generalization performance, Batchnormalization has been widely used in modern convolutional neural networks. Let zin and zout be the input and output of the BN layer, B represents the current batch data, and the formula of the BN layer is as follows:
[0083]
[0084]
[0085] in, and Representative batch The mean and variance of the data, γ and β are respectively trainable scaling and translation parameters. Most of the current convolutional neural networks have a BN layer inserted after the convolution layer, and the BN layer has scaling and translation parameters. Therefore, it is proposed to directly use the γ parameter in the BN layer as a scale factor to measure the importance of each channel. At the same time, in order to make more scale factors in the network close to 0, the L1-norm sparse regularization of the scale factor is added to the loss function, and the network weights and these scale factors are jointly trained. The specific loss function formula is as follows:
[0086]
[0087] Among them, (x, y) represents the input image and the corresponding true classification label respectively, W represents the trainable parameter, the left side of the formula is the usual training loss of the convolutional neural network, and the right side of the formula is the regularization loss of the newly added scale factor, and f is used to balance these two items. Through L1-norm sparse regularization, the value of γ is pushed to zero, allowing this application to identify unimportant channels because each γ corresponds to a specific channel. This helps in channel pruning in the next step. The additional regularization term rarely affects the network performance and will improve the generalization accuracy of the network to a certain extent.
[0088] After training, we can get a model with many scaling factors close to zero, such as Fig.10 . A threshold is obtained by setting a global pruning ratio for the trained model. All channels with a scale factor less than this threshold are pruned by deleting all input and output connections to the channel, thereby obtaining a more compact network. After pruning the trained network, the accuracy of the network will temporarily decrease. However, the present application can compensate by fine-tuning the pruned network. The fine-tuned narrow network is sometimes more accurate than the original unpruned network, and has fewer parameters and runtime memory, as well as fewer computing operations. This allows the model of the present application to be applied to agricultural platforms with limited resources such as mobile devices, wearable devices, or IoT devices.
[0089] Step S400: Acquire crop images, pre-process the crop images and input them into a compressed convolutional neural network model to obtain crop disease recognition results.
[0090] In order to prove the effectiveness of the new attention module proposed in this application, this application conducted a series of comparative experiments on the public dataset CIFAR-10. CIFAR-10 is a benchmark dataset for image classification, which contains 50,000 training set images and 10,000 validation set images from 10 classes, with 6,000 images in each class. This application trains the network model on the training set and uses the validation set to evaluate the effect of the model by calculating the top-1 and top-5 accuracy.
[0091] This application verifies the effectiveness of the attention module proposed in this application on the 6 most representative network models, including Vgg16, ResNet50, EfficientNet, ShuffleNetV2, MobileNetV3 and GhostNet, and the hyperparameters used by all models are consistent. In the data processing part, this application randomly flips the image horizontally, and then randomly crops the image into a 32*32 size image after padding 4 pixels on each side of the image. This application uses a Tesla V100 NVlink graphics card to train each model for 200 cycles, and sets the batch size to 128. This application uses an SGD optimizer with a weight decay of 0.0001 and a momentum of 0.9 to train the model. This application sets the initial learning rate to 0.1, and divides the learning rate by 10 when training to 100 and 150 cycles.
[0092] Table 1 Classification results on the CIFAR-10 dataset
[0093]
[0094] Table 1 reflects the experimental results of different network models on the validation set. The classification performance of all networks with New-CA modules inserted is significantly better than that of networks with CA modules inserted, with lower top1-err and top5-err. This proves that the New-CA module can be used to improve the classification performance of various network models and means that the New-CA module proposed in this application can capture the location information of the target better than the CA module. It is also important to note that the network with New-CA modules inserted has significantly lower top1-err than the network with SE modules inserted, and it is the champion of the ILSVRC2017 classification task.
[0095] This application found that inserting the New-CA module brought very little memory and computational overhead to the original network. The top1-err of the ResNet50+New-CA network is 0.97 lower than that of the ResNet50+CA network, and only 0.01MB of memory space and 0.03MFLOPs of computational effort are spent. The top1-err and top5-err of the MobileNetV3+New-CA network are 1.43 and 0.2 lower than those of MobileNetV3+CA, respectively, and only 0.03MB of memory space and 0.01MFLOPs of computational effort are spent. Fig.11 The error rate curves on the cifar10 validation set with ResNet50 and MobileNetV3 as the baselines are described. It can be clearly seen that the network with the NEW-CA module inserted shows the lowest error rate in both error rate curves. This shows that compared with the CA module, the NEW-CA module has better generalization ability than the baseline model.
[0096] The plant disease dataset after image segmentation is divided into a training set and a validation set, accounting for 80% and 20% respectively. This application uses the training set to train on 6 network models without an attention module, including Vgg16, ResNet50, EfficientNet, ShuffleNetV2, MobileNetV3 and GhostNet. The image sizes of the plant disease dataset used in this article are not the same, and the image sizes collected in the real environment are also different. During the training process, this application randomly crops the image to a size of 650*650, and performs random flipping, random rotation, and affine transformation to increase the diversity of samples. When the trained network model is deployed to the actual environment, this application scales the short side of the real-time image to 700, the aspect ratio remains unchanged, and then the real-time image is uniformly processed to a size of 650*650 by center cropping. This application uses 4 Tesla V100 NVlink graphics cards to train each model in parallel for 200 cycles, and the batchsize of each card is 16 (the effective batch size is 64). The network used the Adam optimizer with an initial learning rate of 0.0001 and used cosine learning rate decay with a final learning rate of 0.00001.
[0097] Table 2 Classification results of plant disease dataset
[0098]
[0099] The performance of different network models on the plant disease validation set is shown in Table 2. This application uses 6 network models to classify plant diseases, including two ordinary convolutional neural networks, namely Vgg16 and ResNet50, and four lightweight convolutional neural networks, namely EfficientNet, ShuffleNetV2, MobileNetV3, and GhostNet. This application can clearly see that Vgg16 has the lowest top-1err of 12.8%. However, its model size and computational size far exceed those of the other five models. Therefore, this application selects ResNet50 from two ordinary networks as the backbone network of this application, and selects GhostNet with the highest accuracy from four lightweight networks with similar model size and computational size as the backbone network of this application. In order to further improve the performance of the network, this application inserts the attention module into the backbone network selected by this application.
[0100] Table 3 Performance of network embedding with different attention modules
[0101]
[0102] The results of networks with different attention modules inserted on the plant disease validation set are shown in Table 5-3. It is clear that compared with networks with other attention modules inserted, the network with the New-CA module inserted has the lowest top1-err and the memory and computational overheads brought by the attention module are negligible. In addition, this once again verifies that the NEW-CA attention module proposed in this application can help the network extract features better than the CA attention module.
[0103] In order for convolutional neural networks to be applied to agricultural production, they need to be able to execute in a timely manner on platforms with limited processor performance and limited memory. Therefore, this application performs model compression on the trained network with the NEW-CA attention module inserted. This application trains the pruned network, and the hyperparameters used in the training are consistent with those used in the plant disease classification experiment.
[0104] Table 4 Results of the pruned network on the plant disease validation set
[0105]
[0106] Table 4 shows the results of different degrees of channel pruning of the trained model in this application. This application can clearly see that model compression can greatly reduce the size and computation of the model and the accuracy will hardly change or even improve. For example, the best performing GhostNet network after compression is 0.2% lower than the top-1err before compression and the model size and computation are only about half of the original.
[0107] The channel pruning algorithm can compress the model according to the set pruning ratio, which can effectively streamline the network. Fig.12 As can be seen in this application, the trained ResNet50 network can reduce the model size and model calculation to 14.8% and 7.9% of the original respectively after using the channel compression algorithm, and the resulting accuracy loss is within an acceptable range. Channel pruning greatly compresses the model, allowing large convolutional neural networks to be applied to agricultural platforms with limited memory and computing power.
[0108] This application mainly attempts to explore a crop disease classification model that can adapt to diverse backgrounds in real environments and can be deployed on agricultural platforms with limited memory and computing. In the data preprocessing stage, this application first uses the GrabCut image segmentation algorithm to ensure the consistency of the data set image and the real image background, so that the trained network model has good robustness when deployed in practice. Then this application proposes a NEW-CA attention module improved from the CA module to improve the accuracy of the classification model, and proves through experiments that it can better help the network extract features. Finally, under the premise of ensuring that the accuracy rate remains almost unchanged, this application greatly reduces the size and computation of the network through the model channel pruning algorithm so that it can be applied to mobile devices and Internet of Things devices. Studies have shown that the model proposed in this application can be well applied to agriculture to identify crop disease categories, solves the problem of high error rate of manual recognition, and thus improves the yield and quality of crops.
[0109] The above is a detailed introduction to a method for identifying crop diseases based on a convolutional neural network provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.
Claims
1. A method for identifying crop diseases based on a convolutional neural network, wherein the convolutional neural network comprises a first convolution module, a plurality of Bottleneck modules, a second convolution module, an average pooling module, a third convolution module and a fully connected layer connected in sequence; the Bottleneck module comprises a first feature extraction module, a New-CA attention module, a second feature extraction module and a shortcut connection module connected in sequence; The New-CA attention module includes an average pooling module in the C direction, an average pooling module in the H direction, an average pooling module in the W direction, a fifth convolution module, a splicing convolution module, a first normalization module, a second normalization module, a sixth convolution module, a seventh convolution module, an eighth convolution module, a first sigmoid function module, a second sigmoid function module, a third sigmoid function module and a combination module, the average pooling module in the C direction, the average pooling module in the H direction and the average pooling module in the W direction are connected to the first feature extraction module, the average pooling module in the C direction, the fifth convolution module, the first normalization module The module, the sixth convolution module and the first sigmoid function module are connected in sequence, the average pooling module in the H direction and the average pooling module in the W direction are connected to the splicing convolution module, the splicing convolution module is connected to the second normalization module, the second normalization module is connected to the seventh convolution module and the eighth convolution module, the seventh convolution module is connected to the second sigmoid function module, the eighth convolution module is connected to the third sigmoid function module, and the first sigmoid function module, the second sigmoid function module and the third sigmoid function module are all connected to the combination module; It is characterized in that The method comprises the following steps: Step S100: obtaining a training data set, and preprocessing the training data set to obtain a preprocessed data set; Step S200: inputting the preprocessed data set into the initial convolutional neural network model for training to obtain a trained convolutional neural network model; Step S300: compressing the trained convolutional neural network model using a channel pruning method to obtain a compressed convolutional neural network model; Step S400: Acquire crop images, pre-process the crop images and input them into the compressed convolutional neural network model to obtain crop disease recognition results.
2. The method according to claim 1, characterized in that Step S100 includes: using the GrabCut algorithm to segment the foreground and background of the image in the data set, and processing the background of the image to black.
3. The method according to claim 2, characterized in that The first convolution module is used to receive the preprocessed image and perform a 3*3 convolution operation to extract features in the image; The multiple Bottleneck modules are used to extract deep features of the image after the convolution operation; The second convolution module is used to use a 1*1 convolution operation to perform a first fusion of deep features; The average pooling module is used to reduce the size of the features after the first fusion using a 7*7 pooling operation; The third convolution module is used to use a 1*1 convolution operation to perform a second fusion of the pooled features; The fully connected layer is used to obtain the classification result according to the features after the second fusion.
4. The method according to claim 3, characterized in that The first feature extraction module is used to extract features of the image after the convolution operation; The New-CA attention module is used to learn the features extracted by the first feature extraction module; The second feature extraction module is used to re-extract the features learned by the New-CA attention module; The shortcut connection module is used to perform a shortcut connection between the feature map output by the second feature extraction module and the feature map input by the first feature extraction module to obtain deep-level features.
5. The method according to claim 4, characterized in that It also includes a fourth convolution module, which is arranged between the New-CA attention module and the second feature extraction module. When the stride is greater than 1, the fourth convolution module is used to change the size of the feature map learned by the New-CA attention module; the second feature extraction module is used to extract features in the feature map after the size is changed.
6. The method according to claim 5, characterized in that The structure of the first feature extraction module is the same as that of the second feature extraction module, which is one of the Vgg16, ResNet50, EfficientNet, ShuffleNetV2, MobileNetV3 and GhostNet models.
7. The method according to claim 6, characterized in that Step S300 includes: Get the scale factors of all channels in the trained convolutional neural network model; All channels whose scale factors are less than a preset threshold and all input and output connections to the channel are deleted to obtain a compressed convolutional neural network model.
