A plant leaf disease classification prediction method and system
By using multiple convolution operations and the combination of MobileNet and InceptionV2 networks in plant leaf disease prediction, the problem of low accuracy in the prior art relying on manual participation and complex background is solved, and more efficient disease classification prediction is achieved.
Patent Information
- Application Number
- CN202111520837.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-13
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-13
AI Technical Summary
The existing methods of plant pest prediction rely on manual participation, have poor universality, and have low image prediction accuracy in complex backgrounds.
Multiple convolution operations are used to combine MobileNet and InceptionV2 networks to extract and fuse the digital images of plant leaves, and the cross entropy loss function is used to predict the probability of disease categories, so as to improve the accuracy of disease classification through a fully connected network.
It improves the accuracy of disease classification, enhances the adaptability and deployment convenience of the system, and reduces the dependence on manual participation.
Smart Images

Figure CN114170489B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target classification under complex backgrounds, and in particular to a method and system for classifying and predicting plant leaf diseases. Background Art
[0002] "Food is the first necessity of the people". As a gift from nature to mankind, food plays an indispensable role in the course of human development. It occupies a fundamental position in the national economy, providing us with basic necessities of life and raw materials for industrial development. With the rapid development of agriculture, the frequency of pests and diseases is increasing, which has a great impact on the yield of crops. In traditional agricultural production, the prevention and control of agricultural pests and diseases are generally discovered by farmers in the middle or even late stage of crop disease. Then, based on their own experience or consulting experts, they can understand the types of diseases and coping methods. However, this method is time-consuming and laborious, and most ordinary farmers do not have the opportunity to get guidance from experts. In this case, how to spread the knowledge of pests and diseases mastered by experts to farmers who need this knowledge more conveniently, quickly, economically and efficiently is an urgent problem to be solved.
[0003] In the past, researchers usually used traditional methods to identify pests and diseases, such as using the K-means clustering method to segment the diseased area, combining the global color histogram, color coherence vector, local binary pattern and SVM (support vector machine) to identify the disease; some methods use the naive Bayes method and the step-size discriminant analysis method to identify crop diseases; and other methods use the spectral prediction of artificial neural networks, use a combination of field photos and remote sensing images to train the network, and then classify the diseases.
[0004] Although the research on plant pest and disease identification based on traditional image processing technology has achieved certain results, the research process is cumbersome, overly dependent on manually designed feature extraction methods, highly subjective, time-consuming and labor-intensive, etc.; it cannot adapt to actual application scenarios with complex backgrounds, and cannot meet the complex situations in actual applications. For example, the traditional image processing method usually requires converting the color space and segmenting the image by manually setting different color thresholds. This approach is highly dependent on artificial prior knowledge and is not universal. In addition, it has a poor effect when the background color and the segmented color are similar. Summary of the invention
[0005] The present invention provides a plant leaf disease classification prediction method and system, which are used to solve the problems that existing plant disease prediction methods rely heavily on manual participation, have poor universality, and have low prediction accuracy for images with complex backgrounds.
[0006] To achieve the above object, the present invention adopts the following technical solutions:
[0007] A first aspect of the present invention provides a method for classifying and predicting plant leaf diseases, the method comprising the following steps:
[0008] Perform multiple convolution operations on the acquired digital image of the plant leaf to obtain a feature vector of each convolution operation;
[0009] Divide each of the feature vectors into multiple parts, perform convolution and / or pooling operations on each part, and fuse the operated parts;
[0010] The fusion result is input into the fully connected network to transform the two-dimensional feature map into a one-dimensional vector, and the cross entropy loss function is used to predict the probability value of the disease category.
[0011] Furthermore, the multiple convolution operations are performed through a MobileNet network, and the multiple convolution operations are specifically 4 times.
[0012] Furthermore, the multiple convolution operations are specifically:
[0013] Perform a convolution operation with 32 convolution kernels, a size of 3*3, a step size of 2, and padding = 1, then perform a BatchNormalization operation, and finally pass the ReLU activation function to output the feature vector Vector1;
[0014] Perform a convolution operation on the feature vector Vector1 with 64 convolution kernels, a size of 3*3, a step size of 1, and padding = 1, then pass through BatchNormalization, and finally pass through the ReLU activation function to output the feature vector Vector2;
[0015] Perform a convolution operation on the feature vector Vector2 with a convolution kernel of 128, a size of 3*3, a step size of 2, and padding = 1, then pass through BatchNormalization, and finally pass through the ReLU activation function to output the feature vector Vector3;
[0016] The feature vector Vector3 is convolved with 256 convolution kernels, a size of 3*3, a step size of 2, and padding = 1, and then BatchNormalization is performed, and finally the ReLU activation function is passed to output the feature vector Vector4.
[0017] Furthermore, the multiple parts are specifically 4, and convolution and / or pooling operations are performed on each of the parts through the InceptionV2 network.
[0018] Furthermore, the specific process of operating the feature vector Vector1 is as follows:
[0019] Perform a four-part operation on the feature vector Vector1;
[0020] The first part performs a convolution operation with a 1*1 convolution kernel and 8 convolution kernels;
[0021] The second part uses global average pooling, followed by a 1*1 convolution kernel with a convolution count of 8.
[0022] The third part performs a convolution operation with a convolution kernel size of 1*1 and a number of 8; then a convolution operation with a convolution kernel size of 1*1 and a number of 8;
[0023] The fourth part performs a convolution operation with a convolution kernel size of 1*1 and a number of 8, then a convolution operation with a convolution kernel size of 1*1 and a number of 8, and finally a convolution operation with a convolution kernel size of 1*1 and a number of 8.
[0024] Furthermore, the specific process of operating the feature vector Vector2 is as follows:
[0025] Perform four-part operations on the feature vector Vector2;
[0026] Perform a convolution operation on the first part with a 1*1 convolution kernel and 16 convolution kernels;
[0027] The second part is operated using global average pooling, followed by a 1*1 convolution kernel and 8 convolution operations;
[0028] The third part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 16, and then subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 16;
[0029] The fourth part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 16, then a convolution operation with a convolution kernel size of 1*1 and a number of 16, and finally a convolution operation with a convolution kernel size of 1*1 and a number of 16.
[0030] Furthermore, the specific process of operating the feature vector Vector3 is as follows:
[0031] Perform a four-part operation on the feature vector Vector3;
[0032] Perform a convolution operation on the first part with a 1*1 convolution kernel and 32 convolution kernels;
[0033] The second part is operated using global average pooling, followed by a 1*1 convolution kernel and 32 convolution operations;
[0034] The third part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 32, and then subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 32;
[0035] For the fourth part, a convolution operation with a convolution kernel size of 1*1 and a number of 32 is first performed, then a convolution operation with a convolution kernel size of 1*1 and a number of 32 is performed, and finally a convolution operation with a convolution kernel size of 1*1 and a number of 32 is performed.
[0036] Furthermore, the specific process of operating the feature vector Vector4 is as follows:
[0037] Perform a four-part operation on the feature vector Vector4;
[0038] Perform a convolution operation on the first part with a 1*1 convolution kernel and 64 convolution kernels;
[0039] The second part is operated using global average pooling, followed by a 1*1 convolution kernel with 64 convolution operations;
[0040] The third part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 64, and then subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 64;
[0041] The fourth part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 64, then a convolution operation with a convolution kernel size of 1*1 and a number of 64, and finally a convolution operation with a convolution kernel size of 1*1 and a number of 64.
[0042] A second aspect of the present invention provides a plant leaf disease classification and prediction system, the system comprising:
[0043] A feature extraction unit is used to perform multiple convolution operations on the acquired digital image of the plant leaf to obtain a feature vector of each convolution operation;
[0044] A feature fusion unit, used to divide each of the feature vectors into multiple parts, perform convolution and / or pooling operations on each part, and fuse the operated parts;
[0045] The classification prediction unit is used to input the fusion result into the fully connected network, convert the two-dimensional feature map into a one-dimensional vector, and use the cross entropy loss function to predict the probability value of the disease category.
[0046] A third aspect of the present invention provides a computer storage medium, wherein the computer storage medium stores computer instructions, and when the computer instructions are executed on the prediction system, the prediction system executes the steps of the prediction method.
[0047] The classification prediction system of the second aspect of the present invention can implement the method in the first aspect and the various implementation methods of the first aspect, and achieve the same effect.
[0048] The effects provided in the content of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:
[0049] The present invention uses MobileNet to extract image features, and uses InceptionV2 to fuse the extracted feature vectors, and changes the number of channels of each processing branch of InceptionV2 to 1 / 4 of the input. In the fusion process, the input features are fused at multiple scales, which not only ensures the adequacy of feature fusion, but also mines the similarities and commonalities between features, and ensures the consistency of the number of feature channels. Combining these two networks improves the accuracy of disease classification and makes deployment and application more convenient. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0051] Figure 1 is a schematic diagram of a flow chart of an embodiment of the method of the present invention;
[0052] Figure 2 It is a schematic diagram of performing InceptionV2 network operation in an embodiment of the method described in the present invention;
[0053] Figure 3 is an example diagram of prediction based on the method of the present invention;
[0054] Figure 4 It is a schematic diagram of the structure of the system embodiment of the present invention. DETAILED DESCRIPTION
[0055] In order to clearly illustrate the technical features of the present solution, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The disclosure below provides many different embodiments or examples for realizing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below. In addition, the present invention may repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. It should be noted that the components illustrated in the accompanying drawings are not necessarily drawn to scale. The present invention omits the description of known components and processing techniques and processes to avoid unnecessary limitations on the present invention.
[0056] Convolutional neural networks are widely used in image classification, target detection, video tracking and other fields. After unremitting exploration, their application in agriculture has proved that convolutional neural networks are also very effective in agriculture. Based on this, we proposed a convolutional neural network that combines MobileNet and the improved InceptionV2, and use this network to extract features and classify diseases. By using the deep separable convolution of MobileNet, this network can greatly reduce the amount of calculation, and the implementation method is simple; the extracted feature vector is input into InceptionV2 for multi-modal feature fusion, which can better fuse features at different levels, and then the fused features are sent to the fully connected network to output the disease classification prediction value. The following is a specific description through an embodiment.
[0057] like Figure 1 As shown, an embodiment of the present invention provides a method for classifying and predicting plant leaf diseases, the method comprising the following steps:
[0058] S1, performing multiple convolution operations on the acquired digital image of the plant leaf to obtain a feature vector of each convolution operation;
[0059] S2, dividing each of the feature vectors into multiple parts, performing convolution and / or pooling operations on each part, and fusing the operated parts;
[0060] S3, the fusion result is input into the fully connected network, the two-dimensional feature map is converted into a one-dimensional vector, and the probability value of the disease category is predicted using the cross entropy loss function.
[0061] In step S1, the multiple convolution operations are performed through the MobileNet network, and the multiple convolution operations are specifically 4 times.
[0062] MobileNet is used to extract features. The specific process can be expressed by the following formula:
[0063] Fe 1=ReBN(Conv1(ReBN(Conv3(input)))), (1)
[0064] Fe 2 =ReBN(Conv1(ReBN(Conv3(Fe 1 )))), (2)
[0065] Fe 3 =ReBN(Conv1(ReBN(Conv3(Fe 2 )))), (3)
[0066] Fe 4 =ReBN(Conv1(ReBN(Conv3(Fe 3 )))), (4)
[0067] Among them, Fe i Represents the feature vector after i layers of convolution, ReBN represents BatchNormalization and ReLU operations, Conv1 represents the convolution operation with a convolution kernel size of 1*1, and Conv3 represents the convolution operation with a convolution kernel size of 3*3.
[0068] In formula (1), the collected digital image of plant leaves is subjected to a convolution operation with a convolution kernel number of 32, a size of 3*3, a step size of 2, and padding = 1, and then a BatchNormalization normalization operation is performed, followed by a 1*1 convolution, and finally a ReLU activation function to output a feature vector Vector1.
[0069] Perform a convolution operation on the feature vector Vector1 with 64 convolution kernels, a size of 3*3, a step size of 1, and padding = 1, then pass through BatchNormalization, then pass through 1*1 convolution, and finally pass through the ReLU activation function to output the feature vector Vector2;
[0070] Perform a convolution operation on the feature vector Vector2 with a convolution kernel number of 128, a size of 3*3, a step size of 2, and padding = 1, then pass through BatchNormalization, then pass through 1*1 convolution, and finally pass through the ReLU activation function to output the feature vector Vector3;
[0071] The feature vector Vector3 is convolved with 256 convolution kernels, a size of 3*3, a step size of 2, and padding = 1. It is then subjected to BatchNormalization, a 1*1 convolution, and finally a ReLU activation function to output the feature vector Vector4.
[0072] In step S2, the multiple parts are specifically 4, and convolution and / or pooling operations are performed on each of the parts through the InceptionV2 network.
[0073] like Figure 2 As shown, the extracted features are subjected to feature fusion operation, specifically, the source input is divided into four parts, the number of channels of each part is 1 / 4 of the number of channels of the source input, and the steps of operating the four parts are as follows:
[0074] F fused =Cat(Conv1(Fe 1 ),(Pool(Conv1(Fe 2 )),Conv1(Conv3(Fe 3 ),Conv1(Conv3(Con v3(Fe 4 )))), (5)
[0075] Among them, F fused Represents the fused feature vector, Conv1 represents the convolution operation with a convolution kernel size of 1*1, Pool represents the global average pooling operation, Conv3 represents the convolution operation with a convolution kernel size of 3*3, and Cat represents the connection operation Concat.
[0076] The feature vector Vector1 is operated in four parts; the first part is convolved with a 1*1 convolution kernel and 8 convolution kernels; the second part is operated using global average pooling, and then a 1*1 convolution kernel and 8 convolution kernels are performed; the third part is convolved with a 1*1 convolution kernel and 8 convolution kernels; then a 1*1 convolution kernel and 8 convolution kernels are performed; the fourth part is convolved with a 1*1 convolution kernel and 8 convolution kernels, and then a 1*1 convolution kernel and 8 convolution kernels are performed, and finally a 1*1 convolution kernel and 8 convolution kernels are performed.
[0077] The specific process of operating the feature vector Vector2 is as follows: perform four-part operation on the feature vector Vector2; perform a convolution operation with a 1*1 convolution kernel and 16 convolution kernels on the first part; use global average pooling to operate on the second part, and then perform a convolution operation with a 1*1 convolution kernel and 8 convolution kernels; perform a convolution operation with a convolution kernel size of 1*1 and 16 on the third part, and then perform a convolution operation with a convolution kernel size of 1*1 and 16; perform a convolution operation with a convolution kernel size of 1*1 and 16 on the fourth part, and then perform a convolution operation with a convolution kernel size of 1*1 and 16, and finally perform a convolution operation with a convolution kernel size of 1*1 and 16.
[0078] The specific process of operating the feature vector Vector3 is as follows: perform four-part operation on the feature vector Vector3; perform a convolution operation with a 1*1 convolution kernel and 32 convolution kernels on the first part; use global average pooling to operate on the second part, and then perform a convolution operation with a 1*1 convolution kernel and 32 convolution kernels; perform a convolution operation with a convolution kernel size of 1*1 and 32 on the third part, and then perform a convolution operation with a convolution kernel size of 1*1 and 32; first perform a convolution operation with a convolution kernel size of 1*1 and 32 on the fourth part, then perform a convolution operation with a convolution kernel size of 1*1 and 32, and finally perform a convolution operation with a convolution kernel size of 1*1 and 32.
[0079] The specific process of operating the feature vector Vector4 is as follows: perform four-part operation on the feature vector Vector4; perform a convolution operation with a 1*1 convolution kernel and 64 convolution kernels on the first part; use global average pooling to operate on the second part, and then perform a convolution operation with a 1*1 convolution kernel and 64 convolution kernels; perform a convolution operation with a convolution kernel size of 1*1 and 64 on the third part, and then perform a convolution operation with a convolution kernel size of 1*1 and 64; perform a convolution operation with a convolution kernel size of 1*1 and 64 on the fourth part, and then perform a convolution operation with a convolution kernel size of 1*1 and 64, and finally perform a convolution operation with a convolution kernel size of 1*1 and 64.
[0080] In step S3, the cross entropy loss function is:
[0081]
[0082] Among them, L represents the overall loss function, L i represents the loss value of the i-th sample, N represents the total number of samples, M represents the number of categories, and y ic represents a sign function (0 or 1), if the true category of sample i is equal to c, it takes 1, otherwise it takes 0, p ic It represents the predicted probability that the observed sample i belongs to category c.
[0083] like Figure 3 As shown, based on the above method of the embodiment of the present invention, Figure 3 The prediction results of disease classification for the apple leaves shown are: the probability of healthy apple leaves is 0.23, the probability of leaf rust is 0.95, and the probability of leaf anthracnose and leaf blight is 0.35.
[0084] like Figure 4As shown, an embodiment of the present invention provides a plant leaf disease classification prediction system, which includes a feature extraction unit 1, a feature fusion unit 2 and a classification prediction unit 3.
[0085] The feature extraction unit 1 is used to perform multiple convolution operations on the acquired digital image of the plant leaf to obtain a feature vector of each convolution operation; the feature fusion unit 2 is used to divide each of the feature vectors into multiple parts, each part is respectively subjected to convolution and / or pooling operations, and the operated parts are fused; the classification prediction unit 3 is used to input the fusion result into a fully connected network, convert the two-dimensional feature map into a one-dimensional vector, and use the cross entropy loss function to predict the probability value of the disease category.
[0086] An embodiment of the present invention further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on the prediction system, the prediction system executes the steps of the prediction method.
[0087] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A classification and prediction method for plant leaf diseases, Its characteristics are: The method comprises the following steps: Perform multiple convolution operations on the acquired digital image of the plant leaf to obtain a feature vector of each convolution operation; Divide each of the feature vectors into multiple parts, perform convolution and / or pooling operations on each part, and fuse the operated parts; The fusion result is input into the fully connected network to transform the two-dimensional feature map into a one-dimensional vector, and the probability value of the disease category is predicted using the cross entropy loss function; The multiple convolution operations are performed through the MobileNet network, and the multiple convolution operations are specifically 4 times; The multiple convolution operations are specifically: Perform a convolution operation with 32 convolution kernels, a size of 3*3, a step size of 2, and padding=1, then perform a BatchNormalization operation, then a 1*1 convolution, and finally a ReLU activation function to output the feature vector Vector1; Perform a convolution operation on the feature vector Vector1 with 64 convolution kernels, a size of 3*3, a step size of 1, and padding=1, then go through Batch Normalization, then 1*1 convolution, and finally go through the ReLU activation function to output the feature vector Vector2; Perform a convolution operation on the feature vector Vector2 with a convolution kernel of 128, a size of 3*3, a step size of 2, and padding=1, then go through Batch Normalization, then a 1*1 convolution, and finally a ReLU activation function to output the feature vector Vector3; Perform a convolution operation on the feature vector Vector3 with a convolution kernel number of 256, a size of 3*3, a step size of 2, and padding=1, then go through Batch Normalization, then 1*1 convolution, and finally pass through the ReLU activation function to output the feature vector Vector4; The multiple parts are specifically 4, and convolution and / or pooling operations are performed on each of the parts through the InceptionV2 network; The specific process of operating the feature vector Vector1 is as follows: Perform a four-part operation on the feature vector Vector1; The first part performs a convolution operation with a 1*1 convolution kernel and 8 convolution kernels; The second part uses global average pooling, followed by a 1*1 convolution kernel with a convolution count of 8. The third part performs a convolution operation with a convolution kernel size of 1*1 and a number of 8; then a convolution operation with a convolution kernel size of 1*1 and a number of 8; The fourth part performs a convolution operation with a convolution kernel size of 1*1 and a number of 8, then a convolution operation with a convolution kernel size of 1*1 and a number of 8, and finally a convolution operation with a convolution kernel size of 1*1 and a number of 8; The specific process of operating the feature vector Vector2 is as follows: Perform four-part operations on the feature vector Vector2; Perform a convolution operation on the first part with a 1*1 convolution kernel and 16 convolution kernels; The second part is operated using global average pooling, followed by a 1*1 convolution kernel and 8 convolution operations; The third part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 16, and then subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 16; The fourth part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 16, then a convolution operation with a convolution kernel size of 1*1 and a number of 16, and finally a convolution operation with a convolution kernel size of 1*1 and a number of 16; The specific process of operating the feature vector Vector3 is as follows: Perform a four-part operation on the feature vector Vector3; Perform a convolution operation on the first part with a 1*1 convolution kernel and 32 convolution kernels; The second part is operated using global average pooling, followed by a 1*1 convolution kernel and 32 convolution operations; The third part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 32, and then subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 32; For the fourth part, first a convolution operation with a convolution kernel size of 1*1 and a number of 32 is performed, then a convolution operation with a convolution kernel size of 1*1 and a number of 32 is performed, and finally a convolution operation with a convolution kernel size of 1*1 and a number of 32 is performed; The specific process of operating the feature vector Vector4 is as follows: Perform a four-part operation on the feature vector Vector4; Perform a convolution operation on the first part with a 1*1 convolution kernel and 64 convolution kernels; The second part is operated using global average pooling, followed by a 1*1 convolution kernel with 64 convolution operations; The third part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 64, and then subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 64; The fourth part is subjected to a convolution operation with a convolution kernel size of 1*1 and a number of 64, then a convolution operation with a convolution kernel size of 1*1 and a number of 64, and finally a convolution operation with a convolution kernel size of 1*1 and a number of 64.
2. A plant leaf disease classification and prediction system, Its characteristics are: The system is used to implement the method according to claim 1, and the system comprises: A feature extraction unit is used to perform multiple convolution operations on the acquired digital image of the plant leaf to obtain a feature vector of each convolution operation; A feature fusion unit, used to divide each of the feature vectors into multiple parts, perform convolution and / or pooling operations on each part, and fuse the operated parts; The classification prediction unit is used to input the fusion result into the fully connected network, convert the two-dimensional feature map into a one-dimensional vector, and use the cross entropy loss function to predict the probability value of the disease category.
3. A computer storage medium having computer instructions stored therein, Its characteristics are: When the computer instructions are executed on the prediction system of claim 2 , the prediction system is caused to perform the steps of the prediction method of claim 1 .
