Crop disease grading method, device and equipment and storage medium

Abstract

The invention discloses a crop disease grading method, device and equipment and a storage medium. The method comprises the following steps: acquiring a crop leaf image; inputting the crop leaf image into a backbone feature extraction network of an encoder layer of a semantic segmentation model to obtain a low-level feature vector and a high-level feature vector; inputting the high-level feature vector into an enhanced feature extraction network of an encoder layer of a semantic segmentation model to obtain a processed high-level feature vector; inputting the low-level feature vector and the processed high-level feature vector into a decoder layer of a semantic segmentation model to obtain a final effective feature vector; inputting the final effective feature vector into a classifier layer of a semantic segmentation model to obtain a category corresponding to each pixel point in the crop leaf image; according to the category corresponding to each pixel point in the crop leaf image, the disease grading information of the crop is determined, so that the disease and the disease grade of the crop leaf are accurately and automatically determined.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a crop disease classification method, device, equipment and storage medium. Background Art

[0002] Crop diseases can lead to reduced crop yields and quality degradation, and even complete crop failure when the disease is severe. Therefore, it is often necessary to classify the disease levels according to the severity of the crop leaf disease, so as to further take appropriate prevention and control measures based on the disease situation.

[0003] At present, the methods for crop disease identification mainly include the following: the crop disease identification method based on manual investigation is that agricultural producers and plant protection experts make visual judgments on diseased leaves based on past experience; the crop disease identification method based on image features is to use artificially designed feature engineering to extract features from the target image of crops and analyze them; the crop disease identification method based on image classification network first uses deep neural network to perform feature learning on the collected diseased crop leaf image samples, and then uses the trained network to automatically extract and identify the target image; the crop disease identification method based on target detection network can accurately locate the position of crop leaves and disease spots.

[0004] However, existing crop disease identification methods can only identify a rectangular area containing leaves or lesions, making it difficult to accurately determine the disease type and disease level of crop leaves. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a crop disease classification method, device, equipment and storage medium, aiming to solve the technical problem that the existing crop disease identification method cannot accurately identify the specific location of the disease, resulting in the inability to determine the disease type and disease grade.

[0006] In order to achieve the above object, the present invention provides a crop disease classification method in a first aspect, comprising:

[0007] Acquire crop leaf images;

[0008] Input the crop leaf image into the backbone feature extraction network of the encoder layer of the semantic segmentation model to obtain a low-level feature vector and a high-level feature vector, wherein the backbone feature extraction network includes four sequentially connected inverse residual layers, the low-level feature vector is the output of the first inverse residual layer of the backbone feature extraction network, and the high-level feature vector is the output of the fourth inverse residual layer of the backbone feature extraction network;

[0009] Input the high-level feature vector into the enhanced feature extraction network of the encoder layer of the semantic segmentation model to obtain a processed high-level feature vector;

[0010] Input the low-level feature vector and the processed high-level feature vector into the decoder layer of the semantic segmentation model to obtain the final valid feature vector;

[0011] The final valid feature vector is input into the classifier layer of the semantic segmentation model to obtain the category corresponding to each pixel in the crop leaf image;

[0012] The crop disease classification information is determined according to the category corresponding to each pixel point in the crop leaf image, wherein the disease classification information includes the disease and the disease grade.

[0013] In an embodiment of the present invention, each inverse residual layer includes a plurality of inverse residual modules, each inverse residual module includes an input layer, a first 1x1 convolution layer, a 3x3 separable convolution layer, a second 1x1 convolution layer and an output layer connected in sequence, wherein the inverse residual module with a convolution kernel step size of 2 further includes a residual connection directly pointing from the input layer to the output layer;

[0014] The inverse residual modules in the second inverse residual layer and the fourth inverse residual layer of the backbone feature extraction network also include an SE attention mechanism module, which is arranged between the 3x3 separable convolution layer and the second 1x1 convolution layer. The SE attention mechanism module includes a pooling layer and two fully connected layers connected in sequence.

[0015] In an embodiment of the present invention, the enhanced feature extraction network includes a third 1x1 convolution layer in parallel, three 3x3 hole convolution layers with different lower expansion rates, a pooling layer, and a fourth 1x1 convolution layer, and the high-level feature vector is input to the enhanced feature extraction network of the encoder layer of the semantic segmentation model to obtain the processed high-level feature vector, including:

[0016] The high-level feature vectors are input into the third 1x1 convolution layer, three 3x3 dilated convolution layers with different dilation rates, and a pooling layer, respectively, to obtain five sets of feature vectors.

[0017] The five groups of feature vectors are concatenated and then input into the fourth 1x1 convolutional layer to obtain the processed high-level feature vector.

[0018] In an embodiment of the present invention, the decoder layer includes an upsampling layer, a fifth 1x1 convolution layer, two depth-separable convolution layers and a 3x3 convolution layer, and the low-level feature vector and the processed high-level feature vector are input into the decoder layer of the semantic segmentation model to obtain the final effective feature vector, including:

[0019] Inputting the processed high-level feature vector to the upsampling layer to upsample the processed high-level feature vector through a bilinear interpolation algorithm to obtain an upsampled high-level feature vector;

[0020] The low-level feature vector is input into the fifth 1x1 convolutional layer for dimensionality reduction to obtain the processed low-level feature vector;

[0021] The upsampled high-level feature vector is concatenated with the processed low-level feature vector, and then passed through two depth-wise separable convolutional layers and a 3x3 convolutional layer for feature fusion to obtain the final effective feature vector.

[0022] In the embodiment of the present invention, the classifier layer includes a sixth 1x1 convolutional layer, and the final valid feature vector is input into the classifier layer of the semantic segmentation model to obtain the category corresponding to each pixel in the crop leaf image, including:

[0023] Input the final effective feature vector to the sixth 1x1 convolutional layer to adjust the channel of the final effective feature vector to obtain the adjusted final effective feature vector;

[0024] The size of the adjusted final effective feature vector is then adjusted to be the same as the height and width of the crop leaf image, and the category corresponding to each pixel in the crop leaf image is obtained.

[0025] In an embodiment of the present invention, determining crop disease classification information according to the category corresponding to each pixel point in the crop leaf image includes:

[0026] Determine the disease, leaf area, and lesion area based on the category corresponding to each pixel in the crop leaf image;

[0027] The leaf area and lesion area of ​​the crops are input into the discriminator to obtain the disease level of the crops.

[0028] In an embodiment of the present invention, obtaining crop leaf images includes:

[0029] Acquire an initial crop leaf image;

[0030] Calculating the ratio between the long side of the initial crop leaf image and a preset length;

[0031] The initial crop leaf image is transformed according to the ratio to obtain the crop leaf image.

[0032] A second aspect of the present invention provides a crop disease classification device, comprising:

[0033] An acquisition module, used for acquiring images of crop leaves;

[0034] A backbone feature extraction module, used for inputting the crop leaf image into the backbone feature extraction network of the encoder layer of the semantic segmentation model to obtain a low-level feature vector and a high-level feature vector, wherein the backbone feature extraction network includes four sequentially connected inverse residual layers, the low-level feature vector is the output of the first inverse residual layer of the backbone feature extraction network, and the high-level feature vector is the output of the fourth inverse residual layer of the backbone feature extraction network;

[0035] An enhanced feature extraction module, used for inputting the high-level feature vector into the enhanced feature extraction network of the encoder layer of the semantic segmentation model to obtain a processed high-level feature vector;

[0036] The feature fusion module is used to input the low-level feature vector and the processed high-level feature vector into the decoder layer of the semantic segmentation model to obtain the final valid feature vector;

[0037] The classification module is used to input the final valid feature vector into the classifier layer of the semantic segmentation model to obtain the category corresponding to each pixel in the crop leaf image;

[0038] The determination module is used to determine the crop disease classification information according to the category corresponding to each pixel point in the crop leaf image, wherein the disease classification information includes the disease and the disease grade.

[0039] A third aspect of the present invention provides a computing device, comprising a processor and a memory, wherein the memory stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the crop disease grading method as described in the first aspect are implemented.

[0040] A fourth aspect of the present invention provides a machine-readable storage medium having a program or instruction stored thereon, and when the program or instruction is executed by a processor, the steps of the crop disease grading method as described in the first aspect are implemented.

[0041] Through the above technical solution, the semantic segmentation model is used to perform pixel-level classification of the leaf surface and disease spots of crop leaves in the crop leaf image, accurately identify the specific location of the disease, and classify the different types of visible disease characteristics on the leaf surface differently, so as to accurately and automatically determine what kind of disease the leaf has suffered and its disease level, greatly reducing the workload of people. At the same time, the semantic segmentation model in this solution can provide a higher resolution feature map by using four sequentially connected inverse residual layers as the backbone feature extraction network and adjusting the extraction position of the low-level feature vector, thereby increasing the gap between the low-level feature vector and the high-level feature vector, so that the difference between the fused feature vectors is more obvious and easier to distinguish, reducing the volume of model parameters while improving the calculation accuracy.

[0042] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0044] Figure 1 A schematic diagram of a process flow of a crop disease classification method according to an embodiment of the present invention is shown;

[0045] Figure 2 A structural diagram of a semantic segmentation model for a crop disease classification method according to an embodiment of the present invention is schematically shown;

[0046] Figure 3 A schematic diagram of a structure of an inverse residual module according to an embodiment of the present invention is shown;

[0047] Figure 4 A schematic diagram showing a structure of another inverse residual module according to an embodiment of the present invention;

[0048] Figure 5 A schematic diagram showing a structure of another inverse residual module according to an embodiment of the present invention;

[0049] Figure 6 A schematic diagram schematically shows the category corresponding to each pixel in a crop leaf image output by a semantic segmentation model according to an embodiment of the present invention;

[0050] Figure 7 The structural block diagram of the crop disease classification device according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0051] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0052] It should be noted that if the implementation methods of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), such directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0053] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0054] Figure 1 The schematic diagram of the process of the crop disease classification method according to the embodiment of the present invention is shown as follows: Figure 1 As shown, in an embodiment of the present invention, a method for grading crop diseases is provided, comprising the following steps:

[0055] Step S110: Acquire crop leaf images.

[0056] Crops refer to various plants cultivated in agriculture, including food crops, cash crops, etc. Edible crops are one of the basic food sources for humans. Once crop diseases affect crop yield and quality, they will affect human normal life. Therefore, it is necessary to monitor crop diseases. The present invention needs to first obtain leaf images of crops, so as to monitor crop diseases based on the semantic segmentation model.

[0057] Specifically, step S110 includes:

[0058] Acquire an initial crop leaf image;

[0059] Calculating the ratio between the long side of the initial crop leaf image and a preset length;

[0060] The initial crop leaf image is transformed according to the ratio to obtain the crop leaf image.

[0061] It is understandable that, considering that the sizes of the initial crop leaf images may be different, and the semantic segmentation model requires the input of an image of a fixed size, direct deformation may cause abnormal deformations such as image stretching and compression, and cropping may cause partial loss of image information. In an embodiment of the present invention, by calculating the ratio between the long side of the initial crop leaf image and the preset length, the ratio is used to calculate the new size of the crop leaf image and the original size of the image is deformed to a specified size, the crop leaf image can be scaled without deformation, wherein the preset length is the input size of the semantic segmentation model of the embodiment of the present invention, which can be set according to actual needs.

[0062] Step S120: Input the crop leaf image into the backbone feature extraction network of the encoder layer of the semantic segmentation model to obtain a low-level feature vector and a high-level feature vector, wherein the backbone feature extraction network includes four sequentially connected inverse residual layers, the low-level feature vector is the output of the first inverse residual layer of the backbone feature extraction network, and the high-level feature vector is the output of the fourth inverse residual layer of the backbone feature extraction network.

[0063] The encoder layer of the semantic segmentation model can provide rich semantic information, and feature extraction is an important step in the semantic segmentation process and the basis of the entire semantic segmentation model. A good feature extraction network can significantly improve the performance of the algorithm. Figure 2 In the embodiment of the present invention, the encoder layer 210 includes a backbone feature extraction network 211 and an enhanced feature extraction network 212. The backbone feature extraction network 211 includes four sequentially connected inverse residual layers, which has the characteristics of being lightweight, so that the semantic segmentation model occupies less computing resources while having better accuracy, and can draw conclusions faster.

[0064] Specifically, considering that the area of ​​crop disease spots is small, the extraction position of the low-level feature vector in the embodiment of the present invention is the output of the first inverse residual layer. The feature size of the output of the first inverse residual layer is larger than the feature size of the output of other inverse residual layers. By reducing the number of downsampling times, a feature map with higher resolution can be provided. At the same time, the gap between the low-level feature vector and the high-level feature vector is increased, so that the difference between the fused feature vectors is more obvious and easier to distinguish.

[0065] At the same time, the backbone feature extraction network 211 uses a new h-swish activation function, which can be expressed as formula (1):

[0066]

[0067] In this formula, ReLu6=min(max(x,0),6).

[0068] This activation function can improve the network accuracy while simplifying the derivation process, thereby accelerating the network reasoning calculation process.

[0069] In an embodiment of the present invention, each inverse residual layer includes a plurality of inverse residual modules, each inverse residual module includes an input layer, a first 1x1 convolution layer, a 3x3 separable convolution layer, a second 1x1 convolution layer and an output layer connected in sequence, wherein the inverse residual module with a convolution kernel step size of 2 further includes a residual connection directly pointing from the input layer to the output layer;

[0070] The inverse residual modules in the second inverse residual layer and the fourth inverse residual layer of the backbone feature extraction network also include an SE attention mechanism module, which is arranged between the 3x3 separable convolution layer and the second 1x1 convolution layer. The SE attention mechanism module includes a pooling layer and two fully connected layers connected in sequence.

[0071] A preferred structural stacking method of the backbone feature extraction network is shown in Table 1. In Table 1, each inverse residual layer corresponds to n rows of data, indicating that the inverse residual layer includes corresponding n inverse residual modules, and each row of data represents the information of an inverse residual module. For example, the first inverse residual layer corresponds to 4 rows of data, indicating that the first inverse residual layer includes 4 inverse residual modules, wherein the output channel size of the first inverse residual module is 16, the SE attention mechanism module is not included, and the convolution kernel step size is 2. It should be noted that the first two inverse residual modules in the first inverse residual layer are adjustment modules before entering the backbone feature extraction network, and may not actually be included in the four inverse residual layers of the backbone feature extraction network.

[0072] Table 1

[0073]

[0074] In Table 1, the output channel size represents the convolution kernel parameter of an inverse residual module, which is used to determine the size of the feature vector output by the inverse residual module. The size of the input feature vector of each inverse residual layer is equal to the size of the feature vector output by the previous inverse residual layer.

[0075] Specifically, the inverse residual module retains the residual connection of the traditional residual module and changes the original input feature vector processing steps from using 1x1 convolution for dimensionality reduction → using 3x3 convolution for feature extraction → using 1x1 convolution for dimensionality increase to using 1x1 convolution for dimensionality increase → using 3x3 separable convolution for feature extraction → using 1x1 convolution for dimensionality reduction, so it is called an inverse residual module. This module can greatly reduce the number of parameters involved in the calculation while ensuring that the recognition accuracy does not drop too much, thereby accelerating the calculation process. The calculation process of each inverse residual module can be written as formula (2):

[0076] x l+1 =f(x l ,W l )+x l (2)

[0077] In this formula, x is the feature vector of layer l, f(.) is the convolution part of the inverse residual module, and W l is the weight matrix of layer l.

[0078] After stacking the inverse residual modules, it can be written as formula (3):

[0079]

[0080] In this formula, l represents the total number of layers in the network, F(.) represents the inverse residual convolution function in the entire network, and W i Represents the weight of the corresponding layer.

[0081] At the same time, the inverse residual module with a convolution kernel stride of 1 does not include a residual connection directly from the input layer to the output layer, see Figure 3 , Figure 3 The schematic diagram shows a structural diagram of an inverse residual module according to an embodiment of the present invention (excluding the residual connection directly pointing from the input layer to the output layer). The function of such an inverse residual module is to deepen the depth of the feature extraction network to extract more features; the inverse residual module with a convolution kernel step size of 2 includes a residual connection directly pointing from the input layer to the output layer, see Figure 4 , Figure 4 The structure diagram of another inverse residual module according to an embodiment of the present invention is schematically shown (including a residual connection directly pointing from the input layer to the output layer). The function of this inverse residual module is to adjust the number of channels of the feature vector and achieve downsampling.

[0082] On this basis, the inverse residual module in the second and fourth inverse residual layers of the backbone feature extraction network also includes a SE (Squeeze and Excitation) attention mechanism module, see Figure 5 , Figure 5 The structure diagram of another inverse residual module according to an embodiment of the present invention (including the SE attention mechanism module) is schematically shown. Figure 5 As shown in the figure, there is no direct channel between the 3x3 separable convolution layer of the inverse residual module containing the SE attention mechanism module and the second 1x1 convolution layer. The two are connected through the SE attention mechanism module. The SE attention mechanism module includes a pooling layer and two fully connected layers connected in sequence. After the 3x3 separable convolution layer extracts the feature vector, the feature vector is copied and input into the SE attention mechanism module. The module will pool each channel of the feature vector, and then pass through two fully connected layers to obtain the output vector. In this output vector, the more important the feature is, the greater the weight assigned to it, and the less important the feature is, the smaller the weight assigned to it. Finally, the output vector will be multiplied element by element with the feature vector to obtain a new feature matrix before continuing with the next step.

[0083] Step S130: input the high-level feature vector into the enhanced feature extraction network of the encoder layer of the semantic segmentation model to obtain a processed high-level feature vector.

[0084] The enhanced feature extraction network 212 is a spatial pyramid pooling module, which is used to introduce multi-scale information and improve the accuracy of segmentation boundaries.

[0085] For details, please refer to Figure 2 In the embodiment of the present invention, the enhanced feature extraction network 212 includes a third 1x1 convolutional layer, three 3x3 dilated convolutional layers with different dilation rates, a pooling layer and a fourth 1x1 convolutional layer.

[0086] Step S130 includes:

[0087] The high-level feature vectors are input into the third 1x1 convolution layer, three 3x3 dilated convolution layers with different dilation rates, and a pooling layer, respectively, to obtain five sets of feature vectors.

[0088] The five groups of feature vectors are concatenated and then input into the fourth 1x1 convolutional layer to obtain the processed high-level feature vector.

[0089] In the enhanced feature extraction network 212, the five structures of the third 1x1 convolutional layer, three 3x3 hole convolutional layers with different lower expansion rates, and the pooling layer are arranged in parallel and processed in parallel. After the high-level feature vectors are processed by these five structures respectively, five different feature vectors will be obtained. After these five groups of feature vectors are spliced ​​together, the number of channels is adjusted through the fourth 1x1 convolutional layer to obtain the processed high-level feature vectors.

[0090] Step S140: input the low-level feature vector and the processed high-level feature vector into the decoder layer of the semantic segmentation model to obtain a final valid feature vector.

[0091] The main task of the decoder layer of the semantic segmentation model is to process and concatenate the low-level feature vectors and high-level feature vectors that have been downsampled by the encoder layer, and then extract the final effective feature vector that is rich in semantic information and image classification information, and finally complete the pixel-level image classification task through the classifier layer.

[0092] For details, please refer to Figure 2 In this embodiment of the present invention, the decoder layer 220 includes an upsampling layer, a fifth 1x1 convolutional layer, two depth-separable convolutional layers and a 3x3 convolutional layer, and step S140 includes:

[0093] Inputting the processed high-level feature vector to the upsampling layer to upsample the processed high-level feature vector through a bilinear interpolation algorithm to obtain an upsampled high-level feature vector;

[0094] The low-level feature vector is input into the fifth 1x1 convolutional layer for dimensionality reduction to obtain the processed low-level feature vector;

[0095] The upsampled high-level feature vector is concatenated with the processed low-level feature vector, and then passed through two depth-wise separable convolutional layers and a 3x3 convolutional layer for feature fusion to obtain the final effective feature vector.

[0096] On this basis, the decoder layer 220 upsamples the size of the processed high-level feature vector to 4 times the original size through a bilinear interpolation algorithm, and uses the fifth 1x1 convolution layer to reduce the dimension of the low-level feature vector, so that the low-level feature vector is aligned with the high-level feature vector after upsampling, and the two vectors are spliced, and then further feature fusion is performed through two depth-separable convolution layers and a 3x3 convolution layer in sequence to obtain the final effective feature vector. It can be understood that this final effective feature vector is a concentration of the entire crop leaf image features.

[0097] Step S150: input the final valid feature vector into the classifier layer of the semantic segmentation model to obtain the category corresponding to each pixel in the crop leaf image.

[0098] After training, the classifier layer of the semantic segmentation model can automatically complete the pixel-level image classification task. The classifier layer can adjust the size of the final effective feature vector output by the decoder layer to be consistent with the resolution of the original image, and output the category corresponding to each pixel in the original image.

[0099] Please also read Figure 6 , Figure 6 The categories corresponding to each pixel in the crop leaf images output by the four groups of semantic segmentation models provided by the embodiment of the present invention are shown, wherein each group of images is the category corresponding to each pixel in the crop leaf images. The black part is the background, and the middle outline represents the crop leaf. The part with a smaller gray value is the normal part of the crop leaf, and the part with a larger gray value is the part where the crop leaf has a disease. The result output after classification by the semantic segmentation model more accurately describes most of the diseased spots on the crop leaf surface.

[0100] For details, please refer to Figure 2 In this embodiment of the present invention, the classifier layer 230 includes a sixth 1x1 convolutional layer, and step S150 includes:

[0101] Input the final effective feature vector to the sixth 1x1 convolutional layer to adjust the channel of the final effective feature vector to obtain the adjusted final effective feature vector;

[0102] The size of the adjusted final effective feature vector is then adjusted to be the same as the height and width of the crop leaf image, and the category corresponding to each pixel in the crop leaf image is obtained.

[0103] Specifically, the classifier layer 230 uses the sixth 1x1 convolutional layer to adjust the channel of the final effective feature vector to be consistent with the number of classification categories. The data in the channel corresponds to the category information of the pixel point. Therefore, the channel needs to be aligned with the number of classification categories so that the category can be distinguished by the activation function. After adjusting the channel, the size of the final effective feature vector is adjusted to the same as the height and width of the initial crop leaf image to obtain the category corresponding to each pixel point in the crop leaf image.

[0104] Step S160: determining crop disease classification information according to the category corresponding to each pixel point in the crop leaf image, wherein the disease classification information includes the disease and the disease grade.

[0105] The disease type can be identified by the different shapes and colors of the lesions in the crop leaf image. The different shapes and colors of the lesions are different in the feature vector. The different categories can be distinguished by the activation function, so as to judge the different disease types of the crop leaves. For example, the lesions of rice blast are gray-black, and the lesions of rice sheath blight are irregular moire spots. Different categories of lesions can be declared during model training. The number of disease levels can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0106] Specifically, step S160 includes:

[0107] Determine the disease, leaf area, and lesion area based on the category corresponding to each pixel in the crop leaf image;

[0108] The leaf area and lesion area of ​​the crops are input into the discriminator to obtain the disease level of the crops.

[0109] In an embodiment of the present invention, the disease, leaf area and lesion area are determined according to the category corresponding to each pixel in the crop leaf image, and the leaf area and lesion area of ​​the crop leaf image are input into the discriminator. The discriminator obtains the crop disease grading information according to the calculated ratio of the leaf area to the lesion area and the crop disease grading standard, and outputs the crop disease grading information and an image that integrates the category corresponding to each pixel in the crop leaf image. The crop disease grading standard can refer to Table 2.

[0110] Table 2

[0111] The proportion of lesions to leaf area Crop disease level 0％ Level Zero 0.1％～2％ Level 1~Level 4 2％～10％ Level 5 10.1％～25％ Level 6 25.1％～50％ Level 7 50.1％～75％ Level 8 75.1％～100％ Level 9

[0112] The crop disease classification method provided by the embodiment of the present invention uses a semantic segmentation model to perform pixel-level classification on the leaf surface and disease spots of crop leaves in the crop leaf image, accurately identifies the specific location of the disease, and can classify the different types of visible disease characteristics on the leaf surface differently, thereby accurately and automatically determining what kind of disease the leaf has suffered and its disease level, greatly reducing the workload of people. At the same time, the semantic segmentation model in this scheme can provide a feature map with higher resolution by using four sequentially connected inverse residual layers as the backbone feature extraction network and adjusting the extraction position of the low-level feature vector, thereby increasing the gap between the low-level feature vector and the high-level feature vector, so that the difference between the fused feature vectors is more obvious and easier to distinguish, reducing the volume of model parameters while improving the calculation accuracy.

[0113] Corresponding to the above method embodiment, see Figure 7 , Figure 7 The structural block diagram of the crop disease classification device according to an embodiment of the present invention is schematically shown. Figure 7 As shown, in an embodiment of the present invention, the crop disease classification device includes:

[0114] An acquisition module 10 is used to acquire crop leaf images;

[0115] A backbone feature extraction module 20 is used to input the crop leaf image into the backbone feature extraction network of the encoder layer of the semantic segmentation model to obtain a low-level feature vector and a high-level feature vector, wherein the backbone feature extraction network includes four sequentially connected inverse residual layers, the low-level feature vector is the output of the first inverse residual layer of the backbone feature extraction network, and the high-level feature vector is the output of the fourth inverse residual layer of the backbone feature extraction network;

[0116] An enhanced feature extraction module 30, for inputting the high-level feature vector into an enhanced feature extraction network of an encoder layer of a semantic segmentation model to obtain a processed high-level feature vector;

[0117] A feature fusion module 40 is used to input the low-level feature vector and the processed high-level feature vector into the decoder layer of the semantic segmentation model to obtain a final effective feature vector;

[0118] A classification module 50 is used to input the final valid feature vector into the classifier layer of the semantic segmentation model to obtain the category corresponding to each pixel in the crop leaf image;

[0119] The determination module 60 is used to determine the crop disease classification information according to the category corresponding to each pixel point in the crop leaf image, wherein the disease classification information includes the disease and the disease grade.

[0120] The crop disease grading device provided in the embodiment of the present invention can implement each process of the crop disease grading method in the method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0121] An embodiment of the present invention further provides a computing device, including a processor and a memory, wherein the memory stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the above-mentioned crop disease grading method are implemented.

[0122] An embodiment of the present invention further provides a machine-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the above-mentioned crop disease grading method are implemented.

[0123] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0124] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0127] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0128] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0129] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0130] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0131] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for grading crop diseases, characterized in that: include: Acquire crop leaf images; Inputting the crop leaf image into a backbone feature extraction network of an encoder layer of a semantic segmentation model to obtain a low-level feature vector and a high-level feature vector, wherein the backbone feature extraction network includes four sequentially connected inverse residual layers, the low-level feature vector is an output of a first inverse residual layer of the backbone feature extraction network, and the high-level feature vector is an output of a fourth inverse residual layer of the backbone feature extraction network; Inputting the high-level feature vector into the enhanced feature extraction network of the encoder layer of the semantic segmentation model to obtain a processed high-level feature vector; Inputting the low-level feature vector and the processed high-level feature vector into the decoder layer of the semantic segmentation model to obtain a final valid feature vector; Inputting the final effective feature vector into the classifier layer of the semantic segmentation model to obtain the category corresponding to each pixel in the crop leaf image; The disease classification information of the crop is determined according to the category corresponding to each pixel point in the crop leaf image, wherein the disease classification information includes the disease and the disease grade.

2. The crop disease classification method according to claim 1, characterized in that: Each of the inverse residual layers includes a plurality of inverse residual modules, each of the inverse residual modules includes an input layer, a first 1x1 convolution layer, a 3x3 separable convolution layer, a second 1x1 convolution layer and an output layer connected in sequence, wherein the inverse residual module with a convolution kernel step size of 2 further includes a residual connection directly pointing from the input layer to the output layer; The inverse residual module in the second inverse residual layer and the fourth inverse residual layer of the backbone feature extraction network also includes an SE attention mechanism module, which is arranged between the 3x3 separable convolution layer and the second 1x1 convolution layer, and the SE attention mechanism module includes a pooling layer and two fully connected layers connected in sequence.

3. The crop disease classification method according to claim 1, characterized in that: The enhanced feature extraction network includes a third 1x1 convolutional layer in parallel, three 3x3 hole convolutional layers with different lower dilation rates, a pooling layer and a fourth 1x1 convolutional layer, and the enhanced feature extraction network that inputs the high-level feature vector into the encoder layer of the semantic segmentation model to obtain a processed high-level feature vector includes: Inputting the high-level feature vectors into the third 1x1 convolution layer, the three 3x3 dilated convolution layers with different lower dilation rates, and the pooling layer, respectively, to obtain five groups of feature vectors; The five groups of feature vectors are concatenated and then input into the fourth 1x1 convolutional layer to obtain a processed high-level feature vector.

4. The crop disease classification method according to claim 1, characterized in that: The decoder layer includes an upsampling layer, a fifth 1x1 convolution layer, two depth-separable convolution layers and a 3x3 convolution layer, and the low-level feature vector and the processed high-level feature vector are input into the decoder layer of the semantic segmentation model to obtain a final valid feature vector, including: Inputting the processed high-level feature vector to the upsampling layer to upsample the processed high-level feature vector by a bilinear interpolation algorithm to obtain an upsampled high-level feature vector; Inputting the low-level feature vector into the fifth 1x1 convolutional layer for dimensionality reduction to obtain a processed low-level feature vector; The upsampled high-level feature vector is concatenated with the processed low-level feature vector, and then sequentially passed through the two depth-separable convolutional layers and the 3x3 convolutional layer for feature fusion to obtain a final effective feature vector.

5. The crop disease classification method according to claim 1, characterized in that: The classifier layer includes a sixth 1x1 convolutional layer, and the final valid feature vector is input into the classifier layer of the semantic segmentation model to obtain the category corresponding to each pixel in the crop leaf image, including: Inputting the final effective feature vector to the sixth 1×1 convolutional layer to adjust the channel of the final effective feature vector to obtain an adjusted final effective feature vector; The size of the adjusted final effective feature vector is then adjusted to be the same as the height and width of the crop leaf image, so as to obtain the category corresponding to each pixel point in the crop leaf image.

6. The crop disease classification method according to claim 1, characterized in that: The step of determining the crop disease classification information according to the category corresponding to each pixel point in the crop leaf image includes: Determine the disease, leaf area and lesion area according to the category corresponding to each pixel point in the crop leaf image; The leaf area and the diseased spot area of ​​the crop are input into a discriminator to obtain the disease grade of the crop.

7. The crop disease classification method according to claim 1, characterized in that: The step of acquiring crop leaf images comprises: Acquire an initial crop leaf image; Calculating a ratio between a long side of the initial crop leaf image and a preset length; The initial crop leaf image is converted according to the ratio to obtain the crop leaf image.

8. A crop disease classification device, characterized in that: include: An acquisition module, used for acquiring crop leaf images; A backbone feature extraction module, used for inputting the crop leaf image into a backbone feature extraction network of an encoder layer of a semantic segmentation model to obtain a low-level feature vector and a high-level feature vector, wherein the backbone feature extraction network includes four sequentially connected inverse residual layers, the low-level feature vector is the output of the first inverse residual layer of the backbone feature extraction network, and the high-level feature vector is the output of the fourth inverse residual layer of the backbone feature extraction network; An enhanced feature extraction module, used for inputting the high-level feature vector into an enhanced feature extraction network of an encoder layer of the semantic segmentation model to obtain a processed high-level feature vector; A feature fusion module, used for inputting the low-level feature vector and the processed high-level feature vector into the decoder layer of the semantic segmentation model to obtain a final effective feature vector; A classification module, used for inputting the final valid feature vector into the classifier layer of the semantic segmentation model to obtain the category corresponding to each pixel in the crop leaf image; The determination module is used to determine the disease classification information of the crop according to the category corresponding to each pixel point in the crop leaf image, wherein the disease classification information includes the disease and the disease grade.

9. A computing device, characterized in that The method comprises a processor and a memory, wherein the memory stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the crop disease grading method according to any one of claims 1 to 7 are implemented.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the crop disease grading method according to any one of claims 1 to 7 are implemented.

Images