Fine-grained leaf disease identification method based on attention deep neural network

Abstract

The invention discloses a fine-grained leaf disease identification method based on an attention deep neural network. The method comprises the following steps: constructing an identification model, a recombination-generation model and a discrimination model; a data set is constructed; training the model; testing the model; and identifying the actual to-be-detected picture by using the identification model which completes the test. According to the method, the difficulty of fine-grained recognition is enhanced by utilizing the construction of the recombination-generation model and adopting a recombination-generation mode, the expert knowledge is obtained by training the classification model, and the recombination-generation method is only used during training, so that the complexity of the network is not increased, and no extra knowledge is needed during the recognition period; except a feedforward stage of a standard classification network, the model does not participate in operation in a reasoning stage, additional calculation overhead and storage consumption are not introduced in actual model deployment and production environments, and compared with a traditional classification network, the generalization ability of the method is enhanced, and the recognition precision is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of leaf disease recognition, and in particular to a fine-grained leaf disease recognition method based on an attention deep neural network. Background Art

[0002] In the identification of crop leaf diseases in agriculture, there are great similarities between the images of different types of leaf diseases. The same type of diseases are in different stages, which makes the images very different, that is, the inter-class differences are small and the intra-class differences are large. This brings great difficulties to the actual identification. This is a typical fine-grained recognition problem.

[0003] Leaf disease images have large intra-class differences but small inter-class differences. It is extremely important to perform fine feature representation of the local area of ​​the target. Complex networks can extract fine features to a certain extent, but they occupy a large amount of computer memory and consume a lot of computing resources.

[0004] In order to further improve the recognition accuracy, we regard the crop disease recognition problem as a fine-grained classification problem. In view of the complex background and many interferences in the actual environment of the greenhouse, the small diseased area, the unclear difference from wilting and soil, and the low recognition accuracy, we propose a fine-grained leaf disease recognition method based on attention deep neural network. Summary of the invention

[0005] The purpose of the present invention is to provide a fine-grained leaf disease recognition method based on an attention deep neural network to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a fine-grained leaf disease recognition method based on an attention deep neural network, comprising the following steps:

[0007] Step 1: Construct recognition model, recombination-generation model and discrimination model;

[0008] Step 2: Collect healthy and diseased leaf images and construct a dataset, and divide the dataset into a training set and a test set;

[0009] Step 3: Use the training set in the data set to train the recognition model, the recombination-generation model, and the discriminant model;

[0010] Step 4: Test the trained recognition model, recombination-generation model, and discriminant model using the test set in the data set;

[0011] Step 5: Use the tested recognition model to identify the actual image to be detected to obtain the recognition result.

[0012] Preferably, the recognition model includes an encoding network module, an attention network module and a classification network module. The encoding network module is used to extract the features of the input image and obtain two vectors of length 256, and then obtain the latent vector z through re-parameterized skill sampling. The classification network module is used to determine the category.

[0013] Preferably, the encoding network module is composed of a series of convolutional layers, and the encoding network module includes a Conv submodule, a 4layers submodule, a Scaleblock submodule, a Reducemean submodule, a Scale_fc submodule and a FC submodule.

[0014] Preferably, the 4layers submodule is composed of a Scale unit, a SENet unit and a Downsample unit, the Scale submodule is used to extract features, and the Downsample unit is used to reduce the size of each feature map and increase the number of channels.

[0015] Preferably, the recombination-generation model includes an encoding network module, an attention network module, a recombination network module and a generation network module. The recombination network module is used to disrupt the spatial distribution of local areas, divide the original input image into blocks and then recombine them. The working method of the recombination network module is to divide the input image into many local areas, and then disrupt them through the recombination mechanism and recombine them into one picture.

[0016] Preferably, the specific working steps of the reorganization network module are: for a picture I of size 128×128, divide it into N×N sub-regions, each region is marked as R ij , which is used to shuffle these local areas in the plane area, first swap the columns of the same row, then swap the row data of the same column, and finally reassemble the image according to the rearranged array.

[0017] Preferably, the generative network module uses a generative network to restore the original spatial layout of the local regions to simulate the semantic correlation between the local regions.

[0018] Preferably, the generating network module is used to convert the vector into The latent vector of size 256 is reduced to an image of size 128×128×3.

[0019] Preferably, the discriminant model includes an encoding network module, an attention module and a discriminant network module, and the discriminant network module is used to distinguish between generated images and real images.

[0020] Preferably, the types of the data set include healthy tomato leaves, tomato bacterial spot disease leaves, tomato early blight disease leaves, tomato late blight disease leaves, tomato leaf mold disease leaves, tomato mosaic virus leaves, tomato leaf spot disease leaves, tomato ring spot disease leaves, tomato two-spotted spider mite disease leaves and tomato yellow smut disease leaves, and the data set includes 70% training set and 30% test set.

[0021] Technical effects and advantages of the present invention:

[0022] The present invention utilizes the construction of a recombination-generation model and adopts a recombination-generation method to enhance the difficulty of fine-grained recognition and train a classification model to acquire expert knowledge. Since the recombination-generation method is only used during training, it does not increase the complexity of the network and does not require any additional knowledge during recognition. Except for the feedforward stage of the standard classification network, the model does not participate in the calculation in the inference stage. In the actual model deployment and production environment, no additional computing overhead and storage consumption are introduced. Compared with the traditional classification network, the generalization ability of the proposed method is enhanced and the recognition accuracy is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] The present invention provides Figure 1 The fine-grained leaf disease recognition method based on attention deep neural network shown in the figure includes the following steps:

[0026] Step 1: Construct recognition model, recombination-generation model and discrimination model;

[0027] Step 2: Collect healthy and diseased leaf images and construct a dataset, and divide the dataset into a training set and a test set;

[0028] Step 3: Use the training set in the data set to train the recognition model, the recombination-generation model, and the discriminant model;

[0029] Step 4: Test the trained recognition model, recombination-generation model, and discriminant model using the test set in the data set;

[0030] Step 5: Use the tested recognition model to identify the actual image to be detected to obtain the recognition result.

[0031] This method constructs a recombination-generation model. Since the recombination-generation method is only used during training, it does not increase the complexity of the network and does not require any additional knowledge during recognition. Except for the feedforward stage of the standard classification network, the model does not participate in the calculation during the inference stage. In the actual model deployment and production environment, no additional computing overhead and storage consumption will be introduced. Compared with the traditional classification network, the generalization ability of the proposed method is enhanced and the recognition accuracy is further improved.

[0032] Among them, the recognition model includes an encoding network module, an attention network module and a classification network module. The encoding network module is used to extract the features of the input image and obtain two vectors of length 256, and then obtain the latent vector z through re-parameterized skill sampling. The classification network module is used to determine the category.

[0033] Among them, the encoding network module is composed of a series of convolutional layers. The encoding network module includes Conv submodule, 4layers submodule, Scaleblock submodule, Reducemean submodule, Scale_fc submodule and FC submodule. The 4layers submodule is composed of Scale unit, SENet unit and Downsample unit. The Scale submodule is used to extract features. The Downsample unit is used to reduce the size of each feature map and increase the number of channels. The input of the model is a 128×128×3 image. After the Conv submodule layer, the size of the vector becomes 128×128×16. After the 4layers submodule, the size becomes 8×8×256. The Reducemean unit is global pooling. The Scale_fc submodule is used to better obtain global information. The input and output sizes of each layer of the encoding network module are as follows:

[0034]

[0035] Among them, the reconstruction-generation model includes an encoding network module, an attention network module, a reconstruction network module and a generation network module. The reconstruction network module is a fixed operation process and does not require training. The reconstruction network module is used to disrupt the spatial distribution of local areas and recombine the original input image after dividing it into blocks. The working method of the reconstruction network module is to divide the input image into many local areas, and then disrupt them through the recombination mechanism and recombine them into one picture.

[0036] The specific working steps of the reorganization network module are as follows: for a 128×128 image I, divide it into N×N sub-regions, each region is marked as R ij, i and j represent the row and column indices respectively, 1<i, j<N, which is used to disrupt these local areas in the plane area. Specifically, a random vector q is generated for the jth row of R j , where the value of the i-th element is q j，i＝i+r , r is a random variable in the range [-k, k], which obeys the uniform distribution of U(k, -k), k is an adjustable parameter, 1≤k<N, which defines the adjacent range of the transformable area, and then the array q j Sorting is performed to obtain a new arrangement of the regions in the jth row. This shuffling method destroys the global structure and ensures that the local regions are recombined within an adjustable range. Since the global structure is destroyed, in order to recognize these randomly shuffled images, the classification network must seek to discriminate regions and then learn the subtle changes of these categories. In practical applications, the method of rearranging first and then rearranging rows is adopted. That is, after dividing the image into N columns, the columns of the same row are first exchanged, and then the row data of the same column are exchanged, and finally the image is recombined according to the rearranged array.

[0037] The generative network module uses the generative network to restore the original spatial layout of the local area to simulate the semantic correlation between the local areas. The generative network module is used to convert the vector into The latent vector of size 256 is reduced to an image of size 128×128×3.

[0038] Among them, the discriminant model includes an encoding network module, an attention module and a discriminant network module, and the discriminant network module is used to distinguish between generated images and real images.

[0039] The crop leaf disease recognition method consists of three parts. During training, all three parts are involved in back propagation and forward reasoning, so the loss also consists of three parts. Specifically, we hope to minimize the following objectives:

[0040] L=L cls +L rec +L dls ;

[0041] In this model, the goal of the recognition model is to make the cross entropy loss L cls Minimum, which represents the accuracy of classification. The purpose of the reorganization-generation model is to keep the generated image as similar to the original image as possible, that is, to minimize L rec , which is used to reduce the introduced noise so that the network can reconstruct the damaged image and increase the semantic relevance between local information. The goal of the discriminant model is to try to distinguish between the generated image and the real image, which is the adversarial loss L dls , the specific algorithm is as follows:

[0042] Initialize the model parameters θ e ,θc ,θ g ,θ d ,

[0043] When training:

[0044] For the recognition model:

[0045] x real ←batch of images sampledfromthe dataset

[0046]

[0047] z real ←z μ real +εz σ real with ε∈N(0,Id)

[0048]

[0049] For the recombination-generation model:

[0050] x rec ←R(z real )

[0051]

[0052] z rec ←z μ rec +εz σ rec with ε∈N(0,Id)

[0053]

[0054] For the discriminant model:

[0055]

[0056] Calculate the gradient of the loss and update the parameters:

[0057]

[0058] End the loop.

[0059] In the algorithm, θ e ,θ c ,θ g ,θ dare the parameters of the encoding network module, classification network module, generation network module and discrimination network module respectively. First, they are initialized. During training, the recognition model, the recombination-generation model and the discrimination model are calculated and updated respectively. For the recognition model, data x is first collected from the training set. real , the vector z is obtained through the encoding network module μ real , z σ real , we get z by reparameterization real， After the recognition model, the category label C is obtained label , for the recombination-generation model, z real After destruction, you get x rec , the vector z is obtained through the encoding network module μ rec , z σ rec On the one hand, the network module is generated to obtain On the other hand, we can obtain z by reparameterization rec , and then get the category label C after the recognition model label , for the discriminant model, respectively rec and z real The discriminant network obtains the label of whether it is a real picture or a generated picture, and then calculates the gradient of the loss and updates the parameters.

[0060] The types of data sets include healthy tomato leaves (TH), tomato bacterial spot leaves (TBS), tomato early blight leaves (TEB), tomato late blight leaves (TLB), tomato leaf mold leaves (TLM), tomato mosaic virus leaves (TMV), tomato leaf spot (TSLS), tomato ring spot (TTS), tomato two-spotted spider mite leaves (TTSSM) and tomato yellow mold leaves (TYLCV). The data set includes 70% training sets and 30% test sets, which are used to train recognition models, recombination-generation models, and discriminant models, as well as to evaluate the performance of recognition models, recombination-generation models, and discriminant models.

[0061] Only the network model used in the training process is changed, and the proposed model is compared with other convolutional neural networks. The classification accuracy of the test set is used as the main evaluation indicator of the experimental results. The higher the classification accuracy, the better the performance of the model and the stronger the generalization ability. By comparing the recognition accuracy of tomato leaf diseases of different networks, the following results are obtained:

[0062]

[0063] The above table compares the recognition accuracy of tomato leaf diseases using different networks. It can be seen from the table that the fine-grained disease recognition method based on the attention deep neural network has improved the recognition accuracy on multiple categories of test sets of two different crops, reflecting the effectiveness of using the attention mechanism and the recombination-generation method to improve network performance. The improved convolutional neural network is compared with several more advanced convolutional neural networks such as ResNet-50

[38] , MobileNet-V2

[92] , Bilinear CNN

[94] , DCL based on ResNet-50

[95] , and PMG based on ResNet-50

[96] , and these networks are used to diagnose and identify diseased tomato leaves.

[0064] The proposed fine-grained disease recognition model based on attention deep neural network consists of three parts, namely recognition model, discrimination model and recombination-generation model. In order to verify the effectiveness of each model, the following ablation experiments are performed, where "i" is the "recognition model" without "attention network", "a" represents "attention network", "d" represents "discrimination network", "r" represents "recombination-generation network", "i+a" is the "recognition model" including "attention network", "i+d+r" is the "recognition model" including "discrimination network" and "recombination-generation network", and "i+a+d+r" is the "recognition model" including "attention network", "discrimination network" and "recombination-generation network", that is, the leaf disease fine-grained recognition network based on recombination-generation proposed in this paper;

[0065]

[0066] The above table lists the classification accuracy of all diseased leaves of tomatoes with different diseases and different module combinations by our method. It can be seen from the table that all models in our method are effective and significantly improve the classification accuracy. The proposed combination method produces the best results.

[0067]

[0068] The table above lists the confusion matrix of tomato leaf disease identification indicators using this method. It shows in detail how many different diseases are identified as other categories. It can be seen from the table that most disease types can be correctly identified, and the number of each disease identified as other categories can be seen.

[0069] The use of deep neural networks to identify crop leaf diseases has been widely studied in recent years. In deep learning research, in order to pursue higher performance, it is usually necessary to study powerful and complex model networks, but complex networks are constrained by storage space and computing resources, and there is redundancy in the parameters of complex network structures. For the tomato leaf disease images collected in greenhouse scenes, the same type of disease images are quite different, and there are great similarities between different types of disease images, that is, the intra-class differences are large and the inter-class differences are small, which brings great difficulties to actual identification. Therefore, the sophisticated feature representation of the local area of ​​the target plays a key role in fine-grained classification tasks.

[0070] In order to further improve the recognition accuracy without introducing complex networks and adding too many parameters, the current deep neural network has a low accuracy in crop disease recognition in greenhouse scenarios. The crop disease recognition problem is regarded as a fine-grained classification problem, and the attention mechanism is introduced. A leaf disease fine-grained recognition model based on attention convolutional neural network is proposed. The recombinant generation network and the discriminant network are introduced to enhance the classification effect and improve the recognition accuracy.

[0071] The attention mechanism is used in the classification network to increase the recognition ability. In order to learn the discriminative areas and features, a reorganization-generation network is added during training to force the classification network to learn the highly discriminative parts of the image instead of focusing on global features. In order to correctly identify these damaged images, an attention mechanism is added on the basis of the general network classification model, and the discriminative network is used to apply adversarial loss to distinguish the generated images from the original images to suppress the noise introduced by the damage. The experimental results show that the recognition accuracy of the method can reach a high level. The recognition accuracy of tomato leaf diseases is increased from 92.58% of 4stage to 94.35%, which is also higher than 93.07% of 5stage. It can be seen that the proposed fine-grained recognition network can improve the recognition accuracy. Since the reorganization-generation method and the discriminative network are only used during training, they will not increase the complexity of the network. No additional knowledge is required during recognition. Except for the feedforward stage of the standard classification network, the model does not participate in the calculation in the inference stage, and no additional computing overhead and storage consumption will be introduced in the actual model deployment and production environment.

[0072] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fine-grained leaf disease recognition method based on attention deep neural network, characterized in that: The following steps are involved: Step 1: Construct recognition model, recombination-generation model and discrimination model; Step 2: Collect healthy and diseased leaf images and construct a dataset, and divide the dataset into a training set and a test set; Step 3: Use the training set in the data set to train the recognition model, the recombination-generation model, and the discriminant model; Step 4: Test the trained recognition model, recombination-generation model, and discriminant model using the test set in the data set; Step 5: Use the tested recognition model to identify the actual image to be detected to obtain the recognition result.

2. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 1 is characterized in that: The recognition model includes an encoding network module, an attention network module and a classification network module. The encoding network module is used to extract the features of the input image and obtain two vectors of length 256, and then obtain the latent vector z through re-parameterized skill sampling. The classification network module is used to determine the category.

3. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 2 is characterized in that: The encoding network module is composed of a series of convolutional layers, and the encoding network module includes a Conv submodule, a 4layers submodule, a Scaleblock submodule, a Reducemean submodule, a Scale_fc submodule and a FC submodule.

4. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 3 is characterized in that: The 4layers submodule is composed of a Scale unit, a SENet unit and a Downsample unit. The Scale submodule is used to extract features, and the Downsample unit is used to reduce the size of each feature map and increase the number of channels.

5. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 1 is characterized in that: The reconstruction-generation model includes an encoding network module, an attention network module, a reconstruction network module and a generation network module. The reconstruction network module is used to disrupt the spatial distribution of local areas, divide the original input image into blocks and then recombine them. The working method of the reconstruction network module is to divide the input image into many local areas, and then disrupt them through the reconstruction mechanism and recombine them into one picture.

6. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 5 is characterized in that: The specific working steps of the reorganization network module are as follows: for a picture I of size 128×128, it is divided into N×N sub-regions, each region is marked as R ij , which is used to shuffle these local areas in the plane area, first swap the columns of the same row, then swap the row data of the same column, and finally reassemble the image according to the rearranged array.

7. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 5 is characterized in that: The generative network module uses a generative network to restore the original spatial layout of the local regions to simulate the semantic correlation between the local regions.

8. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 7 is characterized in that: The generating network module is used to convert the vector into The latent vector of size 256 is reduced to an image of size 128×128×3.

9. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 1 is characterized in that: The discriminant model includes an encoding network module, an attention module and a discriminant network module, and the discriminant network module is used to distinguish between generated images and real images.

10. The fine-grained leaf disease recognition method based on attention deep neural network according to claim 1 is characterized in that: The types of the data set include healthy tomato leaves, tomato bacterial spot disease leaves, tomato early blight disease leaves, tomato late blight disease leaves, tomato leaf mold disease leaves, tomato mosaic virus leaves, tomato leaf spot disease leaves, tomato ring spot disease leaves, tomato two-spotted spider mite disease leaves and tomato yellow smut disease leaves, and the data set includes 70% training set and 30% test set.

Images