A RAML-based identification method for multiple diseases of crop leaves with multiple infections

Abstract

The present invention relates to the fields of agricultural artificial intelligence and plant protection, and specifically, provides a method for identifying multiple diseases of crop leaves with composite infectiousness based on RAML. The method solves the problem that the generalization performance of the crop disease recognition model in an actual planting environment is low, and the accuracy of identifying multiple diseases in an image is low at the same time. The method comprises: firstly collecting crop disease images in an actual planting environment, annotating them with one-hot encoding after data cleaning, dividing the annotated data into a training set, a validation set and a test set, and then inputting them into a method for identifying multiple diseases of crop leaves with composite infectiousness based on RAML provided by the present invention for model training and saving the trained model. The model constructs an attention mechanism module by connecting the channel attention mechanism and the spatial attention mechanism in parallel, and embeds it into the ResNet-50 model architecture, and then combines it with the Sigmoid activation function to simultaneously predict multiple disease categories present in the leaf image.

Description

Technical Field

[0001] The present invention relates to the field of agricultural artificial intelligence and plant protection, and in particular, to a method for identifying compound infectious multi-diseases of crop leaves based on RAML. In particular, a method for identifying compound infectious multi-diseases of crop leaves based on residual attention mechanism and multi-label learning is provided. Background Art

[0002] It is of great value to timely and accurately identify crop diseases and control their spread in the crop production process. Although many crop disease recognition methods based on machine learning and computer vision have been proposed, it is still difficult to meet the production practice requirements in actual planting environments. The main reasons are that there is a certain gap between the training data used in the experimental environment and the image data collected in the actual planting environment, which makes it difficult for the high-precision model trained in the experimental environment to adapt to the requirements of the actual complex environment; second, there may be a combination of multiple pathogens in the actual planting environment, resulting in the coexistence of multiple diseases, and most of the existing models only identify one major disease category in one image, and do not pay enough attention to other concurrent diseases, so it is difficult to achieve the desired effect. To this end, the present invention provides a method for identifying multiple diseases of compound infection of crop leaves based on RAML, which can effectively solve the above problems. The relevant professional terms and research progress involved in the present invention are as follows:

[0003] 1. Professional terminology:

[0004] (1) RAML. RAML is the abbreviation of Residual Attention and Multi-label Learning, which is a method that combines the residual attention mechanism with multi-label learning. It is a deep learning method that uses the residual neural network ResNet-50 as the framework and embeds the channel attention mechanism and spatial attention mechanism with multi-label learning technology.

[0005] (2) Attention mechanism. That is, the attention mechanism, whose main task is to invest more resources in target-related tasks. Attention mechanism plays a very important role in the field of vision, and is mainly divided into channel attention mechanism, spatial attention mechanism, branch attention mechanism and temporal attention mechanism. Among them, channel attention mechanism is an object selection process, which can adaptively determine the weight of each channel to determine the content that needs attention; spatial attention mainly solves the problem of "where to pay attention", and can adaptively adjust the spatial area of ​​attention to predict the most relevant and important spatial position; branch attention mechanism mainly solves the problem of "what to pay attention to", mainly using a multi-branch structure, through a dynamic branch selection method, to select important branches from different candidate branches to solve the problem that needs attention; temporal attention mechanism mainly solves the problem of "when to pay attention", and is widely used in video sequences.

[0006] (3) Multi-label learning. Multi-label learning refers to the situation where a single instance corresponds to a set of labels. Multi-label learning aims to predict the correct label set on a single instance as accurately as possible, which has important application value in practical applications such as scene analysis, image annotation, and plant disease recognition.

[0007] (4) Complex infectious diseases. This refers to the situation where crops are infected by multiple pathogens, resulting in two or more diseases occurring at the same time. Such crops will show multiple disease characteristics.

[0008] 2. Analysis of the current status of related research at home and abroad

[0009] At present, the work related to crop disease recognition at home and abroad is mainly divided into three types: methods based on traditional machine learning, methods based on deep learning, and methods combining traditional machine learning with deep learning. In the application of traditional machine learning methods, there are the following representative advances: Ahmad et al. [2] proposed a crop disease recognition method based on leaf image features, which was carried out through image preprocessing, diseased leaf area segmentation, feature calculation based on gray level co-occurrence matrix (GLCM), feature selection and classification. Finally, six color features and twenty-two texture features were calculated, and finally support vector machine was used to classify plant diseases; Mojjada et al. [3] proposed a segmentation algorithm for automatic detection or classification of plant leaf diseases for leaf disease detection; Li et al. [4] proposed two traditional machine learning methods, CNN-KSVM (shallow CNN with Kernel support vector machine) and SCNN-RF (shallow CNN with Random Forest) for leaf disease recognition. These two methods extracted shallow features from deep learning and input them into support vector machine or random forest classifier for crop disease recognition. In recent years, the use of deep learning for disease recognition has achieved good results, with the following representative results: Fan et al. [5] proposed a crop disease recognition method based on transfer learning and feature fusion; Maheswaran et al. [6] proposed a crop disease recognition method for lightweight devices; Chen et al. [7] added an attention mechanism to a lightweight network for disease recognition; Mkonyi et al. [8] proposed a tomato disease recognition method based on a deep convolutional neural network. In addition, there are some research results that combine deep learning with traditional machine learning. For example, Sethy et al. [9] proposed a rice disease recognition method that combines deep convolutional neural networks with support vector machines. This method extracts the intermediate layer features of deep convolutional neural networks such as ResNet-50, VGG-16 and VGG-19, and then inputs them into support vector machines for classification, achieving good results.

[0010] In addition, the progress of the disclosed patent literature related to crop disease identification is as follows: Patent document (application number: 2021100768105) discloses a potato disease identification method based on convolutional neural network. This method uses image enhancement method to construct a potato disease dataset for convolutional neural network (CNN) training, and then uses median filtering and image standardization methods to pre-process the collected potato disease images. Finally, the potato disease and insect pest dataset is used to construct a disease recognition classifier based on convolutional neural network and use the transfer learning algorithm to load the pre-trained model parameters for model training, which is used to identify potato late blight, early blight, anthracnose and other diseases; Patent document (application number: 2022102954466) discloses a crop disease recognition method based on convolutional neural network. This method uses channel pruning method to compress the trained convolutional neural network model to obtain a compressed convolutional neural network model; obtain crop images For example, the crop image is preprocessed and then input into the compressed convolutional neural network model to obtain the crop disease recognition result; the patent document (application number: 2021105741630) discloses a small sample plant disease recognition method based on transfer learning and self-learning. The method first obtains a small sample data set in the target domain, introduces transfer learning for pre-training, and then introduces and self-learning expansion samples for progressive training and updates the model; the patent document (application number: 2020104425701) discloses a crop disease image processing method based on an attention mechanism. The method is based on a dual-path network structure and constructs a dual-path bidirectional feature level fusion network model based on the attention mechanism to obtain the crop disease image after feature fusion. And post-process it.

[0011] 3. Existing problems

[0012] The above methods play an important role in understanding the relevant processes of crop disease identification, but they also have the following shortcomings:

[0013] (1) From the current public literature, there are few studies on the identification of complex infectious diseases, that is, few consider the situation where two or more diseases occur simultaneously on a crop leaf image. Most of the current public crop disease identification methods are based on only one disease label per image, that is, a single-label multi-classification problem. However, in the actual planting process, one disease often occurs along with another disease. Accurately identifying and effectively controlling multiple diseases that appear simultaneously on crops is of great significance to increasing crop yields and income.

[0014] (2) Lack of data collected in real environments for model training. Most of the existing results are based on crop disease images collected in laboratory environments. It is difficult to achieve ideal recognition accuracy in real environments using these images to train models.

[0015] (3) The model is less innovative and lacks a training model that integrates multiple attention mechanisms. Some literature has introduced attention mechanisms for feature screening, but they are also directly embedded in the existing attention mechanisms. There is a lack of models that integrate multiple attention mechanisms for crop disease features in order to give full play to the advantages of multiple attention mechanisms and further improve recognition accuracy.

[0016] 4. Problems to be solved by the present invention

[0017] (1) In response to the lack of research related to the identification of complex infectious diseases, the present invention provides a crop disease identification method based on RAML, which can effectively solve the problem of coexistence of multiple diseases in a single image. This method can simultaneously and accurately identify the categories of multiple complex infectious diseases occurring in a single image.

[0018] (2) In order to solve the problem of lack of data collected in actual environment for model training, the present invention uses leaf disease images collected in actual environment to train the model. The collected data is as comprehensive as possible, including complex environments such as different lighting and angles. The model trained with data collected in actual environment has strong generalization performance and can effectively adapt to disease identification work in actual planting environment.

[0019] (3) In response to the problems of low model innovation and lack of multi-attention mechanism fusion model, the present invention provides a residual multi-attention mechanism module, which can effectively combine the advantages of channel attention mechanism and spatial attention mechanism, and fuse the weights calculated by the two attention mechanisms, so as to further enhance the feature expression ability of the model and improve the accuracy of crop disease recognition.

[0020] The following are the relevant literatures retrieved:

[0021] [1] Zhou Changjian, Song Jia, Xiang Wensheng. Research progress on crop disease identification based on artificial intelligence. Journal of Plant Protection, 2022, 49(01): 316-324. doi:

[0022] 10.13802 / j.cnki.zwbhxb.2022.2022803.

[0023] [2]N.Ahmad,H.M.S.Asif,G.Saleem,M.U.Younus,S.Anwar,and M.R.Anjum,“LeafImage-Based PlantDisease Identification Using Color and Texture Features,”WIRELESS PERSONAL COMMUNICATIONS,

[0024] vol.121,no.2,pp.1139–1168,Nov.2021,doi:10.1007 / s11277-021-09054-2.

[0025] [3]R.K.Mojjada,K.Kiran Kumar,A.Yadav,and B.V.V.Satya Vara Prasad,“Detection of plant leaf diseaseusing digital image processing,”MaterialsToday:Proceedings,Dec.2020,doi:10.1016 / j.matpr.2020.11.115.[4]Y.Li,J.Nie,andX.Chao,“Do we really need deep CNN for plant diseases identification?,”Computers andElectronics in Agriculture,vol.178,p.105803,Nov.2020,doi:10.1016 / j.compag.2020.105803.

[0026] [5]X.Fan,P.Luo,Y.Mu,R.Zhou,T.Tjahjadi,and Y.Ren,“Leaf image basedplant disease identification usingtransfer learning and feature fusion,”Computers and Electronics in Agriculture,vol.196,p.106892,May

[0027] 2022,doi:10.1016 / j.compag.2022.106892.

[0028] [6]U. Maheswaran, R. Babu Kallam, B. Arathi, K. Prawan, and G. Anitha, “Efficient plant leaf disease identification Material Fabrication using lightweight device,” Materials Today: Proceedings, vol. 47, pp.

[0029] 381–386, Jan. 2021, doi:10.1016 / j.matpr.2021.04.576.

[0030] [7]J. Chen, D. Zhang, A. Zeb, and Y. A. Nanehkaran, “Identification of rice plant diseases using lightweight attention networks,” Expert Systems with Applications, vol. 169, p. 114514, May 2021, doi:

[0031] 10.1016 / j.eswa.2020.114514.

[0032] [8]L. Mkonyi et al., “Early identification of Tuta absoluta in tomato plants using deep learning,” Scientific African, vol. 10, p. e00590, Nov. 2020, doi:10.1016 / j.sciaf.2020.e00590.

[0033] [9]P. K. Sethy, N. K. Barpanda, A. K. Rath, and S. K. Behera, “Deep feature based rice leaf disease identification using support vector machine,” Computers and Electronics in Agriculture, vol. 175, p. 105527, Aug. 2020, doi:10.1016 / j.compag.2020.105527. Summary of the Invention

[0034] The present invention provides a RAML-based crop leaf compound infectious multi-disease identification method for solving the problem of crop compound infectious multi-disease identification in an actual planting environment. By collecting crop leaf disease images in an actual planting environment and marking different disease categories; inputting them into the model provided by the present invention for training, and embedding different attention modules into the model to increase channel and spatial feature screening during the training process, enhance the differences and fine-grained features for distinguishing diseases, and improve the accuracy of prediction of different diseases; finally, the trained model is tested and parameters are adjusted in an actual environment until the ideal effect is achieved. The present invention is tested on a variety of crop disease data sets, and the detection accuracy is significantly improved compared to the published results. The main contents of the present invention are:

[0035] 1. The technical solution adopted by the present invention to solve the problem of identifying the compound infectious diseases of crops under complex background is: a method for identifying compound infectious diseases of crop leaves based on RAML, the method specifically comprising the following steps:

[0036] (1) Collect leaf disease images of crops in different growth cycles and under various growth environment conditions, perform data cleaning on the collected images, remove low-resolution, blurred, and invalid images, and use one-hot encoding to annotate the crop varieties and disease categories. Please refer to the attached Figure 1 ,The case where a leaf image contains only one disease is also labeled using one-hot encoding to establish a database of ,complex infectious leaf diseases of crops.

[0037] (2) The labeled crop disease images are divided into training set / validation set / test set in a ratio of 70% / 15% / 15%, and the training set is enhanced to further expand the number of training sets. The image enhancement methods include but are not limited to horizontal / vertical flipping, left / right flipping, adding noise, etc. No image enhancement operation is performed on the validation set and the test set.

[0038] (3) Design the attention mechanism module attention block, please refer to the attached Figure 2 , input the input feature F for Maxpooling and AvgPooling operations respectively, and then perform 1D convolution Conv1D respectively and then add the matrices to obtain a new feature matrix, which is then input into the spatial attention and channel attention for feature screening to obtain the weight value G C and G S , the weight value G after screening C and G SAfter the expansion operation, the total attention weight G is added together, and then multiplied by the input feature F to obtain the feature vector with the attention weight. Finally, a residual connection is performed with the input feature F to obtain the output feature R of the attention module attention block.

[0039] (4) Construct a RAML-based identification model for compound infectious multi-diseases on crop leaves. Figure 3 The overall architecture of the model adopts the ResNet-50 model architecture. In the residual attention block, after three layers of two-dimensional convolution Conv2D and batch normalization Batch Normalization operations, it is input into the attention mechanism module attention block designed in step (3), and then its output feature R is residually connected with the original input feature of the model. After passing through the ReLU activation function and the Global Average Pooling operation, it is input into the Fully Connected Layer. Finally, after passing through the Sigmoid activation function, different disease categories in the disease image can be output simultaneously.

[0040] (5) Train and save the model. Input the training set data obtained in step (2) into step (4) for model training, and use the validation set divided in step (2) for cross-validation until the model no longer converges, and save the trained model.

[0041] (6) After the model training is completed, the model saved in step (5) is deployed in the system for disease identification. It can simultaneously identify multiple disease categories involved in the training set, including single diseases and complex infectious multi-disease categories.

[0042] 2. The crop leaf disease data collected in the RAML-based crop leaf complex infectious multi-disease identification method refers to the leaf disease data collected by image acquisition equipment in an actual planting environment. The image acquisition equipment includes but is not limited to mobile phones, tablets, digital cameras, cameras, etc.; the labeling process is that plant protection experts label the crop name and disease type of each image.

[0043] 3. In the RAML-based crop leaf complex infectious multi-disease identification method, the data set is divided and sampled using the Stratified K-Fold Cross-Validation method so that the proportion of samples in each category is the same as that of the entire data set to prevent the imbalance of the data set from affecting the performance of the model, where K is an integer greater than 0 that optimizes the model training effect.

[0044] 4. In the RAML-based crop leaf complex infectious multi-disease identification method, image enhancement of the training set is performed to increase the number of training set samples to prevent overfitting. The image enhancement methods used in the present invention include but are not limited to horizontal / vertical flipping, left / right flipping, adding noise, etc. During the training process, the present invention expands the training set to 5 times the original training set.

[0045] 5. The design of the attention block in the RAML-based crop leaf composite infectious multi-disease identification method is the core of the present invention. Figure 2 , let the input feature be F, Maxpool and Avgpool represent Maxpooling and Avgpooling operations respectively, conv represents 1D convolution conv1D operation, CA and SA represent channel attention mechanism and spatial attention mechanism respectively, E refers to the expansion operation expand, G C and G S They are the channel attention weight and spatial attention weight after the expansion operation, which are used to ensure the consistency of feature sizes between input and output vectors. Sigmond is the activation function that can convert G C and G S The added weight value is mapped to the interval (0, 1). is a matrix addition operation, which can add corresponding elements in the matrix. Then G C and G S The calculation methods are: The total attention weight G can be expressed as: The output of the final attention mechanism module attention block is the connection between the features optimized by the total attention weight G and the features of the initial features F, which can be expressed as: Where R is the feature output by the attention block. It is a matrix multiplication operation, which is used to multiply the attention weights by the input features to obtain the features optimized by the attention mechanism.

[0046] 6. The loss function used in the model of the RAML-based crop leaf composite infectious multi-disease identification method is Binary cross-entropy, the activation function used in the residual attention block is ReLU, and the activation function used in the fully connected layer is Sigmoid. The model is trained until it no longer converges, and the trained model is saved.

[0047] 7. The validation set and test set used in the RAML-based crop leaf composite infectious multi-disease identification method do not undergo image enhancement operations to ensure the rigor of the test results.

[0048] 8. After the model training of the RAML-based crop leaf complex infectious multi-disease identification method is completed and the effect is tested using a test set, it can be deployed in a crop disease identification system to predict unlabeled crop disease categories. This method can be deployed on local smart terminals, including but not limited to mobile phones, laptops, and tablets, and can also be deployed on a remote server.

[0049] This invention is aimed at the major needs of smart plant protection technology and the cutting-edge development of artificial intelligence, breaking through the technical bottleneck of identifying compound infectious diseases of crops in actual planting environments, aiming to improve the robustness and generalization performance of the model and establish an objective, efficient and intelligent method for identifying multiple crop diseases. This invention can reduce the workload of plant protection personnel, provide new ideas and methods for crop disease identification, and provide data support for crop disease detection and precision pesticide application technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 : Crop disease examples and one-hot encoding diagram

[0051] Figure 2 :Schematic diagram of attention block module

[0052] Figure 3 :Architecture diagram of the crop leaf composite infectious multi-disease identification model based on RAML DETAILED DESCRIPTION

[0053] In order to facilitate a better understanding of the purpose, technical solutions and advantages of the present invention, the identification of complex infectious diseases on apple leaves is taken as an example, and the present invention is further described in combination with the accompanying drawings and specific examples. Those skilled in the art can easily understand other advantages and functions of the present invention from the contents disclosed in this specification, but the present invention is not limited in any form. It should be pointed out that for ordinary technicians in this field, several changes and improvements can be made without departing from the concept of the present invention, which all belong to the scope of protection of the present invention. In the absence of conflict, the following embodiments and features in the embodiments can be extended to the identification of leaf diseases of all categories of crops.

[0054] According to the RAML-based crop leaf composite infectious multi-disease identification method provided by the present invention, please refer to the attached Figure 3 , the main implementation method includes the following steps:

[0055] Step 1: Data collection and labeling. Data is the basis of model training, and the method described in the present invention is no exception. In order to enable the model in the RAML-based crop leaf compound infectious multi-disease identification method described in the present invention to achieve good recognition effect in an actual planting environment, this example takes apple leaf disease collection as an example, and collects apple leaf disease images in apple plantations under various growth cycles and different weather and lighting environments. The collected images are cleaned to remove low-resolution, blurred, and invalid image data. This example ultimately collects 21,631 images of healthy conditions and 4 types of apple leaf diseases in different growth cycles, and uses one-hot encoding for image labeling. Please refer to the attached figure. Figure 1 Finally, the annotation results of the collected leaf images in this example are as follows: 4624 healthy leaf images (healthy, marked with "0"), 4831 leaf gray spot (frog eye leaf spot, marked with "1"), 1684 leaf powdery mildew (powderymildew, marked with "2"), 2860 leaf rust (rust, marked with "3"), 4826 leaf scab (scab, marked with "4"), 1920 leaf gray spot and rust disease images (frog eye and rust, marked with "1, 3"), 886 leaf gray spot and scab disease images (frog eye and scab, marked with "1, 4"). The annotated images and their annotations are stored in the crop composite infectious leaf disease database.

[0056] Step 2: Divide the training set, validation set and test set, and perform image enhancement. The data set marked in step 1 is subjected to the Stratified K-Fold Cross-Validation method, where K is an integer greater than 0 that optimizes the model training effect; according to the number of diseases in each category, it is divided into a training set at a ratio of 70%, a validation set at a ratio of 15%, and a test set at a ratio of 15%, so that the sample ratio in each category is the same as the ratio of the entire data set to prevent the imbalance of the data set from affecting the model. Subsequently, the divided training set is subjected to image enhancement to prevent the occurrence of overfitting. The image enhancement method used in this implementation example mainly includes horizontal / vertical flipping, left / right flipping, adding noise, etc. In this example, the training set is expanded to 5 times the original training set during the training process. The image enhancement operation adopted by the present invention includes but is not limited to the above image enhancement methods and quantities, and other image enhancement operations and the number of enhanced training sets can be selected according to the specific disease type and quantity. In order to ensure the rigor of the experimental results and the effectiveness of the present invention, the validation set and the test set are not subjected to image enhancement.

[0057] Step 3: Design the attention block. Please refer to the attached Figure 2 , this implementation example uses the channel attention mechanism channel attention and the spatial attention mechanism spatial attention in parallel. The features are subjected to Maxpooling and Avgpooling operations respectively before being input into the two attention mechanism modules to filter out redundant information. Let the input feature be F, Maxpool and Avgpool represent Maxpooling and Avgpooling operations respectively, conv represents 1D convolution conv1D operation, CA and SA represent channel attention mechanism channel attention and spatial attention mechanism spatial attention respectively, E refers to the expansion operation expand, G C and G S They are the channel attention weight and spatial attention weight after the expansion operation, which are used to ensure the consistency of feature sizes between input and output vectors. Sigmond is the activation function that can convert G C and G S The added weight value is mapped to the interval (0, 1). is an addition operation, which means adding the corresponding elements in the matrix. Then G C and G S The calculation methods are: The total attention weight G can be expressed as: The output of the final attention mechanism module attention block is the connection between the features optimized by the total attention weight G and the features of the initial features F, which can be expressed as: Where R is the feature output by the attention block of the attention mechanism module; It is a matrix multiplication operation, which is used to multiply the attention weights by the input features to obtain the features optimized by the attention mechanism.

[0058] Step 4: Construct a RAML-based identification model for compound infectious diseases of crop leaves. Figure 3The overall architecture of the model adopts the ResNet-50 model architecture. In the residual attention block, after three layers of two-dimensional convolution Conv2D and batch normalization Batch Normalization, it is input into the attention mechanism module attentionblock designed in step 3. Its output feature R is residually connected with the original input feature of the model, and then after the ReLU activation function, the global average pooling operation is performed and then input into the fully connected layer Fully Connected Layer. Finally, after the Sigmoid activation function, multiple disease categories in the disease image can be output simultaneously. The specific operating environment and experimental parameter settings of this specific implementation example model are as follows: This implementation example runs on the Cent OS7.5 Linux operating system, the development language is Python, the deep learning framework used is pytorch 1.7, and 4 GeForce RTX TM 2080Ti accelerates the calculation. When the image is input to the model, it is automatically reset to RGB 3-channel 224×224 pixel size. The training batch is 64, which refers to the number of images that the model can process in each iteration. The number of iterations, epoches, is set to 200. The optimization function is Adam, which combines the advantages of the two stochastic gradient descent methods, AdaGrad and RMSProp, and designs independent adaptive learning rates for different parameters by calculating the first-order moment estimate and the second-order moment estimate of the gradient. The activation function used in the residual attentionblock is ReLU, and its formula can be expressed as: It takes the maximum value between 0 and the predicted value x, which is used to alleviate the gradient vanishing situation in the deep convolutional neural network. The activation function used in the fully connected layer is Sigmoid, which has a natural advantage in processing multi-label learning classification. Its calculation formula can be expressed as: Where x is the label probability calculated by the prediction model, and S(x) is the output of the activation function. When the predicted probability value of input x increases, S(x) tends to 1; when the predicted probability value of input x decreases, S(x) tends to 0. In the multi-label classification process, if there are multiple diseases coexisting, the one-hot encoding corresponding to the diseased leaves will have multiple elements with values ​​set to 1, and the element value without disease features will be 0. This implementation example uses the binary cross-entropy loss function, and its formula can be expressed as: The purpose is: when the sample is positive, y = 1, then BCE_Loss = -log(P(y)), when P(y) is larger, the value of the loss function is smaller, and the ideal situation is when P(y) = 1, the value of the loss function is 0; when the sample is a negative example, y = 0, then BCE_Loss = log(P(y)), when P(y) is smaller, the value of the loss function is smaller, and the ideal situation is when P(y) = 0, the value of the loss function is 0;

[0059] Step 5: Train and save the model. Input the training set data obtained in step 2 into the RAML-based crop leaf composite infectious multi-disease recognition model constructed in step 4 for training, and use the validation set divided in step 2 for cross-validation until the model no longer converges, and save the trained model.

[0060] Step 6: After the model training is completed, the model saved in step 5 is deployed in the system for unknown disease identification. It can simultaneously identify multiple disease categories involved in the training set, including single disease categories and complex infectious multi-disease categories.

[0061] It is known to those skilled in the art that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various modules provided by the present invention can be considered as a hardware component, and the modules used to implement various programs can also be considered as structures within the hardware component; the modules used to implement various functions can also be considered as both software programs for implementing the method and structures within the hardware component.

[0062] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various changes or modifications within the scope of the claims, which will not affect the essence of the present invention.

Claims

1. A RAML-based method for identifying multiple diseases of crop leaves with multiple infections, characterized in that: The following steps are involved: S1: Collect leaf disease images of crops in different growth cycles and under various growth environment conditions, clean the collected images, remove low-resolution, blurred, and invalid image data, and annotate the collected images using one-hot encoding. The annotation content is the crop name and disease category. If a leaf image contains only one disease, it is also annotated using one-hot encoding to establish a database of compound infectious leaf diseases of crops; S2: Divide the labeled crop disease images into training set / validation set / test set in a ratio of 70% / 15% / 15%, and perform image enhancement on the training set to further expand the number of training sets. Image enhancement methods include but are not limited to horizontal / vertical flipping, left / right flipping, and adding noise. No image enhancement operation is performed on the validation set and the test set. S3: Design the attention mechanism module attention block, input the input feature F into the attention mechanism module attention block, and obtain the output feature R of the attention mechanism module attention block; S4: Construct a crop leaf compound infectious multi-disease recognition model based on RAML. The overall architecture of the model adopts the ResNet-50 model architecture. The attention block designed in step S3 is embedded in the residual attention block of the ResNet-50 architecture. Its output feature R is residually connected with the original input feature of the model, and then after the ReLU activation function and the Global Average Pooling operation, it is input into the Fully Connected Layer. Finally, after the Sigmoid activation function, multiple disease categories in the disease image can be output simultaneously. S5: training and saving the model, inputting the training set data obtained in step S2 into the RAML-based crop leaf composite infectious multi-disease recognition model constructed in step S4 for training, and cross-validating with the validation set divided in step 2 until the model no longer converges, and saving the trained model; S6: After the model training is completed, the model saved in step S5 is deployed in the system to identify unknown diseases, and multiple disease categories involved in the training set can be identified at the same time, including single disease categories and complex infectious multi-disease categories; The designed attention mechanism module attention block adopts the channel attention mechanism and the spatial attention mechanism in parallel. The features are subjected to Maxpooling and Avgpooling operations before being input into the two attention mechanism modules. Let the input feature be F, Maxpool and Avgpool represent Maxpooling and Avgpooling operations respectively; conv represents 1D convolution conv1D operation; CA and SA represent channel attention mechanism and spatial attention mechanism respectively; E refers to the expansion operation expand; G C and G S They are the channel attention weight and spatial attention weight after the expansion operation expand; Sigmond is the activation function that can convert G C and G S The added weight value is mapped to the interval (0, 1); is an addition operation, which is to add the corresponding elements in the matrix; then G C and G S The calculation methods are: The total attention weight G can be expressed as: The output of the final attention mechanism module attention block is the connection between the features optimized by the total attention weight G and the features of the initial features F, which can be expressed as: Where R is the feature output by the attention block of the attention mechanism module; It is a matrix multiplication operation, which is used to multiply the attention weights by the input features to obtain the features optimized by the attention mechanism.

2. According to claim 1, a RAML-based method for identifying multiple diseases of crop leaves with multiple infections, characterized in that: The collected crop disease images refer to leaf disease data collected in an actual planting environment. The image collection devices include but are not limited to mobile phones, tablets, digital cameras, and webcams.

3. According to claim 1, a RAML-based method for identifying multiple diseases of crop leaves with multiple infections, characterized in that: The data set is divided into training set, validation set and test set, and the Stratified K-Fold Cross-Validation method is used for sampling, so that the proportion of samples in each category is the same as that of the entire data set to prevent the imbalance of the data set from affecting the performance of the model. K is an integer greater than 0 that optimizes the model training effect.

4. According to claim 1, a RAML-based method for identifying multiple diseases of crop leaves with multiple infections, characterized in that: The overall architecture of the model adopts the ResNet-50 model architecture; in the residual attention block, after three layers of two-dimensional convolution Conv2D and batch normalization Batch Normalization, it is input into the attention mechanism module attentionblock; it is then connected with the input features through residual connection and global average pooling GlobalAveragePooling operation, and then input into the fully connected layer Fully Connected Layer; finally, after the Sigmoid activation function, multiple disease categories in the disease image can be output simultaneously; the specific parameter settings of the model are as follows: when the image is input into the model, it is automatically reset to RGB 3 channel 224×224 pixel size, the training batch is set to 64, which refers to the number of images that the model can process per iteration; the number of iterations epoches is set to 200; the optimization function is Adam, which designs independent adaptive learning rates for different parameters by calculating the first-order moment estimate and second-order moment estimate of the gradient; the activation function used in the residual attention block is ReLU, and its formula can be expressed as: It takes the maximum value between 0 and the predicted value x, which is used to alleviate the gradient disappearance in the deep convolutional neural network. The activation function used in the fully connected layer is Sigmoid, and its calculation formula is: Where x is the label probability calculated by the prediction model, and S(x) is the output of the activation function. When the predicted probability value of input x becomes larger and larger, S(x) tends to 1; when the predicted probability value of input x becomes smaller and smaller, S(x) tends to 0. The loss function used is binary cross-entropy, and its formula can be expressed as: The purpose is: when the sample is positive, y=1, at this time BCE_Loss=-log(P(y)), when P(y) is larger, the value of the loss function is smaller, and the ideal situation is when P(y)=1, the value of the loss function is 0; when the sample is a negative example, y=0, at this time BCE_Loss=log(P(y)), when P(y) is smaller, the value of the loss function is smaller, and the ideal situation is when P(y)=0, the value of the loss function is 0.

5. According to claim 1, a RAML-based method for identifying multiple diseases of crop leaves with multiple infections, characterized in that: The method can be deployed on a local side, including but not limited to a mobile phone, a laptop computer, and a tablet computer, or on a remote server side.

Images