A multi-label strategy-based method for plant disease and severity identification

Abstract

A method for identifying plant diseases and their severity based on a multi-label strategy includes: 1. constructing a training set, a validation set, and a test set; 2. building a multi-label classification network; 3. selecting a batch of crop disease images and inputting them into a multi-label feature extraction module. Feature extraction is performed on the crop disease images to obtain features containing disease spatial information. These features are input into an information fusion and communication module for modeling to obtain disease semantic features. The disease semantic features and spatial features are fused through a multi-scale information fusion module and input into a multi-label prediction and decoding module to obtain a multi-label classification result; 4. iteratively training the multi-label classification network; and 5. testing the multi-label classification network. This invention can equally identify crop species, diseases, and severity. The multi-label classification approach combines the advantages of a multi-task network with the low classification complexity of a single-task network and the lack of additional branches.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a plant disease and severity recognition method based on a multi-label strategy. Background Art

[0002] Crop diseases, such as powdery mildew, early blight, and late blight, are a major challenge facing global food production. These diseases can impair photosynthesis by affecting leaf health, thereby impacting crop growth and even causing crop failure. Automatic disease diagnosis and early detection are crucial for crop production. Therefore, crop disease detection has become a critical issue that needs to be addressed in intelligent agriculture.

[0003] Traditionally, disease identification relies primarily on manual observation and the observer's own experience. However, this manual identification method is not only time-consuming and labor-intensive, but also offers limited accuracy. To address this issue, technical experts have been dedicated to developing methods for automated crop disease identification. With the advancement of computer vision technology, new approaches to crop disease detection have emerged. Applying computer vision to disease detection can significantly improve detection speed and accuracy, reduce manual intervention, and lower costs. For example, support vector machines, self-organizing features, and nearest neighbor classification have demonstrated excellent performance in plant disease identification. Computer vision-based crop disease identification primarily utilizes machine learning techniques. Compared to manual observation, machine learning can manually construct crop disease signatures, enabling automated disease detection. However, classic machine learning methods require strict disease expertise and manual feature construction. Furthermore, their models suffer from poor robustness and low recognition accuracy in complex environments, limiting their application in intelligent disease identification. With the continuous improvement of computer performance and advancements in deep learning algorithms, deep learning methods have gradually replaced traditional machine learning algorithms and become mainstream in the field of disease detection. Deep learning models can automatically generate model parameters based on large amounts of data and learn disease characteristics, resulting in improved model accuracy and robustness in complex environments. Convolutional neural networks (CNNs) and transformer modules are the most widely used. For example, R. Amanda trained the Inception v3 network, achieving excellent results in cassava disease detection. Qiaokang Liang proposed the PD2SE-Net framework for multi-task classification of disease and severity diagnosis, and Fengyi Wang proposed the improved swinT framework for cucumber disease identification by improving the swin transformer.

[0004] However, a careful analysis of these papers reveals that they combine crop, disease, and severity labels into a new crop-disease-severity label for classification, or use multi-branch models to classify crop, disease, and severity separately. Both of these approaches are single-label classification methods. The former increases the number of classifications and the difficulty of classification; the latter uses multiple branches to classify crop, disease, and severity separately, which increases model parameters and training difficulty. Summary of the Invention

[0005] The present invention provides a plant disease and severity identification method based on a multi-label strategy to solve the technical problem that the existing classification method uses a single label for classification, which increases the difficulty of classification.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] The present invention provides a method for identifying plant diseases and their severity based on a multi-label strategy, comprising the following steps:

[0008] S1. Collect multiple crop disease images to form a crop disease dataset, then divide the crop disease dataset into a training set, a validation set, and a test set; label all crop disease images in the training set and the validation set to form a multi-label dataset;

[0009] S2. Construct an LDI-NET network. The LDI-NET network includes a multi-label feature extraction module, an information fusion communication module, a multi-scale information fusion module, and a multi-label prediction decoding module connected in sequence.

[0010] S3. Randomly select a batch of multi-label crop disease images from the training set and input them into the multi-label feature extraction module. The multi-label feature extraction module extracts features from the multi-label crop disease images to obtain features with disease spatial information. The features with disease spatial information are input into the information fusion and communication module for modeling to obtain disease semantic features. The multi-scale information fusion module fuses the disease semantic features with the features with disease spatial information and inputs them into the multi-label prediction and decoding module to obtain multi-label classification results.

[0011] S4. Establish a total loss function, calculate the multi-label classification results and the corresponding multi-label loss values, repeat S3 to S4 until the LDI-NET network converges, use the validation set for verification, select the set of weights with the best accuracy in the validation set as the weights of the LDI-NET network, and obtain the trained LDI-NET network;

[0012] S5. Use the test set to test the trained LDI-NET network.

[0013] Furthermore, the multi-label feature extraction module includes an image block embedding module, several Transformer modules and convolutional layers connected in sequence;

[0014] The information fusion and communication module includes a first multi-head self-attention mechanism and a first multi-layer perceptron connected in sequence from left to right; the input end and the output end of the first self-attention mechanism and the first multi-layer perceptron are respectively connected to the normalization and the residual;

[0015] The multi-label prediction decoding module includes a third multi-head self-attention mechanism, a second multi-head self-attention mechanism and a second multi-layer perceptron connected in sequence from bottom to top.

[0016] Furthermore, the number of Transformer modules in the multi-label feature extraction module is 12, and the 12 Transformer modules are connected in sequence.

[0017] Furthermore, the first multilayer perceptron is composed of a stack of linear layers, GELU activation functions, and Dropout.

[0018] Furthermore, the S3 specifically includes the following steps:

[0019] S31. Randomly select a batch of crop disease images with multiple labels from the training set and input them into the multi-label feature extraction module;

[0020] S32. The image block embedding module within the multi-label feature extraction module divides the multi-label crop disease image into image blocks in pixel space, and superimposes a trainable parameter on the image blocks as a position code. The image blocks with the superimposed position code are then flattened and input into several Transformer modules for global feature extraction. The extracted features are then input into a convolutional layer for local feature extraction to obtain features with spatial information of the disease.

[0021] S33, input the features with disease space information into the information fusion communication module, and use the learnable parameter matrix W Q1 、W K1 、W V1 Obtain the first query matrix Q1, the first key matrix K1, and the first value matrix V1, calculate the similarity between the first query matrix Q1 and the first key matrix K1, and combine them through the first multi-head self-attention mechanism, and finally obtain the disease semantic features through the first multi-layer perceptron transformation;

[0022] S34, inputting the features with disease spatial information obtained in S32 and the disease semantic features obtained in S33 into multi-scale information fusion for fusion, to obtain a fusion result X2;

[0023] S35. Input the fusion result X2 into the multi-label prediction decoding module, and combine it with the initialization feature code X3 in the multi-label prediction decoding module to obtain the multi-label classification result.

[0024] Furthermore, the S32 is expressed by the following formula:

[0025] Output=Conv2d(TB...TB(Conv2d(image))

[0026] Among them, Output represents the features containing disease spatial information, Conv2d(.) represents the convolution operation of the convolution layer, TB represents the Transformer module, and image represents crop disease data.

[0027] Furthermore, the S33 is specifically expressed by the following formula:

[0028] Q1=X1W Q1

[0029] K1=X1W K1

[0030] V1=X1W V1

[0031]

[0032]

[0033] Multihead1=Concat(head1,head2,…,head n )W O

[0034] Among them, X1 represents the input sequence, that is, the characteristics of the disease space information; W Q1 、W K1 、W V1 They are the learnable parameter matrix of the first query matrix Q1, the learnable parameter matrix of the first key matrix K1, and the learnable parameter matrix of the first value matrix V1; Attention(.) represents the single-head attention mechanism; softmax(.) represents the normalization process; K1 T represents the transposed matrix of the first key matrix K1; d k Indicates the dimension of the first key matrix K1; Concat(.) indicates the concatenation operation; head i 、h n Respectively represent the i-th self-attention and the n-th self-attention; W O Represents a learnable mapping matrix; Multihead1 represents the first multi-head self-attention mechanism; W i Q1、 and They respectively represent the i-th learnable parameter matrix of the first query matrix Q1, the i-th learnable parameter matrix of the first key matrix K1, and the i-th learnable parameter matrix of the first value matrix V1.

[0035] Furthermore, the S35 specifically includes the following steps:

[0036] S351, using the adaptive initialization feature code X3 stored in the multi-label prediction decoding module as the input of the third multi-head self-attention mechanism, through the learnable matrix W Q3 、W K3 、W V3 Obtain the third query matrix Q3, the third key matrix K3 and the third value matrix V3; calculate the similarity between the third query matrix Q3 and the third key matrix K3 and combine them through the third multi-head self-attention mechanism to obtain the output result of the third multi-head self-attention mechanism

[0037] S352, then the result obtained by the third multi-head attention mechanism Input into formula (1) to obtain the second query matrix Q2. Formula (1) is as follows:

[0038]

[0039] Wherein, WQ2 represents the learnable parameter matrix of the second query matrix Q2;

[0040] S353: Input the fusion result X2 obtained in S34 into formula (2) and formula (3) to obtain the second key matrix K2 and the second value matrix V2; formula (2) and formula (3) are specifically as follows:

[0041] K2=X2W K2 (2)

[0042] V2=X2W V2 (3)

[0043] Wherein, WK2 represents the learnable parameter matrix of the second key matrix K2; WV2 represents the learnable parameter matrix of the second value matrix V2;

[0044] Input the second query matrix Q2, the second key matrix K2, and the second value matrix V2 into formulas (4) to (6) to obtain the output result of the second multi-head attention mechanism:

[0045]

[0046]

[0047] Multihead2=Concat(head1,head2,…,head n )W O (6)

[0048] Among them, Multihead2 represents the second multi-head attention mechanism; K2 T represents the transposed matrix of the second key matrix K2; W i Q2 、 and They represent the i-th learnable parameter matrix of the second query matrix Q2, the i-th learnable parameter matrix of the second key matrix K2, and the i-th learnable parameter matrix of the second value matrix V2 respectively;

[0049] S354. Input the output result of the second multi-head self-attention mechanism into the second multi-layer perceptron to obtain a multi-label classification result.

[0050] Furthermore, the S351 is expressed using the following formula:

[0051] K3=X3W K3

[0052] V3=X3W V3

[0053] Q3=X3W Q3

[0054]

[0055]

[0056] Multihead3=Concat(head1,head2,…,head n )W O

[0057] Among them, W Q3 、W K3 、W V3 W represent the learnable parameter matrix of the third query matrix Q3, the learnable parameter matrix of the third key matrix K3 and the learnable parameter matrix of the third value matrix V3 respectively; i Q3 、 They represent the i-th learnable parameter matrix of the third query matrix Q3, the i-th learnable parameter matrix of the third key matrix K3, and the i-th learnable parameter matrix of the third value matrix V3 respectively; Multihead3 represents the third multi-head self-attention mechanism; K3 T represents the transposed matrix of the third key matrix K3.

[0058] Beneficial effects of the present invention:

[0059] 1. This paper proposes an end-to-end trainable LDI-NET network (leaf disease identification multi-label classification network), which can simultaneously identify the type, disease and severity of plant leaves, which is more convenient and practical;

[0060] 2. The LDI-NET network in the present invention uses multi-label classification. Multi-label classification is different from single-label classification. It can reduce classification complexity without increasing model branches. The multi-label classification method combines the advantages of low classification complexity of multi-task networks and no additional branches of single-task networks.

[0061] 3. The LDI-NET network of the present invention is an end-to-end trainable network that can autonomously extract required features through learning during training;

[0062] 4. The LDI-NET network in the present invention uses CNN (convolutional layer) to extract local features and Transformer module to extract global features, which has stronger feature extraction capabilities;

[0063] 5. In order to enhance the global information extraction capability and reduce invalid information interference, the present invention designs a residual connection structure in the information fusion and communication module of the LDI-NET network to connect the low-dimensional features with disease spatial information and the high-dimensional disease semantic features, further improving the feature expression capability of the module. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a structural schematic diagram of the present invention;

[0065] Figure 2 This is the structure diagram of the multi-label feature extraction module. DETAILED DESCRIPTION

[0066] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The accompanying drawings illustrate preferred embodiments of the present invention. However, the present invention may be implemented in many other forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present disclosure.

[0067] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0068] Reference Figure 1 , the embodiment of the present application provides a plant disease and severity identification method based on a multi-label strategy, comprising the following steps:

[0069] S1. Collect multiple crop disease images to form a crop disease dataset, then divide the crop disease dataset into a training set, a validation set, and a test set; label all crop disease images in the training set and the validation set to form a multi-label dataset;

[0070] S2. Construct an LDI-NET network. The LDI-NET network includes a multi-label feature extraction module, an information fusion communication module, a multi-scale information fusion module, and a multi-label prediction decoding module connected in sequence.

[0071] S3. Randomly select a batch of multi-labeled crop disease images from the training set and input them into the multi-label feature extraction module. The multi-label feature extraction module extracts features from the multi-labeled crop disease images to obtain low-dimensional features with disease spatial information. These low-dimensional features with disease spatial information are input into the information fusion and communication module for modeling to obtain high-dimensional disease semantic features. This process promotes information exchange between disease features, can pay more attention to the mutual connection between the three types of features: crop, disease, and severity, and improves the ability of the multi-label feature extraction module to identify disease features.

[0072] The multi-scale information fusion module fuses high-dimensional disease semantic features with low-dimensional features containing disease spatial information and inputs them into the multi-label prediction decoding module to obtain multi-label classification results;

[0073] S4. Establish a cross entropy loss function, calculate the multi-label classification results and the corresponding multi-label loss values, repeat S3 to S4 until the LDI-NET network converges, use the validation set for verification, select the set of weights with the highest accuracy in the validation set as the weights of the LDI-NET network, and obtain the trained LDI-NET network;

[0074] S5. Use the test set to test the trained LDI-NET network.

[0075] Reference Figure 2 ,In some embodiments, the multi-label feature extraction module includes an image patch embedding module, 12 Transformer modules and a convolutional layer connected in sequence;

[0076] The information fusion and communication module includes a first multi-head self-attention mechanism (MSA) and a first multi-layer perceptron (MLP) connected from left to right; the input and output ends of the first self-attention mechanism and the first multi-layer perceptron are respectively connected to normalization (Norm) and residual; by adding normalization and residual, the degradation of the information fusion and communication module during training can be effectively prevented and the convergence of the LDI-NET network can be accelerated.

[0077] In order to enhance the global information extraction capability and reduce invalid information interference, the present invention designs a residual connection structure in the information fusion and communication module of the LDI-NET network to connect the low-dimensional features with disease spatial information and the high-dimensional disease semantic features, further improving the feature expression capability of the information fusion and communication module.

[0078] The multi-label prediction decoding module includes a third multi-head self-attention mechanism, a second multi-head self-attention mechanism and a second multi-layer perceptron connected in sequence from bottom to top.

[0079] In some embodiments, the first multilayer perceptron (MLP) is composed of a stack of linear layers, GELU activation functions, and dropout (a neural network unit is temporarily discarded from the network with a certain probability during training). While networks using a pure attention mechanism can experience rank collapse when training crop disease classification models, the MLP can control the convergence rate by increasing the Pschitz constant, preventing the information expression capability of the self-attention network from decaying exponentially with increasing depth, which would cause degradation of the information fusion and communication module. Furthermore, the MLP structure further transforms the extracted crop disease label features through fully connected layers, increasing the expressive power of the information fusion and communication module and resulting in more accurate multi-label crop disease classification results.

[0080] In some embodiments, S3 specifically includes the following steps:

[0081] S31. Randomly select a batch of crop disease images with multiple labels from the training set and input them into the multi-label feature extraction module;

[0082] S32. The image block embedding module within the multi-label feature extraction module divides the multi-label crop disease image into image blocks in pixel space. To preserve the position information of the original image, a trainable parameter is superimposed on the image block as a position code. The image block with the superimposed position code is then flattened into sequence data that is easy to process by the Transformer module and input into several Transformer modules for global feature extraction. The extracted features are then input into a convolutional layer for local feature extraction to obtain low-dimensional features with spatial information of the disease.

[0083] In the multi-label feature extraction module, CNN (convolutional layer) captures local features, but is limited by the size of the receptive field. CNN has many deficiencies in global information interaction and efficient feature representation. The self-attention mechanism in the Transformer module can better extract long-range information by utilizing the global relationship between spatial pixels, thereby achieving efficient feature representation, but it has many deficiencies in local feature extraction. In order to solve the above problems, the present invention combines the Transformer module and CNN to design a multi-label feature extraction module. This module can combine the advantages of CNN and Transformer modules to efficiently extract local and global features, thereby obtaining richer semantic information about diseases and facilitating multi-label classification of diseases.

[0084] S33, input the features with disease space information into the information fusion communication module, and use the learnable parameter matrix W Q1 、W K1 、W V1 Obtain the first query matrix Q1, the first key matrix K1, and the first value matrix V1, calculate the similarity between the first query matrix Q1 and the first key matrix K1, and combine them through the first multi-head self-attention mechanism, and finally obtain the disease semantic features through the first multi-layer perceptron transformation;

[0085] The information fusion and communication module can encode the input crop disease features through a hidden information matrix and explore the relationship between the feature information of different crop disease types. This helps to promote information exchange between features and better express features.

[0086] In the information fusion and communication module, the connections established between the three types of information (crop, disease, and severity) in the output of the multi-label feature extraction module cannot fully express their relationship. The information fusion and communication module models features containing spatial disease information, and the input features are passed through the MSA and MLP. The MSA facilitates information exchange between disease information blocks, focusing more on the connections between the three types of features (crop, disease, and severity), thereby improving the multi-label classification model's ability to identify disease features. The model is then optimized using the MLP structure to prevent model degradation in the information fusion and communication module during training of the multi-label disease classification model. The fully connected layers of the MLP structure also further transform the crop disease features, allowing the multi-label disease data to better fit the model and enhance the feature expression capabilities of the information fusion and communication module.

[0087] S34, inputting the low-dimensional features with disease spatial information obtained in S32 and the high-dimensional disease semantic features obtained in S33 into multi-scale information fusion for fusion, to obtain a fusion result X2;

[0088] S35. Input the fusion result X2 into the multi-label prediction decoding module, and combine it with the initialization feature code X3 in the multi-label prediction decoding module to obtain the multi-label classification result.

[0089] In some embodiments, the S32 is expressed by the following formula:

[0090] Output=Conv2d(TB...TB(Conv2d(image))

[0091] Among them, Output represents the features containing disease spatial information, Conv2d(.) represents the convolution operation of the convolution layer, TB represents the Transformer module, and image represents crop disease data.

[0092] In some embodiments, the S33 is expressed by the following formula:

[0093] Q1=X1W Q1

[0094] K1=X1W K1

[0095] V1=X1W V1

[0096]

[0097]

[0098] Multihead1=Concat(head1,head2,…,head n )W O

[0099] Among them, X1 represents the input sequence, that is, the low-dimensional feature with disease space information; W Q1 、W K1 、W C1 They are the learnable parameter matrix of the first query matrix Q1, the learnable parameter matrix of the first key matrix K1, and the learnable parameter matrix of the first value matrix V1; Attention(.) represents the single-head attention mechanism; softmax(.) represents the normalization process; K1 T represents the transposed matrix of the first key matrix K1; d k Indicates the dimension of the first key matrix K1; Concat(.) indicates the concatenation operation; head i 、head n Respectively represent the i-th self-attention and the n-th self-attention; W ORepresents a learnable mapping matrix; Multihead represents multi-head attention. In order to improve the performance of the self-attention mechanism, multiple self-attention heads are i Concat splicing is performed, i represents the identity of self-attention in multi-head attention, through the learnable mapping matrix W O Get the first multi-head self-attention mechanism Multihead1. i Q1 、 and They respectively represent the i-th learnable parameter matrix of the first query matrix Q1, the i-th learnable parameter matrix of the first key matrix K1, and the i-th learnable parameter matrix of the first value matrix V1.

[0100] In some embodiments, the S35 specifically includes the following steps:

[0101] S351, using the adaptive initialization feature code X3 stored in the multi-label prediction decoding module as the input of the third multi-head self-attention mechanism, through the learnable parameter matrix W Q3 、W K3 、W V3 Obtain the third query matrix Q3, the third key matrix K3 and the third value matrix V3; calculate the similarity between the third query matrix Q3 and the third key matrix K3 and combine them through the third multi-head self-attention mechanism to obtain the output result of the third multi-head self-attention mechanism The formula is as follows:

[0102] Q3=X3W Q3

[0103] K3=X3Q K3

[0104] V3=X3W V3

[0105]

[0106]

[0107] Multihead3=Concat(head1,head2,…,head n )W O

[0108] Among them, W Q3 、W K3 、W V3 W represent the learnable parameter matrix of the third query matrix Q3, the learnable parameter matrix of the third key matrix K3 and the learnable parameter matrix of the third value matrix V3 respectively; i Q3 、 They represent the i-th learnable parameter matrix of the third query matrix Q3, the i-th learnable parameter matrix of the third key matrix K3, and the i-th learnable parameter matrix of the third value matrix V3 respectively; Multihead3 represents the third multi-head self-attention mechanism; K3 T represents the transposed matrix of the third key matrix K3;

[0109] S352, then the result obtained by the third multi-head attention mechanism Input into formula (1) to obtain the second query matrix Q2. Formula (1) is as follows:

[0110]

[0111] Among them, W Q2 represents the learnable parameter matrix of the second query matrix Q2;

[0112] S353: Input the fusion result X2 obtained in S34 into formula (2) and formula (3) to obtain the second key matrix K2 and the second value matrix V2; formula (2) and formula (3) are specifically as follows:

[0113] K2=X2W K2 (2)

[0114] V2=X2W V2 (3)

[0115] Among them, W K2 W represents the learnable parameter matrix of the second key matrix K2; V2 Represents the learnable parameter matrix of the second value matrix V2;

[0116] Input the second query matrix Q2, the second key matrix K2, and the second value matrix V2 into formulas (4) to (6) to obtain the output result of the second multi-head attention mechanism:

[0117]

[0118]

[0119] Multihead2=Concat(head1,head2,…,head n )W O (6)

[0120] Among them, Multihead2 represents the second multi-head attention mechanism; K2 T represents the transposed matrix of the second key matrix K2; W i Q2 、 and They represent the i-th learnable parameter matrix of the second query matrix Q2, the i-th learnable parameter matrix of the second key matrix K2, and the i-th learnable parameter matrix of the second value matrix V2 respectively;

[0121] S354. Input the output result of the second multi-head self-attention mechanism into the second multi-layer perceptron to obtain a multi-label classification result.

[0122] The LDI-NET network in the present invention uses multi-label classification. Multi-label classification is different from single-label classification. It can reduce the classification complexity without increasing the model branches. The multi-label classification method combines the low classification complexity of the multi-task network and the lack of additional branches of the single-task network.

[0123] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A plant disease and severity identification method based on a multi-label strategy, characterized in that: The steps include: S1. Collect multiple crop disease images to form a crop disease dataset, then divide the crop disease dataset into a training set, a validation set, and a test set; label all crop disease images in the training set and the validation set to form a multi-label dataset; S2. Construct an LDI-NET network. The LDI-NET network includes a multi-label feature extraction module, an information fusion communication module, a multi-scale information fusion module, and a multi-label prediction decoding module connected in sequence. S3. Randomly select a batch of multi-label crop disease images from the training set and input them into the multi-label feature extraction module. The multi-label feature extraction module extracts features from the multi-label crop disease images to obtain features with disease spatial information. The features with disease spatial information are input into the information fusion and communication module for modeling to obtain disease semantic features. The multi-scale information fusion module fuses the disease semantic features with the features with disease spatial information and inputs the fusion and inputs the fusion into the multi-label prediction and decoding module to obtain the multi-label classification results. S4. Establish a total loss function, calculate the multi-label classification results and the corresponding multi-label loss values, repeat S3 to S4 until the LDI-NET network converges, use the validation set for verification, select the set of weights with the best accuracy in the validation set as the weights of the LDI-NET network, and obtain the trained LDI-NET network; S5. Use the test set to test the trained LDI-NET network; The multi-label feature extraction module includes an image block embedding module, several Transformer modules and convolutional layers connected in sequence; The information fusion and communication module includes a first multi-head self-attention mechanism and a first multi-layer perceptron connected in sequence from left to right; the input end and the output end of the first self-attention mechanism and the first multi-layer perceptron are respectively connected to the normalization and the residual; The multi-label prediction decoding module includes a third multi-head self-attention mechanism, a second multi-head self-attention mechanism and a second multi-layer perceptron connected in sequence from bottom to top.

2. The plant disease and severity identification method according to claim 1, characterized in that: The number of Transformer modules in the multi-label feature extraction module is 12, and the 12 Transformer modules are connected in sequence.

3. The plant disease and severity identification method according to claim 1, characterized in that: The first multi-layer perceptron is composed of a stack of linear layers, GELU activation functions, and Dropout.

4. The plant disease and severity identification method according to claim 1, characterized in that: The S3 specifically includes the following steps: S31. Randomly select a batch of crop disease images with multiple labels from the training set and input them into the multi-label feature extraction module; S32. The image block embedding module within the multi-label feature extraction module divides the multi-label crop disease image into image blocks in pixel space, and superimposes a trainable parameter on the image blocks as a position code. The image blocks with the superimposed position code are then flattened and input into several Transformer modules for global feature extraction. The extracted features are then input into a convolutional layer for local feature extraction to obtain features with spatial information of the disease. S33, input the features with disease space information into the information fusion communication module, and use the learnable parameter matrix 、 、 Get the first query matrix , first bond matrix and the first value matrix , calculate the first query matrix and the first bond matrix The similarities between them are combined through the first multi-head self-attention mechanism, and finally the semantic features of the disease are obtained through the first multi-layer perceptron transformation; S34: Input the features with disease spatial information obtained in S32 and the disease semantic features obtained in S33 into multi-scale information fusion for fusion to obtain the fusion result. ; S35, fusion results Input to the multi-label prediction decoding module and combine it with the initialized feature encoding in the multi-label prediction decoding module , and obtain the multi-label classification results.

5. The method for identifying plant diseases and their severity according to claim 4, characterized in that: The S32 is expressed by the formula as follows: ； in, Represents the features containing the spatial information of the disease, represents the convolution operation of the convolution layer, Represents the Transformer module, Represents crop disease data.

6. The method for identifying plant diseases and their severity according to claim 5, characterized in that: The S33 is specifically expressed by the formula as follows: ； ； ； ； ； ； in, Represents the input sequence, i.e., the characteristics of the disease spatial information; 、 、 The first query matrix The learnable parameter matrix and the first bond matrix The learnable parameter matrix and the first value matrix The learnable parameter matrix of Represents a single-head attention mechanism; Indicates normalization processing; Represents the first bond matrix The transposed matrix of Represents the first bond matrix Dimensions; Represents a splicing operation; 、 Represent the i-th self-attention and the n-th self-attention respectively; represents the learnable mapping matrix; Represents the first multi-head self-attention mechanism; 、 and Represent the first query matrix The i-th learnable parameter matrix, the first bond matrix The i-th learnable parameter matrix and the first value matrix The i-th learnable parameter matrix of .

7. The method for identifying plant diseases and their severity according to claim 6, characterized in that: The S35 specifically includes the following steps: S351, using the adaptive initialization feature encoding stored in the multi-label prediction decoding module As the input of the third multi-head self-attention mechanism, through the learnable matrix 、 、 Get the third query matrix , the third bond matrix and the third value matrix ; Calculate the third query matrix and the third bond matrix The similarity is combined through the third multi-head self-attention mechanism to obtain the output result of the third multi-head self-attention mechanism ; S352, then the result obtained by the third multi-head attention mechanism Input into formula (1) to obtain the second query matrix , formula (1) is as follows: （1） in, represents the second query matrix The learnable parameter matrix of S353, the fusion result obtained in S34 Input into formula (2) and formula (3) to obtain the second bond matrix and the second value matrix ; Formula (2) and Formula (3) are as follows: （2） （3） in, Represents the second bond matrix The learnable parameter matrix of Represents the second value matrix The learnable parameter matrix of The second query matrix , the second bond matrix and the second value matrix Input into formula (4) to formula (6) to obtain the output result of the second multi-head attention mechanism: （4） （5） （6） in, represents the second multi-head attention mechanism; Represents the second bond matrix The transposed matrix of 、 and Represent the second query matrix The i-th learnable parameter matrix, the second key matrix The i-th learnable parameter matrix and the second value matrix The i-th learnable parameter matrix; S354. Input the output result of the second multi-head self-attention mechanism into the second multi-layer perceptron to obtain a multi-label classification result.

8. The method for identifying plant diseases and their severity according to claim 7, characterized in that: The S351 is expressed using the following formula: ； ； ； ； ； ； in, 、 、 Represent the third query matrix The learnable parameter matrix and the third bond matrix The learnable parameter matrix and the third value matrix The learnable parameter matrix of 、 、 Represent the third query matrix The i-th learnable parameter matrix, the third key matrix The i-th learnable parameter matrix, the third value matrix The i-th learnable parameter matrix; Represents the third multi-head self-attention mechanism; Represents the third bond matrix The transposed matrix of .

Images