Plant disease and severity identification method based on multi-label strategy

Abstract

The invention relates to the field of agricultural information, and discloses a plant disease and severity identification method based on a multi-label strategy, and the method comprises the steps: 1, building a plant disease data set, and carrying out the multi-dimensional label labeling of each sample in the data set; 2, counting the occurrence frequency of the label combination, and generating a label co-occurrence matrix C; 3, constructing a multi-label embedded network based on the label co-occurrence matrix C; 4, constructing a label association model by using the graph neural network; 5, constructing a multi-task branch network; step 6, constructing a multi-level step-by-step refining classification model; and step 7, in the model training process, performing hyper-parameter tuning based on Bayesian optimization, and dynamically adjusting task loss weight to improve recognition performance. According to the method, the problems of multi-task learning, complex label association, multi-level detailed classification, edge deployment and the like in multi-label plant disease identification are effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of agricultural information technology, and in particular to a plant disease and severity identification method based on a multi-label strategy. Background Art

[0002] In modern agricultural production, early detection and precise management of plant diseases are crucial for improving crop yield and quality and reducing pesticide use. However, plant diseases are numerous and their symptoms are complex. Different diseases can manifest in various forms at different growth stages, and the same disease can also exhibit varying degrees of symptoms in different environments and host plants. This complexity poses significant challenges to traditional manual detection, which not only requires specialized knowledge but is also costly and inefficient to implement across large areas of farmland. Furthermore, ensuring the accuracy and consistency of detection results is difficult. Therefore, automated plant disease identification based on computer vision and deep learning has gradually become a key research direction in intelligent agriculture, providing new technical means for early disease diagnosis and precise prevention and control.

[0003] With the development of deep learning and image recognition technology, by building large-scale disease image datasets and training them with deep learning models, effective features can be extracted from plant disease images to achieve automatic identification of disease types. Although existing deep learning models have achieved good results in single disease identification, plant disease identification tasks still face some significant technical bottlenecks and challenges:

[0004] Challenges of multi-label disease recognition

[0005] Plant diseases often manifest as multiple diseases occurring simultaneously with varying degrees of severity, resulting in a complex combination of disease type and severity labels. Existing recognition models can typically only handle single disease labels, making it difficult to effectively address complex scenarios where multiple diseases coexist. In multi-label disease recognition, disease type and severity often have complex correlations and interactions. Simple label combinations or independent label learning methods cannot fully express the interdependence between diseases, resulting in insufficient recognition accuracy and generalization capabilities in multi-label scenarios. In actual agricultural production, this deficiency limits the application effectiveness of automatic recognition systems, making it difficult to provide accurate judgments on disease type and severity.

[0006] The need for hierarchical identification of disease types and severity

[0007] In the actual process of disease prevention and control, accurately identifying the type of disease is only the first step; further determining the severity of the disease is crucial for developing precise prevention and control measures. Different disease types and severities require different treatment plans. For example, mild diseases may only require monitoring, while severe diseases may require immediate action. However, most existing models use a flat, single-level classification structure and lack the ability to refine multi-level classification. This makes it impossible to simultaneously identify both the type and severity of the disease. As a result, the model's predictions fail to fully reflect the actual disease situation and cannot provide highly accurate disease prevention and control information for agricultural production.

[0008] Dynamic tuning and parameter balance issues in multi-task learning

[0009] The performance of deep learning models depends heavily on the proper configuration of hyperparameters, especially in multi-task learning scenarios where multiple tasks are processed simultaneously. Disease type and severity classification tasks have varying demands on model resources. Achieving a dynamic balance between tasks and improving overall optimization results in multi-task learning is a key technical challenge. Traditional hyperparameter configuration methods are typically static and manually set, lacking a mechanism for dynamically adjusting hyperparameters based on task requirements. This can easily lead to problems such as unbalanced learning between tasks and insufficient training. This static configuration model makes it difficult for the model to adaptively optimize various tasks during training, limiting the effectiveness of multi-task learning in disease identification tasks.

[0010] Therefore, the present invention proposes a plant disease and severity identification method based on a multi-label strategy to address the shortcomings of the existing technology. Summary of the Invention

[0011] In response to the shortcomings of the existing technology, the present invention provides a plant disease and severity identification method based on a multi-label strategy, which effectively solves the difficulties in multi-task learning, complex label association, multi-level refined classification and edge deployment in multi-label plant disease identification.

[0012] To achieve the above objectives, the present invention is implemented through the following technical solutions: a plant disease and severity identification method based on a multi-label strategy, comprising the following steps:

[0013] Step 1: Create a plant disease dataset and perform multi-dimensional labeling on each sample in the dataset. The label includes the disease type label and the severity label, and generate a combined label matrix Y(x) = {(L i ,S j )}, where L i The disease type label and severity label are used to represent the combined relationship between the type and severity of plant diseases;

[0014] Step 2: Count the occurrence frequencies of the label combinations and generate a label co-occurrence matrix C. The co-occurrence matrix is ​​used to represent the correlation between the disease type label and the severity label, where the matrix element C ij Indicates the disease type label L i and severity label S j The joint occurrence probability of

[0015] Step 3: Based on the label co-occurrence matrix C, a multi-label embedding network is constructed to map the combination of disease type label and severity label into a high-dimensional embedding space to obtain the embedding vector of each combination label.

[0016] Step 4: Use graph neural networks to build a label association model. Based on the label co-occurrence matrix, iteratively update the embedding vectors of disease type labels and severity labels to capture the nonlinear association between disease type labels and severity labels.

[0017] Step 5: Construct a multi-task branch network, which includes a shared feature layer, a disease type branch, and a severity branch. The shared feature layer is used to extract global features of plant disease images. The global features are input into the disease type branch and the severity branch respectively to generate corresponding disease type prediction results and severity prediction results.

[0018] Step 6: Construct a multi-level and step-by-step classification model to identify the type and severity of plant diseases layer by layer through a two-level classification model;

[0019] Step 7: During model training, perform hyperparameter tuning based on Bayesian optimization and dynamically adjust task loss weights to improve recognition performance.

[0020] Step 8: Compress the trained model and deploy it to edge devices to achieve real-time inference for plant disease identification.

[0021] Preferably, in step 2, the method for generating the label co-occurrence matrix C includes determining the matrix element C based on the combined frequency of different disease type labels and severity labels. ij The value of the matrix element C ij The calculation method is:

[0022]

[0023] Among them, C ij Indicates the disease type label L i and severity label S j The joint occurrence probability of is used to establish the statistical association relationship between multiple labels.

[0024] Preferably, in step 3, the multi-label embedding network maps each disease type label and severity label combination into a high-dimensional embedding vector through an embedding layer. The mapping formula of the embedding layer is:

[0025]

[0026] Among them, the embedding vector As the feature input of the disease label combination, it retains the semantic information of the disease type label and severity label, and provides input features for the subsequent label association model.

[0027] Preferably, in step 4, the graph neural network performs feature propagation and update through the label graph G = (V, E) generated by the label co-occurrence matrix to capture the relationship between label nodes. The update process of the label node feature is implemented by the following formula:

[0028]

[0029] in, is the feature representation of node v in the kth layer, represents the set of neighbor nodes of node v, deg(v) represents the degree of node v, W (k) is the weight matrix of the kth layer, which is used to propagate the embedded features of nodes in the graph neural network.

[0030] Preferably, in step 5, the total loss function of the multi-task branch network is Loss function branched by disease type and the loss function of the severity branch The weighted combination of is defined as:

[0031]

[0032] Among them, λ type and λ severity are the weight coefficients of the disease type task and the severity task, respectively, which control the learning proportion of the multi-task branch network between different tasks to achieve collaborative learning of disease type labels and severity labels.

[0033] Preferably, the loss function of the disease type branch is The cross entropy loss is used, which is defined as:

[0034]

[0035] The loss function of the severity branch is the weighted cross entropy loss, defined as:

[0036]

[0037] Among them, β j Represents the weights of labels of different severity levels, which are used to adjust the learning difficulty of different severity levels during training.

[0038] Preferably, in step 6, a multi-level and gradually refined classification model is constructed, and the types and severity of plant diseases are gradually refined and identified through a two-level classification model. The specific steps are as follows:

[0039] The global features of the plant disease images are extracted through the shared feature layer, and the shared features are input into the disease type branch and the severity branch;

[0040] The disease type branch is trained using the cross entropy loss function, and the severity branch is trained using the weighted cross entropy loss function;

[0041] The weight β of the weighted cross-partial loss function j The learning for balancing labels of different severity is defined as:

[0042]

[0043] Among them, p j Represents the model's response to severity label S j The predicted probability, β j is the weight of the label.

[0044] Preferably, in the step of dynamically adjusting task loss, the dynamic double adjustment coefficient λ dynamic The calculation formula is:

[0045]

[0046] in, is the task loss weight after the t+1th iteration, and η is the adjustment step size.

[0047] Preferably, deploying the trained model to the edge device includes the following steps:

[0048] Compress the trained model, including pruning, quantization, and knowledge distillation, to reduce the computational complexity of the model;

[0049] Deploy the compressed model to the edge device and implement real-time inference through the inference acceleration engine.

[0050] Preferably, a plant disease and severity identification device based on a multi-label strategy is characterized by comprising:

[0051] A data acquisition module is used to collect plant disease image data and pre-process the data to generate a multi-dimensional label matrix with disease type labels and severity labels;

[0052] A multi-label embedding module, connected to the data acquisition module, is used to map the disease type label and severity label combination into a high-dimensional embedding vector to form a multi-label feature representation;

[0053] A label association modeling module, connected to the multi-label embedding module, is used to construct a label graph based on the label co-occurrence matrix and perform association modeling on the label embedding vectors through a graph neural network to capture the relationship between different disease type labels and severity labels;

[0054] a multi-task branch module, connected to the label association modeling module, for predicting the disease type and severity based on the global features of the plant disease image extracted by the shared feature layer, and generating a disease type recognition result and a severity classification result;

[0055] A multi-level classification module, connected to the multi-task branch module, is used to further refine the classification of disease severity based on the preliminary classification results of disease types, forming a layer-by-layer disease classification system;

[0056] A dynamic optimization module, connected to the multi-task branch module and the multi-level classification module, for optimizing the model parameters in the multi-task branch module and the multi-level classification module through an automatic tuning strategy to improve recognition accuracy and model convergence speed;

[0057] The edge computing module is connected to the multi-task branch module and is used to deploy the compressed model to the edge device and perform real-time recognition of plant disease images through the inference acceleration engine.

[0058] The present invention provides a method for identifying plant diseases and their severity based on a multi-label strategy. It has the following beneficial effects:

[0059] 1. The present invention maps disease type labels and severity labels to a high-dimensional embedding space by adopting multi-label embedding and label association modeling technology based on graph neural networks, and constructs a label graph structure to capture the potential association between disease type and severity labels. This technical solution achieves the technical effect of effectively expressing the semantic relationship of disease labels and deeply exploring the nonlinear association between labels, so that the model can more accurately identify the type of disease and its severity. Compared with the label combination or independent label learning solutions adopted in the prior art, the present invention solves the problem of being unable to capture complex dependencies between labels through label association modeling, improves the accuracy and robustness of multi-label learning, and especially ensures the stability and recognition effect of the model in complex multi-label scenarios.

[0060] 2. The present invention innovatively adopts a collaborative solution of multi-task branch network and multi-level step-by-step refinement classification model, extracts global features of plant disease images through shared feature layers, and performs deep learning on independent branches on disease type and severity tasks, thereby realizing coarse classification of disease types and refined classification of severity through a two-level classification structure. This technical solution achieves the technical effect of simultaneously optimizing the classification of disease types and severity in the same model, and can ensure the improvement of the overall accuracy of disease type and severity identification in multi-label scenarios. Compared with the existing solutions of single classifiers or lack of layer-by-layer refinement classification design, the technical solution of the present invention solves the problems of reduced classification accuracy and label confusion caused by task coupling, so that the classification of disease types and severity is optimized layer by layer, achieving a dual improvement in classification accuracy and refinement capability.

[0061] 3. The present invention introduces a strategy for automatic tuning of hyperparameters and dynamic adjustment of task loss weights based on Bayesian optimization, which enables the model to adjust key hyperparameters such as learning rate and task weight in real time during training, thereby optimizing the balance and stability of multiple tasks. This technical solution achieves the technical effect of adaptively optimizing the learning rate of each task during training, so that the model can converge faster and achieve a learning balance between the tasks of identifying the type and severity of the disease. Compared with the solution of statically configuring hyperparameters and fixing task weights in the prior art, the technical solution of the present invention effectively solves the problems of learning imbalance and local optimality caused by the lack of dynamic adjustment in traditional methods, improves the convergence efficiency and training stability of the multi-task model, and ensures a better synergistic effect in the multi-task learning of disease types and severity.

[0062] 4. The present invention uses compression technologies such as model pruning, quantization, and knowledge distillation, and deploys them on edge devices in combination with an inference acceleration engine, which significantly reduces the computational complexity and storage requirements of the model, enabling the model to run efficiently and perform real-time reasoning on resource-constrained edge devices. This technical solution achieves the technical effect of rapid response to disease identification in agricultural sites, enabling the identification results of disease types and severity to be presented in real time with low latency. Compared to the existing solutions of deploying models in the cloud or without compression, the present invention effectively solves the problems of limited computing resources and high network latency of edge devices through local edge deployment, and realizes fast and low-power disease identification applications in real-time monitoring scenarios such as farmland or greenhouses, providing more timely and accurate technical support for on-site disease management. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Flowchart of the method for plant disease and severity identification based on a multi-tag strategy. DETAILED DESCRIPTION

[0064] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0065] Example:

[0066] Please see the attached Figure 1 The embodiment of the present invention provides a method for identifying plant diseases and their severity based on a multi-label strategy, comprising the following steps:

[0067] Step 1: Create a plant disease dataset and annotate each sample in the dataset with multi-dimensional labels. The labels include disease type labels and severity labels. Generate a combined label matrix Y(x) = {(L i ,S j )}, where L i The disease type label and severity label are used to represent the combined relationship between the type and severity of plant diseases;

[0068] First, accurately identifying the type and severity of plant diseases is a key technology for achieving precision disease prevention and control in agriculture. Step S1 in this invention provides the data foundation for subsequent multi-label learning. By constructing a plant disease dataset and performing multi-dimensional labeling, it lays the information foundation for the joint identification of disease type and severity. The core of this step lies in not only acquiring diverse disease image data but also labeling each image sample with a disease type label and severity label, thereby generating a disease combination label matrix that reflects the complex relationship between disease type and severity.

[0069] In this embodiment, the specific implementation of step S1 includes the following key technical steps:

[0070] Dataset construction

[0071] In this example, the plant disease dataset was constructed by collecting and organizing image samples of various plant diseases. The dataset contains a large number of images of plant diseases, representing various plant disease types (such as leaf spot, downy mildew, and rust) and varying degrees of severity. The image sample data can be sourced from actual disease photos collected in the field or from publicly available disease datasets to ensure dataset diversity and coverage.

[0072] Data preprocessing

[0073] To ensure the consistency and quality of the image samples, this embodiment performs standardization on the collected image data. The standardization process includes image resizing, color space conversion, and denoising. Standardized image data eliminates impurities and noise in the samples, making the dataset more representative and robust.

[0074] Multi-dimensional label annotation

[0075] In this embodiment, a multi-dimensional labeling system is constructed to achieve joint recognition of disease type and severity. Specifically, the label of each image sample consists of two parts: the disease type label and the severity label, forming a two-dimensional label combination. Assume that the dataset contains m disease type labels (L1, L2, ..., L m ) and nnn severity labels (S1, S2, ..., S n ), then the label of each image sample x can be expressed as:

[0076] Y(x)={(L i ,S j )|i∈[1,m],j∈[1,n]}

[0077] Among them, the label L i represents the type of disease, such as "leaf spot", "downy mildew", etc., while S j Severity labels such as "mild," "moderate," and "severe" ensure that each sample describes both the type and severity of the disease.

[0078] Combined label matrix generation

[0079] In order to facilitate the model to handle the joint relationship between disease type and severity in the multi-label learning process, a combined label matrix is ​​generated in this embodiment. Each element y in the combined label matrix ij Indicates the disease type label L i and severity label S j The combination of labels constitutes the matrix Y:

[0080] Y=[y ij ] m×n

[0081] Among them, y ij ∈{0,1}, when y ij =1 indicates that the sample contains disease type L i And the severity is S j ; When y ij = 0 means that the sample does not contain the combination of disease type and severity. This combined label matrix provides the basic structure for the multi-label learning of the subsequent model, enabling the model to learn the relationship between disease type and severity from the multidimensional feature space.

[0082] Tag statistical analysis

[0083] To further understand the distribution of disease types and severity in the dataset, this example performs a statistical analysis on the combined labels and calculates the frequency of occurrence of each combination of disease type and severity. ij The frequency of occurrence in the data set is the frequency (L i ,S j ), we statistically generate label distribution data to support subsequent label association modeling and co-occurrence matrix construction. This step ensures a reasonable distribution of disease type and severity labels in the dataset, preventing the model from being affected by label imbalance during training.

[0084] Through the specific implementation of the above step S1, the present invention can establish a multi-dimensionally labeled dataset that combines the combined relationship between plant disease types and severity, providing high-quality input data for subsequent multi-label learning.

[0085] Step 2: Count the occurrence frequencies of the label combinations and generate a label co-occurrence matrix C. The co-occurrence matrix is ​​used to represent the correlation between the disease type label and the severity label, where the matrix element C ij Indicates the disease type label L i and severity label S j The joint occurrence probability of

[0086] To enable the joint learning of disease types and severities in a multi-label plant disease recognition model, in step 2, the present invention generates a label co-occurrence matrix by statistically analyzing the combination of multi-dimensional labels. This co-occurrence matrix quantifies the correlation between disease type labels and severity labels, providing co-occurrence information between disease type and severity, enabling the model to better understand and utilize these label relationships during training. By constructing this label co-occurrence matrix, the model can capture the joint distribution characteristics of plant disease types and severities, thereby enhancing the model's learning capabilities in multi-label scenarios.

[0087] In this embodiment, the specific implementation of step 2 includes the following key steps:

[0088] Tag combination frequency statistics

[0089] In the embodiment, first, the multi-dimensional label matrix Y(x) generated in step 1 is i ,S j )} to perform statistics and calculate the label L of each disease type in the data set i With severity label S jSpecifically, for each pair of combined labels (L i ,S j ), count the number of times it appears in the data set, recorded as frequency (L i ,S j ). This statistical process is used to quantify the correlation between disease types and severity, providing basic data for the construction of the label co-occurrence matrix.

[0090] Tag co-occurrence matrix generation

[0091] Based on the statistical results of the label combination frequency, a label co-occurrence matrix C is generated in this embodiment. The label co-occurrence matrix is ​​an m×n matrix, where m is the number of disease type labels and n is the number of severity labels. Each element in the matrix C ij Indicates the disease type label L i and severity label S j The joint occurrence probability in the data set. The specific calculation formula is as follows:

[0092]

[0093] The denominator is the total frequency of all disease type labels and severity labels in the dataset. Through this formula, the element value C of the label co-occurrence matrix ij It is normalized into a joint probability value to reflect the association between the disease type label and the severity label.

[0094] Characteristic Analysis of Co-occurrence Matrix

[0095] In this embodiment, the distribution characteristics of disease type and severity labels in the data set can be further analyzed through the label co-occurrence matrix C. The diagonal elements of the label co-occurrence matrix (i.e., C ii ) represents the strength of association between the same disease type label and different severity labels, while the off-diagonal elements show the co-occurrence associations between different disease type labels. This analysis helps us understand which disease type labels and severity labels co-occur more frequently in the dataset and provides guidance for subsequent label association modeling.

[0096] Label weight initialization

[0097] In this embodiment, according to the statistical results of the label co-occurrence matrix, a corresponding weight is set for each combination label to perform weighted processing in the subsequent loss function. The calculation formula for weight initialization is as follows:

[0098]

[0099] Among them, α ij Is the disease type label L iand severity label S j The weight of the combined labels is used to dynamically adjust the label learning ratio during training. For low-frequency label combinations, the weight value is relatively high to increase the model's attention to these label combinations during training.

[0100] Through step 2 of this embodiment, the generated label co-occurrence matrix C can provide the model with co-occurrence information between disease types and severity, and provide support for subsequent multi-label embedding and association modeling, ensuring that the model can better learn the dependency relationship between labels in a multi-label scenario.

[0101] Step 3: Based on the label co-occurrence matrix C, a multi-label embedding network is constructed to map the combination of disease type label and severity label into a high-dimensional embedding space to obtain the embedding vector of each combination label.

[0102] In a multi-label plant disease recognition model, to fully understand and utilize the combined information of disease type and severity labels, the present invention introduces a multi-label embedding network in step three. This network maps the combination of disease type and severity labels into a high-dimensional embedding vector, constructing a multi-label feature space. This embedding vector, used as input feature for subsequent models, not only preserves the semantic information of the labels but also provides structured input for subsequent label association modeling, enabling the model to learn the complex relationship between disease type and severity.

[0103] Label embedding network construction

[0104] In this embodiment, a multi-label embedding network is constructed using the label combination information provided by the label co-occurrence matrix C generated in step 2. The multi-label embedding network maps the combination of disease type label and severity label to a high-dimensional embedding space through the embedding layer, and generates an embedding vector for each combination label. The mapping relationship of the embedding layer is expressed as:

[0105]

[0106] in, Label the disease type L i and severity label S j The representation of the combined labels in the embedding space is shown in Figure 2, where d represents the embedding dimension. The embedding dimension d is selected based on the number and complexity of the disease type and severity labels to ensure that the embedding vector can fully express the semantic information of the labels.

[0107] Embedding vector initialization

[0108] In order to ensure the convergence and training effect of the model, the embedding vector Initialized. The generation of the initial embedding vector is completed by random initialization and double correction is performed in combination with the label. Specifically, based on the label weight α in step 2 ij , the embedding vectors of low-frequency label combinations are given a larger initialization value range so that the model can pay attention to these low-frequency combinations faster and improve label balance.

[0109] Embedding layer training

[0110] In this embodiment, the label embedding vector During the network training process, it is constantly adjusted to adapt to the specific recognition task. The model updates the embedding layer weights through the backpropagation algorithm, so that the embedding vector can not only represent the characteristic information of each combined label, but also optimize the label representation according to the actual distribution of the sample. During the training process, the weight update formula of the embedding layer is:

[0111]

[0112] in, is the combined label (L i ,S j ), η is the learning rate, is the total loss function of the model. By continuously updating the embedding vector, the model can learn the potential association between different disease types and severity levels.

[0113] Embedding vector normalization

[0114] To prevent the embedding vectors from diverging in high-dimensional space, in this embodiment, all embedding vectors Normalization is performed. The normalized embedding vector satisfies:

[0115]

[0116] Through normalization, the embedding vectors of all combined labels have the same vector modulus, avoiding numerical instability in subsequent label association modeling and improving the training stability of the model.

[0117] The multi-label embedding vector generated by step 3 in this embodiment is This effectively characterizes the combined features of disease type and severity labels, providing the input foundation for label association modeling in subsequent steps. The embedded network empowers the model with stronger feature representation capabilities in multi-label scenarios, laying the foundation for the joint identification of plant disease type and severity.

[0118] Step 4: Use graph neural networks to build a label association model. Based on the label co-occurrence matrix, iteratively update the embedding vectors of disease type labels and severity labels to capture the nonlinear association between disease type labels and severity labels.

[0119] In the multi-label recognition of plant disease types and severity, to fully capture the potential associations between disease type and severity labels, step four introduces a label association modeling method based on a graph neural network (GNN). By constructing the combined labels of disease type and severity labels into a graph structure and using the GNN to propagate and update information between label nodes, the model can learn the nonlinear dependencies between multi-label combinations. The association information generated in step four can effectively help the model improve its recognition capabilities in multi-label scenarios and strengthen the model's joint learning of disease type and severity labels.

[0120] In this embodiment, the specific implementation of step 4 includes the following key technical steps:

[0121] Label graph construction

[0122] In this embodiment, a multi-label graph structure is constructed using the label combination information provided by the label co-occurrence matrix C generated in step 2.

[0123] The node set V of the label graph G = (V, E) contains all the combination labels of disease type labels and severity labels, and the edge set E represents the co-occurrence relationship between the combination labels. The edge weight between any two nodes v and u in the graph is determined by the corresponding element C in the label co-occurrence matrix. vu Determines the co-occurrence strength of the combined tags.

[0124] Feature initialization of label nodes

[0125] After the label graph is constructed, this embodiment initializes the features of the label node. The initial features of the label node are the embedding vectors generated in step 3. That is, the feature vector of each node corresponds to the representation of the combined label in the embedding space. The purpose of node feature initialization is to provide the initial features of the label for the subsequent graph neural network, so that the graph neural network can effectively transfer and update information between label nodes.

[0126] Graph Neural Network Construction

[0127] In this embodiment, a graph neural network (GNN) model is constructed to propagate and update feature information between label nodes. The GNN uses a multi-layer information propagation mechanism to transfer features and perform nonlinear transformations between label nodes, learning the associations between different label combinations. The update formula for the GNN is as follows:

[0128]

[0129] in, represents the feature representation of the label node v in the k+1th layer, represents the set of neighbor nodes of node v, deg(v) represents the degree of node v, W (k) is the weight matrix of the kth layer, and σ is the activation function. This formula generates the updated label node features of the first layer through information transmission and feature aggregation of neighbor nodes.

[0130] Multi-layer feature update

[0131] The graph neural network in this embodiment employs a multi-layered architecture, with each layer sequentially updating the features of labeled nodes, gradually enhancing the representation of associations between labeled nodes. At each layer, the features of a labeled node are updated based on the features of its neighboring nodes, enabling labeled nodes to aggregate label information from multi-hop neighbors. Through these multi-layer feature updates, the features of each labeled node gradually incorporate more contextual information, enhancing the representation of nonlinear associations between labels.

[0132] Label feature normalization

[0133] To ensure the training stability of the graph neural network and the controllability of the eigenvalues, this embodiment normalizes the features of the label nodes after each layer of feature update. The normalized label node features satisfy:

[0134]

[0135] Through normalization, it is ensured that the feature vectors of all label nodes will not cause numerical instability due to transfer and aggregation during the update process, thereby improving the convergence of the model.

[0136] In step 4 of this example, a graph neural network is used to model associations between label nodes, enabling the model to learn the complex relationships between disease type and severity labels from the state graph structure. The label association information generated in this step not only effectively enriches the semantic information of the label features but also provides a reliable foundation for the model's multi-label learning.

[0137] Step 5: Construct a multi-task branch network, which includes a shared feature layer, a disease type branch, and a severity branch. The shared feature layer is used to extract global features of plant disease images. The global features are input into the disease type branch and the severity branch respectively to generate corresponding disease type prediction results and severity prediction results.

[0138] In the multi-label plant disease recognition model, a multi-task branch network is introduced in step five to achieve joint prediction of disease type and severity. By designating task branches for disease type and severity within the model structure, the model can separately identify disease type and classify severity based on shared features, thereby improving recognition accuracy. Furthermore, through multi-task collaborative learning, information sharing between disease type and severity is achieved, further enhancing the model's generalization and prediction performance.

[0139] In this embodiment, the specific implementation of step S5 includes the following key technical steps:

[0140] The overall architecture of the multi-task branch network

[0141] In this embodiment, the multi-task branch network includes a shared feature layer and two task branches: a disease type branch and a severity branch. The shared feature layer is used to extract global features from the input image, providing input features for subsequent task branches. The shared feature layer extracts global information from plant disease images, allowing the model to utilize basic image features in different task branches.

[0142] Feature extraction of shared feature layers

[0143] In the shared feature layer, this embodiment uses a convolutional neural network (CNN) architecture to extract features from the input plant disease image. The extracted global features, denoted as F, are used in the subsequent disease type and severity branches. The shared feature layer design not only saves computing resources but also enables feature sharing between disease type and severity.

[0144] Structure and loss function design of disease category branches

[0145] The disease type branch is used to classify the disease type. In this embodiment, the disease type branch receives the output F of the shared feature layer and performs further processing to generate a prediction result of the disease type label. The loss function of the disease type branch is: The cross entropy loss is used, which is defined as follows:

[0146]

[0147] Among them, m represents the number of labels of disease types, y i Indicator variable representing the true disease type label, p i Represents the probability of disease type predicted by the model. Through cross entropy loss, the model can effectively learn the classification characteristics of disease types.

[0148] Structure and loss function design of severity branch

[0149] The severity branch is used to predict the severity of the disease. In this embodiment, the severity branch also receives the output F of the shared feature layer and generates a prediction result of the severity label based on the image features. The loss function of the severity branch is is a weighted cross entropy loss used to balance the imbalanced distribution of labels of different severity in the dataset and is defined as follows:

[0150]

[0151] Where n represents the number of severity labels, β j is the weight coefficient of the severity label, y j Indicator variable representing the true severity label, p j Represents the severity probability predicted by the model. The weighted cross entropy loss can adjust the learning intensity of different severity labels to ensure that low-frequency labels can receive sufficient attention.

[0152] Definition and optimization of multi-task loss function

[0153] In order to achieve collaborative learning of disease type and severity tasks, a multi-task total loss function is designed in this embodiment. By the loss of disease species and severity of losses Weighted summation is performed to achieve joint optimization of the two tasks. The multi-task total loss function is defined as follows:

[0154]

[0155] Among them, λ type and λ severity is a weight coefficient used to control the learning ratio of the disease type task and the severity task. By optimizing the multi-task total loss function, the model can achieve a learning balance between different tasks and achieve joint prediction of disease type and severity.

[0156] Through step 5 of this example, the constructed multi-task branch network can learn the characteristics of disease type and severity separately based on shared features, ensuring the accuracy and stability of multi-label recognition. The multi-task learning design enables the model to achieve synergistic improvements in predicting disease type and severity, providing more accurate label prediction results for subsequent steps.

[0157] Step 6: Construct a multi-level and step-by-step classification model to identify the type and severity of plant diseases layer by layer through a two-level classification model;

[0158] To further improve the prediction accuracy of disease type and severity in the multi-label plant disease recognition model, step six introduces a multi-level, stepwise refinement classification model. This two-level classification structure allows the model to first perform a preliminary classification of disease type and then refine the severity classification based on this initial classification. This allows the model to better distinguish between multiple disease types and their varying severity levels. This two-level classification model reduces confusion between different disease types and severity levels, ensuring the model's predictive effectiveness in complex labeling scenarios.

[0159] In this embodiment, the specific implementation of step six includes the following key technical steps:

[0160] First-level classification model: coarse classification of disease types

[0161] In this embodiment, a first-level classification model is first constructed to perform a rough classification of the disease types. The input of the first-level model is the global feature F obtained from the shared feature layer or the multi-task branch network. The model uses a simple fully connected network or convolutional layer structure to quickly identify the type of plant disease. The output of the first-level classification model is the preliminary prediction result of the disease type label. It provides basic information for subsequent refinement of severity classification.

[0162] First-level classification loss function

[0163] In the first level classification, this embodiment adopts the cross entropy loss function To optimize the rough classification of disease types.

[0164] as follows:

[0165]

[0166] Among them, m represents the number of disease type labels, y i is the real disease type label, p i is the model's predicted probability for the disease type. Cross-entropy loss can effectively measure the accuracy of disease classification and guide the first-level model to quickly converge to the optimal classification effect of the disease type.

[0167] Second-level classification model: detailed classification of severity

[0168] After obtaining the first-level classification results, this embodiment uses the second-level classification model to further classify the severity of the disease. The second-level model is based on the first-level disease type prediction results. Combine it with the shared feature F or the output feature of the disease type branch to further predict the severity of the disease. The output of the severity classification is the prediction result of the severity label The model differentiates the severity more specifically based on the ethnological information of the disease.

[0169] Second-level classification loss function

[0170] In the second level classification, this embodiment adopts the weighted cross entropy loss function To optimize the classification effect of severity. Since the distribution of labels of different severity levels in the dataset may be unbalanced, weighted cross entropy can be used to balance the labels of each severity level.

[0171] The learning intensity is defined as follows:

[0172]

[0173] Where n is the number of severity labels, β j is the weight coefficient for each severity label, y j is the true severity label, p j is the model's predicted probability of severity. By using the weighted cross-entropy loss function, the model can better respond to low-frequency labels in severity classification and improve the recognition effect of low-frequency severity labels.

[0174] Layer-by-layer refinement of multi-level classification loss function

[0175] This embodiment constructs a multi-level total loss function by combining the losses of the first-level and second-level classification models. To achieve joint optimization of disease types and severity. The multi-level total loss function is defined as follows:

[0176]

[0177] Among them, λ level-1 and λ level-2 is the weight coefficient for the first- and second-level classification models, controlling the learning weights for the coarse classification of disease types and the fine classification of severity. Through the multi-level total loss function, the model achieves a learning balance between different classification levels, improving the joint recognition of disease types and severity.

[0178] The multi-layered, progressively refined classification model constructed in step 6 of this embodiment can first perform a coarse classification of disease types, then refine the classification by severity, thereby improving recognition accuracy when combining multiple disease types and varying severity levels. This layer-by-layer refinement provides the model with more refined prediction capabilities in complex labeling scenarios.

[0179] Step 7: During model training, perform hyperparameter tuning based on Bayesian optimization and dynamically adjust task loss weights to improve recognition performance.

[0180] To further optimize model performance and ensure high accuracy across diverse tasks in a multi-label plant disease type and severity recognition model, step seven introduces a dynamic optimization and hyperparameter tuning strategy. Through Bayesian optimization-based automatic hyperparameter tuning and dynamic adjustment of multi-task loss weights, the model adaptively adjusts key parameters during training, improving training convergence and balancing multi-task learning, thereby achieving joint optimization of disease type and severity recognition.

[0181] In this embodiment, the specific implementation of step seven includes the following key technical steps:

[0182] Automatic hyperparameter tuning with Bayesian optimization

[0183] In this embodiment, in order to automatically find the best hyperparameter combination suitable for the model during training, the Bayesian optimization algorithm is used for hyperparameter tuning. Bayesian optimization constructs a probabilistic model of the hyperparameters and iteratively optimizes them based on the prior distribution and observed data to find the hyperparameters that minimize the model loss function. Specifically, Bayesian optimization evaluates the combination of hyperparameters Θ in each iteration to minimize the total multi-task loss. For the goal:

[0184]

[0185] in, is the total multi-task loss function, X is the training sample, and Θ includes hyperparameters such as the learning rate, regularization parameter, and loss weight. Bayesian optimization determines a set of optimal hyperparameter combinations in the early stages of model training, providing an initial advantage for model convergence.

[0186] Dynamic adjustment of multi-task loss weights

[0187] In this embodiment, in order to achieve a dynamic balance between the disease type and severity tasks in the multi-task branch, a dynamic adjustment strategy of the loss weight is adopted. During the training process of the model, the disease type loss weight λ is adjusted according to the progress of task learning. type and severity loss weight λ severity In the initial stage, the loss weights for disease type and severity can be set to the same value. As training progresses, they are dynamically adjusted based on the convergence speed of each task to ensure that the learning effects of the two tasks are balanced. The dynamic weight adjustment formula is as follows:

[0188]

[0189] in, is the task loss weight after the t+1th iteration, and η is the adjustment step size. By dynamically adjusting the weights, the model maintains a reasonable training intensity in learning disease types and severities, avoiding overfitting of a single task that affects overall model performance.

[0190] Dynamic learning rate adjustment

[0191] This embodiment also introduces a dynamic learning rate adjustment strategy, which allows the learning rate to change dynamically during the training process. In the early stages of training, the model uses a higher learning rate to accelerate convergence; as training progresses, the learning rate is gradually reduced to improve accuracy.

[0192] The update formula of dynamic learning rate is:

[0193] η (t+1) =η (t) ×(1-decay rate)

[0194] Among them, η (t+1) The learning rate after the t+1th iteration is represented by the decay rate coefficient. Dynamically adjusting the learning rate ensures that the model converges to the optimal solution more stably in the later stages of training.

[0195] Optimization of multi-task total loss function

[0196] Under the combined effects of Bayesian optimization, dynamic loss weight adjustment, and dynamic learning rate adjustment, this embodiment ultimately optimizes the multi-task total loss function To achieve collaborative optimization of disease type and severity tasks. The multi-task total loss function is defined as follows:

[0197]

[0198] By type and λ severity By dynamically adjusting the and optimization of the learning rate, the model can achieve balanced learning on the tasks of disease type and severity, ensuring that the two tasks promote each other in joint optimization.

[0199] Through step seven of this embodiment, the model has the ability to automatically adjust hyperparameters in a multi-task environment, and realizes the coordinated optimization of disease type and severity tasks under the dynamic optimization of loss weight and learning rate, thereby improving the recognition accuracy and training stability of the model in multi-label scenarios.

[0200] Step 8: Compress the trained model and deploy it to edge devices to achieve real-time inference for plant disease identification.

[0201] In the multi-label plant disease type and severity identification model, step eight introduces model compression and edge deployment technologies to achieve efficient inference on edge devices. By performing compression operations such as pruning, quantization, and knowledge distillation on the model, the model's parameter count and computational complexity are reduced, adapting it to the computational constraints of edge devices. Subsequently, the optimized model is deployed on the edge device using an inference acceleration engine, enabling real-time disease identification and inference, thus meeting the needs of real-time agricultural field monitoring.

[0202] In this embodiment, the specific implementation of step S8 includes the following key technical steps:

[0203] Model compression

[0204] In this embodiment, to adapt the model to the computing resources of edge devices, the trained model is compressed. Compression methods include pruning, quantization, and knowledge distillation to reduce the computational complexity and storage requirements of the model.

[0205] Pruning: Pruning is a technique used to reduce redundant parameters in a model while retaining the main weight connections, thereby reducing the model size. During the pruning process, some weights are removed based on their contribution to reduce computational overhead.

[0206] Quantization: Quantization technology converts the model's floating-point weights into low-order integers (such as 8 bits), significantly reducing storage space. The quantized model can be executed more efficiently on edge devices while reducing memory and bandwidth requirements.

[0207] Knowledge distillation: Through knowledge distillation technology, the complex teacher model knowledge is transferred to the lightweight student model. Knowledge distillation loss function The definition is as follows:

[0208]

[0209] Among them, p teacher (k) represents the predicted probability of the teacher model on category k, p student (k) represents the predicted probability of the student model on category k. By optimizing this distillation loss, the student model can retain the classification ability of the teacher model while reducing the number of parameters.

[0210] Inference acceleration

[0211] To further improve the inference speed on edge devices, this embodiment uses an inference acceleration engine (such as TensorRT or MNN) to optimize the deployment of the compressed model. The inference acceleration engine improves the running efficiency of the model on specific hardware through a series of optimization technologies, such as operator fusion, memory sharing, and parallel computing. Specifically, the inference acceleration engine merges multiple operation levels in the model, and improves the inference speed and computing efficiency of edge devices by reducing the number of computing nodes and storage reads.

[0212] Model deployment to edge devices

[0213] In this embodiment, the compressed and accelerated optimized model is deployed on edge devices to achieve real-time identification of plant diseases. Edge devices typically use low-power, high-performance embedded devices (such as NVIDIA Jetson or Raspberry Pi) that can run continuously in field environments. The deployment process includes loading the optimized model onto the device and configuring the inference engine to support fast loading and execution, allowing the model to run locally without relying on cloud resources, meeting the real-time requirements of agricultural sites.

[0214] Model update mechanism

[0215] To ensure the long-term applicability and accuracy of the model in applications, this embodiment incorporates a model update mechanism. By collecting and annotating new disease data in the cloud, the model is regularly retrained and optimized, and the updated model is pushed to edge devices for replacement and update. This model update mechanism enables the model to adapt to changing disease types and severity, helping to improve system performance in diverse environments.

[0216] Through step eight of this embodiment, the trained model, through compression techniques such as pruning, quantization, and knowledge distillation, is adapted to the computing environment of edge devices. The inference acceleration engine enables efficient deployment, thus meeting the real-time monitoring needs of agricultural sites. The design of the model update mechanism ensures the long-term applicability of the model, enabling it to maintain high recognition accuracy as actual data changes.

[0217] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying plant diseases and their severity based on a multi-label strategy, characterized in that: The following steps are involved: Step 1: Create a plant disease dataset and annotate each sample in the dataset with multi-dimensional labels; Step 2: Count the occurrence frequency of tag combinations and generate a tag co-occurrence matrix C; Step 3: Based on the label co-occurrence matrix C, a multi-label embedding network is constructed to map the combination of disease type label and severity label to a high-dimensional embedding space to obtain the embedding vector of each combined label. Step 4: Use graph neural networks to build a label association model. Based on the label co-occurrence matrix, iteratively update the embedding vectors of disease type labels and severity labels to capture the nonlinear association between disease type labels and severity labels. Step 5: construct a multi-task branch network, the network including a shared feature layer, a disease type branch, and a severity branch; Step 6: Construct a multi-level and step-by-step classification model to identify the types and severity of plant diseases layer by layer through a two-level classification model; Step 7: During model training, hyperparameters are tuned based on Bayesian optimization, and task loss weights are dynamically adjusted to improve recognition performance; Step 8: Compress the trained model and deploy it to the edge device to achieve real-time inference of plant disease identification.

2. A method for identifying plant diseases and their severity based on a multi-label strategy according to claim 1, characterized in that: In the step 2, the method for generating the label co-occurrence matrix C includes determining the matrix element C based on the combination frequency of different disease type labels and severity labels. ij The value of the matrix element C ij The calculation method is: Among them, C ij Indicates the disease type label L i and severity label S j The joint occurrence probability of is used to establish the statistical association relationship between multiple labels.

3. A method for identifying plant diseases and their severity based on a multi-label strategy according to claim 1, characterized in that: In step 3, the multi-label embedding network maps each disease type label and severity label combination into a high-dimensional embedding vector through an embedding layer. The mapping formula of the embedding layer is: Among them, the embedding vector As the feature input of the disease label combination, it retains the semantic information of the disease type label and the severity label, and provides input features for the subsequent label association model.

4. A method for identifying plant diseases and severity based on a multi-label strategy according to claim 1, characterized in that: In step 4, the graph neural network performs feature propagation and update through the label graph G=(V, E) generated by the label co-occurrence matrix to capture the relationship between label nodes. The update process of the label node feature is implemented by the following formula in, is the feature representation of node v at layer k, represents the set of neighbor nodes of node v, deg(v) represents the degree of node v, W (k) is the weight matrix of the kth layer, which is used to propagate the embedded features of nodes in the graph neural network.

5. A method for identifying plant diseases and their severity based on a multi-label strategy according to claim 1, characterized in that: In step 5, the total loss function of the multi-task branch network Loss function branched by disease type and the loss function of the severity branch The weighted combination of is defined as: Among them, λ type and λ severity are the weight coefficients of the disease type task and the severity task, respectively, which control the learning proportion of the multi-task branch network between different tasks to achieve collaborative learning of disease type labels and severity labels.

6. A method for identifying plant diseases and their severity based on a multi-label strategy according to claim 5, characterized in that: The loss function of the disease type branch The cross entropy loss is used, which is defined as: The loss function of the severity branch is the weighted cross entropy loss, defined as: Among them, β j Represents the weights of labels of different severity, which are used to adjust the learning difficulty of different severity levels during training.

7. A method for identifying plant diseases and their severity based on a multi-label strategy according to claim 1, characterized in that: In step 6, a multi-level and gradually refined classification model is constructed, and the types and severity of plant diseases are gradually refined and identified through a two-level classification model. The specific steps are as follows: The global features of the plant disease images are extracted through the shared feature layer, and the shared features are input into the disease type branch and the severity branch; The disease type branch is trained using the cross entropy loss function, and the severity branch is trained using the weighted cross entropy loss function; The weight β of the weighted cross partial loss function j The learning for balancing labels of different severity is defined as: Among them, p j Represents the model's severity label S j The predicted probability, β j is the weight of the label.

8. A method for identifying plant diseases and their severity based on a multi-label strategy according to claim 1, characterized in that: In the step of dynamically adjusting task loss, the dynamic double adjustment coefficient λ dynamic The calculation formula is: in, is the task loss weight after the t+1th iteration, and η is the adjustment step size.

9. A method for identifying plant diseases and their severity based on a multi-label strategy according to claim 1, characterized in that: Deploying the trained model to the edge device includes the following steps: Compress the trained model, including pruning, quantization, and knowledge distillation, to reduce the computational complexity of the model; Deploy the compressed model to the edge device and achieve real-time reasoning through the inference acceleration engine.

10. A plant disease and severity identification device based on a multi-label strategy, applied to the method according to any one of claims 1 to 9, characterized in that: include: A data acquisition module is used to collect plant disease image data and pre-process the data to generate a multi-dimensional label matrix with disease type labels and severity labels; A multi-label embedding module, connected to the data acquisition module, is used to map the disease type label and the severity label combination into a high-dimensional embedding vector to form a multi-label feature representation; A label association modeling module, connected to the multi-label embedding module, is used to construct a label graph based on the label co-occurrence matrix, and to perform association modeling on the label embedding vectors through a graph neural network to capture the relationship between different disease type labels and severity labels; A multi-task branch module, connected to the label association modeling module, is used to predict the disease type and severity respectively according to the global features of the plant disease image extracted by the shared feature layer, and generate a disease type recognition result and a severity classification result; A multi-level classification module, connected to the multi-task branch module, is used to further refine the classification of disease severity based on the preliminary classification results of disease types, so as to form a layer-by-layer refined disease classification system; A dynamic optimization module, connected to the multi-task branch module and the multi-level classification module, for optimizing the model parameters in the multi-task branch module and the multi-level classification module through an automatic tuning strategy to improve recognition accuracy and model convergence speed; The edge computing module is connected to the multi-task branch module and is used to deploy the compressed model to the edge device and perform real-time recognition of plant disease images through the inference acceleration engine.

Images