An integrated method for long-tail disease identification and pesticide application based on disease-pesticide knowledge graph

Abstract

The present invention discloses an integrated method for long-tail disease identification and medication based on a disease pesticide knowledge graph, including: constructing a disease pesticide information database, a disease pesticide knowledge graph; building a recognition model, obtaining long-tail disease data, uniformly processing the data, one-stage model training, strengthening prototype contrast learning, and two-stage model training using class balance loss and tau Norm to adjust the long-tail disease classifier; according to the classification results obtained by the long-tail disease classifier, querying the corresponding pesticide from the disease pesticide knowledge graph. The present invention can realize accurate identification of long-tail diseases and medication recommendations.

Description

Technical Field

[0001] The present invention belongs to the field of agricultural information technology, and specifically relates to an integrated method for long-tail disease identification and medication based on a disease and pesticide knowledge graph. Background Art

[0002] As a basic industry of the national economy, agriculture is facing increasingly severe challenges, among which crop disease prevention and control is one of the key links. With climate change and the complexity of the farming environment, the various diseases suffered by crops are increasing, so it is necessary to establish an integrated technology for crop disease identification and drug use.

[0003] As a structured method for organizing and storing information in the form of a graph, knowledge graphs represent entities through nodes and the relationships between these entities through edges. They are widely used in various industries. However, the current pesticide knowledge graphs are limited to the mainstream diseases of specific crops, focusing on professional information in a single dimension, and cannot meet the requirements of integrated crop disease identification and drug use technology.

[0004] At the same time, due to the crop area and regional issues, the existing disease data has a serious long-tail phenomenon, which leads to the recognition model trained by this method to achieve high accuracy on common diseases of common crops, but when the recognition range is expanded, the overall accuracy of the recognition model will drop sharply. In this regard, the computer field adopts methods such as two-stage training, class rebalancing, and resampling, but the agricultural field has its own particularities, especially in plant disease recognition. Compared with traditional computer vision data sets, plant disease data has higher inter-class similarity. For example, wheat and rice, two crops, have a high similarity. It is undoubtedly more difficult to distinguish different diseases on these two crops on this basis. Traditional long-tail algorithms focus on balancing the quantitative gap between classes and ignore the further extraction of fine-grained features of samples. Therefore, the direct migration of computer field methods to the agricultural field is not effective. Taking BCL (balanced contrastive learning) as an example, as a classic method in the long-tail field of computers, it introduces class averaging and class completion mechanisms based on contrastive learning to prevent the tail class from lacking contrast due to insufficient samples. These improvements enable BCL to maintain the balance of feature representation on long-tail data, effectively improving the model's recognition performance for tail classes. For example, Chinese patent CN117593626A discloses a method for reducing the long-tail effect of a citrus tree leaf disease detection model. This method directly migrates the long-tail solution in the computer field to the field of disease recognition. It lacks special adjustments for tasks in the agricultural field and fails to fully capture the fine-grained features of the disease, resulting in poor disease recognition results. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide an integrated method for long-tail disease identification and medication based on a disease and pesticide knowledge graph that can achieve accurate identification of long-tail diseases and recommend medication.

[0006] The technical solution of the present invention:

[0007] The present invention provides a long-tail disease identification and pesticide application integrated method based on a disease and pesticide knowledge graph, comprising the following steps:

[0008] Step 1. Build a disease and pesticide information database:

[0009] Step 1.1 Data acquisition: Integrate websites and knowledge bases related to diseases, crops, and pesticides, obtain semi-structured and unstructured knowledge from web pages, and save them in the local database; Step 1.2 Data cleaning: Process the knowledge obtained from different data sources and unify the language;

[0010] Step 1.3 Data supplementation: Use relevant literature to supplement professional knowledge of pesticides according to their active ingredients;

[0011] Step 1.4 Data alignment: Link and align data from different sources through designated entities to build a disease and pesticide information database;

[0012] Step 2. Construction of disease and pesticide knowledge graph:

[0013] Step 2.1. Construction of the knowledge graph of pests and diseases: Complete the knowledge graph of pests and diseases ontology, including pest ontology, pesticide ontology, crop ontology and active ingredient ontology;

[0014] Step 2.2 Filling in the knowledge graph content: According to the disease pesticide knowledge graph ontology in step 2.1, extract triples from the disease pesticide information database in step 1.4, and use the graph database neo4j to complete the data filling of the disease pesticide knowledge graph;

[0015] Step 3: Identification model building:

[0016] Step 3.1 Dataset preprocessing: Obtain long-tail disease data and process the data uniformly;

[0017] Step 3.2: One-stage model training uses the ResNet32 network as the baseline network and uses cross entropy loss and weight decay to learn image features;

[0018] Step 3.3 Strengthen prototype contrastive learning: Generate category prototypes through classifier weights, calculate the cosine similarity matrix between prototypes, and use mutual exclusion constraints to minimize the similarity between category prototypes based on the similarity between prototypes. Strengthen the distance between prototypes so that contrastive learning can further learn fine-grained features in diseases.

[0019] Step 3.4 Two-stage model training: Use class balance loss and tau norm to adjust the long-tail disease classifier;

[0020] Step 4. Pesticide query:

[0021] According to the classification results obtained by the long-tail disease classifier in step 3.4, the corresponding pesticides are queried from the disease-pesticide knowledge graph constructed in step 2.

[0022] In the above-mentioned integrated method for long-tail disease identification and pesticide application based on disease-pesticide knowledge graph, in step 1.1, a web crawler script is written in Python language to save semi-structured and unstructured knowledge in a local database.

[0023] The above-mentioned integrated method for long-tail disease identification and medication based on the disease-pesticide knowledge graph, wherein the pesticide supplement professional knowledge in step 1.3 is to supplement the structural category, specific type and mode of action of fungicide pesticides; to supplement the action mechanism, structural category, action property and mode of action of herbicide pesticides; and to supplement the mode of action and structural information of herbicide pesticides.

[0024] The above-mentioned integrated method for long-tail disease identification and medication based on disease and pesticide knowledge graph, wherein the data obtained from each data source in step 1.4 is aligned by aligning the data of the pesticide information registration network with the names of pests and diseases in the agricultural pest and disease image and text database and the names of pests and diseases in the garden network through the control objects, aligning the pest and disease entities in the garden network and the agricultural pest and disease image and text database to ensure the uniqueness of the entities; aligning the crop types in the pesticide information registration network with the Chinese names in Baidu Encyclopedia; aligning the active ingredient names in the pesticide information registration network with the ISO common names in the pesticide name network and the IUPACName or CAS Name in PubChem.

[0025] Compared with the prior art, the present invention has obvious beneficial effects. It can be seen from the above technical solutions that a regularization optimization term is added to the loss function of BCL in the construction of the recognition model of the present invention, which is specifically used to minimize the cosine similarity between prototypes of different categories. This method increases the distance between prototypes so that each category has a more independent representation in the feature space. This method effectively avoids the overlapping representation of the head class and the tail class, improves the classification accuracy of the model, can achieve fine-grained distinction between crop diseases, and ultimately improves the recognition effect of long-tail diseases. The knowledge graph construction of the present invention constructs a complete multi-source disease pesticide knowledge graph, has rich professional knowledge, and provides a relatively complete disease solution. The present invention not only constructs a complete disease pesticide knowledge graph, but also adds enhanced prototype comparative learning for agricultural diseases to the traditional two-stage algorithm, which can effectively alleviate the long-tail effect caused by data imbalance in actual agricultural production, thereby providing accurate pesticide use recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of the present invention;

[0027] Figure 2 It is the ontology graph;

[0028] Figure 3 This is a partial knowledge graph of rice;

[0029] Figure 4 It is a reinforced prototype contrast learning diagram;

[0030] Figure 5 This is a comparison chart of accuracy. DETAILED DESCRIPTION

[0031] The following is a detailed description of the specific implementation method and beneficial effects of a long-tail disease identification and medication technology based on a disease pesticide knowledge graph proposed by the present invention in conjunction with the accompanying drawings and preferred embodiments:

[0032] See also Figure 1 , a long-tail disease identification and pesticide application integrated method based on disease and pesticide knowledge graph, comprising the following steps:

[0033] Step 1. Build a disease pesticide information database. Pesticide disease information will be collected through network mining technology, the data will be cleaned, pesticide professional knowledge will be further supplemented based on pesticide chemical data, and knowledge alignment of multi-source data will be completed.

[0034] Step 1.1 Data acquisition: The pesticide, crop, disease and active ingredient information from websites such as the Pesticide Information Registration Network, Garden Network, Baidu Encyclopedia, Agricultural Pest and Weed Picture and Text Database, Pesticide Name Network, and PubChem were integrated, and a web crawler script was written in Python to save the semi-structured and unstructured knowledge in the local database.

[0035] Step 1.2 Data cleaning: Process the knowledge obtained from different data sources, including null values ​​and redundant spaces in the data. At the same time, since the language of the website and the knowledge base is not unified, the present invention uses DeepL to translate the content into American English to ensure data consistency.

[0036] Step 1.3 Data supplement: Through Tang Chuchi’s pesticide chemistry books and IRAC, FRAC and HRAC websites, according to the active ingredients of the pesticides, the structural category, specific type and mode of action of fungicides are supplemented; the mechanism of action, structural category, action properties and mode of action of herbicides are supplemented; and the mode of action and structural information of herbicides are supplemented.

[0037] Step 1.4 Data alignment: Align the data of the pesticide information registration network with the names of pests and diseases in the agricultural pest and disease image database and the names of pests and diseases in the garden network through the control objects, align the pest and disease entities in the garden network and the agricultural pest and disease image database to ensure the uniqueness of the entities; align the crop types in the pesticide information registration network with the Chinese names in Baidu Encyclopedia; align the active ingredient names in the pesticide information registration network with the ISO common names in the pesticide name network, the IUPACName or CAS Name in PubChem, etc. Align the data obtained from various data sources, so as to realize the interconnection of knowledge and build a pest and disease pesticide information database.

[0038] Step 2. Construction of disease and pesticide knowledge graph

[0039] The knowledge graph ontology is constructed, and the content filling and connection of the knowledge graph are completed through neo4j.

[0040] Step 2.1. Construction of knowledge graph of pests and pesticides: Based on the data in step 1, the present invention uses four ontologies, namely pesticide ontology, crop ontology, disease ontology and active ingredient ontology. Figure 2, taking pesticide PD20100107 as an example, starting from the pesticide node, it is subdivided step by step into inorganic copper fungicides according to its own chemical properties, and then the corresponding mode of action and target are determined according to the active ingredients it contains, followed by the corresponding relationship between it and specific pesticides and crop diseases. Among them, the pesticide ontology will contain different attributes, but all contain information such as toxicity, usage, and expiration date; the attributes of the crop ontology are mainly obtained from Baidu Encyclopedia, containing category information such as order, family, and genus; the disease ontology is mainly obtained from the Garden Network and the Agricultural Diseases and Insects and Weeds Picture and Text Database, mainly including symptoms, pathogens, causes of occurrence and other attributes; the active ingredient ontology is mainly obtained from the Pesticide Name Network and PubChem, mainly including mode of action, main target, chemical name and other attributes. According to the subsequent long-tail disease identification and medication needs, this knowledge graph mainly defines three main relationships: cure, suffer from, and apply to these three relationships, which are convenient for the model to query the disease medication in the future. At the same time, there are also including and determining these relationships, in order to better express the hierarchical structure between pesticides and the intrinsic connection between the structure, action site and action mode in the active ingredients.

[0041] Step 2.2 Filling the knowledge graph of pests and pesticides: According to the knowledge graph of pests and pesticides in 2.1, extract key triple data from the information database of pests and pesticides, and write the extracted triples into the knowledge graph. Figure 3 This is a knowledge graph of rice diseases and pesticides. The green nodes represent the crop, i.e. rice, the orange nodes represent the pesticides, including the relevant attributes of the pesticides, and the blue nodes represent the disease nodes. Specifically, the pyneo library in python is used to connect to the neo4j database, and the data is updated to the constructed disease and pesticide knowledge graph ontology by reading the triple information.

[0042] Step 3. Recognition model construction

[0043] Using public datasets in the plant field and a reinforced prototype contrast learning module, the model uses two-stage training. The first stage learns the disease image features, and the second stage learns the classifier effect.

[0044] Step 3.1 Dataset preprocessing: Extract disease images of staple crops from the dataset provided by PlantPad, and obtain 47 diseases of 7 staple crops in total. The 7 staple crops are "corn", "potato", "rice", "sorghum", "soybean", "sweet potato", and "wheat". Some disease images are shown in Figure 5, showing 4 diseases of each staple crop. Before the image is input into the model, it is unified into a 32*32 image size, randomly cropped and rotated, and normalized using the distribution of ImageNet.

[0045] Step 3.2: One-stage model training: Use cross entropy loss, weight decay learning and reinforced prototype contrast learning to extract image features.

[0046] The first-stage model training mainly uses the ResNet32 network as the baseline network, and uses cross entropy loss and weight decay to learn the features of the image. Among them, cross entropy loss, as a classic image loss function, performs well in classification tasks, but in long-tail tasks, due to too many head class samples, the model weights fit the head class. Weight decay constrains the network by limiting the growth of network weights. In the network model of long-tail learning, since the weight of the head class is too large, the model is punished more severely for high weights by setting a penalty term. The weight decay calculation method is:

[0047]

[0048] This part is the traditional cross entropy loss function. is a penalty item, Represents the weight of the kth item. Due to the characteristics of long-tail distribution, weight decay works well in constraining model weights.

[0049] In order to further focus on the fine-grained features in long-tail diseases, the present invention adopts enhanced prototype contrast learning.

[0050] Step 3.3 Strengthen prototype contrast learning:

[0051] For the specific implementation of the enhanced prototype contrast learning module, see Figure 4

[0052] For the input sample , through three changes , get the positive sample pair ,in The function is a random combination of data enhancements, and then the enhanced samples are concatenated in 0 dimension and sent to the model training. It is a combination of image enhancement methods. Specifically The image is cropped to 32x32 using random cropping, with a cropping ratio between 0.08 and 1; randomly flipped horizontally; randomly applied color changes so that the brightness, contrast, and saturation of the image vary with an intensity of 0.4. The random enhancement method is used, with parameters n as 10 and m as 2; the image is converted to a tensor; and the image is processed using normalization. The image is cropped to 32x32 using random cropping with a cropping ratio between 0.08 and 1; horizontal flipping is performed randomly; the strength is 0.4 and is performed with a probability of 80%; the random enhancement method is used with parameters n as 10 and m as 2; the image is converted to grayscale with a probability of 20%; the image is converted to a tensor; and the image is processed using normalization. The image is cropped to 32x32 using random cropping, with a cropping ratio between 0.08 and 1; randomly flipped horizontally; the strength is 0.4 and executed with 80% probability; the image is converted to grayscale with 20% probability; the image is converted to a tensor; and the image is processed using standardization. Among them, the random enhancement method refers to a series of random image enhancement (color change, rotation, cropping, flipping) operations to increase the diversity of samples. The intensity of these operations and the randomness of the application are determined by configuring the parameters of the enhancement strength (m) and the number of operation layers (n).

[0053] Feature extraction: The model receives the input image and extracts features through the encoder. The extracted features are flattened and mapped through a fully connected layer to generate feature representations for enhanced prototype contrastive learning.

[0054] Classification prediction: The flattened features are used to generate logits for class predictions through a linear classifier. At the same time, the classifier weights are converted into class prototypes through a fully connected layer to measure the relationship between different classes.

[0055] Prototype calculation: Generate category prototypes through normalized classifier weights and calculate the similarity matrix between prototypes. The similarity matrix is ​​used to optimize the distribution of prototypes between classes to avoid adjacent categories being too close. The cosine similarity between prototypes is:

[0056]

[0057] represents the similarity between category i and category j, represents the prototype of the i-th class and has been L2 normalized (i.e. ).

[0058] In order to minimize the similarity between different prototypes, the loss function is defined as:

[0059]

[0060] represents the prototype of the ith class, Separation loss for the prototype.

[0061] The overall loss function of the last stage is defined as:

[0062] ,

[0063] Represents the total loss of the model. represents the classic balanced contrastive learning loss, represents the cross entropy loss of the model classification head, represents the enhanced prototype loss, and The three losses are constrained simultaneously to ensure that the model can fully extract sufficient information of each category in one stage of training.

[0064] A classic method for solving long tail problems in computer science. The input is a disease image enhanced by data. , the specific implementation is as follows The balanced contrastive learning (BCL) loss function achieves the category balance of long-tail distribution data by selecting appropriate positive and negative samples and weighting them. In the formula, the positive sample set comes from the sample Other samples of the same category and the center vector of the category ; Such a design encourages the similarity of samples with samples of the same category in the numerator (through the inner product , thereby improving the consistency within the class. The negative sample set comes from samples of other categories. (gather ), and the similarity between positive and negative samples is also determined by the inner product. The balancing factor in the denominator The inverse of the number of class samples is used as the weight to reduce the influence of large classes and make small classes occupy a larger proportion in the loss function. is the cross entropy loss function, and the input is the disease picture enhanced by data , the specific implementation is as follows:

[0065] , the cross entropy loss function (CE) guides the training of the model by measuring the difference between the model prediction probability distribution and the true label distribution, where the predicted probability of each category is , is the category index, is the total number of categories, is the sample label.

[0066] Experimental verification When , the feature learning effect is best.

[0067] Step 3.4 Two-stage model training: The two-stage model training is to fine-tune the classification head of the model based on the first stage, so that it can learn the long-tail disease classification. Since the first-stage training is directly performed on the long-tail data set, even with weight decay, the classification accuracy will be low due to the low sampling frequency of the tail class. Therefore, class balance loss is added to the second-stage training, and the class weight of each class is adjusted according to the frequency of occurrence to force it to learn the characteristics of the minority class. In order to further constrain the model, the present invention uses tau Norm to further adjust the weight of the model. The tau Norm calculation method is as follows:

[0068]

[0069] Representation vector of norm, Indicates The absolute value of the elements. Unlike the traditional L2 regularization method, tau Norm sets the hyperparameters , by changing the model parameter vector Sum and open The square root is used to complete the category weights When The hyperparameter is 1, which means that tau Norm is equivalent to L2 Norm. The adjustment can obtain better constraint effect than L2 Norm.

[0070] Since the main goal of the second stage is to adjust the classification head of the model, the other parameters of the model need to be frozen and the loss function of the second stage also needs to be adjusted.

[0071] The weight of is set to 0, It needs to be further adjusted to class balance loss, and the categories are weighted according to the number of samples in each batch. This can improve the model's learning ability for small categories and thus improve the classification performance.

[0072] The accuracy of the model can be found in Figure 5, the horizontal axis represents each category, the vertical axis represents the accuracy, the blue legend represents the original model, the orange legend represents the improved model, and the red curve represents the overall data distribution. By adding the enhanced prototype contrast loss, compared with the original computer field method, it can be seen that the accuracy of the tail class has been greatly improved on the basis of a slight decrease in the accuracy of the head class. The superiority of the model of the present invention in processing long-tail data is demonstrated under an imbalance ratio of 100. It can be seen that the present invention has a very obvious improvement effect compared with the baseline model, and the classification accuracy of the tail class has been greatly improved without excessively losing the accuracy of the head class.

[0073] Step 4. Pesticide Query

[0074] After obtaining the classification results of the image, you can use Cypher statements to query the corresponding pesticide data in the graph database. Taking wheat rust as an example, the query statement is as follows:

[0075] match (d:`Specific pesticide`)-[r1:Cure]-(n:`Plant disease`)-[r2:`Existsin`]-(p:Crop),(d)-[r3:`Act on`]-(p) where p.name='rice' and n.name='rust' return dd

[0076] Specific pesticide refers to pesticides, Plant disease refers to diseases, and Crop refers to crops; r1 represents the relationship between pesticides and diseases, r2 represents the relationship between diseases and crops, and r3 represents the relationship between pesticides and crops. The model recognizes the image, obtains the corresponding crop name n and disease name p, and searches the disease pesticide knowledge graph constructed in step 2 to obtain the treatment method for the corresponding crop disease.

[0077] Through the above steps, the present invention realizes an integrated technology of long-tail disease identification and pesticide application based on the disease-pesticide knowledge graph, which alleviates the data imbalance problem prevalent in current smart agriculture practice and provides an effective disease treatment method.

[0078] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A long-tail disease identification and pesticide application integrated method based on disease and pesticide knowledge graph, comprising the following steps: Step 1. Build a disease and pesticide information database Step 1.1 Data acquisition: Integrate websites and knowledge bases related to diseases, crops, and pesticides, obtain semi-structured and unstructured knowledge from web pages, and save them in the local database; Step 1.2 Data cleaning: Process the knowledge obtained from different data sources and unify the language; Step 1.3 Data supplementation: Use relevant literature to supplement professional knowledge of pesticides according to their active ingredients; Step 1.4 Data alignment: Link and align data from different sources through designated entities to build a disease and pesticide information database; Step 2. Construction of disease and pesticide knowledge graph Step 2.

1. Knowledge graph ontology construction: Complete the knowledge graph ontology, including pest and disease ontology, pesticide ontology, crop ontology and active ingredient ontology; Step 2.2 Knowledge graph content filling: According to the knowledge graph ontology in step 2.1, extract triples from the disease pesticide information database in step 1.4, and use the graph database neo4j to complete the knowledge graph data filling; Step 3: Identification model building Step 3.1 Dataset preprocessing: Obtain long-tail disease data and process the data uniformly; Step 3.2: One-stage model training uses the ResNet32 network as the baseline network and uses cross entropy loss and weight decay to learn image features; Step 3.3 Strengthen prototype contrastive learning: Generate category prototypes through classifier weights, calculate the cosine similarity matrix between prototypes, and use mutual exclusion constraints to minimize the similarity between category prototypes based on the similarity between prototypes. Strengthen the distance between prototypes so that contrastive learning can further learn fine-grained features in diseases. Step 3.4 Two-stage model training: Class balance loss is added during the two-stage training. The class weights of each class are adjusted according to the frequency of occurrence of each class to force it to learn the characteristics of the minority class. The weights of the model are further adjusted using tau Norm. The calculation method of tauNorm is as follows: Representation vector of norm, Indicates The absolute value of the elements of the model parameter vector Sum and open The square root is used to complete the category weights Adjustment of The other parameters of the model need to be frozen, and the loss function of the second stage needs to be adjusted; , The weight of is set to 0, Adjusted to class-balanced loss, weighting classes according to the number of samples in each batch; Step 4. Pesticide search According to the classification results obtained by the long-tail disease classifier in step 3.4, the corresponding pesticides are queried from the disease-pesticide knowledge graph constructed in step 2.

2. A long-tail disease identification and pesticide application integrated method based on disease and pesticide knowledge graph as described in claim 1, wherein in step 1.1, a web crawler script is written in Python language to save semi-structured and unstructured knowledge in a local database.

3. An integrated method for long-tail disease identification and medication based on a disease-pesticide knowledge graph as described in claim 1, wherein the pesticide supplement professional knowledge in step 1.3 is to supplement the structural category, specific type and mode of action of fungicides; to supplement the action mechanism, structural category, action properties and mode of action of herbicides; and to supplement the mode of action and structural information of herbicides.

4. A method for integrating long-tail disease identification and medication based on disease and pesticide knowledge graph as claimed in claim 1, wherein the data obtained from each data source in step 1.4 is aligned by aligning the data of the pesticide information registration network with the names of pests and diseases in the agricultural pest and disease graphic database and the names of pests and diseases in the garden network through the control objects, aligning the pest and disease entities in the garden network and the agricultural pest and disease graphic database to ensure the uniqueness of the entities; aligning the crop types in the pesticide information registration network with the Chinese names in Baidu Encyclopedia; aligning the active ingredient names in the pesticide information registration network with the ISO common names in the pesticide name network and the IUPAC Name or CAS Name in PubChem.

Images