A knowledge graph-based pest and disease diagnosis system and method thereof
By combining knowledge graphs and deep learning, the pest and disease diagnosis system solves the problems of insufficient accuracy in pest and disease detection and lack of diagnostic suggestions in existing technologies, and achieves efficient and accurate output of pest and disease diagnosis and control suggestions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIANGSU UNIV OF SCI & TECH
- Filing Date
- 2022-07-14
- Publication Date
- 2026-04-17
AI Technical Summary
Existing deep learning-based pest and disease detection technologies cannot improve accuracy after model training is completed, and cannot provide diagnostic suggestions. They have low accuracy, and agricultural pest and disease detection lacks diversified methods and resources.
By combining knowledge graphs and deep learning, a pest and disease diagnosis system is constructed, including a knowledge graph module and a pest and disease diagnosis module. The deep learning sub-module is used to extract and learn features from pest and disease images, and the results are compared with text data in the knowledge graph module to output diagnosis results and prevention and control suggestions.
It achieves highly accurate diagnosis of pests and diseases from images and can provide corresponding prevention and control suggestions, reducing reliance on manual identification and resource scarcity, and improving the efficiency and accuracy of agricultural pest and disease detection.
Smart Images
Figure CN115391549B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of knowledge graph technology, and involves the intersection of deep learning and knowledge graph technologies, particularly a knowledge graph-based pest and disease diagnosis system and method. Background Technology
[0002] Compared with developed agricultural countries in the West, my country's agricultural competitiveness is weak. In the field of pest and disease prevention and detection, there is little practical application. The control work is structurally simple, and it is usually judged and identified by local planting personnel (plant protection bureau). The labor cost is high, and there is a shortage of specialized training personnel, which cannot be popularized to farmers. Therefore, diversified control methods are becoming increasingly urgent.
[0003] Currently, besides manual methods for identifying pests and diseases, there are also some high-tech detection methods, the most important of which is pest and disease detection based on deep learning. Deep learning-based pest and disease detection methods mainly use convolutional neural networks to extract features from large datasets of pest and disease images. Popular convolutional neural networks include ResNet and DenseNet. The extracted features are automatically learned, and then trained through multiple iterations, continuously adjusting model parameters such as the learning rate and loss function. By comparing the results of multiple training iterations, the model with the best performance is selected. The resulting neural network model can then more accurately identify pest and disease images. However, current deep learning-based pest and disease detection still has the following shortcomings: First, current deep learning-based pest and disease detection technology can only detect pest and disease images using pre-trained models. Once the model is trained, its accuracy cannot be improved. To improve the model's accuracy, it must be retrained, and its accuracy cannot be improved during use. Second, deep learning-based pest and disease detection technology can only identify the type of pest or disease in the image, but cannot provide corresponding diagnostic suggestions. Third, although the accuracy of deep learning-based pest and disease detection technology is gradually increasing with the continuous advancement of science and technology, overall, the accuracy of this technology in pest and disease detection is not particularly high and needs further improvement. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a knowledge graph-based pest and disease diagnosis system and method. This system utilizes knowledge graphs and combines them with deep learning to detect and diagnose pests and diseases. Users can more easily learn about pests and diseases through this system, which is beneficial for achieving agricultural informatization.
[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution.
[0006] The present invention discloses a knowledge graph-based pest and disease diagnosis system, characterized in that it includes a knowledge graph module and a pest and disease diagnosis module; the pest and disease diagnosis module includes a pest and disease image input submodule, a deep learning submodule, and a diagnosis output submodule; the deep learning submodule is connected to the knowledge graph module, the pest and disease image input submodule, and the diagnosis output submodule, and the diagnosis output submodule is connected to the deep learning submodule and the knowledge graph module;
[0007] The deep learning submodule extracts features from the pest and disease images received from the pest and disease image input submodule, identifies the pests and diseases using the feature data stored in the knowledge graph module, and outputs the identification results to the diagnosis output submodule. The deep learning submodule also learns features from pest and disease images not stored in the knowledge graph module, supplements the knowledge graph module with the learning results, and updates the knowledge graph module. The diagnosis output submodule compares the identification results from the deep learning submodule with the pest and disease text data stored in the knowledge graph module to obtain the diagnosis results and control suggestions for the type of pest and disease, and then outputs them.
[0008] This invention discloses a knowledge graph-based pest and disease diagnosis method, employing a knowledge graph-based pest and disease diagnosis system, including a knowledge graph module and a pest and disease diagnosis module. The pest and disease diagnosis module comprises a pest and disease image input submodule, a deep learning submodule, and a diagnosis output submodule. The deep learning submodule is connected to the knowledge graph module, the pest and disease image input submodule, and the diagnosis output submodule; the diagnosis output submodule is connected to both the deep learning submodule and the knowledge graph module. The deep learning submodule extracts features from pest and disease images received from the pest and disease image input submodule, identifies pests and diseases using feature data stored in the knowledge graph module, and outputs the identification results to the diagnosis output submodule. The deep learning submodule also learns features from pest and disease images not stored in the knowledge graph module, supplements the knowledge graph module with the learning results, and updates the knowledge graph module. The diagnosis output submodule compares the identification results from the deep learning submodule with the pest and disease text data stored in the knowledge graph module to obtain a diagnosis result and control suggestions for the type of pest and disease, and outputs these suggestions.
[0009] The method includes the following steps:
[0010] Step 1: Predefine the entities and relationships between entities in the knowledge graph of pest and disease diagnosis, and construct a schema diagram of the knowledge graph of pest and disease diagnosis.
[0011] Step 2: Based on the schema diagram of the pest and disease diagnosis knowledge graph constructed in Step 1, acquire knowledge by obtaining text data and dataset images of pests and diseases: Obtain specific data on entities and relationships required for pest and disease diagnosis from the Internet, including text data and a large number of pest and disease dataset images. Each pest and disease dataset image has a corresponding pest and disease label. Based on this, further obtain the links between entities according to the relationship categories defined in the schema diagram of the pest and disease diagnosis knowledge graph.
[0012] Step 3: Preprocess the large number of pest and disease dataset images obtained in Step 2, extract the features of the preprocessed pest and disease dataset images, and finally classify the features of the preprocessed and extracted pest and disease dataset images.
[0013] Step 4: Perform knowledge fusion processing on the text data of pests and diseases obtained in Step 2;
[0014] Step 5: Store the image feature data of the pest and disease dataset classified in Step 3 and the text data of the pest and disease after knowledge fusion in Step 4 into the Neo4j database to complete the construction of the knowledge graph module.
[0015] Step 6: Input the pest and disease images to be diagnosed through the pest and disease image input submodule of the pest and disease diagnosis module, and perform image size adjustment and filtering preprocessing on the images. Then, send the preprocessed pest and disease images into the deep learning submodule.
[0016] Step 7: The deep learning submodule receives the preprocessed pest and disease images from the pest and disease image input submodule, and extracts features from the pest and disease images through convolution operations. Then, it retrieves the extracted pest and disease image features and selects whether to optimize the pest and disease image features based on the retrieval results. If not, it directly sends the pest and disease labels obtained from the retrieval results to the diagnostic output submodule.
[0017] Step 8: The diagnostic output submodule outputs the diagnostic results of pests and diseases and their prevention and control suggestions.
[0018] Specifically, step 1 includes the following process:
[0019] Step 1.1 Predefine entities and relationships between entities in the knowledge graph of pest and disease diagnosis: Based on the stages, characteristics, damage, and treatment measures of pests and diseases, define entities as follows: early stage crop disease, late stage crop disease, early stage crop disease characteristics, late stage crop disease characteristics, early stage measures, early stage drugs, late stage measures, late stage drugs, disease description, disease cause, and disease damage; define the relationship between disease and characteristics as belonging, the relationship between disease and measures as resolution, the relationship between drugs and measures as usage, the relationship between disease description and disease as belonging, and the relationship between disease and disease cause as inducing relationship;
[0020] Step 1.2 Constructing the schema diagram of the knowledge graph for disease and pest diagnosis: Based on the defined entities and the relationships between entities, each entity is associated through the relationships between entities; including: associating the entities of early crop diseases and early crop characteristics that have a relationship; through one entity and its relationship, the other entity can be queried; finally, all entities and their relationships are associated to form the schema diagram of the knowledge graph for disease and pest diagnosis.
[0021] Specifically, step 3 includes the following processes:
[0022] Step 3.1 Preprocessing a large number of pest and disease dataset images: First, crop the size of the pest and disease dataset images to 224*224 images. Then, use Gaussian filtering to filter the cropped pest and disease dataset images to reduce Gaussian noise in the cropped pest and disease dataset images.
[0023] Step 3.2: Extract features from the preprocessed pest and disease dataset images: Project the features of the preprocessed pest and disease dataset images through convolution operations, and reshape the projected features into N blocks to obtain a rich set of intermediate features; pass this set of intermediate features through a self-attention path and a convolution path using a fully connected layer respectively; obtain the features of the self-attention path and the features of the fully connected layer convolution path respectively; finally, add the features of the self-attention path and the features of the fully connected layer convolution path to obtain the final pest and disease dataset image features;
[0024] Step 3.3: Classify the preprocessed and extracted pest and disease dataset image features: Divide the extracted final pest and disease dataset image features into training, validation, and test sets according to a certain ratio. Use these three datasets to train and optimize the neural network of the deep learning module. The neural network structure is: convolutional layer, pooling layer, activation layer, fully connected layer, and classification layer. During training, the learning rate is dynamically adjusted using the momentum method. Based on the training results, classify the final pest and disease dataset image features to obtain multiple pest and disease image feature libraries, and take the mean image feature of each class image library as the index.
[0025] Specifically, step 4 includes the following process:
[0026] First, the acquired pest and disease text data is subjected to entity alignment processing, which involves finding equivalent instances among numerous data sets. This process begins by normalizing the original pest and disease text data. Then, the attribute similarity or entity similarity of the normalized pest and disease text data is calculated to link the records of two entities. Finally, the Limes entity matching tool is used to match the entities.
[0027] Specifically, step 5 includes the following process:
[0028] The image features of the classified pest and disease datasets, the index using the mean image features, the labels of each pest and disease dataset, and the pest and disease text data processed by knowledge fusion are stored in the Neo4j database according to the corresponding formats to complete the construction of the knowledge graph module.
[0029] Specifically, step 6 includes the following process:
[0030] The pest and disease image input submodule resizes the input pest and disease images to be diagnosed, adjusting their size to 224*224 pixels. The resized images are then filtered using a Gaussian filter. Finally, the resized and filtered images are fed into the deep learning submodule.
[0031] Specifically, step 7 includes the following process:
[0032] Step 7.1 The deep learning submodule extracts features from the received preprocessed pest and disease images through convolution operations. It projects the extracted pest and disease image features and reshapes the projected features into N blocks to obtain a rich set of intermediate features. This set of intermediate features is then passed through a self-attention path and a convolutional path using a fully connected layer to obtain the features of the self-attention path and the features of the fully connected layer convolutional path, respectively. Finally, the features of the self-attention path and the features of the fully connected layer convolutional path are added together to obtain the final pest and disease dataset image features.
[0033] Step 7.2 The deep learning submodule retrieves the extracted pest and disease image features, setting a threshold for image similarity measurement before retrieval. It then uses feature matching to match the pest and disease image features with features in the pest and disease diagnosis knowledge graph. First, the pest and disease image features and the features in the pest and disease diagnosis knowledge graph are described using vectors. Next, the Euclidean distance between the feature vector of the pest and disease image and the feature vectors of various libraries in the pest and disease diagnosis knowledge graph is calculated. This Euclidean distance is used to determine which library or libraries the pest and disease image features belong to. Then, in these retrieved libraries, the feature similarity measurement between the pest and disease image feature vector and the feature vectors in the libraries is calculated. If all retrieval results are within the pre-set threshold, the library containing the final query image, i.e., the corresponding pest and disease label, is obtained based on the optimal result of the feature similarity measurement, and this pest and disease label is sent to the diagnosis output submodule.
[0034] Step 7.3 If all search results are greater than the preset threshold, the features of the pest and disease image need to be further optimized. The features of the pest and disease image are fed into the trained neural network model. The neural network learns the features of the image. After learning, it can find out which type of image feature library the features of the pest and disease image belong to and store the features of the pest and disease image in the corresponding image feature library, that is, in the pest and disease knowledge graph module.
[0035] Specifically, step 8 includes the following processes:
[0036] The diagnosis output submodule compares the pest and disease tags output by the deep learning submodule with the pest and disease text data stored in the knowledge graph module to find the text data of the pest and disease. Based on the entities and entity relationships in the pattern graph of the pest and disease diagnosis knowledge graph, it outputs the diagnosis results of the pest and disease, including the cause and damage of the pest and disease. It also proposes corresponding prevention and control suggestions for the pest and disease, including prevention and control measures and drugs.
[0037] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0038] 1. Compared to existing knowledge graph technologies that only store text data and deep learning-based pest detection technologies that can only identify pest images, this invention, in addition to storing pest text data, also stores pest image feature data in a knowledge graph. It extracts features from a large dataset of pest images using deep learning technology and stores these extracted features in the knowledge graph. Therefore, this invention not only enables more accurate diagnosis of pest images but also provides reasonable control suggestions for each type of pest. This allows staff to quickly and easily identify the type of pest and disease and, based on the provided control suggestions, treat crops promptly to minimize losses.
[0039] 2. This invention, during diagnosis, optimizes the features of pest and disease images not currently stored in the knowledge graph module using deep learning technology after setting a threshold, and then stores them in the knowledge graph. The knowledge graph is continuously updated to continuously reduce diagnostic errors. Furthermore, this technology does not require professional personnel to periodically update the system; data is automatically supplemented and updated as staff use the system, thus reducing economic costs. Attached Figure Description
[0040] Figure 1 This is a block diagram of a knowledge graph-based pest and disease diagnosis system according to the present invention.
[0041] Figure 2 This is a flowchart of a knowledge graph-based method for diagnosing pests and diseases according to the present invention. Detailed Implementation
[0042] The present invention will now be described in further detail with reference to the accompanying drawings.
[0043] like Figure 1 As shown, the present invention discloses a knowledge graph-based pest and disease diagnosis system, comprising a knowledge graph module and a pest and disease diagnosis module. The pest and disease diagnosis module includes three sub-modules: pest and disease image input, deep learning, and diagnostic output. The deep learning sub-module is connected to the knowledge graph module, the pest and disease image input sub-module, and the diagnostic output sub-module; the diagnostic output sub-module is connected to both the deep learning sub-module and the knowledge graph module.
[0044] The deep learning submodule extracts features from the pest and disease images received from the pest and disease image input submodule, identifies the pests and diseases using the feature data stored in the knowledge graph module, and outputs the identification results to the diagnosis output submodule. The deep learning submodule also learns features from pest and disease images not stored in the knowledge graph module, adding the learning results to the knowledge graph module and updating it. The diagnosis output submodule, connected to both the deep learning submodule and the knowledge graph module, compares the identification results from the deep learning submodule with the pest and disease text data stored in the knowledge graph module to obtain diagnostic results and prevention and control suggestions for the specific pest or disease, and then outputs these suggestions.
[0045] like Figure 2 As shown, a knowledge graph-based method for diagnosing pests and diseases according to the present invention includes the following steps:
[0046] Step 1: Predefine the entities and relationships between entities in the knowledge graph of pest and disease diagnosis, and construct a schema diagram of the knowledge graph of pest and disease diagnosis.
[0047] Step 2: Based on the pattern diagram of the knowledge graph for disease and pest diagnosis constructed in Step 1, acquire knowledge, obtain text data and dataset images of diseases and pests.
[0048] Step 3: Preprocess the large number of pest and disease dataset images obtained in Step 2, extract features from the preprocessed pest and disease dataset images, and finally classify the preprocessed and extracted pest and disease dataset image features.
[0049] Step 4: Perform knowledge fusion processing on the text data of pests and diseases obtained in Step 2;
[0050] Step 5: Store the image feature data of the pest and disease dataset classified in Step 3 and the text data of the pest and disease after knowledge fusion in Step 4 into the Neo4j database to complete the construction of the knowledge graph module.
[0051] Step 6: Input the pest and disease images to be diagnosed through the pest and disease image input submodule of the pest and disease diagnosis module, and perform image size adjustment and filtering preprocessing on the images. Then, send the preprocessed pest and disease images into the deep learning submodule.
[0052] Step 7: The deep learning submodule receives the preprocessed pest and disease images from the pest and disease image input submodule, and extracts features from the pest and disease images through convolution operations. Then, it retrieves the extracted pest and disease image features and selects whether to optimize the pest and disease image features based on the retrieval results. If not, it directly sends the pest and disease labels obtained from the retrieval results to the diagnostic output submodule.
[0053] Step 8: The diagnostic output submodule outputs the diagnostic results of pests and diseases and their prevention and control suggestions.
[0054] Step 1 includes the following process:
[0055] Step 1.1: Predefine the entities and relationships between them in the pest and disease diagnosis knowledge graph. Based on the stages, characteristics, damage, and treatment measures of pests and diseases, define the entities as follows: early-stage crop disease, late-stage crop disease, early-stage crop disease characteristics, late-stage crop disease characteristics, early-stage measures, early-stage pesticides, late-stage measures, late-stage pesticides, disease description, disease cause, and disease damage. Define the relationship between disease and characteristics as belonging, between disease and measures as a solution, between pesticides and measures as a use, between disease description and disease as belonging, and between disease and disease cause as an inducing relationship.
[0056] Step 1.2: Construct the schema diagram of the pest and disease diagnosis knowledge graph. Based on the entities and relationships defined in Step 1.1, we can associate each entity through these relationships. For example, if the relationship between early-stage crop diseases and early-stage crop characteristics is one of association, then these two entities can be associated. By using one entity and its relationship, we can query the other entity. Finally, after associating all entities and their relationships, the schema diagram of the pest and disease diagnosis knowledge graph is formed.
[0057] Step 2 includes the following process:
[0058] The system obtains specific data on entities and relationships required for pest and disease diagnosis from different data sources on the Internet. These different data sources refer to text data from agricultural information platforms such as Baidu Encyclopedia and Agricultural Knowledge Network, as well as a large number of pest and disease dataset images. Each pest and disease dataset image has a corresponding pest and disease label. Based on this, the system further obtains the links between entities according to the relationship categories defined in the pattern graph of the pest and disease diagnosis knowledge graph.
[0059] Step 3 includes the following process:
[0060] Step 3.1: Preprocess the large number of pest and disease dataset images. Image preprocessing can eliminate irrelevant information in the images, restore useful real information, enhance the detectability of relevant information, and simplify the data to the maximum extent, thereby improving the reliability of feature extraction, matching, and recognition. First, the images in these pest and disease datasets are cropped to a size of 224*224, because the input images in deep learning are mostly 224*224. Next, Gaussian filtering is applied to the cropped pest and disease dataset images. This reduces Gaussian noise in the cropped images, and Gaussian filtering has less blurring and better preserves the overall details of the images.
[0061] Step 3.2: Extract features from the preprocessed pest and disease dataset images. First, project the features of the preprocessed pest and disease dataset images through convolution operations, and reshape the projected features into N blocks, thus obtaining a rich set of intermediate features. Next, pass this set of intermediate features through a self-attention path and a convolution path using a fully connected layer, respectively, to obtain the features of the self-attention path and the features of the fully connected layer convolution path. Finally, add the features of the self-attention path and the features of the fully connected layer convolution path to obtain the final pest and disease dataset image features.
[0062] Step 3.3: Classify the preprocessed and extracted pest and disease dataset image features. The final pest and disease dataset image features extracted in Step 3.2 are divided into training, validation, and test sets according to a certain ratio. These three datasets are used to train and optimize the neural network of the deep learning module. The network structure is as follows: convolutional layer, pooling layer, activation layer, fully connected layer, and classification layer. During training, the learning rate is dynamically adjusted using the momentum method. Based on the training results, the final pest and disease dataset image features are classified to obtain multiple pest and disease image feature libraries, and the mean image feature of each class is used as the index.
[0063] Step 4 includes the following process:
[0064] First, the acquired pest and disease text data undergoes entity alignment processing, which involves finding equivalent instances among numerous data points. This process begins by normalizing the original pest and disease text data, for example, using syntactic or data normalization. Next, the attribute similarity or entity similarity of the normalized pest and disease text data is calculated to link the records of two entities. Finally, the Limes entity matching tool is used to match the entities. Limes is a metric space-based entity matching discovery framework suitable for linking large-scale data.
[0065] Step 5 includes the following process:
[0066] The knowledge graph module is constructed by storing the image features of the categorized pest and disease datasets, the index using mean image features, the labels for each pest and disease dataset, and the text data of pests and diseases processed through knowledge fusion, in the corresponding locations within the Neo4j database according to their respective formats. Therefore, this knowledge graph module not only contains text data of pests and diseases but also image features of the pest and disease datasets. Compared to other technologies that can only diagnose pests and diseases from text data, this technology can also diagnose pests and diseases from images and provide control measures for that type of pest and disease. This makes it more practical and suitable for daily use by staff.
[0067] Step 6 includes the following process:
[0068] The pest and disease image input submodule first resizes the input images of pests and diseases to be diagnosed, adjusting their size to 224*224 pixels. Next, it applies a Gaussian filter to the resized image, significantly reducing Gaussian noise while preserving overall image detail. Finally, the resized and filtered image is fed into the deep learning submodule.
[0069] Step 7 includes the following process:
[0070] Step 7.1: The deep learning submodule extracts features from the received preprocessed pest and disease images through convolution operations. The extracted pest and disease image features are projected and reshaped into N blocks, thus obtaining a rich set of intermediate features. Then, this set of intermediate features is passed through a self-attention path and a convolutional path using a fully connected layer, respectively, to obtain the features of the self-attention path and the features of the fully connected layer convolutional path. Finally, the features of the self-attention path and the features of the fully connected layer convolutional path are added together to obtain the final pest and disease dataset image features.
[0071] Step 7.2: The deep learning submodule also retrieves the extracted pest and disease image features. Before retrieval, a threshold for image similarity measurement is set. A feature matching method is used to match the pest and disease image features with features in the pest and disease diagnosis knowledge graph. First, the pest and disease image features and features in the pest and disease diagnosis knowledge graph are described using vectors. Next, the Euclidean distance between the feature vector of the pest and disease image and the feature vectors of various libraries in the pest and disease diagnosis knowledge graph is calculated. The result of this distance calculation yields the library or several libraries to which the pest and disease image features belong. Then, in these retrieved libraries, the feature similarity measurement between the feature vector of the pest and disease image and the feature vectors in the libraries is calculated respectively. If the retrieval results are all within the pre-set threshold, the library to which the final query image belongs, i.e., the corresponding pest and disease label, is obtained based on the optimal result of the feature similarity measurement, and this pest and disease label is sent to the diagnosis output submodule.
[0072] Step 7.3: If all search results exceed the preset threshold, then the pest and disease image features need further optimization. These features are fed into a trained neural network model. The neural network learns from these features, identifying which image feature library the feature belongs to and storing it in the corresponding library—the pest and disease knowledge graph module. This allows for continuous updates to the pest and disease knowledge graph, thereby improving diagnostic accuracy. Furthermore, this technology eliminates the need for regular system updates by professionals; data is automatically updated as staff use the system, reducing costs.
[0073] Step 8 includes the following process:
[0074] The diagnostic output submodule compares the pest and disease tags output by the deep learning submodule with the pest and disease text data stored in the knowledge graph module to identify the relevant text data. Based on the entities and entity relationships in the pest and disease diagnostic knowledge graph, it outputs the diagnostic results for this type of pest and disease, such as the cause and damage. It also provides corresponding control suggestions, such as control measures and pesticides. Through the output pest and disease diagnostic results and control suggestions, staff can more quickly and promptly determine the type of pest and disease and, based on the given control suggestions, treat crops in a timely manner to minimize losses.
Claims
1. A knowledge graph-based pest and disease diagnosis method, characterized in that, A knowledge graph-based pest and disease diagnosis system is adopted, including a knowledge graph module and a pest and disease diagnosis module. The pest and disease diagnosis module comprises a pest and disease image input submodule, a deep learning submodule, and a diagnosis output submodule. The deep learning submodule is connected to the knowledge graph module, the pest and disease image input submodule, and the diagnosis output submodule; the diagnosis output submodule is connected to both the deep learning submodule and the knowledge graph module. The deep learning submodule extracts features from pest and disease images received from the pest and disease image input submodule, identifies pests and diseases using feature data stored in the knowledge graph module, and outputs the identification results to the diagnosis output submodule. The deep learning submodule also learns features from pest and disease images not stored in the knowledge graph module, supplements the knowledge graph module with the learning results, and updates the knowledge graph module. The diagnosis output submodule compares the identification results from the deep learning submodule with the pest and disease text data stored in the knowledge graph module to obtain the diagnosis results and control suggestions, and outputs them. The method includes the following steps: Step 1: Predefine the entities and relationships between entities in the knowledge graph of pest and disease diagnosis, and construct a schema diagram of the knowledge graph of pest and disease diagnosis. Step 2: Based on the pattern diagram of the knowledge graph for disease and pest diagnosis constructed in Step 1, acquire knowledge, obtain text data and dataset images of diseases and pests. Step 3: Preprocess the large number of pest and disease dataset images obtained in Step 2, extract the features of the preprocessed pest and disease dataset images, and finally classify the preprocessed and extracted pest and disease dataset image features. Step 4: Perform knowledge fusion processing on the text data of pests and diseases obtained in Step 2; Step 5: Store the image feature data of the pest and disease dataset classified in Step 3 and the text data of the pest and disease after knowledge fusion in Step 4 into the Neo4j database to complete the construction of the knowledge graph module. Step 6: Input the pest and disease images to be diagnosed through the pest and disease image input submodule of the pest and disease diagnosis module, and perform image size adjustment and filtering preprocessing on the images. Then, send the preprocessed pest and disease images into the deep learning submodule. Step 7: The deep learning submodule receives the preprocessed pest and disease images from the pest and disease image input submodule, and extracts features from the pest and disease images through convolution operations. Then, it retrieves the extracted pest and disease image features and selects whether to optimize the pest and disease image features based on the retrieval results. If not, it directly sends the pest and disease labels obtained from the retrieval results to the diagnostic output submodule. Step 8: The diagnostic output submodule outputs the diagnostic results of pests and diseases and their prevention and control suggestions. 2.The pest diagnosis method based on the knowledge graph according to claim 1, characterized in that, The process of step 1 includes: Step 1.1 Predefine entities and relationships between entities in the knowledge graph of pest and disease diagnosis: Based on the stages, characteristics, damage, and treatment measures of pests and diseases, define entities as follows: early stage crop disease, late stage crop disease, early stage crop disease characteristics, late stage crop disease characteristics, early stage measures, early stage drugs, late stage measures, late stage drugs, disease description, disease cause, and disease damage; define the relationship between disease and characteristics as belonging, the relationship between disease and measures as a solution, the relationship between drugs and measures as a use, the relationship between disease description and disease as belonging, and the relationship between disease and disease cause as an inducing relationship; Step 1.2 Constructing the schema diagram of the knowledge graph for disease and pest diagnosis: Based on the defined entities and the relationships between entities, each entity is associated through the relationships between entities; including: associating the entities of early crop diseases and early crop characteristics that have a relationship; through one entity and its relationship, the other entity can be queried; finally, all entities and their relationships are associated to form the schema diagram of the knowledge graph for disease and pest diagnosis.
3. The method for diagnosing pests and diseases based on knowledge graphs according to claim 1, characterized in that, Step 2 includes the following process: The system retrieves specific data on entities and relationships required for pest and disease diagnosis from the internet, including text data and a large number of pest and disease dataset images. Each pest and disease dataset image has a corresponding pest and disease label. Based on this, the system further retrieves the links between entities according to the relationship categories defined in the pattern graph of the pest and disease diagnosis knowledge graph. 4.The pest diagnosis method based on the knowledge graph according to claim 1, characterized in that, Step 3 includes the following process: Step 3.1 Preprocessing a large number of pest and disease dataset images: First, crop the size of the pest and disease dataset images to 224*224 images. Then, use Gaussian filtering to filter the cropped pest and disease dataset images to reduce Gaussian noise in the cropped pest and disease dataset images. Step 3.2: Extract features from the preprocessed pest and disease dataset images: Project the features of the preprocessed pest and disease dataset images through convolution operations, and reshape the projected features into N blocks to obtain a rich set of intermediate features; pass this set of intermediate features through a self-attention path and a convolution path using a fully connected layer respectively; obtain the features of the self-attention path and the features of the fully connected layer convolution path respectively; finally, add the features of the self-attention path and the features of the fully connected layer convolution path to obtain the final pest and disease dataset image features; Step 3.3: Classify the preprocessed and extracted pest and disease dataset image features: Divide the extracted final pest and disease dataset image features into training, validation, and test sets according to a certain ratio. Use these three datasets to train and optimize the neural network of the deep learning module. The neural network structure is: convolutional layer, pooling layer, activation layer, fully connected layer, and classification layer. During training, the learning rate is dynamically adjusted using the momentum method. Based on the training results, classify the final pest and disease dataset image features to obtain multiple pest and disease image feature libraries, and take the mean image feature of each class image library as the index. 5.The knowledge graph-based pest and disease diagnosis method of claim 1, wherein, Step 4 includes the following process: First, the acquired pest and disease text data is subjected to entity alignment processing, which involves finding equivalent instances among numerous data sets. This process begins by normalizing the original pest and disease text data. Then, the attribute similarity or entity similarity of the normalized pest and disease text data is calculated to link the records of two entities. Finally, the Limes entity matching tool is used to match the entities. 6.The knowledge graph-based pest and disease diagnosis method of claim 1, wherein, Step 5 includes the following process: The image features of the classified pest and disease datasets, the index using the mean image features, the labels of each pest and disease dataset, and the pest and disease text data processed by knowledge fusion are stored in the Neo4j database according to the corresponding formats to complete the construction of the knowledge graph module. 7.The knowledge graph-based pest and disease diagnosis method of claim 1, wherein, Step 6 includes the following process: The pest and disease image input submodule resizes the input pest and disease images to be diagnosed, adjusting their size to 224*224 pixels. The resized images are then filtered using a Gaussian filter. Finally, the resized and filtered images are fed into the deep learning submodule. 8.The knowledge graph-based pest and disease diagnosis method of claim 1, wherein, The process of step 7 includes: Step 7.1 The deep learning submodule extracts features from the received preprocessed pest and disease images through convolution operations. It projects the extracted pest and disease image features and reshapes the projected features into N blocks to obtain a rich set of intermediate features. This set of intermediate features is then passed through a self-attention path and a convolutional path using a fully connected layer to obtain the features of the self-attention path and the features of the convolutional path using the fully connected layer, respectively. Finally, the features of the self-attention path and the features of the convolutional path using the fully connected layer are added together to obtain the final pest and disease dataset image features. Step 7.2 The deep learning submodule retrieves the extracted pest and disease image features, setting a threshold for image similarity measurement before retrieval. It then uses feature matching to match the pest and disease image features with features in the pest and disease diagnosis knowledge graph. First, the pest and disease image features and the features in the pest and disease diagnosis knowledge graph are described using vectors. Next, the Euclidean distance between the feature vector of the pest and disease image and the feature vectors of various libraries in the pest and disease diagnosis knowledge graph is calculated. This Euclidean distance is used to determine which library or libraries the pest and disease image features belong to. Then, within these retrieved libraries, the feature similarity measurement between the pest and disease image feature vector and the feature vectors in the libraries is calculated. If all retrieval results are within the pre-set threshold, the library containing the final query image, i.e., the corresponding pest and disease label, is obtained based on the optimal result of the feature similarity measurement, and this pest and disease label is sent to the diagnosis output submodule. Step 7.3 If all search results exceed the preset threshold, the features of the pest and disease image need to be further optimized. The features of the pest and disease image are then fed into the trained neural network model. The neural network learns the features of the image. After learning, it can find out which category of image feature library the features of the pest and disease image belong to and store the features of the pest and disease image in the corresponding image feature library, that is, in the pest and disease knowledge graph module.
9. The method for diagnosing pests and diseases based on knowledge graphs according to claim 1, characterized in that, The process of step 8 includes: The diagnosis output submodule compares the pest and disease tags output by the deep learning submodule with the pest and disease text data stored in the knowledge graph module to find the text data of the pest and disease. Based on the entities and entity relationships in the pattern graph of the pest and disease diagnosis knowledge graph, it outputs the diagnosis results for this type of pest and disease, including the cause and damage of the pest and disease. It also proposes corresponding prevention and control suggestions for the pest and disease, including prevention and control measures and drugs.