Disease and pest image matching case system based on knowledge graph

Abstract

The invention relates to the technical field of knowledge reasoning analysis, in particular to a disease and insect pest picture matching case system based on a knowledge graph. The system comprises a disease and insect pest knowledge graph construction module, a to-be-matched visual semantic fusion module, a disease and insect pest case similarity measurement module and a disease and insect pest case prevention and treatment suggestion module, can collect a disease and insect pest knowledge data set corresponding to the agricultural field, extracts hosts and disease and insect pest entities, and constructs a disease and insect pest knowledge graph at the same time. Generating a pest and disease knowledge graph; obtaining a to-be-matched disease and insect pest picture, and performing histogram equalization processing and visual semantic feature fusion to generate a to-be-matched disease and insect pest visual semantic knowledge feature vector; and performing case similarity calculation and case matching sorting processing on the corresponding entity semantic knowledge in the disease and insect pest knowledge graph, and generating a corresponding disease and insect pest case prevention and treatment suggestion scheme so as to execute corresponding disease and insect pest case prevention and treatment suggestion work. According to the invention, accurate case matching of the disease and pest pictures can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge reasoning and analysis, and particularly to a pest and disease picture matching case system based on a knowledge graph. Background Art

[0002] In recent years, automatic identification of pests and diseases based on images has become an important research direction. By taking pictures of pests and diseases of crops and combining computer vision and deep learning algorithms, automatic classification and diagnosis of pests and diseases can be achieved. In addition, the method of matching pest and disease pictures based on a knowledge graph has gradually become a new research trend. A knowledge graph is a graph structure that represents things and their relationships through nodes and edges, and can comprehensively express and integrate knowledge in the field. In the diagnosis of pests and diseases, the knowledge graph can structure and semanticize information such as the causes, symptoms, transmission routes, and environmental conditions of pests and diseases, providing knowledge support for the accurate diagnosis of pests and diseases. By combining the image features of pests and diseases with the relevant information in the knowledge graph, the accuracy and reliability of pest and disease diagnosis can be improved. However, due to the complex background of pest and disease images, variable shooting angles, and the intertwining of disease symptoms with other environmental factors, the existing pest and disease analysis methods based on knowledge graphs have problems such as ineffective fusion of images and knowledge graph data and high mis-matching rates. Summary of the Invention

[0003] Based on this, it is necessary for the present invention to provide a pest and disease picture matching case system based on a knowledge graph to solve at least one of the above technical problems.

[0004] To achieve the above object, a pest and disease picture matching case system based on a knowledge graph includes the following modules:

[0005] A pest and disease knowledge graph construction module, which is used to collect a pest and disease knowledge data set corresponding to the agricultural field, extract host and pest and disease entities from the pest and disease knowledge data set to obtain host plant entities and pest and disease entities corresponding to the agricultural field; construct a pest and disease knowledge graph for the pest and disease knowledge data set based on the host plant entities and pest and disease entities corresponding to the agricultural field to generate a pest and disease knowledge graph;

[0006] A to-be-matched visual semantic fusion module, which is used to obtain a to-be-matched pest and disease picture, perform histogram equalization processing on the to-be-matched pest and disease picture to generate a to-be-matched standard pest and disease picture sample; perform visual semantic feature fusion on the to-be-matched standard pest and disease picture sample to generate a to-be-matched pest and disease visual semantic knowledge feature vector;

[0007] The pest and disease case similarity measurement module is used to perform case similarity calculation on the corresponding entity semantics in the pest and disease knowledge graph based on the visual semantic knowledge feature vector of the pest and disease to be matched, so as to obtain the case entity semantic similarity between the to-be-matched picture sample and each case in the knowledge graph;

[0008] The pest and disease case prevention and control suggestion module is used to perform case matching and sorting on the corresponding pest and disease cases in the pest and disease knowledge graph based on the case entity semantic similarity between the to-be-matched picture sample and each case in the knowledge graph, so as to generate the case matching result corresponding to the to-be-matched pest and disease picture sample; generate the corresponding pest and disease case prevention and control suggestion plan according to the case matching result corresponding to the to-be-matched pest and disease picture sample, including the prevention and control suggestions corresponding to similar and dissimilar matches, so as to execute the corresponding pest and disease case prevention and control suggestion work.

[0009] Furthermore, the pest and disease knowledge graph construction module includes the following functions:

[0010] Collect the pest and disease knowledge data set corresponding to the agricultural field;

[0011] Extract the host plant entities from the pest and disease knowledge data set to obtain the host plant entities corresponding to the agricultural field;

[0012] Extract the pest and disease entities from the pest and disease knowledge data set to obtain the pest and disease entities corresponding to the agricultural field;

[0013] Perform entity knowledge attribute analysis on the host plant entities and pest and disease entities corresponding to the agricultural field to obtain the entity knowledge attributes corresponding to each host plant entity and the corresponding pest and disease entity;

[0014] Based on the entity knowledge attributes corresponding to each host plant entity and the corresponding pest and disease entity, extract the entity knowledge association relationship between the host plant entities and pest and disease entities corresponding to the agricultural field, so as to obtain the entity knowledge relationship between each host plant entity and the corresponding pest and disease entity, including parasitic relationship, causal relationship and prevention and control relationship;

[0015] Based on the entity knowledge relationship between each host plant entity and the corresponding pest and disease entity, and combined with the corresponding host plant entity and pest and disease entity, construct the pest and disease knowledge graph for the pest and disease prevention and control knowledge in the pest and disease knowledge data set, taking the corresponding host plant entity and pest and disease entity as knowledge nodes, the entity knowledge relationship as edges, and marking the corresponding prevention and control knowledge at each pest and disease entity node, so as to generate the pest and disease knowledge graph.

[0016] Further, the pest and disease knowledge dataset specifically includes data on the names, classifications, case symptom manifestations, host plants, occurrence environments, and control methods corresponding to various pests and diseases in the agricultural field.

[0017] Further, the to-be-matched visual semantic fusion module includes the following functions:

[0018] Obtain the to-be-matched pest and disease picture;

[0019] Grayscale the to-be-matched pest and disease picture to generate a grayscale image of the to-be-matched pest and disease; analyze the pixel blurriness of the grayscale image of the to-be-matched pest and disease to obtain the pixel blurriness of the to-be-matched pest and disease image.

[0020] Perform pixel blurring noise reduction on the to-be-matched pest and disease picture based on the pixel blurriness of the to-be-matched pest and disease image to generate a noise-reduced picture of the to-be-matched pest and disease.

[0021] Perform histogram equalization on the noise-reduced picture of the to-be-matched pest and disease to generate a standard picture sample of the to-be-matched pest and disease.

[0022] Perform visual semantic feature fusion on the standard picture sample of the to-be-matched pest and disease to generate a visual semantic knowledge feature vector of the to-be-matched pest and disease.

[0023] Further, the performing visual semantic feature fusion on the standard picture sample of the to-be-matched pest and disease includes:

[0024] Extract the visual features of the to-be-matched pest and disease from the standard picture sample of the to-be-matched pest and disease to obtain the visual features of the to-be-matched pest and disease, including the lesion shape, contour, color change features of the crops in the picture, and the category and quantity features of the pests.

[0025] Perform visual semantic concept mining and analysis on the visual features of the to-be-matched pest and disease by combining the knowledge in the pest and disease field to establish a mapping relationship between the visual features of the pest and disease and the semantic concepts of the pest and disease, and obtain a set of visual semantic concepts of the to-be-matched pest and disease.

[0026] Perform visual semantic feature fusion on each visual semantic concept in the set of visual semantic concepts of the to-be-matched pest and disease to generate a visual semantic knowledge feature vector of the to-be-matched pest and disease.

[0027] Further, the performing visual semantic feature fusion on each visual semantic concept in the set of visual semantic concepts of the to-be-matched pest and disease includes:

[0028] Perform semantic word segmentation and embedding processing on each visual semantic concept in the set of visual semantic concepts of the to-be-matched pest and disease to obtain a visual semantic word embedding sub-vector corresponding to each to-be-matched visual semantic concept.

[0029] Use the text encoder corresponding to the Transformer to perform additional context-aware encoding on the visual semantic word embedding sub-vectors corresponding to each visual semantic concept to be matched, so as to generate the word context-aware sub-vectors corresponding to each visual semantic concept to be matched;

[0030] Obtain the corresponding visual semantic entities to be matched through the word context-aware sub-vectors corresponding to each visual semantic concept to be matched, and construct a semantic relationship network between the visual semantic entities to be matched corresponding to each visual semantic concept to be matched, so as to analyze the semantic logical relationships between each semantic concept, including causal relationships, parallel relationships, and subordinate relationships, and connect them into the corresponding semantic relationship network to generate the visual semantic relationship network to be matched;

[0031] Determine the association weights between different visual semantic concepts to be matched according to the visual semantic relationship network to be matched, and perform visual semantic feature fusion on the visual semantic word embedding sub-vectors corresponding to each visual semantic concept to be matched based on the association weights between different visual semantic concepts to be matched, so as to generate the visual semantic knowledge feature vectors of the pests and diseases to be matched.

[0032] Further, the pests and diseases case similarity measurement module includes the following functions:

[0033] Obtain the visual semantic entity attributes corresponding to the visual semantic knowledge to be matched through the visual semantic knowledge feature vectors of the pests and diseases to be matched;

[0034] Obtain the corresponding entity semantic path lengths through each entity semantic knowledge in the pests and diseases knowledge graph;

[0035] Based on the visual semantic entity attributes corresponding to the visual semantic knowledge to be matched and the corresponding entity semantic path lengths in the pests and diseases knowledge graph, use the case entity semantic similarity calculation formula to perform case similarity calculation on the corresponding entity semantic knowledge attributes in the pests and diseases knowledge graph, so as to obtain the case entity semantic similarity between the to-be-matched picture sample and each case in the knowledge graph.

[0036] Further, the case entity semantic similarity calculation formula is specifically:

[0037]

[0038] In the formula, S b is the case entity semantic similarity between the to-be-matched picture sample and the case in the knowledge graph, α is the entity semantic similarity weight coefficient, exp is the exponential function, s1 is the visual semantic entity attribute corresponding to the visual semantic knowledge to be matched, s2 is the corresponding entity semantic knowledge attribute in the pests and diseases knowledge graph, σ is the visual semantic matching attenuation factor, L pis the length of the entity semantic path corresponding to the pest knowledge graph, L m is the maximum path length between all entities in the pest and disease knowledge graph, D is the case entity expansion, and η is the correction coefficient of the case entity semantic similarity.

[0039] Furthermore, the pest and disease case prevention and control suggestion module includes the following functions:

[0040] Based on the semantic similarity of the case entities corresponding to the image sample to be matched and each case in the knowledge graph, the case matching judgment is performed on the corresponding pest and disease cases in the pest and disease knowledge graph. If the semantic similarity of the corresponding case entity is greater than or equal to the preset case semantic similarity threshold, the case matching result is considered to be similar between the image sample to be matched and the corresponding case in the pest and disease knowledge graph; if the semantic similarity of the corresponding case entity is less than the preset case semantic similarity threshold, the case matching result is considered to be dissimilar between the image sample to be matched and any case in the pest and disease knowledge graph, so as to generate a case matching result corresponding to the image sample to be matched;

[0041] If the case matching result is a similar match, the first five corresponding cases in the pest knowledge graph are sorted from high to low and their corresponding prevention and control knowledge references are integrated as the pest control suggestions corresponding to the image sample to be matched, and the case control suggestions corresponding to the similar match are generated to execute the corresponding similar pest case control suggestions;

[0042] If the case matching result is a dissimilar match, the prevention and control knowledge corresponding to the first 10 cases in the pest and disease knowledge graph is obtained in descending order, and based on the prevention and control knowledge corresponding to the first 10 cases, the prevention and control fine-tuning suggestion analysis is performed on the pest and disease cases corresponding to the image sample to be matched, and the case prevention and control suggestions corresponding to the dissimilar match are generated to execute the corresponding prevention and control suggestion work for the dissimilar pest and disease cases.

[0043] Furthermore, the case prevention and control recommendations corresponding to the matching dissimilarities specifically use the causal relationship and inference rules of the prevention and control knowledge corresponding to the first 10 cases in the pest and disease knowledge graph to derive the prevention and control fine-tuning recommendations corresponding to the image sample to be matched, including adjusting the dosage and frequency of the agent according to the severity of the pest and disease, and updating and optimizing the corresponding pest and disease knowledge graph after the corresponding prevention and control fine-tuning recommendations are verified by experts.

[0044] Beneficial effects of the present invention:

[0045] The pest and disease picture matching case system based on the knowledge graph proposed by the present invention is generally composed of a pest and disease knowledge graph construction module, a visual semantic fusion module to be matched, a pest and disease case similarity measurement module, and a pest and disease case prevention and control suggestion module. Compared with the prior art, the beneficial effect of this application is that by constructing a pest and disease knowledge graph, it is possible to better understand the relationship between pests and diseases and host plants, and provide data support for precise prevention and control. Collect pest and disease knowledge data sets in related agricultural fields. These data can be sourced from agricultural research literature, agricultural production practices, and online agricultural platforms, covering a large amount of information related to pests and diseases, including the types of pests and diseases, occurrence patterns, and impacts on host plants. Through systematic collation and analysis of these data, a detailed pest and disease knowledge database can be formed. Next, extraction of host and pest and disease entities is carried out. The host plant entity refers to the plant invaded by pests and diseases, while the pest and disease entity refers to harmful organisms such as specific pests or pathogens. Through natural language processing techniques (such as named entity recognition, relation extraction, etc.), specific pest and disease entities and host plant entities can be identified from pest and disease-related literature, and redundant information can be refined into structured and semantic content. At the same time, by constructing a pest and disease knowledge graph based on the identified entity information, the relationship between the host plant and the corresponding pest and disease entities can be clearly displayed, forming a visual knowledge system to help researchers and agricultural producers quickly identify potential pest and disease problems and guide decision-making, enabling subsequent pest and disease matching prevention and control strategies to be formulated more precisely. Information on the pest and disease risks of host plants and targeted prevention and control measures can be obtained by querying the knowledge graph. Secondly, by analyzing real-time pest and disease pictures, the types and development stages of pests and diseases can be accurately identified, providing a visual basis for prevention and control work. These pictures can be from real-time shooting at agricultural production sites or from various pest and disease databases, covering pictures of various crops and their common pests and diseases. By performing histogram equalization processing on the pest and disease pictures to be matched, this processing step can improve the contrast of the image, making the details in the image clearer, thereby enhancing the recognizability of pest and disease characteristics. After image preprocessing, it enters the visual semantic feature fusion stage. By using deep learning techniques (such as convolutional neural networks, CNN), visual features can be extracted from pest and disease pictures, including information such as texture, shape, and color. These features can reflect the appearance characteristics of pests and diseases and be associated with the corresponding pest and disease types. At the same time, by combining semantic information (such as the names and symptoms of pests and diseases), visual features and semantic features are fused to generate visual semantic knowledge feature vectors of pests and diseases. These vectors not only contain the morphological information of pests and diseases but also contain semantic content related to pests and diseases, enhancing the recognition accuracy of the images.Then, a similarity calculation method based on visual semantic feature vectors is used to match the pest and disease pictures to be matched with the cases in the knowledge graph. Through the visual semantic knowledge feature vectors obtained in the previous step, the features of the pictures to be matched can be compared with the features of each case in the knowledge graph. It can quantify the similarity between the pest and disease pictures and each case in the knowledge graph. This similarity metric enables the quantification of the correlation between the pest and disease pictures and the known cases in the database, thereby providing a reference for matching and prevention and control suggestions. In this way, the most relevant cases can be efficiently screened out, and prevention and control suggestions can be provided based on the similarity. It can also effectively integrate the images with the knowledge graph data and improve the matching efficiency. Finally, based on the similarity between the pest and disease pictures and the cases calculated previously, case matching sorting is performed. The purpose of this process is to sort the pictures to be matched with the most similar cases in the knowledge graph in order to generate corresponding prevention and control suggestions. Once the case matching sorting is completed, corresponding prevention and control suggestions can be generated. According to the similarity level, targeted prevention and control measures will be provided. For cases with high matching degrees, the system will recommend similar prevention and control methods, which are usually based on existing research and practical experience; for cases with low matching degrees, different prevention and control suggestions will be provided to verify and update the corresponding knowledge graph, helping to identify potential pest and disease types and response strategies. This process not only makes pest and disease prevention and control more accurate, but also can propose customized prevention and control plans according to the characteristics of different crops, thereby better reducing the mis-matching rate corresponding to the pest and disease picture cases. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Other features, objects, and advantages of the present invention will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0047] Figure 1 It is a schematic diagram of the modules of the pest and disease picture matching case system based on the knowledge graph of the present invention;

[0048] Figure 2 is Figure 1 a schematic diagram of the functional flow of the pest and disease knowledge graph construction module in

[0049] Figure 3 is Figure 1 a schematic diagram of the functional flow of the visual semantic fusion module to be matched in DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The technical system of the present invention will be clearly and completely described below with reference to the drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0051] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0052] It should be understood that although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0053] To achieve the above object, please refer to Figures 1 to 3 , the present invention provides a pest and disease picture matching case system based on a knowledge graph, and the system includes the following modules:

[0054] A pest and disease knowledge graph construction module, which is used to collect a pest and disease knowledge data set corresponding to the agricultural field, extract host and pest and disease entities from the pest and disease knowledge data set to obtain host plant entities and pest and disease entities corresponding to the agricultural field; construct a pest and disease knowledge graph based on the host plant entities and pest and disease entities corresponding to the agricultural field in the pest and disease knowledge data set to generate a pest and disease knowledge graph;

[0055] A visual semantic fusion module to be matched, which is used to obtain a pest and disease picture to be matched, perform histogram equalization processing on the pest and disease picture to be matched to generate a standard picture sample of the pest and disease to be matched; perform visual semantic feature fusion on the standard picture sample of the pest and disease to be matched to generate a visual semantic knowledge feature vector of the pest and disease to be matched;

[0056] A pest and disease case similarity measurement module, which is used to perform case similarity calculation on the entity semantic knowledge corresponding to the pest and disease knowledge graph based on the visual semantic knowledge feature vector of the pest and disease to be matched to obtain the case entity semantic similarity between the picture sample to be matched and each case in the knowledge graph;

[0057] The pest and disease case prevention and control suggestion module is used to perform case matching and sorting processing on the corresponding pest and disease cases in the pest and disease knowledge graph based on the semantic similarity of the case entities corresponding to each case in the knowledge graph and the picture sample to be matched, so as to generate a case matching result corresponding to the pest and disease picture sample to be matched; generate a corresponding pest and disease case prevention and control suggestion plan according to the case matching result corresponding to the pest and disease picture sample to be matched, including case prevention and control suggestions corresponding to similar matches and dissimilar matches, so as to execute the corresponding pest and disease case prevention and control suggestion work.

[0058] In the embodiment of the present invention, please refer to Figure 1 As shown, it is a module schematic diagram of the pest and disease picture matching case system based on the knowledge graph of the present invention. In this example, the pest and disease picture matching case system based on the knowledge graph includes the following modules:

[0059] S1: The pest and disease knowledge graph construction module is used to collect the pest and disease knowledge data set corresponding to the agricultural field, and extract the host and pest and disease entities from the pest and disease knowledge data set to obtain the host plant entities and pest and disease entities corresponding to the agricultural field; construct the pest and disease knowledge graph based on the host plant entities and pest and disease entities corresponding to the agricultural field to generate the pest and disease knowledge graph;

[0060] In the embodiment of the present invention, text data containing plant host information and pest and disease characteristics is collected through multiple channels such as relevant agricultural literature, scientific research reports, and professional websites. For the collected data, the named entity recognition (NER) method in natural language processing technology is used to perform entity extraction on the data, and the host plants and pest and disease entities are identified. For example, deep learning models such as conditional random fields (CRF) or BERT are used to train the text, extract the host plants and pest and disease entities, and label them. After this processing, the entity information of the host plants and pest and disease can be obtained. Subsequently, this information is sorted in the form of triples (host plant - pest and disease - prevention method) to construct the pest and disease knowledge graph. On this basis, a graph construction tool (such as Neo4j or RDF) is used to store the relationships between the host plant and pest and disease entities to form a structured pest and disease knowledge graph.

[0061] S2: The visual semantic fusion module to be matched is used to obtain the pest and disease picture to be matched, and perform histogram equalization processing on the pest and disease picture to be matched to generate a standard picture sample of the pest and disease to be matched; perform visual semantic feature fusion on the standard picture sample of the pest and disease to be matched to generate a visual semantic knowledge feature vector of the pest and disease to be matched;

[0062] In the embodiments of the present invention, by obtaining pest and disease pictures to be matched from an image library in the agricultural field, the image sources may include farmland monitoring devices, drone photography, etc. Subsequently, the histogram equalization method is used to preprocess the pictures to be matched to enhance the contrast of the images and ensure that the pest and disease features in the images are clearer. Histogram equalization equalizes the pixel distribution of the images, making the brightness distribution of the images more uniform, thereby improving the effect of feature extraction. Specifically, when operating, an image processing library such as OpenCV is used to perform histogram equalization processing, so as to generate a standardized pest and disease picture sample to be matched. Then, the image processed by histogram equalization is subjected to visual semantic feature extraction. By using a deep convolutional neural network (CNN), such as ResNet or VGG, the visual features of the image are extracted and fused with the semantic information in the image. The fusion method can form a unified visual semantic feature vector by combining the pixel features of the image and text information (such as pest and disease names, symptoms, etc.). This vector contains the detailed features and semantic information of the pest and disease images, and finally generates a visual semantic knowledge feature vector of the pest and disease to be matched.

[0063] S3: The pest and disease case similarity measurement module is used to perform case similarity calculation on the corresponding entity semantic knowledge in the pest and disease knowledge graph based on the visual semantic knowledge feature vector of the pest and disease to be matched, so as to obtain the case entity semantic similarity corresponding to each case in the knowledge graph and the picture sample to be matched.

[0064] In the embodiments of the present invention, by calculating the visual semantic knowledge feature vector corresponding to the pest and disease picture to be matched and matching it with the entity semantic information in the pest and disease knowledge graph, by calculating the similarity between the to-be-matched feature vector and the semantic features of each pest and disease entity in the knowledge graph, methods such as cosine similarity or Euclidean distance or the calculation method corresponding to the semantic knowledge attributes in the present invention are used to calculate the similarity between the to-be-matched picture and each case entity in the knowledge graph. Specifically, when implementing, first, the features of each pest and disease entity in the pest and disease knowledge graph are extracted to form a semantic feature vector. Then, by calculating the similarity between these semantic feature vectors and the visual semantic feature vector of the to-be-matched image, the similarity value of each case is obtained. In this way, the case most similar to the pest and disease picture to be matched can be effectively screened out, and finally the case entity semantic similarity corresponding to each case in the knowledge graph and the picture sample to be matched is obtained.

[0065] S4: Pest and disease case prevention and control advice module, which is used to perform case matching and sorting processing on the corresponding pest and disease cases in the pest and disease knowledge graph based on the semantic similarity of the case entities corresponding to each case in the knowledge graph and the to-be-matched picture sample, so as to generate a case matching result corresponding to the to-be-matched pest and disease picture sample; generate a corresponding pest and disease case prevention and control advice plan according to the case matching result corresponding to the to-be-matched pest and disease picture sample, including case prevention and control advice corresponding to similar matches and dissimilar matches, so as to execute the corresponding pest and disease case prevention and control advice work.

[0066] In the embodiment of the present invention, by performing case matching and sorting according to the similarity values between the previously obtained pest and disease images and each pest and disease entity in the knowledge graph. In this step, all matching results are sorted through a sorting algorithm (such as an inverted index based on cosine similarity) to obtain the most relevant pest and disease cases. For cases with high similarity, they are directly classified as matching cases, and corresponding prevention and control advice is generated according to the prevention and control methods of the pests and diseases. For cases with relatively low similarity, they can be regarded as dissimilar cases, and a series of prevention and control advice is generated in combination with other information in the graph. These prevention and control advice include chemical pesticides or biological control methods recommended according to the types of pests and diseases, or planting management measures, etc., to ensure that users can take scientific and effective prevention and control measures according to the actual pest and disease problems, and finally execute the corresponding pest and disease case prevention and control advice work.

[0067] Further, the pest and disease knowledge graph construction module includes the following functions:

[0068] Collect the pest and disease knowledge data set corresponding to the agricultural field;

[0069] Extract the host plant entities from the pest and disease knowledge data set to obtain the host plant entities corresponding to the agricultural field;

[0070] Extract the pest and disease entities from the pest and disease knowledge data set to obtain the pest and disease entities corresponding to the agricultural field;

[0071] Perform entity knowledge attribute analysis on the host plant entities and pest and disease entities corresponding to the agricultural field to obtain the entity knowledge attributes corresponding to each host plant entity and the corresponding pest and disease entity;

[0072] Extract the entity knowledge association relationships between the host plant entities and pest and disease entities corresponding to the agricultural field based on the entity knowledge attributes corresponding to each host plant entity and the corresponding pest and disease entity to obtain the entity knowledge relationships between each host plant entity and the corresponding pest and disease entity, including parasitic relationships, causal relationships, and prevention and control relationships;

[0073] Construct a pest and disease knowledge graph for the pest and disease prevention knowledge in the pest and disease knowledge dataset based on the entity knowledge relationships between each host plant entity and the corresponding pest and disease entity, combined with the corresponding host plant entity and pest and disease entity. Use the corresponding host plant entity and pest and disease entity as knowledge nodes, the entity knowledge relationship as an edge, and label the corresponding prevention knowledge at each pest and disease entity node to generate the pest and disease knowledge graph.

[0074] As an embodiment of the present invention, refer to Figure 2 shown in Figure 1 the functional flowchart of the pest and disease knowledge graph construction module in

[0075] S11: Collect the pest and disease knowledge dataset corresponding to the agricultural field;

[0076] In the embodiment of the present invention, based on various resources such as relevant agricultural literature, databases, expert knowledge, and agronomic survey data in the agricultural field, the dataset should include the names, classifications, case symptom manifestations, host plants, occurrence environments, and prevention methods of various pests and diseases in the agricultural field. During the collection process, public data resources in the agricultural field can be relied on, such as agricultural data centers, databases of scientific research institutions, etc.; data can also be supplemented through methods such as questionnaires of agricultural experts and summaries of farmers' experiences. To ensure the accuracy and comprehensiveness of the dataset, each pest and disease needs to be carefully labeled. Especially in terms of symptom manifestations, typical visible symptoms should be listed, such as leaf discoloration, wilting, lesions, etc., and according to different pest and disease types, describe their impacts on host plants. At the same time, the prevention method part should include various methods such as biological control, chemical control, and agronomic control. For each prevention method, the specific dosage, timing, and method of use need to be listed in detail, and finally, the pest and disease knowledge dataset corresponding to the agricultural field is obtained.

[0077] S12: Extract host plant entities from the pest and disease knowledge dataset to obtain host plant entities corresponding to the agricultural field;

[0078] In the embodiments of the present invention, after collecting the pest and disease knowledge dataset, host plant entity extraction is performed. This step requires the use of natural language processing techniques, specifically including the named entity recognition (NER) method to identify the host plant names in the dataset. First, a named entity recognition model based on deep learning (such as BERT, BiLSTM-CRF, etc.) is used to train the model to identify the plant entities mentioned in agricultural pest and disease knowledge. These entities may appear in symptom descriptions, occurrence environment descriptions, or control methods. To improve the accuracy of entity recognition, a domain-specific vocabulary and an agricultural domain term dictionary are used to ensure that the model can correctly identify plant names such as "rice", "wheat", "apple tree", etc. At the same time, some common host plant names are supplemented through rule and dictionary matching to avoid omission or misidentification, and finally, the corresponding host plant entities in the agricultural domain are obtained.

[0079] S13: Perform pest and disease entity extraction on the pest and disease knowledge dataset to obtain the corresponding pest and disease entities in the agricultural domain;

[0080] In the embodiments of the present invention, after extracting the host plant entities, the next step is to extract the pest and disease entities. Similar to the host plant entity extraction, the pest and disease entity extraction also relies on the named entity recognition technology. By using a pre-trained NER model or a rule-based method, the pest and disease names involved in the dataset can be identified. The pest and disease entities can be identified by parsing the text data related to pests and diseases (such as symptom manifestations, occurrence environment descriptions, etc.). During this process, special attention should be paid to the fact that there are multiple forms of expression for pest and disease names, such as "rice blast" and "rice rice blast". The model needs to be able to handle issues such as synonyms, spelling mistakes, and different naming methods. Therefore, techniques such as reinforcement learning can be used to fine-tune the NER model to improve the recognition performance in specific agricultural domain texts, and finally, the corresponding pest and disease entities in the agricultural domain are obtained.

[0081] S14: Perform entity knowledge attribute analysis between the host plant entities and pest and disease entities corresponding to the agricultural domain to obtain the corresponding entity knowledge attributes between each host plant entity and the corresponding pest and disease entity;

[0082] In an embodiment of the present invention, once the host plant entity and the pest entity have been extracted, the next step is to analyze the entity knowledge attributes between the host plant entity and the pest entity. The main task of this step is to identify the relationship between the host plant and the pest and extract the corresponding attribute information. For example, by analyzing the symptom manifestations and the growth environment of the host plant, it is identified whether a certain pest is particularly likely to occur in a specific environment. This process relies on machine learning and knowledge graph construction technologies. Clustering algorithms, association rule mining and other methods can be used to mine the connections between plants and pests. This analysis not only helps to understand the corresponding relationships between different pests and host plants, but also can extract the conditions for the occurrence of pests, such as environmental factors like temperature and humidity, as well as the specific impacts of pests on plants, further improving the accuracy of the model, and finally obtaining the corresponding entity knowledge attributes between each host plant entity and the corresponding pest entity.

[0083] S15: Extract the entity knowledge association relationship between the host plant entity corresponding to the agricultural field and the pest entity based on the corresponding entity knowledge attributes between each host plant entity and the corresponding pest entity, so as to obtain the entity knowledge relationship between each host plant entity and the corresponding pest entity, including parasitic relationship, causal relationship and control relationship;

[0084] In an embodiment of the present invention, by extracting the knowledge relationship between entities from the entity knowledge attributes between the host plant entity and the pest entity, mainly including parasitic relationship, causal relationship and control relationship. To achieve this goal, a relationship-based graph model (such as an RDF graph model) can be constructed to analyze the association between the host plant and the pest in the dataset. Specifically, natural language processing technologies such as dependency syntax analysis and entity relationship extraction can be used to automatically identify and classify the relationships between different entities. The parasitic relationship reflects how the pest attaches to the host plant, the causal relationship indicates how the pest causes plant diseases or death, and the control relationship associates the control methods with the countermeasures for pests. During the extraction process, combined with the context information and the agricultural expert knowledge base, the accuracy and rationality of the relationship are ensured, and finally the corresponding entity knowledge attributes between each host plant entity and the corresponding pest entity are obtained.

[0085] S16: Construct a pest knowledge graph for the pest control knowledge in the pest knowledge dataset based on the entity knowledge relationship between each host plant entity and the corresponding pest entity and in combination with the corresponding host plant entity and the pest entity, taking the corresponding host plant entity and the pest entity as knowledge nodes, the entity knowledge relationship as edges, and annotating the corresponding control knowledge at each pest entity node to generate a pest knowledge graph.

[0086] In an embodiment of the present invention, after entity knowledge relation extraction is completed, a knowledge graph corresponding to pests and diseases in this field is constructed. The core of this step is to organize host plant entities, pest and disease entities, and the relationships between them in the form of a graph. Each host plant entity and pest and disease entity will serve as a node in the knowledge graph, and the entity relationships between the nodes (such as parasitic relationships, causal relationships, control relationships, etc.) will serve as edges in the graph, forming a directed graph. Each pest and disease entity node is attached with detailed control knowledge tags, such as control methods, dosages, timing, and other information. To construct the graph, a graph database tool, such as Neo4j, is usually selected, or a graph processing framework, such as Apache TinkerPop, is used for graph storage and query. When constructing the graph, combined with manual expert review to ensure the accuracy of control knowledge, and through regular updates to ensure the timeliness of the pest and disease knowledge graph, and finally generate the pest and disease knowledge graph.

[0087] Further, the pest and disease knowledge dataset specifically includes data such as the names, classifications, case symptom manifestations, host plants, occurrence environments, and control methods corresponding to various pests and diseases in the agricultural field.

[0088] Further, the to-be-matched visual semantic fusion module includes the following functions:

[0089] Obtain the to-be-matched pest and disease picture;

[0090] Perform grayscale processing on the to-be-matched pest and disease picture to generate a grayscale picture of the to-be-matched pest and disease; perform pixel blurriness analysis on the grayscale picture of the to-be-matched pest and disease to obtain the pixel blurriness of the to-be-matched pest and disease picture;

[0091] Perform pixel blur noise reduction on the to-be-matched pest and disease picture based on the pixel blurriness of the to-be-matched pest and disease picture to generate a noise-reduced picture of the to-be-matched pest and disease;

[0092] Perform histogram equalization processing on the noise-reduced picture of the to-be-matched pest and disease to generate a standard picture sample of the to-be-matched pest and disease;

[0093] Perform visual semantic feature fusion on the standard picture sample of the to-be-matched pest and disease to generate a visual semantic knowledge feature vector of the to-be-matched pest and disease.

[0094] As an embodiment of the present invention, refer to Figure 3 shown, for Figure 1 the functional flowchart of the to-be-matched visual semantic fusion module in, in this embodiment, the to-be-matched visual semantic fusion module includes the following functions:

[0095] S21: Obtain the to-be-matched pest and disease picture;

[0096] In the embodiments of the present invention, in the process of matching pest and disease pictures, first, a high-resolution imaging device (such as a digital camera, a high-definition scanner, etc.) is required to obtain the pest and disease pictures to be matched. The obtained pictures should cover the pest and disease situations under different perspectives and different illuminations to ensure the diversity and integrity of the pictures to be matched. After the pictures are obtained, it should be ensured that the resolution of the pictures meets the requirements of subsequent image processing and feature extraction. Generally, the resolution is required to be not less than 1920x1080 pixels to ensure the accurate capture of details. For different types of pests and diseases, relevant attributes of each picture, such as the imaging device, ambient light conditions, shooting time, etc., need to be recorded separately, and finally, the pest and disease pictures to be matched are obtained.

[0097] S22: Perform grayscale processing on the pest and disease pictures to be matched to generate grayscale pest and disease pictures to be matched; perform pixel blurriness analysis on the grayscale pest and disease pictures to be matched to obtain the pixel blurriness of the pest and disease pictures to be matched.

[0098] In the embodiments of the present invention, by performing grayscale processing on the obtained pest and disease pictures to be matched, the grayscale processing process reduces the complexity of the image by converting the color image into a black-and-white image, reduces color interference, and enhances the morphological features of pests and diseases. The grayscale processing is performed by weighted averaging the RGB values of each pixel. The formula is: I_gray = 0.299×R + 0.587×G + 0.114×B, where R, G, and B are the values of the red, green, and blue channels respectively, and I_gray is the value of the generated grayscale image. After generating the grayscale image, further perform pixel blurriness analysis on it. Specifically, by calculating the neighborhood variance of each pixel point, the smoothness of the image area is evaluated. The higher the blurriness, the more blurred the detailed information in the image; the lower the blurriness means the more clear the details of the image. Finally, the pixel blurriness of the pest and disease pictures to be matched is obtained.

[0099] S23: Perform pixel blur noise reduction on the pest and disease pictures to be matched based on the pixel blurriness of the pest and disease pictures to be matched to generate noise-reduced pest and disease pictures to be matched.

[0100] In the embodiment of the present invention, by selecting an appropriate noise reduction algorithm to process the pest and disease picture to be matched according to the previously obtained pixel blurriness, common noise reduction techniques include median filtering, Gaussian filtering, bilateral filtering, etc. For images with high blurriness, Gaussian filtering is used for blurring noise reduction. The specific operation is to perform weighted averaging on the neighboring pixels around each pixel point and use the Gaussian kernel function to smooth the noise. For images with low blurriness, median filtering is used to retain the edge information of the image and filter out the noise. During the processing, the noise reduction algorithm automatically selects the appropriate filter kernel size and type according to the blurriness value of the image to achieve the best noise reduction effect. The picture generated after noise reduction processing should have good detail retention, the noise is effectively suppressed, and the image clarity is improved. Finally, a pest and disease noise reduction picture to be matched is generated.

[0101] S24: Perform histogram equalization on the pest and disease noise reduction picture to be matched to generate a standard picture sample of the pest and disease to be matched;

[0102] In the embodiment of the present invention, by performing histogram equalization on the pest and disease noise reduction picture to be matched, this method adjusts the distribution of pixel values in the image to make the gray level distribution of the image more uniform, reduces the phenomenon of uneven brightness, and improves the detail performance of the image. The specific implementation is to first count the gray histogram of the image, calculate the cumulative distribution function (CDF), and then adjust the gray value of each pixel point through the CDF. Finally, an image after histogram equalization is generated. During this process, an adaptive equalization technique is adopted to avoid over-enhancing certain image regions. The pest and disease picture after histogram equalization processing has clearer colors and more distinct details. Finally, a standard picture sample of the pest and disease to be matched is generated.

[0103] S25: Perform visual semantic feature fusion on the standard picture sample of the pest and disease to be matched to generate a visual semantic knowledge feature vector of the pest and disease to be matched.

[0104] In an embodiment of the present invention, based on the generated standard pest and disease picture samples, their visual features are extracted and fused with semantic features. The extraction of visual features is usually completed by a convolutional neural network (CNN). CNN can learn high-level visual features from images, such as textures, shapes, and edges. Commonly used network structures such as VGG and ResNet can automatically extract the deep features of images after training. At the same time, through natural language processing technology, semantic information about pests and diseases, such as types, symptoms, and harms, is obtained from the knowledge graph. These semantic information are fused with the visual features of the images to form a unified knowledge feature vector. During the feature fusion process, a weighted fusion strategy is adopted to effectively combine visual features and semantic features, enhancing the semantic understanding ability of pest and disease images. The generated visual-semantic knowledge feature vector of the pest and disease to be matched not only contains the visual information of the image but also includes semantic-level knowledge such as the types, symptoms, and harm levels of pests and diseases, and finally, a visual-semantic knowledge feature vector of the pest and disease to be matched is generated by fusion.

[0105] Further, the visual-semantic feature fusion of the standard picture sample of the pest and disease to be matched includes:

[0106] Extract the visual features of the pest and disease from the standard picture sample of the pest and disease to be matched to obtain the visual features of the pest and disease to be matched, including the shape, contour, color change features of the crop disease spots corresponding to the crops in the picture and the category and quantity features of the pests.

[0107] In an embodiment of the present invention, by extracting the visual features of the pest and disease from the standard picture sample of the pest and disease to be matched, the purpose is to extract visual features from the picture that can distinguish different types of pests and diseases. First, the picture is preprocessed through image processing algorithms, including denoising, enhancing contrast, and standardizing the size to ensure the quality of the picture. Then, a deep convolutional neural network (CNN) is used to extract features from the image. CNN can automatically learn features such as the shape, contour, and color change of the disease spots. For the disease spot features, extract their shapes (such as circular, oval, irregular), edge contours (such as sharp, blurred), and color changes (such as brown, yellow, or green changes). For the pest features, object detection technologies (such as YOLO or Faster R-CNN) are used to locate the positions of the pests, and the quantity and types of the pests are calculated. In this way, the visual features of the disease spots and pests in the picture to be matched can be accurately extracted, and a feature vector containing information such as the shape, contour, color change of the disease spots and the category and quantity of the pests is generated, and finally, the visual features of the pest and disease to be matched are obtained.

[0108] Preferably, by combining the knowledge in the field of pests and diseases, visual semantic concept mining and analysis are carried out on the visual characteristics of pests and diseases to be matched, so as to establish a mapping relationship between the visual characteristics of pests and diseases and the semantic concepts of pests and diseases, and obtain the set of visual semantic concepts of pests and diseases to be matched;

[0109] In the embodiment of the present invention, during the process of analyzing the visual characteristics of pests and diseases to be matched, by combining the knowledge in the field of pests and diseases, semantic concept mining and analysis are carried out on the visual characteristics, so as to compare the visual characteristics extracted by the image recognition model with the classification information of diseases and pests in the existing knowledge graph of pests and diseases. First, according to the domain knowledge, the visual characteristics of each type of pest and disease are determined. For example, different types of disease spots may present different colors, shapes, distributions, etc. Then, semantic analysis tools (such as natural language processing technology and association rule analysis methods) are used to label the visual characteristics of pests and diseases, and a mapping is established according to the correspondence between the labels and the pests and diseases. For example, by analyzing the relationship between the characteristics such as the shape and color of the disease spot and a certain type of pest and disease, a mapping relationship such as "yellow disease spot = a certain disease" is formed. In this way, semantic concept mining can be carried out on the visual characteristics of pests and diseases to be matched. For example, a specific disease spot shape is semantically represented as "round necrotic spot", and a certain color feature is semantically represented as "leaf chlorosis", etc. Finally, the set of visual semantic concepts of pests and diseases to be matched is obtained.

[0110] Preferably, visual semantic feature fusion is carried out on each visual semantic concept in the set of visual semantic concepts of pests and diseases to be matched, so as to generate a visual semantic knowledge feature vector of pests and diseases to be matched.

[0111] In the embodiment of the present invention, by performing feature fusion on each visual semantic concept in the set of visual semantic concepts of pests and diseases to be matched, first, for each visual semantic concept of pests and diseases to be matched, multi-dimensional fusion of its corresponding visual features and semantic labels is carried out. By means of weighted average or splicing, the extracted visual features such as the shape, color, and contour of the disease spot are combined with the known semantic information (such as disease type, pest species, quantity, etc.) to generate a comprehensive feature vector. This feature vector includes the mapping relationship between the visual characteristics of pests and diseases and the semantic concepts, and can effectively represent the overall characteristics of pests and diseases. Specifically, classification algorithms such as support vector machine (SVM) and decision tree can be used to further process the fused features to ensure that they can be accurately compared and matched with the features in the pest and disease database during the subsequent matching process. The generated visual semantic knowledge feature vector not only contains the specific visual features of the image, but also fuses the semantic information of pests and diseases, and finally fuses to generate a visual semantic knowledge feature vector of pests and diseases to be matched.

[0112] Further, the visual semantic feature fusion for each pest and disease visual semantic concept in the set of pest and disease visual semantic concepts to be matched includes:

[0113] Performing semantic word segmentation and embedding processing on each pest and disease visual semantic concept in the set of pest and disease visual semantic concepts to be matched to obtain a visual semantic word embedding sub-vector corresponding to each pest and disease visual semantic concept to be matched;

[0114] In the embodiments of the present invention, for each pest and disease visual semantic concept to be matched, semantic word segmentation and embedding processing are first performed. Specifically, a pre-trained word vector model (such as Word2Vec, GloVe, or BERT) is used to process the pest and disease visual semantic concepts. First, the pest and disease visual semantic concepts are transformed into a set of words or phrases through semantic word segmentation technology. Each word or phrase represents a specific pest and disease feature or attribute. Then, using word vector embedding technology, each word or phrase is mapped to a high-dimensional vector. These vectors contain the position of the word in the semantic space and can better represent its meaning. The dimension of the embedding vector is usually set between 300 and 768 dimensions, and the specific dimension depends on the pre-trained model used. Through this process, a word embedding sub-vector of each pest and disease visual semantic concept to be matched can be obtained, and finally a visual semantic word embedding sub-vector corresponding to each pest and disease visual semantic concept to be matched is obtained.

[0115] Preferably, using the text encoder corresponding to Transformer to perform additional context-aware encoding on the visual semantic word embedding sub-vectors corresponding to each pest and disease visual semantic concept to be matched to generate a word context-aware sub-vector corresponding to each pest and disease visual semantic concept to be matched;

[0116] In the embodiments of the present invention, by performing context-aware encoding on the visual semantic word embedding sub-vectors of each pest and disease visual semantic concept to be matched, the text encoder module in the Transformer architecture (such as the encoder part in the BERT model) is used to perform additional context-aware encoding. First, the input visual semantic word embedding sub-vectors are processed through a multi-layer self-attention mechanism. The self-attention mechanism can capture the context relationship between words. By considering the mutual influence of each word and other words, more contextually relevant word vectors are generated. In this process, each word vector not only retains its independent semantic information but also fuses its information in the sentence or context. After encoding, a context-aware sub-vector corresponding to each visual semantic concept is obtained. These vectors have stronger context understanding ability and can help the system identify the fine-grained relationships between words. Finally, a word context-aware sub-vector corresponding to each pest and disease visual semantic concept to be matched is encoded and generated.

[0117] Preferably, the corresponding visual semantic entity to be matched is obtained through the word context-aware sub-vector corresponding to each visual semantic concept to be matched, and a semantic relationship network is constructed among the visual semantic entities to be matched corresponding to each visual semantic concept to be matched, so as to analyze the semantic logical relationships among each semantic concept, including causal relationship, parallel relationship and subordinate relationship, and connect them into a corresponding semantic relationship network to generate a visual semantic relationship network to be matched;

[0118] In the embodiment of the present invention, a semantic relationship network is further constructed based on the context-aware sub-vector of each visual semantic concept. Specifically, first, the specific pest and disease entities corresponding to each visual semantic concept are identified through the context-aware sub-vector. These entities usually represent specific pest and disease types or specific symptoms of pests and diseases in the image. Then, the system analyzes the semantic logical relationships (such as causal relationship, parallel relationship, subordinate relationship, etc.) between each visual semantic concept and its corresponding entity, and connects these entity nodes in the form of a graph model to form a semantic relationship network. For example, in some cases, the occurrence of a specific pest and disease may cause the lesion of some plants, thus forming a causal relationship; in other cases, two different pests and diseases share similar visual features, forming a parallel relationship. All these relationships will be encoded in the form of a graph to generate a visual semantic relationship network to be matched, describing the mutual associations among each pest and disease concept and entity. Furthermore, for example, there is a causal relationship between "germ infection" and "appearance of disease spots on leaves", "fungal diseases" and "bacterial diseases" are in a parallel relationship, and "powdery mildew" is subordinate to "fungal diseases". Using graph theory knowledge, taking semantic concepts as nodes and semantic relationships as edges, a semantic relationship network graph is constructed, and finally a visual semantic relationship network to be matched is generated.

[0119] Preferably, the association weight between different visual semantic concepts to be matched is determined according to the visual semantic relationship network to be matched, and the visual semantic feature fusion is performed on the visual semantic word embedding sub-vector corresponding to each visual semantic concept to be matched based on the association weight between different visual semantic concepts to be matched, so as to generate a visual semantic knowledge feature vector of pests and diseases to be matched.

[0120] In an embodiment of the present invention, the association weights between each visual semantic concept to be matched are determined through a visual semantic relationship network to be matched. The calculation of the association weights is usually based on the relationship strength, semantic similarity, and context information between entities. Specifically, methods such as graph neural network (GNN) algorithms based on graph convolutional network (GCN) or graph attention network (GAT) can be used to calculate the association strength between each node through the network structure. After calculating the association weights, the visual semantic word embedding sub-vectors are fused based on these weights. The fusion methods usually include weighted average, concatenation, or multi-layer perceptron (MLP), etc. Among them, the weights determine the importance of each visual semantic word embedding sub-vector in the final fusion. Through this process, a comprehensive visual semantic knowledge feature vector is generated. This vector combines the original embedding information of each visual semantic concept and its relationship information in the knowledge graph, thereby effectively improving the accuracy of pest and disease identification and matching, and finally generating a visual semantic knowledge feature vector of the pest and disease to be matched.

[0121] Furthermore, the pest and disease case similarity measurement module includes the following functions:

[0122] Obtain the visual semantic entity attributes corresponding to the visual semantic knowledge to be matched through the visual semantic knowledge feature vector of the pest and disease to be matched;

[0123] In an embodiment of the present invention, the semantic information of the pest and disease picture to be matched is obtained through the visual feature vector of the pest and disease picture to be matched. The specific operation is as follows: Using a deep learning algorithm, an image feature extraction network (such as ResNet, VGG, etc.) is used to extract features from the pest and disease picture to generate a visual feature vector of the picture. Subsequently, through semantic embedding technology, the image feature vector is mapped to a pre-trained pest and disease visual semantic knowledge base, which contains semantic knowledge features of various pests and diseases. Using similarity calculation (such as cosine similarity), the pest and disease knowledge feature vector with the highest similarity to the feature of the pest and disease picture to be matched is found, and then the visual semantic entity attributes corresponding to the picture are determined, such as the type, symptoms, occurrence stage, etc. of the pest and disease, and finally the visual semantic entity attributes corresponding to the visual semantic knowledge to be matched are obtained.

[0124] Preferably, the corresponding entity semantic path length is obtained through each entity semantic knowledge in the pest and disease knowledge graph;

[0125] In the embodiments of the present invention, semantic knowledge paths of each entity in the pest and disease knowledge graph are extracted. Each entity in the knowledge graph has a path representing its relationship, and the length of the path indicates the distance between this entity and other related entities. Specifically, each entity in the knowledge graph is connected to several other entities through multiple layers of relationships, and the path length is the number of relationship layers required to go from this entity to the target entity. In the implementation process, through graph algorithms (such as graph traversal algorithms or shortest path algorithms), the semantic path lengths between the pest and disease entity to be matched and other entities in the graph are calculated, and these path lengths will be used as the basic data for subsequent calculations to evaluate the relative importance and relevance of each entity in the knowledge graph, and finally the corresponding entity semantic path lengths are obtained.

[0126] Preferably, based on the visual semantic entity attributes corresponding to the visual semantic knowledge to be matched and the entity semantic path lengths corresponding in the pest and disease knowledge graph, a case entity semantic similarity calculation formula is used to perform case similarity calculation on the corresponding entity semantic knowledge attributes in the pest and disease knowledge graph, so as to obtain the case entity semantic similarity between the picture sample to be matched and each case in the knowledge graph.

[0127] In the embodiments of the present invention, by combining the visual semantic entity attributes corresponding to the visual semantic knowledge to be matched, the corresponding entity semantic knowledge attributes in the pest and disease knowledge graph, the visual semantic matching attenuation factor, the corresponding entity semantic path lengths in the pest and disease knowledge graph, the maximum path length between all entities in the pest and disease knowledge graph, the case entity expansion degree, the entity semantic similarity weight coefficient, and relevant parameters, a suitable case entity semantic similarity calculation formula is constructed to perform case corresponding similarity calculation, so as to quantitatively obtain the case entity semantic similarity between the picture sample to be matched and each case in the knowledge graph. In addition, this case entity semantic similarity calculation formula can also use any similarity measurement method in the art (such as cosine similarity or semantic matching algorithms of deep learning) to replace the process of case similarity calculation, and is not limited to this case entity semantic similarity calculation formula.

[0128] Furthermore, the case entity semantic similarity calculation formula is specifically:

[0129]

[0130] In the formula, S b is the case entity semantic similarity between the picture sample to be matched and the case in the knowledge graph, α is the entity semantic similarity weight coefficient, exp is the exponential function, s1 is the visual semantic entity attribute corresponding to the visual semantic knowledge to be matched, s2 is the corresponding entity semantic knowledge attribute in the pest and disease knowledge graph, σ is the visual semantic matching attenuation factor, L pis the corresponding entity semantic path length in the pest and disease knowledge graph, L m is the maximum path length between all entities in the pest and disease knowledge graph, D is the case entity expansion degree, and η is the correction coefficient of the case entity semantic similarity.

[0131] The present invention obtains a case entity semantic similarity calculation formula through the use of a specific mathematical model and verification, which is used to calculate the case similarity of the corresponding entity semantic knowledge attributes in the pest and disease knowledge graph. This case entity semantic similarity calculation formula considers the visual semantic entity attributes (such as pest and disease characteristics) of the picture sample to be matched and each case in the pest and disease knowledge graph, and can carefully calculate the similarity between the cases in the graph and the picture to be matched. In this way, the accuracy and matching efficiency of pest and disease diagnosis can be improved. By introducing a weight coefficient and a visual semantic matching attenuation factor, the formula can not only calculate the similarity according to the attribute differences between entities, but also attenuate according to the change of similarity, making more accurate matching possible. For example, the greater the gap between the visual semantic knowledge attributes to be matched and the attributes in the pest and disease knowledge graph, the greater the influence of the attenuation factor, and the smaller the calculated similarity, avoiding irrelevant or incorrect matching. The factor of path length enhances the structural information of the knowledge graph by controlling the influence of path length on similarity, ensuring that the calculation result is still reliable in the case of a relatively complex entity semantic relationship in the knowledge graph. Through the path length, the formula not only considers semantic similarity, but also considers the hierarchical structure of these similarities in the knowledge graph, making the matching of complex relationships more delicate. By considering the case entity expansion degree, the similarity calculation of pest and disease entities can be adjusted to adapt to the expansion and coverage of the knowledge graph in different situations, avoiding overmatching and under-matching. Through the expansion degree, the formula can adjust the sparsity or density of the relationships between cases in the graph, thereby adjusting the sensitivity and accuracy of matching under different knowledge graph structures. In addition, by introducing a correction coefficient to finally correct the similarity, the calculation result can better meet the actual needs. For example, the importance or confidence of pest and disease cases in certain situations can be adjusted through the correction coefficient, improving the rationality and reliability of the final matching result. In summary, the formula fully considers the case entity semantic similarity S between the picture sample to be matched and the cases in the knowledge graph b , the entity semantic similarity weight coefficient α, the exponential function exp, the visual semantic entity attribute s1 corresponding to the visual semantic knowledge to be matched, the entity semantic knowledge attribute s2 corresponding in the pest and disease knowledge graph, the visual semantic matching attenuation factor σ, and the corresponding entity semantic path length L in the pest and disease knowledge graph p , the maximum path length L between all entities in the pest and disease knowledge graph m, the case entity expansion degree D, the correction coefficient η of the case entity semantic similarity, and the case entity semantic similarity S corresponding between the to-be-matched picture sample and the cases in the knowledge graph b The mutual correlation relationship between the above parameters constitutes a functional relationship This formula can realize the case similarity calculation process of the entity semantic knowledge attributes corresponding in the pest and disease knowledge graph. At the same time, by introducing the correction coefficient η of the case entity semantic similarity, it can be adjusted according to the error situation in the calculation process, thereby improving the accuracy and applicability of the case entity semantic similarity calculation formula.

[0132] Furthermore, the pest and disease case prevention and control suggestion module includes the following functions:

[0133] Based on the case entity semantic similarity corresponding between the to-be-matched picture sample and each case in the knowledge graph, perform case matching judgment on the corresponding pest and disease cases in the pest and disease knowledge graph. If the corresponding case entity semantic similarity is greater than or equal to the preset case semantic similarity threshold, it is considered that the case matching result is that the to-be-matched picture sample matches and is similar to the corresponding case in the pest and disease knowledge graph; if the corresponding case entity semantic similarity is less than the preset case semantic similarity threshold, it is considered that the case matching result is that the to-be-matched picture sample does not match and is similar to any case in the pest and disease knowledge graph, so as to generate the case matching result corresponding to the to-be-matched pest and disease picture sample.

[0134] In the embodiment of the present invention, by combining the case entity semantic similarity corresponding between the to-be-matched picture sample and each case in the knowledge graph obtained by previous quantification and comparing it with the corresponding case semantic similarity threshold, when the entity semantic similarity between the to-be-matched picture sample and a certain case is greater than or equal to the preset case semantic similarity threshold, it is considered that the case matches and is similar to the to-be-matched picture sample. If the corresponding case entity semantic similarity is less than the preset case semantic similarity threshold, it is considered that the match is not similar. According to the matching result of each case, finally generate the case matching result corresponding to the to-be-matched pest and disease picture sample.

[0135] Preferably, if the case matching result is matching and similar, then sort the top 5 corresponding cases in the pest and disease knowledge graph from high to low and fuse their corresponding prevention and control knowledge references as the pest and disease prevention and control suggestions corresponding to the to-be-matched picture sample, generate the case prevention and control suggestions corresponding to the matching similarity, so as to execute the corresponding matching and similar pest and disease case prevention and control suggestion work;

[0136] In an embodiment of the present invention, if the obtained case matching result is "matched and similar", then the result is further processed. First, it is necessary to sort the cases in the pest and disease knowledge graph according to semantic similarity, and select the top five cases with the highest semantic similarity to the image sample to be matched. Sorting algorithms such as quicksort or mergesort can be used to sort by similarity from high to low to ensure that the most similar cases are ranked at the top. Subsequently, the prevention and control knowledge of these five cases is extracted. The prevention and control knowledge usually includes content such as the use of pesticides, application time, and application method. The specific information in the case (such as the type of pest and disease, the affected part, etc.) can be combined to optimize the prevention and control suggestions to ensure their pertinence. The prevention and control knowledge of these five cases is integrated to form the final pest and disease prevention and control suggestions. The content of the prevention and control suggestions can include the recommended types of drugs, dosage, application frequency, etc., and corresponding treatment plans for the pests and diseases of the image sample to be matched are provided based on this knowledge. Finally, the case prevention and control suggestions corresponding to the matched similarity are generated to implement the corresponding work of preventing and controlling the pests and diseases of the matched similarity cases.

[0137] Preferably, if the case matching result is "not matched and similar", then the prevention and control knowledge corresponding to the top 10 cases in the pest and disease knowledge graph is obtained by sorting from high to low, and the prevention and control fine-tuning suggestions for the pest and disease cases corresponding to the image sample to be matched are analyzed based on the prevention and control knowledge corresponding to the top 10 cases, and the case prevention and control suggestions corresponding to the not matched and similar are generated to implement the corresponding work of preventing and controlling the pests and diseases of the not matched and similar cases.

[0138] In an embodiment of the present invention, if the obtained case matching result is "not matched and similar", then it is sorted from high to low according to similarity, and the top ten most similar cases in the pest and disease knowledge graph are selected. The prevention and control knowledge of these cases can include different types of pesticide selection, application techniques, environmental factors, etc. At this stage, the integration of prevention and control knowledge serves as the basis for fine-tuning analysis. First, based on the prevention and control knowledge of the top ten cases, personalized fine-tuning is carried out considering the types of pests and diseases, the severity of the disease, and other relevant factors. For example, in the case of the pests and diseases of the image sample to be matched, the dosage and application frequency of the pesticide are adjusted according to the severity of the disease. This process combines the causal relationships and inference rules in the pest and disease knowledge graph. For example, the occurrence of certain pests and diseases may be related to specific climatic conditions, or too high an application frequency may lead to drug resistance. After the fine-tuning suggestions are generated, they need to be optimized and confirmed through an expert verification process. The experts will review these suggestions and put forward modification opinions to ensure that the final suggestions are accurate and effective. The fine-tuning suggestions verified by the experts can timely update and optimize the pest and disease knowledge graph so that it can provide more accurate prevention and control solutions in the future. Finally, the case prevention and control suggestions corresponding to the not matched and similar are generated to implement the corresponding work of preventing and controlling the pests and diseases of the not matched and similar cases.

[0139] Further, the specific prevention and control suggestions for the cases corresponding to the matching dissimilarities are to derive the corresponding prevention and control fine-tuning suggestions for the to-be-matched picture sample by using the causal relationships and inference rules of the prevention and control knowledge of the first 10 cases in the pest and disease knowledge graph, including adjusting the dosage and application frequency of the medicament according to the severity corresponding to the pest and disease, and updating and optimizing the corresponding pest and disease knowledge graph after verifying the corresponding prevention and control fine-tuning suggestions by experts.

[0140] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the application documents are intended to be included in the present invention.

[0141] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A pest and disease image matching case system based on knowledge graph, characterized in that: Includes the following modules: The pest and disease knowledge graph construction module is used to collect the pest and disease knowledge dataset corresponding to the agricultural field, and extract the host and pest and disease entities from the pest and disease knowledge dataset to obtain the host plant entities and pest and disease entities corresponding to the agricultural field; Based on the host plant entities and pest entities corresponding to the agricultural field, the pest knowledge dataset is used to construct a pest knowledge graph to generate a pest knowledge graph; The to-be-matched visual semantic fusion module is used to obtain the to-be-matched pest and disease pictures, and perform histogram equalization on the to-be-matched pest and disease pictures to generate to-be-matched pest and disease standard picture samples; perform visual semantic feature fusion on the to-be-matched pest and disease standard picture samples to generate to-be-matched pest and disease visual semantic knowledge feature vectors; The pest and disease case similarity measurement module is used to calculate the case similarity of the corresponding entity semantic knowledge in the pest and disease knowledge graph based on the visual semantic knowledge feature vector of the pest and disease to be matched, so as to obtain the case entity semantic similarity between the image sample to be matched and each case in the knowledge graph; The pest and disease case prevention and control recommendation module is used to perform case matching and sorting processing on the corresponding pest and disease cases in the pest and disease knowledge graph based on the semantic similarity of the case entities corresponding to the image samples to be matched and each case in the knowledge graph, so as to generate case matching results corresponding to the pest and disease image samples to be matched; and generate corresponding pest and disease case prevention and control recommendation plans based on the case matching results corresponding to the pest and disease image samples to be matched, including case prevention and control recommendations corresponding to matching similarities and matching dissimilarities, so as to execute the corresponding pest and disease case prevention and control recommendation work.

2. The pest and disease picture matching case system based on knowledge graph according to claim 1 is characterized in that: The pest and disease knowledge graph construction module includes the following functions: Collect pest and disease knowledge datasets corresponding to the agricultural field; Extract host plant entities from the pest and disease knowledge dataset to obtain the corresponding host plant entities in the agricultural field; Extract pest and disease entities from the pest and disease knowledge dataset to obtain the pest and disease entities corresponding to the agricultural field; Perform entity knowledge attribute analysis on the host plant entities and pest and disease entities corresponding to the agricultural field to obtain the corresponding entity knowledge attributes between each host plant entity and the corresponding pest and disease entity; Based on the corresponding entity knowledge attributes between each host plant entity and the corresponding pest entity, the entity knowledge association relationship between the host plant entity and the pest entity corresponding to the agricultural field is extracted to obtain the entity knowledge relationship between each host plant entity and the corresponding pest entity, including parasitic relationship, causative relationship and control relationship; Based on the entity knowledge relationship between each host plant entity and the corresponding pest and disease entity and in combination with the corresponding host plant entity and pest and disease entity, a pest and disease knowledge graph is constructed for the prevention and control knowledge in the pest and disease knowledge dataset, so that the corresponding host plant entity and pest and disease entity are used as knowledge nodes, the entity knowledge relationship is used as the edge, and the corresponding prevention and control knowledge is marked at each pest and disease entity node to generate a pest and disease knowledge graph.

3. The pest and disease picture matching case system based on knowledge graph according to claim 2 is characterized in that: The pest and disease knowledge data set specifically includes the names, classifications, case symptoms, host plants, occurrence environments, and control method data corresponding to various pests and diseases in the agricultural field.

4. The pest and disease picture matching case system based on knowledge graph according to claim 1 is characterized in that: The visual semantic fusion module to be matched includes the following functions: Obtain the pest and disease pictures to be matched; The pest and disease picture to be matched is grayed to generate a grayscale image of the pest and disease to be matched; pixel fuzziness analysis is performed on the grayscale image of the pest and disease to be matched to obtain pixel fuzziness of the image of the pest and disease to be matched; Based on the pixel fuzziness of the to-be-matched pest and disease image, pixel fuzzy denoising is performed on the to-be-matched pest and disease image to generate a denoised pest and disease image to be matched; Performing histogram equalization on the pest and disease denoising images to be matched to generate pest and disease standard image samples to be matched; Visual semantic feature fusion is performed on the standard image samples of the pests and diseases to be matched to generate the visual semantic knowledge feature vector of the pests and diseases to be matched.

5. The pest and disease picture matching case system based on knowledge graph according to claim 4 is characterized in that: The visual semantic feature fusion of the matched pest and disease standard picture samples includes: Extract pest visual features from the pest standard image samples to be matched, and obtain the pest visual features to be matched, including the shape, outline, and color change characteristics of the disease spots corresponding to the crops in the images, and the category and quantity characteristics of the pests; By combining pest and disease domain knowledge to conduct visual semantic concept mining and analysis on the visual features of pests and diseases to be matched, a mapping relationship between the visual features of pests and diseases and the semantic concepts of pests and diseases is established, and a set of visual semantic concepts of pests and diseases to be matched is obtained; Visual semantic feature fusion is performed on each pest and disease visual semantic concept in the pest and disease visual semantic concept set to be matched, so as to generate a pest and disease visual semantic knowledge feature vector.

6. The pest and disease picture matching case system based on knowledge graph according to claim 5 is characterized in that: The visual semantic feature fusion of each pest visual semantic concept in the pest visual semantic concept set to be matched comprises: Perform semantic word segmentation and embedding processing on each pest and disease visual semantic concept in the pest and disease visual semantic concept set to be matched, so as to obtain a visual semantic word embedding sub-vector corresponding to each visual semantic concept to be matched; Use the text encoder corresponding to the Transformer to perform additional context-aware encoding on the visual semantic word embedding sub-vector corresponding to each visual semantic concept to be matched, so as to generate a word context-aware sub-vector corresponding to each visual semantic concept to be matched; The corresponding visual semantic entity to be matched is obtained through the word context-aware sub-vector corresponding to each visual semantic concept to be matched, and a semantic relationship network is constructed between the visual semantic entities to be matched corresponding to each visual semantic concept to be matched, so as to analyze the semantic logical relationship between each semantic concept, including causal relationship, parallel relationship and subordinate relationship, and connect them into a corresponding semantic relationship network to generate a visual semantic relationship network to be matched; The association weights between different visual semantic concepts to be matched are determined according to the visual semantic relationship network to be matched, and the visual semantic features of the visual semantic word embedding sub-vectors corresponding to each visual semantic concept to be matched are fused based on the association weights between different visual semantic concepts to be matched to generate a visual semantic knowledge feature vector of pests and diseases to be matched.

7. The pest and disease picture matching case system based on knowledge graph according to claim 1, characterized in that: The pest case similarity measurement module includes the following functions: Obtaining visual semantic entity attributes corresponding to the visual semantic knowledge to be matched through the feature vector of the visual semantic knowledge of the pests and diseases to be matched; Obtain the corresponding entity semantic path length through each entity semantic knowledge in the pest and disease knowledge graph; Based on the visual semantic entity attributes corresponding to the visual semantic knowledge to be matched and the corresponding entity semantic path length in the pest and disease knowledge graph, the case similarity calculation formula is used to calculate the corresponding entity semantic knowledge attributes in the pest and disease knowledge graph to obtain the corresponding case entity semantic similarity between the image sample to be matched and each case in the knowledge graph.

8. The pest and disease picture matching case system based on knowledge graph according to claim 7 is characterized in that: The case entity semantic similarity calculation formula is specifically: In the formula, S b is the case entity semantic similarity between the image sample to be matched and the case in the knowledge graph, α is the entity semantic similarity weight coefficient, exp is the exponential function, s1 is the visual semantic entity attribute corresponding to the visual semantic knowledge to be matched, s2 is the corresponding entity semantic knowledge attribute in the pest knowledge graph, σ is the visual semantic matching attenuation factor, L p is the length of the entity semantic path corresponding to the pest knowledge graph, L m is the maximum path length between all entities in the pest and disease knowledge graph, D is the case entity expansion, and η is the correction coefficient of the case entity semantic similarity.

9. The pest and disease picture matching case system based on knowledge graph according to claim 1, characterized in that: The pest and disease case control suggestion module includes the following functions: Based on the semantic similarity of the case entities corresponding to the image sample to be matched and each case in the knowledge graph, the case matching judgment is performed on the corresponding pest and disease cases in the pest and disease knowledge graph. If the semantic similarity of the corresponding case entities is greater than or equal to the preset case semantic similarity threshold, the case matching result is considered to be similar between the image sample to be matched and the corresponding case in the pest and disease knowledge graph; If the semantic similarity of the corresponding case entity is less than the preset case semantic similarity threshold, the case matching result is considered to be a dissimilar match between the image sample to be matched and any case in the pest and disease knowledge graph, so as to generate a case matching result corresponding to the image sample to be matched; If the case matching result is a similar match, the first five corresponding cases in the pest knowledge graph are sorted from high to low and their corresponding prevention and control knowledge references are integrated as the pest control suggestions corresponding to the image sample to be matched, and the case control suggestions corresponding to the similar match are generated to execute the corresponding similar pest case control suggestions; If the case matching result is a dissimilar match, the prevention and control knowledge corresponding to the first 10 cases in the pest and disease knowledge graph is obtained in descending order, and based on the prevention and control knowledge corresponding to the first 10 cases, the prevention and control fine-tuning suggestion analysis is performed on the pest and disease cases corresponding to the image sample to be matched, and the case prevention and control suggestions corresponding to the dissimilar match are generated to execute the corresponding prevention and control suggestion work for the dissimilar pest and disease cases.

10. The pest and disease picture matching case system based on knowledge graph according to claim 9, characterized in that: The matching of the case prevention and control suggestions corresponding to the dissimilar cases specifically uses the causal relationship and inference rules of the prevention and control knowledge corresponding to the first 10 cases in the pest and disease knowledge graph to derive the prevention and control fine-tuning suggestions corresponding to the image sample to be matched, including adjusting the dosage and frequency of the pesticide according to the severity of the pest and disease, and updating and optimizing the corresponding prevention and control fine-tuning suggestions after expert verification.

Images