Agricultural knowledge graph construction and retrieval method

Abstract

The invention discloses an agricultural knowledge graph construction and retrieval method, and relates to the field of knowledge graph technology, information retrieval technology and agricultural information technology. The method aims to cope with data fragmentation and isomerism challenges in agricultural informatization. The method comprises the following steps: firstly, collecting agricultural text data from multiple channels, performing data preprocessing by using SpaCy, and then developing a specific algorithm to integrate multi-source data; and data features in the agricultural field are optimized by training an adaptive deep learning model, and a large-scale corpus is adopted for fine tuning of the pre-training model, so that the accuracy and generalization ability of the pre-training model are improved. In order to construct a detailed agricultural field ontology, entities, attributes and relationships are automatically extracted and fused, and real-time monitoring and updating of data are realized in combination with a crawler technology; user query is analyzed and converted into formalized expression, complex logic operation is executed by means of an inference engine, and visual answers and natural dialogue experience are provided. And finally, in combination with real-time data, historical experience and expert knowledge, accurate decision suggestions are generated, and the agricultural field is accurately served.

Description

Technical Field

[0001] The present invention relates to the fields of knowledge graph technology, information retrieval technology and agricultural information technology, and in particular to a method for constructing and retrieving an agricultural knowledge graph. Background Art

[0002] With the development of agricultural informatization, a large amount of agricultural data is constantly generated, but the fragmentation and heterogeneity of data limit the effective use of data. How to effectively manage and use this data has become an urgent problem to be solved. Traditional retrieval methods are difficult to meet users' needs for comprehensive and accurate acquisition of agricultural knowledge. Therefore, building an agricultural knowledge graph and realizing systematic management and intelligent retrieval of knowledge have become important directions for the development of agricultural informatization.

[0003] At present, the agricultural field has accumulated a huge amount of data resources, including but not limited to crop growth data, pest control knowledge, agricultural policy information, market supply and demand conditions, etc. In order to further support the novelty and creativity of the present invention, the following is a detailed analysis of the specific defects of the closest prior art: (1) Agri-Know: A knowledge base focusing on the agricultural field, integrating a large amount of agricultural data resources, such as crop growth data, pest and disease control, etc. However, it lacks a real-time update mechanism and cannot reflect the latest agricultural research and technological progress in a timely manner; (2) Google Knowledge Graph and Microsoft Academic Knowledge Graph: Although these large general knowledge graphs have powerful natural language processing capabilities, they do not cover enough specific knowledge points in the agricultural field and cannot provide sufficient depth and expertise. (3) China National Knowledge Infrastructure (CNKI) Agricultural Science and Technology Literature Database: It focuses on the collection and organization of literature, but does not do enough to transform unstructured or semi-structured data into structured knowledge. It also needs to strengthen its user experience design. (4) Crop Ontology (CO) and Plant Ontology (PO): They create standardized terminology systems for plant sciences, which facilitate interoperability between different data sources. However, these ontology structures are relatively fixed and difficult to quickly adapt to the emergence of new technologies and concepts in the agricultural field.

[0004] However, most of the data involved in these existing technologies exist in unstructured or semi-structured forms, which are difficult to be effectively used and quickly retrieved, mainly in the following aspects: (1) Limitations of data source integration: Existing technologies can only integrate limited data sources, and the integration process lacks unified data standards and effective cleaning mechanisms, resulting in low data quality and difficulty in supporting high-quality knowledge graph construction. The data formats, standards, and quality of different sources vary greatly, making the data integration process complex and time-consuming; (2) Limitations of model training data: Due to the specificity and professionalism of the agricultural field, labeled data is relatively scarce, which limits the training effect of machine learning models. High-quality, large-scale labeled data is essential for training effective deep learning models, but such data is relatively scarce in the agricultural field, and it is difficult to accurately identify complex entities and relationships in the agricultural field. (3) Practicality issues in knowledge graph construction: Knowledge graphs constructed using existing technologies often lack timeliness and accuracy, making it difficult to update and reflect the latest knowledge in the agricultural field in real time. At the same time, the construction process is complex and time-consuming, requiring professional technology and tool support; (4) Limitations of user interaction and application: The user interaction interface of existing technologies is often not intuitive and easy to use, which leads to problems such as complex operations and unclear results during use. In addition, the application scenarios are relatively limited, making it difficult to meet users' needs for comprehensive and accurate acquisition of agricultural knowledge.

[0005] Therefore, a more comprehensive, efficient and intelligent agricultural knowledge graph construction and retrieval method is needed. This new technology should be able to improve the existing agricultural knowledge graph construction and retrieval methods, starting from data acquisition and processing, model training and optimization, practicality of knowledge graph construction, user interaction and application, etc., to improve the efficiency and accuracy of agricultural knowledge graph construction, and promote its wide application and in-depth development in the agricultural field. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a method for constructing and retrieving an agricultural knowledge graph. The core purpose of the present invention is to integrate multi-source dynamic data fusion with adaptive entity recognition and relationship extraction based on deep learning, precise anomaly detection and early warning system, as well as intelligent question-answering and reasoning engine and agricultural intelligent decision support system based on knowledge graph, to create a method for constructing and retrieving an agricultural knowledge graph. This innovation aims to improve the efficiency and accuracy of agricultural information management, promote the development of intelligent agriculture, promote the sharing and exchange of agricultural knowledge, and enhance the professional quality of agricultural practitioners. The realization of these goals will help promote the transformation, upgrading and sustainable development of the agricultural industry.

[0007] Specifically, the present invention aims to achieve in-depth analysis of agricultural texts through deep learning models, automatically identify and annotate key entities and their relationships; ensure the timeliness and accuracy of knowledge graphs through multi-source data fusion and adaptive evolution mechanisms; provide flexible and accurate agricultural knowledge services through intelligent question-answering and reasoning engines; and finally, provide farmers with accurate and scientific agricultural decision-making support in combination with intelligent decision-making algorithms. This series of technologies jointly promotes the development of agriculture towards intelligence, precision, and efficiency, improves agricultural production efficiency, reduces production costs, and promotes sustainable agricultural development.

[0008] To achieve the above objectives, an agricultural knowledge graph construction and retrieval method is implemented through the following technical solutions: S1: Data source integration and intelligent cleaning, first, collect text data in the agricultural field, including but not limited to government databases, scientific research institutions data, agricultural enterprise data, social media, agricultural forums, etc. Then use natural language processing (NLP) and machine learning technology to automatically identify and clean the noise, redundancy and erroneous information in the data. At the same time, develop special algorithms to deal with the format and encoding problems unique to agricultural data; finally, establish unified data standards, including data formats, encoding rules and semantic specifications, to ensure that data from different data sources can be seamlessly integrated; S2: Adaptive deep learning model training, select deep learning models suitable for agricultural text processing (BERT-based models, Transformer, etc.), and use large-scale agricultural corpora for pre-training. Design adaptive training strategies based on the particularity of agricultural texts. Dynamically adjust the learning rate, introduce domain knowledge to enhance training data, etc., to improve the model's ability to recognize agricultural entities and the accuracy of relationship extraction; S3: Knowledge graph construction and dynamic update, combining professional knowledge in the agricultural field, designing a detailed agricultural field ontology, including multiple sub-fields such as crops, soil, meteorology, pests and diseases, and agricultural policies. Using the trained deep learning model, automatically extract entities, attributes, and relationships from the cleaned data, and integrate knowledge from different sources into the knowledge graph through knowledge fusion technology. Establish a real-time data monitoring and update mechanism, use the Internet of Things (IoT) technology and Web crawler technology to monitor new knowledge, new technologies, and new pest and disease information in the agricultural field in real time, automatically integrate new data into the knowledge graph, and adjust and optimize the ontology structure; S4: Intelligent question-answering and reasoning engine, develop a natural language processing module to parse the user's natural language query and convert it into a form that the knowledge graph can understand. Use semantic analysis and fuzzy matching technology to improve the flexibility and accuracy of queries. Build a reasoning engine to execute complex query logic, such as path finding, pattern matching, etc. Generate easy-to-understand answers based on the reasoning results and provide them to users; S5: Agricultural intelligent decision support application: Develop customized intelligent decision models based on specific needs in the agricultural field. The model can combine real-time data, historical experience and expert knowledge to provide farmers with accurate decision-making suggestions on planting, fertilization, pest control, etc. Apply knowledge graphs and intelligent decision-making models to multiple agricultural fields, such as precision agriculture, pest and disease early warning, agricultural product traceability, and agricultural policy analysis, to improve agricultural production efficiency and management level. By collecting user feedback and evaluating the actual application effect, the accuracy and practicality of the intelligent decision-making model are continuously optimized, and the knowledge graph content is updated to ensure the timeliness and accuracy of the system.

[0009] Preferably, in said S1, it also includes: Collect data from the China Agricultural Information Network and the National Meteorological Information Center, write crawler and data cleaning scripts in Python, and integrate the data into a central database. Use the natural language processing (NLP) tool SpaCy for preliminary data cleaning, including removing stop words, punctuation, non-standard characters, etc. Develop algorithms specifically for agricultural data format and encoding issues, regular expression matching, and parsers for specific fields to ensure that data from different data sources can be seamlessly integrated. Solve the problem of large differences in data format, standards, and quality during data source integration, reduce manual intervention, and improve data integration efficiency.

[0010] Preferably, in said S2, further comprising: To select a suitable deep learning model, you can consider using or developing a pre-trained model specifically for the agricultural field. Pre-training with a large-scale agricultural corpus can collect a large-scale agricultural corpus from a variety of sources, including literature, policy documents, technical reports, etc., and then pre-process the data: clean and pre-process the data to remove noise and redundant information; then use the collected corpus to pre-train the selected deep learning model to better understand the language characteristics of the agricultural field. Designing an adaptive training strategy can dynamically adjust the learning rate, introduce domain knowledge to enhance training data, use pre-trained models in other fields (medicine, biology) as a starting point, migrate to the agricultural field, design a multi-task learning framework, and train multiple related tasks (entity recognition, relationship extraction) at the same time, sharing the underlying representation layer. Regularly evaluate the performance of the model on new data, and update and optimize it as needed. Improve the performance of deep learning models in text processing in the agricultural field to better support the construction and retrieval of agricultural knowledge graphs.

[0011] Preferably, in said S3, further comprising: Build an agricultural knowledge graph containing 1,000 entities and 500 relationships, use the trained deep learning model to automatically extract entities, attributes and relationships, and integrate knowledge from different sources into the knowledge graph through knowledge fusion technology. Use Neo4j graph database for storage and set up scripts to run automatically every day. Update data through Web crawler technology, obtain new knowledge, new technologies and new pests and diseases in the agricultural field in a timely manner, automatically integrate them into the knowledge graph, and adjust and optimize the ontology structure. Through real-time monitoring and dynamic update mechanisms, the timeliness and accuracy of the knowledge graph are guaranteed, overcoming the problem that the existing methods lack an effective dynamic update mechanism.

[0012] Preferably, in said S4, further comprising: The intelligent question-answering and reasoning engine aims to parse the user's natural language query, convert it into a form that the knowledge graph can understand, and use it to execute complex query logic, and finally generate easy-to-understand answers. Through deep understanding of user intent, processing synonyms and near-synonyms, identifying key entities, handling spelling errors and abbreviations, calculating semantic similarity, and then converting user queries into query language that the knowledge graph can understand, plus formulating rules to guide the reasoning process, dynamically adjusting strategies to generate reasoning results, extracting key information from them, ensuring accurate answers, and providing an interface to allow users to further inquire or provide feedback, generating a natural conversation experience.

[0013] Preferably, in said S5, further comprising: Develop an intelligent decision support system to provide farmers with planting recommendations based on real-time meteorological data and historical crop growth data. Apply knowledge graphs to precision agriculture, pest and disease early warning and other fields. Design an intuitive and easy-to-use user interface, deeply understand and flexibly handle user queries, improve the user interaction experience, so that even users without professional backgrounds can easily obtain the required information, and solve the problem of unfriendly user interaction interface and unclear result display.

[0014] Compared with the prior art, the present invention discloses a method for constructing and retrieving an agricultural knowledge graph. The present invention has the following beneficial effects: 1. Automatically identify and annotate key information in the agricultural field, reduce manual intervention, and improve the efficiency and automation of knowledge graph construction; use advanced deep learning models to more accurately identify complex entities and relationships in the agricultural field, improving the accuracy and reliability of knowledge graphs; 2. It can collect and process data from multiple channels in real time to ensure the timeliness and update of the knowledge graph. It integrates multi-source data and covers a wider range of agricultural knowledge, making the knowledge graph more comprehensive and rich. Through the adaptive evolution mechanism of domain ontology, combined with expert knowledge and machine learning algorithms, it realizes the intelligent update and maintenance of ontology, reduces the cost of manual maintenance, and improves the accuracy and stability of the knowledge graph; 3. Through semantic analysis and fuzzy matching technology, the flexibility and accuracy of queries are improved, and user needs can be quickly responded to. Combined with knowledge graph and graph reasoning technology, accurate and insightful answers can be generated to provide users with personalized agricultural knowledge services; 4. Combine real-time data, historical experience and expert knowledge to provide farmers with accurate decision-making suggestions and help them make more scientific and reasonable agricultural decisions. Through intelligent decision support, help farmers optimize the processes of planting, fertilization, pest control, etc., improve agricultural production efficiency and output, and reduce production costs. Through functions such as precision agriculture and pest warning, reduce the excessive use of pesticides and fertilizers, protect the ecological environment, and promote sustainable agricultural development. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative work: Figure 1 It is a flowchart of agricultural data fusion and knowledge graph construction driven by deep learning of the present invention; Figure 2 It is a flow chart of the agricultural intelligent question-answering and decision-making support system of the present invention. DETAILED DESCRIPTION

[0016] Step 1: Adaptive entity recognition and relationship extraction based on deep learning S11 Collect text data in the agricultural field from a variety of sources, including but not limited to government databases, scientific research institutions, agricultural enterprises, social media, agricultural forums, etc., to ensure the diversity and representativeness of the data to cover entities and relationships in different scenarios and contexts. Use natural language processing tools (SpaCy) to clean the text, including removing stop words, punctuation, non-standard characters, etc., and annotate the entities and relationships in the text data. The BIO annotation method (B represents the beginning of the entity, I represents the middle part of the entity, and O represents non-entity) can be used to annotate entities; S12 Select a deep learning model suitable for text processing, a model with BERT or Transformer architecture. Clean and preprocess the corpus to remove noise and redundant information, and use a large-scale general corpus (WikiText) to pre-train the selected model to learn the general features of the language; S13 fine-tunes the pre-trained model on a domain-specific corpus to adapt to the professional terminology and expression habits in the agricultural field. Using labeled training data, the model is trained to recognize entities in the text, and the accuracy of entity recognition is improved by adjusting model parameters and using different optimization algorithms (Adam, SGD, etc.). A multi-task learning framework is designed to train multiple related tasks at the same time, share the underlying representation layer, and improve the generalization ability of the model; S14 trains a relation extraction model between the identified entity pairs to determine the relationship between them, and uses the model's context understanding ability to improve the accuracy of relation extraction, especially when dealing with complex and ambiguous relationships. Use the validation set to evaluate the performance of the model, including precision, recall, and F1 score. Based on the evaluation results, adjust the model structure and parameters, and perform multiple iterations of optimization; S15 deploys the trained model to the server or cloud platform, provides an API interface, and uses the model to automatically identify and extract entities and their relationships in the actual agricultural knowledge graph construction process. At the same time, the model is updated and trained with new data to adapt to the changes and updates of agricultural knowledge. Using the Internet of Things (IoT) technology and Web crawler technology, new knowledge, new technologies, and new pests and diseases information in the agricultural field are monitored in real time, and new data is automatically integrated into the knowledge graph, and the ontology structure is adjusted and optimized to ensure the timeliness and accuracy of the knowledge graph.

[0017] Step 2: Multi-source dynamic data fusion and domain ontology adaptive evolution mechanism S21 determines the data sources that need to be integrated, including agricultural sensor data, literature, policy documents, Internet resources, etc. Collect data from the above data sources in real time or regularly through API interfaces, web crawlers, data import, etc., remove invalid, erroneous and redundant data, including outlier detection and correction, missing value processing, etc., and convert data from different sources into a unified format for subsequent processing; S22 designs domain ontology including concepts such as crops, soil, and meteorology based on the characteristics of the agricultural field, implements the domain ontology using languages ​​such as OWL or RDF, and stores it in a graph database. According to the characteristics of the data and application requirements, select appropriate fusion algorithms, such as weighted average and Kalman filtering, to integrate the results of multi-source data fusion with the domain ontology and update the knowledge graph; S23 designs an adaptive evolution mechanism for the domain ontology to adapt to the updates and changes in agricultural domain knowledge. It collects the needs and suggestions for ontology evolution through user feedback and the knowledge of domain experts. It automatically updates the domain ontology according to preset rules and machine learning algorithms, including adding new concepts, modifying relationships, etc. Domain experts review the automatically updated ontology to ensure the accuracy and rationality of the update; S24 Each stage should have corresponding documentation for subsequent review and tracking. Use strategies such as the Cyclic Acquisition Process to continuously collect new data and information to support the adaptive evolution of the ontology.

[0018] Step 3: Intelligent Question Answering and Reasoning Engine S31 cuts the text into smaller units, marks the part of speech for each word, removes meaningless words in the text, identifies entities in the text (names of people, places, etc.) and extracts the relationship between them, and converts the text into a vector representation for similarity calculation; S32 uses natural language processing technology to understand the intention and meaning of user questions and converts the knowledge, rules and constraints of the problem domain into a form that can be understood and processed by computers. Common knowledge representation methods include logical expressions, semantic networks, expert system rules, etc. S33 selects an appropriate reasoning mechanism for logical reasoning and problem solving based on the input knowledge and problem description. The reasoning mechanism can adopt forward reasoning, backward reasoning, mixed reasoning, etc. S34 fully tests the inference engine to ensure that it can correctly process and solve problems. Based on the test results, the system is optimized and adjusted to improve the performance and accuracy of the system.

[0019] Step 4: Agricultural intelligent decision support system based on knowledge graph S41 builds ontologies covering agricultural fields such as crops, soil, climate, pests and diseases, and integrates them into the knowledge graph. The collected agricultural data is filled into the knowledge graph, including historical data, real-time data and expert knowledge, and the collected data is preprocessed, including data cleaning, normalization and feature extraction; S42 selects an appropriate machine learning model, random forest, support vector machine or neural network, to generate decision recommendations, trains the model using historical data and expert knowledge, and optimizes model parameters to improve decision accuracy; S43 develops a rule engine that converts expert knowledge and the output of machine learning models into specific agricultural decision recommendations, implements decision path planning, and provides farmers with the best decision path based on current conditions and forecast results; S44 is designed with an intuitive user interface, allowing farmers and agricultural researchers to easily input parameters and view decision suggestions. It enables user interaction with the system, including parameter input, decision query and result display; S45 integrates knowledge graph, intelligent decision model and user interface into a unified system and conducts system testing, including unit testing, integration testing and user acceptance testing, to ensure the stability and availability of the system; S46 monitors system performance, including response time and user satisfaction, to ensure the system operates efficiently. Optimizes system functionality and performance based on feedback, including updating the knowledge graph, adjusting decision models, and improving the user interface. The knowledge graph is regularly updated to include new agricultural data, research results, and policy changes.

Claims

1. A method for constructing and retrieving an agricultural knowledge graph, characterized in that: The method comprises the following steps: S1: Data source integration and intelligent cleaning: collect text data in the agricultural field, use natural language processing (NLP) and machine learning technology to automatically identify and clean noise, redundancy and erroneous information in the data, and establish unified data standards; S2: Adaptive deep learning model training: select deep learning models suitable for agricultural text processing, use large-scale agricultural corpora for pre-training, design adaptive training strategies, and improve the model's ability to recognize agricultural entities and the accuracy of relationship extraction; S3: Knowledge graph construction and dynamic update: designing professional knowledge ontology in the agricultural field, using trained deep learning models to automatically extract entities, attributes and relationships from cleaned data, and integrating knowledge from different sources into the knowledge graph through knowledge fusion technology, and establishing a real-time data monitoring and update mechanism; S4: Intelligent question answering and reasoning engine, develop a natural language processing module to parse the user's natural language query, convert it into a form that the knowledge graph can understand, build a reasoning engine to execute complex query logic, and generate easy-to-understand answers; S5: Agricultural intelligent decision support application, develop customized intelligent decision-making models, combine real-time data, historical experience and expert knowledge to provide farmers with accurate decision-making suggestions on planting, fertilization, pest and disease control, etc.

2. The method according to claim 1, characterized in that The S1 further comprises: Data were collected from the China Agricultural Information Network and the National Meteorological Information Center; Use Python to write crawler and data cleaning scripts to integrate data into a central database; Use the natural language processing tool SpaCy for preliminary data cleaning; Develop algorithms specifically for agricultural data formatting and encoding issues to ensure data from different data sources can be seamlessly integrated.

3. The method according to claim 1, characterized in that The S2 further includes: When choosing a deep learning model, consider using or developing pre-trained models specifically for the agricultural domain; Use a large-scale agricultural corpus for pre-training; Design adaptive training strategies, including dynamically adjusting the learning rate, introducing domain knowledge to enhance training data, using pre-trained models from other fields as a starting point for transfer learning, and designing a multi-task learning framework to train multiple related tasks simultaneously.

4. The method according to claim 1, characterized in that: The S3 further includes: Build an agricultural knowledge graph containing multiple entities and relationships; Automatically extract entities, attributes, and relationships using trained deep learning models; Use Neo4j graph database for storage and set up automatic scripts to update data using Web crawler technology; The timeliness and accuracy of the knowledge graph are ensured through real-time monitoring and dynamic update mechanisms.

5. The method according to claim 1, characterized in that The S4 further comprises: Intelligent question answering and reasoning engine processes synonyms and near-synonyms, identifies key entities, handles spelling errors and abbreviations, and calculates semantic similarity by deeply understanding user intent; Convert the user's natural language query into a query language that the knowledge graph can understand; Formulate rules to guide the reasoning process and dynamically adjust strategies to generate reasoning results; Provide an interface to allow users to further inquire or provide feedback.

6. The method according to claim 1, characterized in that The S5 further includes: Develop intelligent decision support systems to provide farmers with planting recommendations based on real-time weather data and historical crop growth data; Apply knowledge graphs to areas such as precision agriculture and pest warning; Design intuitive and easy-to-use user interfaces to improve user interaction experience; Use knowledge graphs and intelligent decision-making models to generate decision-making recommendations and improve agricultural production efficiency and management level.

Images