An agricultural large model evaluation method and system based on a knowledge graph

Abstract

A knowledge graph-based evaluation method and system for large-scale agricultural models, relating to the field of machine learning, addresses the problems of existing large-scale agricultural model evaluation methods, such as lack of factual verification, uninterpretable scoring, and weak structure matching capabilities. The method includes: S1. Constructing an agricultural knowledge base; S2. Constructing an agricultural knowledge graph based on the agricultural knowledge base; S3. Using the agricultural knowledge graph, expanding the original user query's intent and entities to obtain an expanded natural language query; S4. Inputting the expanded natural language query into the large-scale model under test to obtain the output; S5. Combining the factual verification and text similarity verification to generate a comprehensive score, and achieving automated testing and feedback through a scoring platform. This invention has significant application prospects in the field of smart agriculture, and its knowledge graph-driven interpretable evaluation method provides core support for agricultural intelligence.

Description

Technical Field

[0001] This invention relates to the field of machine learning, specifically to a method and system for evaluating large agricultural models based on knowledge graphs. Background Technology

[0002] In recent years, with the development of artificial intelligence technology, large language models have shown great potential in applications such as agricultural knowledge services, intelligent question answering, and agricultural technology decision support. However, the agricultural knowledge system is complex and time-sensitive, involving multi-dimensional knowledge such as crop varieties, cultivation management, pest and disease control, and climate conditions. Model responses often suffer from unstable accuracy, incomplete logical chains, opaque reasoning processes, and lagging knowledge updates. Therefore, a scientific, objective, and interpretable evaluation system is urgently needed to quantify the true capabilities of models in the agricultural field. Existing evaluation methods mainly include:

[0003] 1) Expert scoring. Domain experts review each model's answer, which, while highly accurate, is extremely costly and unsuitable for evaluating large-scale models and frequent iterations.

[0004] 2) Accuracy evaluation based on static test sets. This involves manually constructing a small set of questions to assess the model's accuracy, but this method has limited coverage, a long update cycle, and cannot fully reflect the model's true performance.

[0005] 3) General model benchmark transfer, such as evaluation systems like MMLU and AGIEval, directly transfers general test sets to the agricultural field, but the specialization of agricultural knowledge and regional differences lead to large deviations in results;

[0006] 4) Semantic similarity-driven automatic scoring. This method only calculates the similarity between the language expressions of the generated answer and the standard answer, and cannot verify the correctness of facts and the rationality of logic, which can easily lead to "irrelevant answers" or "seemingly true but actually false" situations.

[0007] In summary, existing methods generally suffer from a lack of factual verification, poor interpretability, insufficient coverage, and inadequate evaluation granularity, making it difficult to meet the application needs of large-scale agricultural models in scientific research, production, and decision-making. To address this, this project proposes a knowledge graph-based evaluation scheme for large-scale agricultural models. This scheme integrates authoritative agricultural data and expert knowledge to construct a queryable and reasonable knowledge graph, abstracting agricultural knowledge into entity-relationship-attribute triples. This enables structured analysis and factual comparison of model responses, while simultaneously combining semantic similarity, logical consistency, and other multi-dimensional indicators for comprehensive scoring. This scheme not only significantly improves the objectivity and coverage of the evaluation but also possesses the capabilities for continuous updating, automated execution, and error interpretability, providing solid data support and methodological assurance for the optimization, iteration, and reliable deployment of agricultural artificial intelligence models. Summary of the Invention

[0008] To address the shortcomings of existing agricultural large-scale model evaluation methods, such as lack of factual verification, uninterpretable scoring, and weak structure matching capabilities, this invention provides a knowledge graph-based agricultural large-scale model evaluation method, comprising:

[0009] S1. Construct an agricultural knowledge base and generate question-and-answer pairs through multi-source agricultural data. The question-and-answer pairs include single-choice questions, multiple-choice questions, and question-and-answer questions, covering the entire life cycle of crop growth.

[0010] S2. Based on the aforementioned agricultural knowledge base, construct an agricultural knowledge graph;

[0011] S3. Using the agricultural knowledge graph, the original user query is expanded in terms of intent and entity to obtain an expanded natural language query, which provides targeted input and verification benchmark for model evaluation;

[0012] S4. Input the expanded natural language query into the large model to be tested to obtain the output, and parse the output, including factual verification and text similarity verification;

[0013] S5. Combine the factual verification and text similarity verification to generate a comprehensive score, and use the scoring platform to achieve automated testing and feedback.

[0014] Furthermore, in S1, the multi-source agricultural data includes: open-source agricultural professional books, crop cultivation guides, agricultural research papers, policy standards, technical manuals, and expert interview records.

[0015] Furthermore, the question-and-answer pairs are generated through a large language model optimized by instructions, and the question bank is organized in a hierarchical structure of "crop-growth stage-operational specifications-conditional constraints".

[0016] Furthermore, S2 specifically includes:

[0017] S21. Define a schema model to unify agricultural entity categories, entity attributes, and relationship types;

[0018] S22. Entity recognition and relation extraction are performed in collaboration between large-model paths and small-model paths;

[0019] S23. Use the XGBoost classifier to verify the credibility of the triples and store them in a graph database.

[0020] Furthermore, in S3, the intent expansion utilizes prompt words to allow the large model to analyze the user's query intent from the user's basic inquiry, thereby expanding to more comprehensive query directions;

[0021] Entity expansion utilizes named entity recognition technology to extract the defined entity and relation types from the knowledge graph as the standard answer for factual verification of the knowledge graph. The final result is a triple of entity-relation-entity type.

[0022] Furthermore, in S4, the factual verification adopts the triplet comparison verification method: the results generated by the model are used to extract semantic triples using the entity recognition and relation extraction module mentioned above and matched with standard triples in the agricultural knowledge graph. The matching logic is to calculate the similarity between triples according to entity-relation-entity.

[0023] The text similarity verification process saves the test answer output by the model and converts it into an embedding vector from a pre-set answer using BERT. Then, the cosine similarity between the two vectors is calculated.

[0024] Furthermore, in S5, a comprehensive score is generated by combining the factual verification and text similarity verification, and automated testing and feedback are achieved through the scoring platform.

[0025] A knowledge graph-based agricultural large-scale model evaluation system is also provided, implemented based on any of the methods described above, including:

[0026] The agricultural knowledge base construction module is used to generate question-and-answer pairs from multi-source agricultural data. The question-and-answer pairs include single-choice questions, multiple-choice questions, and question-and-answer questions, covering the entire life cycle of crop growth.

[0027] An agricultural knowledge graph construction module is used to construct an agricultural knowledge graph based on the agricultural knowledge base.

[0028] The query expansion module is used to expand the original user query by intent and entity using the agricultural knowledge graph, so as to obtain an expanded natural language query, which provides targeted input and verification benchmark for model evaluation.

[0029] The output verification module is used to input the expanded natural language query into the large model under test to obtain the output, and to parse the output, including factual verification and text similarity verification.

[0030] The evaluation and feedback module is used to generate a comprehensive score by combining the factual verification and text similarity verification, and to achieve automated testing and feedback through the scoring platform.

[0031] The beneficial effects of this invention are:

[0032] In the process of building an agricultural knowledge base, this invention generates multiple types of question-and-answer pairs from multiple open-source agricultural books, realizing the systematic accumulation of knowledge in the agricultural field and significantly reducing the cost of manual annotation and organization.

[0033] In the knowledge graph construction process, this invention introduces a prior knowledge system through schema design, unifying entity types, attributes, and relationship categories in the agricultural field, thus achieving standardization and interdisciplinary scalability of the knowledge structure. The collaborative extraction mechanism of large and small models balances complex semantic understanding capabilities with computational efficiency. The large model ensures recall and semantic accuracy, while the small model ensures lightweight speed and rule constraints, improving the overall robustness of knowledge extraction. The introduction of the XGBoost validation module, integrating features such as confidence, schema matching, semantic similarity, and domain dictionary constraints, effectively improves the accuracy and credibility of triples and reduces knowledge noise. The knowledge graph is stored in the form of a graph database, realizing a queryable, reasonable, and scalable knowledge management approach, providing reliable support for intelligent agricultural decision-making, task planning, and model validation.

[0034] During the construction of the evaluation platform, existing technologies are prone to label confusion and low accuracy when dealing with agricultural problems involving multiple overlapping intentions. This invention, by fine-tuning the GPT-4o model and employing a hierarchical attention mechanism, can dynamically weight conflicting labels across 16 categories of agricultural intentions, achieving more accurate joint recognition of multiple intentions. For example, for problems containing multiple intentions, such as "fertilizer needed for rice pest and disease control," it can accurately identify relevant intentions, with a significantly higher accuracy than traditional methods.

[0035] Traditional entity recognition technologies have low recall rates for low-frequency agricultural entities (such as "sheath blight"). This invention employs a BERT-BiLSTM-CRF model combined with AC automaton rule matching and TF-IDF alignment correction. By injecting agricultural dictionary features and rule compensation, it significantly improves the recall rate of low-frequency entities and effectively solves the problem of missed detection of low-frequency entities.

[0036] This invention constructs a BERT-graph verification framework that integrates textual semantics and graph structural features. Through cross-modal similarity calculation, it achieves more accurate credibility assessment with smaller scoring errors and can perform triple-level error localization, providing a more reliable basis for agricultural decision-making.

[0037] The various modules of this invention (intent recognition, entity recognition, knowledge query, and credibility assessment) form a collaborative closed-loop system through a dynamic routing mechanism. The intent recognition result can dynamically constrain the entity extraction range, the number of entities determines the query mode, and the evaluation result feedback optimizes the accuracy of knowledge query. The modules cooperate and promote each other, improving the performance and efficiency of the entire system.

[0038] The invention's architecture has good scalability; for example, it can easily add new agricultural intent tags, entity categories, and query rules, adapting to constantly changing agricultural knowledge and user needs, thus providing a solid foundation for the further development and optimization of the system. Attached Figure Description

[0039] Figure 1 This is a flowchart of existing agricultural large-scale model evaluation methods;

[0040] Figure 2 This is a flowchart of a knowledge graph-based agricultural large-scale model evaluation method according to the present invention. Detailed Implementation

[0041] The technical solution of the present invention will be further described below with reference to embodiments, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention. In the following embodiments, process equipment or devices not specifically specified are all conventional equipment or devices in the art. Unless specifically specified, the technical means used in the embodiments of the present invention are all conventional means well known to those skilled in the art.

[0042] Example 1: This invention provides a method for evaluating large-scale agricultural models based on knowledge graphs, comprising:

[0043] S1. Construct an agricultural knowledge base and generate question-and-answer pairs through multi-source agricultural data. The question-and-answer pairs include single-choice questions, multiple-choice questions, and question-and-answer questions, covering the entire life cycle of crop growth.

[0044] S2. Based on the aforementioned agricultural knowledge base, construct an agricultural knowledge graph;

[0045] S3. Using the agricultural knowledge graph, the original user query is expanded in terms of intent and entity to obtain an expanded natural language query, which provides targeted input and verification benchmark for model evaluation;

[0046] S4. Input the expanded natural language query into the large model to be tested to obtain the output, and parse the output, including factual verification and text similarity verification;

[0047] S5. Combine the factual verification and text similarity verification to generate a comprehensive score, and use the scoring platform to achieve automated testing and feedback.

[0048] Specifically, Figure 1 This is a flowchart of existing technologies, describing the process of building a knowledge base and knowledge graph from multi-source data, and directly using a large model to generate question-and-answer responses for overall scoring. Its limitation lies in the lack of deep domain knowledge involvement in the evaluation. However, Figure 2The core innovation of this invention is fully demonstrated: it Figure 1 Based on the knowledge base, user queries are introduced as triggers, and the evaluation process is broken down into a closed loop of intent recognition, entity recognition, dual-path retrieval, and dual verification.

[0049] In S1, the multi-source agricultural data includes: open-source agricultural professional books, crop cultivation guides, agricultural research papers, policy standards, technical manuals, and expert interview records.

[0050] Furthermore, the question-and-answer pairs are generated through a large language model optimized by instructions, and the question bank is organized in a hierarchical structure of "crop-growth stage-operational specifications-conditional constraints".

[0051] Specifically, the agricultural knowledge base in S1 aims to provide factual evidence and structured knowledge support for evaluating the capabilities of large-scale agricultural language models. Unlike traditional text-based knowledge bases, this knowledge base is a question-and-answer type knowledge system for large-scale model capability verification. It is constructed in the form of questions and standard answers, with authoritative content, broad scope, and a clear organizational structure, directly driving the construction of subsequent knowledge graphs and the generation of evaluation examples. S1 specifically includes:

[0052] S11. Multi-source data acquisition and integration

[0053] Data sources include open-source agricultural professional books, crop cultivation guides, agricultural research papers, policies and standards, technical manuals, expert interview records, etc.

[0054] The data preprocessing stage includes format standardization, chapter division, knowledge point extraction, and duplicate content removal to ensure the integrity, accuracy, and coverage of the data.

[0055] The construction process incorporates domain expert knowledge, transforming abstract technical specifications into assessable knowledge units.

[0056] S12. Question-Answer Pair Generation Based on Large Language Model

[0057] Using a large language model optimized by instructions, and based on the scope of agricultural knowledge and question format specifications, a large-scale agricultural question-and-answer pair is automatically generated from pre-processed books and materials; the generated question-and-answer pairs cover three main question types:

[0058] Multiple choice questions: These are used to precisely test a model's understanding of a single fact, such as "the recommended fertilization method for a certain crop during the jointing stage."

[0059] Multiple choice questions: These are used to assess the model's systematic understanding of complex knowledge points, such as "multiple combined measures for the prevention and control of a certain pest or disease";

[0060] Question-and-answer questions: These are used to test the model's language generation, logical reasoning, and comprehensive decision-making abilities, such as "Given weather conditions and soil conditions, design a suitable irrigation strategy."

[0061] Each question-and-answer pair is accompanied by a standard or reference answer to support subsequent automated comparison and interpretability scoring.

[0062] S13. Knowledge Coverage and Hierarchical Organization

[0063] The knowledge points cover the key stages of the entire life cycle of crops, from sowing, seedling raising, field management, fertilization, plant protection, and harvesting; the question bank is organized according to a hierarchical structure of "crop-growth stage-operational specifications-condition constraints" to achieve a systematic expression of agricultural knowledge; a unique identifier is defined for each question and answer, and an index is built in the database to support subsequent graph construction, fast query and multi-dimensional statistical analysis.

[0064] The agricultural knowledge base constructed using the above method not only possesses factual and authoritative qualities but also, due to its question-and-answer structure, is naturally suitable for evaluating the capabilities of large language models. This design can significantly reduce the cost of manually constructing test sets while improving the diversity and scalability of evaluation samples, providing a solid data foundation for subsequent automated model validation.

[0065] S2 specifically includes:

[0066] S21. Define a schema model to unify agricultural entity categories, entity attributes, and relationship types;

[0067] S22. Entity recognition and relation extraction are performed in collaboration between large-model paths and small-model paths;

[0068] S23. Use the XGBoost classifier to verify the credibility of the triples and store them in a graph database.

[0069] Specifically, S2 achieves automatic extraction of entities, attributes, and semantic relationships from agricultural texts through three major stages: defining a schema model, entity recognition and relation extraction, and knowledge verification and fusion, forming a queryable and reasonable agricultural knowledge graph.

[0070] 1. Define the schema model

[0071] Corpus Sources: Based on standardized agricultural knowledge materials such as "Agricultural Information Cards," and combined with authoritative corpora such as policy documents, crop specifications, and agricultural technology manuals, a high-quality collection of basic knowledge is formed.

[0072] Schema design method: Through manual annotation and dependency parsing, common agricultural entity categories (such as "crops", "pests and diseases", "fertilizers", "equipment", "meteorological conditions"), entity attributes (such as "growth cycle", "disease symptoms", "suitable temperature") and relation types (such as "at stage", "permitted operation", "use equipment").

[0073] Based on these analysis results, an initial schema model for an agricultural knowledge graph was designed;

[0074] By leveraging the Few-shot learning capability of large language models, the schema is generalized to subfields such as agronomy, horticulture, and forestry, forming a unified interdisciplinary knowledge structure template that provides prior constraints for subsequent knowledge extraction.

[0075] Technical innovation: The defined schema model realizes the standardization and unification of agricultural knowledge across multiple disciplines, providing a complete top-level design and expansion capability for agricultural knowledge graphs.

[0076] Entity recognition and relation extraction

[0077] This step employs a dual-path structured extraction mechanism that combines large and small model paths in parallel to achieve high-precision extraction of entities and relationships from complex agricultural texts.

[0078] Large Model Path: Using ChatGPT-4o or other large language models, a two-stage Prompt instruction design is employed: Entity Recognition Stage: Input a predefined schema to guide the model in extracting entity boundaries and types from the text; Relation Extraction Stage: Input a set of candidate relation types and contextual semantics to guide the model in recognizing the logical relationships between entity pairs and generating a set of candidate triples.

[0079] Large model paths excel at handling long texts, complex contextual logic, and implicit relationships, improving the recall and diversity of extraction results.

[0080] Small model path: Uses a lightweight neural network architecture, combined with a domain feature dictionary and shallow rules, to achieve efficient relation extraction; Entity recognition submodule: Uses a BiLSTM-CRF network to capture contextual features and model label dependencies to achieve the global optimal solution for entity boundary recognition and type classification; Relation extraction submodule: Uses a BiLSTM + Attention architecture, uses the attention mechanism to focus on keyword regions to construct relation vectors and perform relation classification prediction;

[0081] Small model path calculation efficiency is high, suitable for rapid processing of large-scale agricultural text and extraction of basic knowledge.

[0082] 3. Multi-source knowledge fusion and verification

[0083] Because of the differences in extraction results between large and small models, this invention introduces the XGBoost classifier as a knowledge verification module to achieve result fusion and conflict determination: verification feature design: model confidence score; matching degree between entity type and schema; contextual semantic matching degree (measured by sentence vector similarity); whether entities and relations appear in the agricultural domain knowledge dictionary;

[0084] Fusion method: Input the above features into the XGBoost classifier to calculate the comprehensive credibility score of each triple; filter and adjudicate candidate triples according to the credibility threshold to form the final credible agricultural knowledge set.

[0085] 4. Atlas storage and retrieval

[0086] The final set of selected triples is stored in a graph database (such as Neo4j); nodes represent agricultural entities, edges represent entity relationships, and all nodes have attribute fields (such as growth cycle, suitable temperature, etc.); a Cypher query interface is provided to enable complex multi-hop queries, knowledge visualization, and reasoning support; dynamic incremental updates of knowledge are supported, and when new agricultural data or Q&A content is added, the graph nodes and relationships can be quickly parsed and updated to maintain the timeliness of the knowledge system.

[0087] Through the above technical solutions, S2 realizes the schematization, automation, and verifiable extraction of agricultural knowledge, and combines the high recall capability of large models with the high precision capability of small models to achieve multi-source information complementarity and the construction of a highly reliable knowledge graph, providing a structured, reasonable, and continuously updated knowledge foundation for the evaluation of large agricultural models.

[0088] In S3, the intent expansion utilizes prompt words to allow the large model to analyze the user's query intent from the user's basic inquiry, thereby expanding to more comprehensive query directions;

[0089] Entity expansion utilizes named entity recognition technology to extract the defined entity and relation types from the knowledge graph as the standard answer for factual verification of the knowledge graph. The final result is a triple of entity-relation-entity type.

[0090] Specifically, here's an example of an intent query:

[0091] For example: Input: "What type of soil is suitable for planting rice?"

[0092] Output: ["Query crop", "Query soil type"] # This question implicitly asks for both the crop and soil type.

[0093] As shown above, the final expanded result is a natural language type. This result is then combined with the user's original query and fed into the large model under test to output the answer to be tested.

[0094] Example of entity query:

[0095] The output is: {'Agricultural Technology': 'Seedbed', 'Agricultural Behavior': 'Sowing'}.

[0096] In S4, the factual verification adopts the triplet comparison verification method: the semantic triples generated by the model are extracted using the entity recognition and relation extraction module mentioned above and matched with the standard triples in the agricultural knowledge graph. The matching logic is to calculate the similarity between triples according to entity-relation-entity.

[0097] The text similarity verification process saves the test answer output by the model and converts it into an embedding vector from a pre-set answer using BERT. Then, the cosine similarity between the two vectors is calculated.

[0098] In S5, a comprehensive score is generated by combining the factual verification and text similarity verification, and automated testing and feedback are achieved through the scoring platform.

[0099] Specifically, the rating platform mainly includes the following modules:

[0100] Initialization and configuration:

[0101] Load a pre-trained agricultural domain model (RoBERTa-RNN)

[0102] Initialize the Streamlit interface and configure the sidebar control panel.

[0103] Provides model selection, dialog window management, and display control options.

[0104] Test data loading:

[0105] First, the test questions and standard answers are read from the QA.json file to verify data integrity and display the loading status.

[0106] Automated testing process:

[0107] Clicking "Start Test" will initiate batch testing. The progress bar will display the test progress in real time (N / M issues).

[0108] Perform the following steps for each question in sequence:

[0109] a. Intent Recognition: Analyzing the Question Intent using the GPT Model

[0110] b. Knowledge Graph Query: Connect to the Neo4j database to retrieve relevant knowledge.

[0111] c. Model Response Generation: Obtaining the responses from the tested large model.

[0112] d. Answer Evaluation: Compare the model's answer with the standard answer to generate a scoring report.

[0113] 4. Results Display and Storage:

[0114] The test results for each question are displayed in real time (question / answer comparison / score).

[0115] The average score of all questions is summarized.

[0116] Generate structured test results (CSV format) and provide a test result download function.

[0117] Exception handling:

[0118] Capture anomalies during the test (such as data processing errors), terminate the test promptly, and display error messages.

[0119] The entire process achieves a complete closed loop from test data loading and automated execution to result analysis, providing intuitive test feedback through progress bars and real-time result displays, and finally generating a downloadable evaluation report.

[0120] A knowledge graph-based agricultural large-scale model evaluation system is also provided, implemented based on any of the methods described above, including:

[0121] The agricultural knowledge base construction module is used to generate question-and-answer pairs from multi-source agricultural data. The question-and-answer pairs include single-choice questions, multiple-choice questions, and question-and-answer questions, covering the entire life cycle of crop growth.

[0122] An agricultural knowledge graph construction module is used to construct an agricultural knowledge graph based on the agricultural knowledge base.

[0123] The query expansion module is used to expand the original user query by intent and entity using the agricultural knowledge graph, so as to obtain an expanded natural language query, which provides targeted input and verification benchmark for model evaluation.

[0124] The output verification module is used to input the expanded natural language query into the large model under test to obtain the output, and to parse the output, including factual verification and text similarity verification.

[0125] The evaluation and feedback module is used to generate a comprehensive score by combining the factual verification and text similarity verification, and to achieve automated testing and feedback through the scoring platform.

Claims

1. A knowledge graph-based agricultural large model evaluation method, characterized in that, include: S1. Construct an agricultural knowledge base and generate question-and-answer pairs through multi-source agricultural data. The question-and-answer pairs include single-choice questions, multiple-choice questions, and question-and-answer questions, covering the entire life cycle of crop growth. S2. Based on the aforementioned agricultural knowledge base, construct an agricultural knowledge graph; S3. Using the agricultural knowledge graph, the original user query is expanded in terms of intent and entity to obtain an expanded natural language query, which provides targeted input and verification benchmark for model evaluation; S4. Input the expanded natural language query into the large model to be tested to obtain the output, and parse the output, including factual verification and text similarity verification; S5. Combine the factual verification and text similarity verification to generate a comprehensive score, and use the scoring platform to achieve automated testing and feedback; S2 specifically includes: S21. Define a schema model to unify agricultural entity categories, entity attributes, and relationship types; S22. Entity recognition and relation extraction are performed in collaboration between large-model paths and small-model paths; S23. Use the XGBoost classifier to verify the credibility of triples and store them in a graph database; The large model path uses ChatGPT-4o or other large language models and is designed with a two-stage Prompt instruction. The large model path is good at handling long texts, complex contextual logic and implicit relationships, and improves the recall and diversity of the extraction results. The small model path uses a lightweight neural network architecture, combined with a domain feature dictionary and shallow rules, to achieve efficient relation extraction; the small model path has high computational efficiency and is suitable for rapid processing of large-scale agricultural text and extraction of basic knowledge. In S3, the intent expansion utilizes prompt words to allow the large model to analyze the user's query intent from the user's basic inquiry, thereby expanding to more comprehensive query directions; Entity expansion utilizes named entity recognition technology to extract pre-defined entities and relation types from the knowledge graph as the standard answer for factual verification of the knowledge graph. The final result is a triple of entity-relation-entity type.

2. The method for evaluating large-scale agricultural models based on knowledge graphs according to claim 1, characterized in that, In S1, the multi-source agricultural data includes: open-source agricultural professional books, crop cultivation guides, agricultural research papers, policy standards, technical manuals, and expert interview records.

3. The method for evaluating large-scale agricultural models based on knowledge graphs according to claim 1, characterized in that, In S1, the question-answer pairs are generated through a large language model optimized by instructions, and the question bank is organized in a hierarchical structure of "crop-growth stage-operational specifications-conditional constraints".

4. The method for evaluating large-scale agricultural models based on knowledge graphs according to claim 1, characterized in that, In S4, the factual verification adopts the triplet comparison verification method: the semantic triples generated by the model are extracted using the entity recognition and relation extraction module mentioned above and matched with the standard triples in the agricultural knowledge graph. The matching logic is to calculate the similarity between triples according to entity-relation-entity type. The text similarity verification process involves saving the test answer output by the model, converting it into an embedding vector along with a pre-set answer using BERT, and then calculating the cosine similarity between the two vectors.

5. A knowledge graph-based agricultural large-scale model evaluation system, implemented based on the method described in any one of claims 1-4, characterized in that, include: The agricultural knowledge base construction module is used to generate question-and-answer pairs from multi-source agricultural data. The question-and-answer pairs include single-choice questions, multiple-choice questions, and question-and-answer questions, covering the entire life cycle of crop growth. An agricultural knowledge graph construction module is used to construct an agricultural knowledge graph based on the agricultural knowledge base. The query expansion module is used to expand the original user query by intent and entity using the agricultural knowledge graph, so as to obtain an expanded natural language query, which provides targeted input and verification benchmark for model evaluation. The output verification module is used to input the expanded natural language query into the large model under test to obtain the output, and to parse the output, including factual verification and text similarity verification. The evaluation and feedback module is used to generate a comprehensive score by combining the factual verification and text similarity verification, and to achieve automated testing and feedback through the scoring platform.

Images