System, method, computer readable storage medium, and computer program product for crop breeding questions and answers

Abstract

The invention relates to a computer system using a computational model. Systems, methods, computer-readable storage media, and computer program products for crop breeding questions and answers are disclosed. A system for crop breeding questions and answers includes a computing resource, a storage unit, and a large language model. The storage unit is configured to store a knowledge graph, the knowledge graph comprising a crop breeding data set. The large language model is configured to: receive input from a user; performing retrieval in the knowledge graph by using a predefined operator to determine a knowledge graph retrieval result; determining the credibility of the knowledge graph retrieval result; selectively performing an additional search to determine a final search result based on the credibility; and generating an answer to the crop breeding question based on the crop breeding question and the final retrieval result. According to the system, the limitation that a large language model lacks professional knowledge for the crop breeding field is overcome, and the accuracy of answering the crop breeding problem by the large language model is improved.

Description

Technical Field

[0001] The present invention generally relates to computer systems utilizing computational models, and more particularly to systems, methods, computer-readable storage media, and computer program products for answering questions in crop breeding. Background Art

[0002] Plant biology research involves complex decision making and requires the integration of massive literature and multi-omics data. The rapid growth of biological data and publications has brought great challenges to extracting knowledge and content understanding about crop breeding. Large Language Model (LLM) is a promising solution to this challenge due to its excellent performance in text generation.

[0003] However, although the general large language model performs well in natural language processing tasks, it is a general artificial intelligence product and does not have the ability to answer highly professional crop breeding questions. For example, the general large language model lacks professional content in the field of crop breeding in its training corpus, which leads to a misunderstanding of professional terms. Taking rice as an example, the semantic parsing accuracy of the general large language model for rice-specific concepts (such as "leaf salt stress response during tillering") is less than 60%.

[0004] In addition, researchers have tried to use knowledge graphs to answer professional questions. Although traditional knowledge graphs can represent gene function annotations, their reasoning capabilities are insufficient. Existing systems only support basic rule-based queries and cannot handle complex questions that require the combination of semantics and omics data (such as "Is a gene involved in drought stress response and has breeding potential").

[0005] There is a need in the art for improved crop breeding question-answering techniques on at least one of the dimensions discussed above. Summary of the invention

[0006] The present invention is provided in order to further improve crop breeding question answering technology by utilizing large language models and knowledge graphs.

[0007] One aspect of the present invention provides a system for crop breeding question answering, comprising: computing resources; a storage unit, the storage unit being configured to store a knowledge graph, the knowledge graph comprising a crop breeding data set; and a large language model, the large language model being configured to call the computing resources to: receive input from a user, the input comprising a crop breeding question; perform a search in the knowledge graph based on the crop breeding question using a predefined operator to determine a knowledge graph retrieval result, the predefined operator comprising an operation of formalized constraints required to retrieve the knowledge graph; determine the credibility of the knowledge graph retrieval result; based on the credibility, selectively perform an additional search to determine a final retrieval result; and generate an answer to the crop breeding question based on the crop breeding question and the final retrieval result.

[0008] In the system as described above, the large language model is configured to call the computing resources to: decompose the crop breeding problem into multiple sub-steps; perform the operation of at least one operator in each of the multiple sub-steps to obtain an intermediate retrieval result of each sub-step; and determine the knowledge graph retrieval result based on the intermediate retrieval result of the last sub-step in the multiple sub-steps.

[0009] In the system as described above, the predefined operator is determined in the following manner: for each evaluation problem in the crop breeding test set, the evaluation problem is decomposed into multiple sub-steps using the large language model, each sub-step includes an operator; it is determined whether the operator list of the large language model covers the operators of the decomposed sub-steps of all evaluation problems, the operator list includes the predefined operator; and the operator list is selectively updated according to the result of the determination; or for each evaluation problem in the crop breeding test set, the evaluation problem is decomposed using the large language model; for the evaluation problem that fails to be decomposed, the expression of the operator in the operator list is selectively adjusted.

[0010] In the system as described above, the predefined operator includes at least one of a knowledge graph query, an arithmetic operation on data, a logical operation on data, and a result output.

[0011] In the system as described above, the large language model is configured to call the computing resources to: determine whether the credibility meets the requirements; in response to the credibility meeting the requirements, return the knowledge graph retrieval result as the final retrieval result; in response to the credibility not meeting the requirements, perform an additional search on the knowledge graph to determine the final retrieval result.

[0012] In the system as described above, in response to the credibility not meeting the requirement, the large language model is configured to call the computing resources to: decompose the crop breeding problem into a multi-level vocabulary, the multi-level vocabulary including a first-level vocabulary and a second-level vocabulary, the second-level vocabulary including superordinate concepts of the first-level vocabulary; use the multi-level vocabulary to search the knowledge graph to determine additional search results; and return the additional search results as the final search results.

[0013] In the system as described above, the large language model is configured to call the computing resource to: determine the complexity of the crop breeding problem; and in response to the complexity exceeding a complexity threshold, retrieve a predetermined number of relationships associated with the crop breeding problem in the knowledge graph, wherein the predetermined number is based on the length of the longest context of the large language model and the length of the text block mapped to the crop breeding dataset.

[0014] In the system as described above, the predetermined number of relationships includes the predetermined number of relationships with the highest priority, wherein the priority of the relationships in the knowledge graph is sorted based on the weight of the relationship determined when establishing the knowledge graph, and the weight of the relationship reflects the closeness between two entity nodes connected by the relationship in the knowledge graph.

[0015] In the system as described above, the input is a first input, and the large language model is configured to call the computing resource to: receive a second input from the user, the second input comprising a second question; determine the relevance of the second question to the field of crop breeding; and in response to the second question being irrelevant to the field of crop breeding, generate an answer to the second question based on the second question.

[0016] In the system as described above, the large language model is configured to call the computing resources to: divide the second question into semantic slots, and if the semantic slots are irrelevant to the field of crop breeding, determine that the second question is irrelevant to the field of crop breeding; or approximately match the dense features of the second question with the entity word list of the knowledge base of the large language model, and if the result of the approximate match is lower than a predetermined threshold, determine that the second question is irrelevant to the field of crop breeding.

[0017] In the system as described above, the large language model is configured to: record historical messages of the user's conversations with the large language model, the historical messages including questions raised by the user; detect the relevance of the questions raised by the user to multiple aspects in the field of crop breeding; and in response to detecting that a threshold number of questions are related to a first aspect among the multiple aspects, increase the weight of the text block associated with the first aspect in the final retrieval result.

[0018] In the system as described above, the crop breeding dataset comprises multi-omics data.

[0019] In the system as described above, the crop breeding data set is organized in the knowledge graph in the form of triples, the triples include entities and relationships, and the entities of the triples of the multi-omics data include at least one of the following: genes, transcriptomes, proteins, events, gene annotations, traits and environmental factors.

[0020] In the system as described above, the large language model is configured to: obtain literature related to crop breeding; extract triples of the crop breeding dataset from the literature; and construct the knowledge graph based on the triples of the crop breeding dataset, wherein the triples are mapped to text blocks in the literature.

[0021] In the system as described above, the triple includes a head entity and a tail entity, and the large language model is configured to: extract a gene name or a variety name from the document as a head entity; when the head entity is a gene name, extract breeding history, trait records or cultivation techniques from the document as a tail entity; when the head entity is a variety name, extract agronomic traits from the document as a tail entity.

[0022] In the system as described above, the knowledge graph is a first knowledge graph, and the system also includes a question-answer extraction model, which is configured to: extract a second knowledge graph from a crop breeding corpus, wherein the crop breeding corpus includes literature related to crop breeding; generate statements based on entities and relationships in the second knowledge graph; use the large language model to predict whether the statement is correct to generate a prediction result, wherein the prediction result indicates the confidence of the large language model in the statement; calculate the difference between the prediction result and the true value of the statement; in response to the difference being greater than an error threshold, determine the statement as a knowledge blind spot of the large language model; and extract a subgraph in the second knowledge graph based on the knowledge blind spot to generate a crop breeding question-answer sample training set for training the large language model.

[0023] Another aspect of the present invention provides a method for crop breeding question answering, comprising: calling computing resources by a large language model to perform the following steps: S101: receiving input from a user, the input comprising a crop breeding question; S102: searching in a knowledge graph based on the crop breeding question using a predefined operator to determine a knowledge graph retrieval result, the knowledge graph comprising a crop breeding data set, the predefined operator comprising operations of formal constraints required to retrieve the knowledge graph; S103: determining the credibility of the knowledge graph retrieval result; S104: based on the credibility, selectively performing additional retrieval to determine a final retrieval result; and S105: generating an answer to the crop breeding question based on the crop breeding question and the final retrieval result.

[0024] As described above, step S102 includes: S1021: decomposing the crop breeding problem into multiple sub-steps; S1022: performing at least one operator operation in each of the multiple sub-steps to obtain an intermediate retrieval result of each sub-step; and S1023: determining the knowledge graph retrieval result based on the intermediate retrieval result of the last sub-step in the multiple sub-steps.

[0025] In the method as described above, the predefined operator includes at least one of a knowledge graph query, an arithmetic operation on data, a logical operation on data, and a result output, and the predefined operator is determined in the following manner: for each evaluation problem in the crop breeding test set, the evaluation problem is decomposed into multiple sub-steps using the large language model, and each sub-step includes an operator; it is determined whether the operator list of the large language model covers the operators of the decomposed sub-steps of all evaluation problems, and the operator list includes the predefined operator; and the operator list is selectively updated according to the result of the judgment; or for each evaluation problem in the crop breeding test set, the evaluation problem is decomposed using the large language model; for the evaluation problem that fails to be decomposed, the expression of the operator in the operator list is selectively adjusted.

[0026] As described above, step S102 includes: S1024: determining the complexity of the crop breeding problem; and S1025: in response to the complexity exceeding a complexity threshold, retrieving a predetermined number of relationships associated with the crop breeding problem in the knowledge graph, the predetermined number being based on the length of the longest context of the large language model and the length of the text block mapped to the crop breeding dataset, the predetermined number of relationships including the predetermined number of relationships with the highest priority, wherein the priority of the relationships in the knowledge graph is sorted based on the weight of the relationships determined when the knowledge graph is established, and the weight of the relationship reflects the closeness between two entity nodes connected by the relationship in the knowledge graph.

[0027] According to the method described above, the large language model calls computing resources to perform the following steps: S201: recording historical messages of the conversation between the user and the large language model, the historical messages including questions raised by the user; S202: detecting the relevance of the questions raised by the user to multiple aspects in the field of crop breeding; and S203: in response to detecting that a threshold number of questions are related to a first aspect among the multiple aspects, increasing the weight of the text block associated with the first aspect in the final retrieval result.

[0028] According to the method described above, the crop breeding dataset includes multi-omics data, and the crop breeding dataset is organized in the knowledge graph in the form of triples, wherein the triples include entities and relationships, and the entities of the triples of the multi-omics data include at least one of the following: genes, transcriptomes, proteins, events, gene annotations, traits, and environmental factors.

[0029] As described above, the large language model is configured to be used for: S401: obtaining literature related to crop breeding; S402: extracting triples of the crop breeding data set from the literature, the triples including a head entity and a tail entity, including: S4021: extracting a gene name or a variety name from the literature as a head entity; S4022: when the head entity is a gene name, extracting a breeding history, a trait record or a cultivation technique from the literature as a tail entity; S4023: when the head entity is a variety name, extracting agronomic traits from the literature as a tail entity; S403: constructing the knowledge graph based on the triples of the crop breeding data set, wherein the triples are mapped to text blocks in the literature.

[0030] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above methods.

[0031] Another aspect of the present invention provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any one of the above methods are implemented.

[0032] The system and method according to the present invention overcome the limitation of the large language model's lack of professional knowledge in the field of crop breeding, and improve the accuracy of the large language model in answering crop breeding questions by searching in the knowledge graph and combining the reasoning ability of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Embodiments of the present invention are described with reference to the accompanying drawings.

[0034] Figure 1is a block diagram of a system for crop breeding question answering according to some embodiments of the present invention.

[0035] Figure 2 is a schematic diagram of triples of a knowledge graph according to some embodiments of the present invention.

[0036] Figure 3 is a schematic diagram of a processing flow of a large language model generating an answer to a user input according to some embodiments of the present invention.

[0037] Figure 4 is a flow chart of a first method for crop breeding question answering according to some embodiments of the present invention.

[0038] Figure 5 is a flow chart of a first process associated with a first method for crop breeding question answering according to some embodiments of the present invention.

[0039] Figure 6 is a flow chart of a second process associated with the first method for crop breeding question answering according to some embodiments of the present invention.

[0040] Figure 7 is a flow chart of a third process associated with the first method for crop breeding question answering according to some embodiments of the present invention.

[0041] Figure 8 is a flow chart of a second method for crop breeding question answering according to some embodiments of the present invention.

[0042] Fig. 9 is a flow chart of a third method for crop breeding question answering according to some embodiments of the present invention.

[0043] Fig.10 is a flow chart of a fourth method for crop breeding question answering according to some embodiments of the present invention.

[0044] Fig.11 is a flowchart of a fifth method for generating a training dataset according to some embodiments of the present invention.

[0045] Fig.12 is a block diagram of a computer-readable storage medium according to some embodiments of the present invention.

[0046] Fig.13 is a block diagram of a computer program product according to some embodiments of the invention. DETAILED DESCRIPTION

[0047] In this application, ordinal numbers "first", "second", "third", etc. are used to distinguish different instances of objects with the same name. Ordinal numbers "first", "second", "third", etc. do not indicate the relative order of the objects indicated in time, space, sorting, or other aspects.

[0048] According to one aspect of the present invention, a system for question-answering in crop breeding is provided.

[0049] Figure 1 is a block diagram of a system 100 for crop breeding question answering according to some embodiments of the present invention.

[0050] System 100 may be a local or remote computer, server, etc. System 100 may include computing resources 110, storage unit 120, and large language model 130. Storage unit 120 may be configured to store a knowledge graph, which may include a crop breeding dataset.

[0051] In some embodiments, the computing resources 110 may include a central processing unit (CPU), a graphics processing unit (GPU), and various other processing units or cores (e.g., arithmetic logic units, integer units, floating point units, tensor units, ray tracing cores, etc.), etc.

[0052] The large language model 130 may be configured to call the computing resource 110 to perform corresponding operations. The large language model 130 may be configured to receive input from a user, the input including a crop breeding question. The large language model 130 may be configured to search the knowledge graph based on the crop breeding question using predefined operators to determine the knowledge graph retrieval results. The predefined operators include operations on formal constraints required to retrieve the knowledge graph. The large language model 130 may be configured to determine the credibility of the knowledge graph retrieval results. The large language model 130 may be configured to selectively perform additional searches based on the credibility to determine the final retrieval results. The large language model 130 may be configured to generate answers to crop breeding questions based on the crop breeding questions and the final retrieval results.

[0053] In some embodiments, the credibility of the knowledge graph search results can be determined based on the results themselves. As an example, the large language model 130 can compare the results with the data in the open source seed industry evaluation set to determine the credibility of the results. As an example, the open source seed industry evaluation set may include at least one of multiple-choice questions, fill-in-the-blank questions, and essay questions, each of which includes an objective answer. The large language model 130 can compare the knowledge graph search results with the objective answers, and determine the credibility of the knowledge graph search results based on the comparison results (e.g., the similarity between the results and the objective answers).

[0054] In some embodiments, the credibility of the knowledge graph search result can be determined based on the process of generating the result. As an example, the large language model 130 can determine whether the credibility of the knowledge graph search result meets the requirements based on the success rate of decomposing the crop breeding problem into predefined operators and executing the operators.

[0055] The present invention proposes a big language model for the field of crop breeding. By combining knowledge graph retrieval with the reasoning ability of the big language model, the dynamic integration of the text data of the big language model itself and the data in the knowledge graph is realized, so that the system can dynamically fuse multi-source data (such as the text data of the big language model and the crop breeding data in the knowledge graph) according to user queries to generate comprehensive and accurate responses, thereby breaking through the limitations of existing general-purpose big language models in dealing with professional problems in crop breeding.

[0056] In some embodiments, the crop breeding dataset may include multi-omics data. Multi-omics data refers to a collection of biological data obtained through multiple omics technologies (such as genomics, transcriptomics, proteomics, metabolomics, etc.). Integrating crop breeding datasets including multi-omics data (such as genomes, transcriptomics, proteomics, etc.) through knowledge graphs provides richer biological data for large language models to answer crop breeding related questions.

[0057] In some embodiments, the crop breeding data set can be organized in a knowledge graph in the form of triples. Triples include entities and relationships, entities can also be called nodes, and relationships can also be called edges. The entities of the triples of multi-omics data can include at least one of the following: genes, transcriptomes, proteins, events, gene annotations, traits, and environmental factors.

[0058] Figure 2 is a schematic diagram of a triple of a knowledge graph according to some embodiments of the present invention. Circles indicate entities of a triple. In some embodiments, the entities of a triple may include genes, transcripts, proteins, events, gene annotations (kegg, go), traits (pheno) and / or environmental factors (env_factor), etc. The numbers next to the circles indicate the number of examples of the corresponding entity. The lines connecting the circles indicate the relationship between the entities.

[0059] It should be understood that Figure 2 It is a statistical diagram of triples of a knowledge graph constructed according to some embodiments of the present invention. For a single triple, the triple itself exists in the form of one entity connected to another entity.

[0060] A triple can be represented in the form of (head entity, relation, tail entity). The following is an example of a triple in a knowledge graph:

[0061] (1) Gene nodes, each piece of information contains the unique identifier of the gene in the map. Gene nodes can be related to GO annotations, Kegg annotations, genes, traits, and transcriptomes:

[0062] Relationship with GO: (AGIS_Os06g001280, in go, GO:0003674);

[0063] Relationship with Kegg: (AGIS_Os07g036840, in kegg, K01126);

[0064] Whether the two genes form an upstream and downstream relationship: (AGIS_Os09g044250, near, AGIS_Os09g044260);

[0065] Whether two genes are co-expressed: (AGIS_Os11g026770, coexpression, AGIS_Os03g050730);

[0066] Relationship between genes and environmental factors: (AGIS_Os05g024200, env_factor_phenotype, Water-related factors);

[0067] Relationship between genes and traits: (AGIS_Os06g001280, Phenotype, interact with chloroplast);

[0068] Relationship between gene and transcriptome: (AGIS_Os02g050860, transcript, transcript of AGIS_Os02g050860);

[0069] (2) Transcriptome node: each gene contains a unique transcriptome, which can be related to genes, proteins, and transcriptome events:

[0070] Relationship between gene and transcriptome: (AGIS_Os02g050860, transcript, transcript of AGIS_Os02g050860);

[0071] Relationship between transcriptome and protein: (transcript of AGIS_Os05g033520, translate,protein of AGIS_Os05g033520);

[0072] Relationship between transcriptome events and transcriptome: (transcriptome_event_0, involves_transcript, transcript of AGIS_Os01g002320);

[0073] (3) Protein nodes can be related to transcriptomes, proteins, and protein events:

[0074] Relationship between transcriptome and protein: (transcript of AGIS_Os05g033520, translate,protein of AGIS_Os05g033520);

[0075] Protein-protein interaction (PPI): (protein of AGIS_Os01g055060, Protein_protein_interaction, protein of AGIS_Os02g053780);

[0076] The relationship between proteins and protein events: (proteome_event_0, involves_protein, proteinof AGIS_Os05g031310);

[0077] (4) Event nodes, including transcriptome events and protein events, can be related to transcriptomes and proteins. In addition, tissue type and growth stage can be regarded as attributes of the node:

[0078] Relationship between transcriptome events and transcriptome: (transcriptome_event_0, involves_transcript, transcript of AGIS_Os01g002320);

[0079] Protein-protein event relationship: (proteome_event_0, involves_protein, proteinof AGIS_Os05g031310);

[0080] The growth stage of the protein event: (proteome_event_0, observed_in_stage, SeedlingStage);

[0081] Tissue type of protein event: (proteome_event_2, observed_in_TissuesOrgans,Root);

[0082] Tissue type of transcriptome event: (transcriptome_event_200919, observed_in_TissuesOrgans, Leaf);

[0083] (5) Kegg annotation nodes can be related to genes:

[0084] Relationship between gene and Kegg: (AGIS_Os07g036840, in kegg, K01126);

[0085] (6) GO annotation nodes can be related to genes:

[0086] Relationship between gene and GO: (AGIS_Os06g001280, in go, GO:0003674);

[0087] (7) Trait nodes can be related to genes and environmental factors:

[0088] Relationship between genes and traits: (AGIS_Os06g001380, Phenotype, affect ethylenehomeostasis);

[0089] Relationship between genes and environmental factors: (AGIS_Os05g024200, env_factor_phenotype, Water-related factors);

[0090] (8) Environmental factors:

[0091] Relationship between genes and environmental factors: (AGIS_Os05g024200, env_factor_phenotype, Water-related factors).

[0092] In some embodiments, the entities of the triplet may include gene name, variety name, breeding history, trait record, cultivation technology, agronomic traits, etc.

[0093] It should be understood that the above are merely examples of triples in the knowledge graph and are not exhaustive, and the scope of the present invention is not limited to this.

[0094] Some embodiments of the present invention use triple structures to represent complex associations between multi-omics data, providing a new paradigm for the structured representation of crop breeding data, making it easier for large language models to retrieve and generate information related to crop breeding problems from massive data, and improving data processing efficiency.

[0095] The knowledge graph can also be mapped to text blocks. Text blocks are stored in a corpus generated based on documents, images, and other content. Each text block has an approximate length, which is called the text block length. Each entity and relationship in the triple in the knowledge graph is mapped to a text block in the corpus. The text block identifier corresponding to the text block can be recorded in each entity and relationship, so that the text block in the corpus can be indexed through the text block identifier recorded in the entity and relationship.

[0096] In some embodiments, the predefined operator may include at least one of a knowledge graph query, an arithmetic operation on data, a logical operation on data, and a result output.

[0097] Table 1 below is an example table of operators. Among them, arithmetic operations on data may include accumulation, etc., and logical operations on data may include counting, comparison, etc. The process of retrieving knowledge graphs is constrained by operators based on formal methods, that is, the operations involved in the process of retrieving knowledge graphs are divided into a limited number of specific types of sub-steps based on the nature of the operations, where each type of sub-step can be represented and executed by "formal expression" (that is, an abstract method of unambiguous and precise description of complex systems or concepts through symbols, rules and mathematical structures, the core of which is to replace the ambiguity of natural language with strictly defined syntax and semantics). This constraint can make the retrieval process highly interpretable. The parameters of each operator are clearly defined, which can reduce the requirements for the context length of large language models, thereby reducing the demand for video memory.

[0098]

[0099] In some embodiments, the large language model 130 may be configured to decompose the crop breeding problem into multiple sub-steps. The large language model 130 may be configured to perform at least one operator operation in each of the multiple sub-steps to obtain an intermediate search result for each sub-step. The large language model 130 may be configured to determine the knowledge graph search result based on the intermediate search result of the last sub-step in the multiple sub-steps.

[0100] The following is an example of searching in a knowledge graph based on a crop breeding problem using predefined operators.

[0101] ===========;

[0102] json "query": "What is the parent relationship between Huang Fengzhan and Feng Xiuzhan?",

[0103] "answer": [

[0104] {"step":"Query the parent relationship between Huang Fengzhan and Feng Xiuzhan","action":"get_spo(s=s1:Rice[Huang Fengzhan], p=p1:Parent relationship, o=o1:Rice[Feng Xiuzhan])"},

[0105] {"step":"Query parent relationship","action":"get(p1)"} ]

[0107] ===========;

[0108] First, the large language model 130 uses predefined operators to decompose the crop breeding problem "What is the parental relationship between Huang Fengzhan and Feng Xiuzhan?" into two steps. Then, the operators in each of the two steps are executed, where variables (such as s1, p1, o1) are used to record the intermediate results of the first step. Finally, in the second step (i.e., the last step), the knowledge graph retrieval result for the crop breeding problem is obtained.

[0109] The following are other examples of searching in the knowledge graph based on crop breeding problems using predefined operators.

[0110] 1) Crop breeding problems involving simple statistics:

[0111] ===========;

[0112] Plain Text

[0113] query: How many AGIS genes are involved in drought stress?

[0114] reasoning: [

[0116] {

[0117] "step": "Query AGIS genes involved in drought stress",

[0118] "action": "get_spo(s=s1:gene[AGIS], p=p1:involved stress type, o=o1:stress type[drought])"

[0119] },

[0120] {

[0121] "step": "Count the number of genes that meet the conditions",

[0122] "action": "count(s1)"

[0123] } ]

[0125] response:

[0126] According to the reference documents provided, the AGIS genes involved in drought stress are as follows:

[0127] 1. AGIS_Os05g040410: Differential expression in different varieties and tissues and organs:

[0128] - In the leaves of variety Nagina-22, it showed upregulation (Up).

[0129] - In the grain embryos of variety Zhonghua11, it showed upregulation (Up).

[0130] - In the glumes of variety YIL100, it showed an upregulation (Up).

[0131] - In the roots of variety Rice 9311, it showed an upregulation (Up).

[0132] 2. AGIS_Os10g034460: In the glumes of variety YIL100, it showed upregulation (Up).

[0133] - In the roots of variety Rice 9311, it showed an upregulation (Up).

[0134] In summary, there are two AGIS genes involved in drought stress.

[0135] ===========;

[0136] 2) Crop breeding issues involving comparison of different genes:

[0137] ===========;

[0138] Plain Text

[0139] query: How many types of environmental responsiveness are similar between AGIS_Os05g040410 and AGIS_Os06g035130?

[0140] logic form: [

[0142] {"step":"Query the environmental responsiveness type of AGIS_Os05g040410","action":"get_spo(s=s1:gene[AGIS_Os05g040410], p=p1:environmental responsiveness type, o=o1)"},

[0143] {"step":"Query the environmental responsiveness type of AGIS_Os06g035130","action":"get_spo(s=s2:gene[AGIS_Os06g035130], p=p2:environmental responsiveness type, o=o2)"},

[0144] {"step":"Calculate the number of similar environmental responsiveness of two genes","action":"compare(set=[o1, o2], op=equal)->equal_count"},

[0145] {"step":"Count the number of similar environmental responsiveness types","action":"count(equal_count)->count_result"} ]

[0147] Intermediate results:

[0148] step1

[0149] According to the reference document, the environmental responsiveness types of AGIS_Os05g040410 include the following:

[0150] 1. Cold Stress Response

[0151] 2. Cadmium Stress Responses and Tolerance in Rice

[0152] 3. Early Chilling Stress Response of Rice

[0153] 4. Rice Drought Stress Response

[0154] 5. Rice heat stress response (Heat Stress)

[0155] 6. Low-Level Gamma Radiation

[0156] A total of 6 different environmental responsiveness.

[0157] step2

[0158] According to the reference document, the environmental responsiveness types of AGIS_Os06g035130 include the following:

[0159] 1. Cadmium Stress Responses and Tolerance in Rice

[0160] 2. Salt Tolerance in Rice

[0161] 3. Rice Drought Stress Response

[0162] 4. Oxidative Stress Response in Rice

[0163] 5. Strigolactone Stress in Rice

[0164] 6. Aerobic Adaptation in Rice

[0165] 7. Rice and Arabidopsis receptor-like kinase gene response (ABA Signaling and Biosynthesis)

[0166] A total of 7 different environmental responsiveness.

[0167] step3

[0168] According to the information provided, AGIS_Os05g040410 has 6 environmental responsiveness and AGIS_Os06g035130 has 7 environmental responsiveness. We need to find out how many types of environmental responsiveness they have in common.

[0169] First, list all environmental responsiveness categories of AGIS_Os05g040410:

[0170] 1. Cold stress response

[0171] 2. Rice cadmium stress response

[0172] 3. Early cold stress response in rice

[0173] 4. Rice drought stress response

[0174] 5. Rice heat stress response

[0175] 6. Low-level gamma radiation response

[0176] Then, list all environmental responsiveness categories of AGIS_Os06g035130:

[0177] 1. Rice cadmium stress response

[0178] 2. Rice salt stress response

[0179] 3. Rice drought stress response

[0180] 4. Oxidative stress response in rice

[0181] 5. Response of rice to gibberellin stress

[0182] 6. Adaptive response of rice shoot roots

[0183] 7. Rice and Arabidopsis receptor-like kinase gene response

[0184] Comparing these two lists, we can see that they have two common types of environmental responsiveness, namely:

[0185] 1. Rice cadmium stress response

[0186] 2. Rice drought stress response

[0187] Therefore, the number of similar environmental responsiveness types between AGIS_Os05g040410 and AGIS_Os06g035130 is 2.

[0188] Final reply:

[0189] How many environmental responsiveness are similar between AGIS_Os05g040410 and AGIS_Os06g035130?

[0190] According to the reference information, the number of similar environmental responsiveness types between AGIS_Os05g040410 and AGIS_Os06g035130 is 2. These common environmental responsiveness include:

[0191] 1. Rice cadmium stress response

[0192] 2. Rice drought stress response

[0193] This was concluded by comparing the environmental responsiveness lists of the two genes.

[0194] ===========;

[0195] Traditional large language models are unfamiliar with professional terms (such as rice varieties) and concepts in the field of crop breeding. When encountering queries containing unfamiliar professional terms and concepts, retrieval of knowledge graphs degenerates into fuzzy matching, which greatly reduces the credibility of the query results.

[0196] From the above examples, we can see that by searching the knowledge graph for crop breeding problems using predefined operators, the search is performed at a finer granularity (each sub-step), making the entire search process more interpretable, increasing the probability of matching professional terms and concepts during the query process, and thus increasing the credibility of the search results. In addition, each operator has clear input parameters, and the maximum context length requirement for large language models is low, thereby reducing the memory requirements for hardware execution.

[0197] In the retrieval process, whether the operator list containing predefined operators includes all the operators required to answer the question is of great significance to the realization of the retrieval process. In order to improve the support of operators for crop breeding problems, the problems in the crop breeding test set can be decomposed to improve the coverage of the operator list for existing problems. The crop breeding test set can be an artificially generated question-answering set used to evaluate the performance of the large language model in answering questions in the field of crop breeding. The specific content of the crop breeding test set is described in detail below.

[0198] In some embodiments, the predefined operator can be determined in the following manner. First, for each evaluation question in the crop breeding test set, the evaluation question is decomposed into multiple sub-steps using the large language model 130, and each sub-step includes an operator. Then, it is determined whether the operator list of the large language model 130 covers the operators of the decomposed sub-steps of all evaluation questions, and the operator list includes the predefined operators. Finally, the operator list is selectively updated according to the result of the determination.

[0199] In some embodiments, the predefined operators may be determined in the following manner: For each evaluation question in the crop breeding test set, the evaluation question is decomposed using the large language model 130. For evaluation questions that fail to be decomposed, the expression of the operators in the operator list is selectively adjusted.

[0200] By improving the coverage of crop breeding issues in the operator list, it helps to improve the accuracy and comprehensiveness of the retrieval results and provide a basis for the large language model to generate better answers.

[0201] Additionally or alternatively, the prompts of the large language model may be optimized to improve the success rate of decomposing the large language model into sub-steps.

[0202] In some embodiments, successful decomposition examples for crop breeding problems may be added to the prompts of the large language model to guide the large language model to perform decomposition in a targeted manner.

[0203] Here is an example of a successful decomposition for a crop breeding problem:

[0204] ===========;

[0205] "query": "What is the parental relationship between Huang Fengzhan and Feng Xiuzhan?",

[0206] "answer": [{"step":"Query the parent relationship between Huang Fengzhan and Feng Xiuzhan","action":"get_spo(s=s1:Rice[Huang Fengzhan], p=p1:Parent relationship, o=o1:Rice[Feng Xiuzhan])"},{"step":"Query the parent relationship","action":"get(p1)"}]

[0207] ===========;

[0208] In some embodiments, a requirement for question decomposition may be added to the prompts of the large language model.

[0209] The following is an example of a requirement for problem decomposition:

[0210] ===========;

[0211] ## Output requirements

[0212] - The output result is a list, each element of the list contains a step and an action. The step is the description of the sub-problem, and the action is the function to be executed.

[0213] - Each sub-question can only execute 1 function, not multiple

[0214] - You don't nest a function inside another function's parameters

[0215] - You will not only output natural language, if there is no suitable function to solve the problem, directly reply "there is no suitable function"

[0216] ## Notes

[0217] - You only need to break down the problem into sub-steps from an NLP perspective, **no need to answer the question itself**

[0218] ===========;

[0219] In some cases, when searching in the knowledge graph using predefined operators, the search results obtained may not meet the requirements. For example, for rare crop breeding problems, there may not be a corresponding predefined operator, resulting in search results that do not meet the requirements. Therefore, in order to improve the accuracy of the search results, it is necessary to judge whether the search results meet the requirements.

[0220] In some embodiments, the large language model 130 may be configured to determine whether the credibility of the knowledge graph retrieval results meets the requirements.

[0221] In some embodiments, in response to the credibility meeting the requirement, the large language model 130 may be configured to return the knowledge graph search result as the final search result. In this case, the search result obtained using the predefined operator is the final search result.

[0222] In some embodiments, in response to the credibility not satisfying the requirement, the large language model 130 may be configured to perform an additional search on the knowledge graph to determine a final search result.

[0223] For example, the large language model 130 may determine whether the credibility of the search result meets the requirement based on the following prompts.

[0224] ===========;

[0225] yaml

[0226] You are a Chinese and English examiner, good at analyzing whether students can get the answers to the test questions based on their own evidence, and giving YES or NO accordingly.

[0227] ## Task

[0228] Please read the test questions and student evidence carefully to determine whether the student evidence can answer the test questions.

[0229] ## Output format requirements

[0230] - If the student's evidence is insufficient, output NO

[0231] - If the student's evidence contains or can deduce the answer to the test question, output YES

[0232] - Before giving the final YES / NO, you will explain why you made this judgment

[0233] ## Notes

[0234] - When checking students' answers, you will notice the difference in units of expression, for example, 1 kg is 2 jin

[0235] ## Example of student answers

[0236] - Output NO after explanation: "It cannot be determined. The information in options A, B, C, and D does not match what is provided in the literature."

[0237] - Output NO after explanation: "Based on the existing information, we cannot determine the breeding mother and father of Koshihikari. However, the literature mentions that Koshihikari was used as a parent and was hybridized with other conventional varieties (lines)"

[0238] - Output YES after explanation: "According to the provided materials, Koshihikari's parents are Kinki 34 (♀) and Hokuriku 4 (♂)"

[0239] ## Exam Questions

[0240] {input_query}

[0241] ## Student basis

[0242] ```txt

[0243] {input_evidence}

[0244] ```

[0245] ===========;

[0246] In some embodiments, the credibility of the knowledge graph search result can be determined based on the result itself. The manner of determining the credibility of the knowledge graph search result based on the result itself is as described above for the large language model 130.

[0247] In some embodiments, the credibility of the knowledge graph retrieval result can be determined based on the process of generating the result. As an example, the large language model 130 can determine whether the credibility of the retrieval result meets the requirements based on the success rate of decomposing the crop breeding problem into predefined operators and executing the operators. For example, the pass@k indicator can be used to evaluate the success rate of the large language model 130 in decomposing and executing the crop breeding problem. For example, pass@k = the number of sub-steps passed / the total number of sub-steps, if the indicator is greater than a predetermined threshold, it indicates that the credibility meets the requirements. Otherwise, it indicates that the credibility does not meet the requirements.

[0248] By judging whether the credibility meets the requirements and responding differently according to different situations, the large language model can obtain more accurate retrieval results as the basis for generating the final answer.

[0249] When the confidence level does not meet the requirements, additional searches such as Graph-based Retrieval-Augmented Generation (GraphRAG) can be performed as a fallback strategy to provide basic answers to crop breeding questions.

[0250] In some embodiments, in response to the credibility not meeting the requirement, the large language model 130 may be configured to decompose the crop breeding problem into a multi-level vocabulary. The multi-level vocabulary may include a first-level vocabulary and a second-level vocabulary, and the second-level vocabulary may include a hypernym of the first-level vocabulary. The large language model 130 may be configured to search in the knowledge graph using the multi-level vocabulary to determine additional search results. The large language model 130 may be configured to return the additional search results as the final search results.

[0251] As an example, first, the crop breeding problem input by the user is decomposed into multi-level vocabulary. The following is an example of the decomposition process:

[0252] ===========;

[0253] Plain Text

[0254] query: "How many years has Zhejiang Radiation 802 been planted?"

[0255] low_level: [Zhejiang Radiation 802, year]

[0256] high_level: [rice variety, planting time]

[0257] ===========;

[0258] Among them, rice variety is the superordinate concept of Zhejiang Radiation 802, and planting time is the superordinate concept of year.

[0259] Then, the first-level vocabulary and the second-level vocabulary are used to retrieve the associated entities and relations in the knowledge graph, and mapped to the text blocks corresponding to the retrieved entities and relations. Therefore, the retrieved entities, relations, and text blocks can be used as the final retrieval results.

[0260] Therefore, when the credibility of the initial search results does not meet the requirements, by performing additional searches, it can be ensured that the search results meet the basic requirements.

[0261] When the crop breeding questions input by users are more complex, the large language model may retrieve several entities, relationships, and corresponding text blocks in the knowledge graph, but there is an upper limit to the context length that the large language model can handle. Therefore, the retrieval process needs to be adjusted according to the complexity of the problem.

[0262] In some embodiments, the large language model 130 may be configured to determine the complexity of the crop breeding question. In response to the complexity exceeding a complexity threshold, the large language model 130 is configured to retrieve a predetermined number of relationships associated with the crop breeding question in the knowledge graph. The predetermined number is based on the length of the longest context of the large language model 130 and the length of the text block mapped with the crop breeding dataset.

[0263] As an example, the Measure of Textual Lexical Diversity (MTLD) metric can be used to measure the complexity of the problem. It evaluates the lexical richness of the text by calculating the frequency and distribution of different words in the text. The calculation formula is:

[0264] (Formula 1)

[0265] in, is the total number of words in the text, is the number of distinct words in the text, It is The frequency of different words in the text.

[0266] The predetermined number can be calculated in the following way. For example, if the length of the longest context of the large language model is llm_max_length, and the length of the single longest text block mapped to the knowledge graph is chunk_length, then the predetermined number top_k = llm_max_length / chunk_length, which corresponds to the maximum number of text blocks that the context of the large language model can accommodate. When there are several candidate relationships for crop breeding problems, by limiting the number of retrievals, the amount of data processing of the large language model can be reduced and the processing efficiency can be improved.

[0267] In some embodiments, the predetermined number of relationships may include a predetermined number of relationships with the highest priority. The priorities of the relationships in the knowledge graph may be sorted based on the weights of the relationships determined when the knowledge graph is established. The weight of the relationship reflects the closeness between the two entity nodes connected by the relationship in the knowledge graph. For example, when establishing the knowledge graph, the closeness between entities may be scored by a large language model to determine the weight of the corresponding relationship.

[0268] The process of performing corresponding searches based on credibility will be discussed below. Figure 3 Further description.

[0269] During the conversation between the user and the large language model, the user's questions may be divergent, or they may be focused on a certain field, or even a certain aspect of a certain field. When the user's questions repeatedly involve a certain aspect of a certain field, the user is likely to have professional knowledge in this area. If the answers provided by the large language model include more content related to this aspect, it will help improve the user's satisfaction with the answer.

[0270] In some embodiments, the large language model 130 may be configured to record historical messages of the user's dialogue with the large language model 130, the historical messages including questions raised by the user. The large language model 130 may be configured to detect the relevance of the questions raised by the user to multiple aspects of the field of crop breeding. The large language model 130 may be configured to, in response to detecting that a threshold number of questions are related to a first aspect of the multiple aspects, increase the weight of the text block associated with the first aspect in the final search results.

[0271] As an example, during the interaction between the user and the large language model 130, the historical messages of the conversation may be recorded, and the specific expression may use ChatML format, for example:

[0272] ===========;

[0273] Plain Text [

[0275] {"role":"user","content":"Hello"},

[0276] {"role":"assistant","content":"Hello! I am Fengdeng. I can provide you with information and services on seed industry, agricultural technology, crop management, etc. How can I help you?"},

[0277] {"role":"user","content":"What is Zhejiang Radiation 802?"},

[0278] {"role":"assistant","content":"Zhefu 802 is a rice mutant variety, bred by the Institute of Nuclear Agronomy of Zhejiang Agricultural University and Yuhang County Agricultural Science Institute.."},

[0279] {"role":"user","content":"What is Yebai?"} ]

[0281] ===========;

[0282] From the above content, it can be seen that the current question "What is Yebai?" and the previous question "What is Zhejiang Radiation 802?" are both related to rice varieties, and it can be considered that the user has knowledge of rice breeding. Therefore, in the subsequent rearrangement calculation of the text blocks in the search results provided to the large language model 130 as context, the text blocks related to rice can be weighted, thereby reducing the replies that are not related to rice breeding. For example, the specific implementation is:

[0283] ===========;

[0284] Python

[0285] from typing import List

[0286] class Chunk:

[0287] def __init__(self, label: str, score: float):

[0288] self.label = label

[0289] self.score = score

[0290] class Query:

[0291] def __init__(self, label: str, default_bias: float = 1.5):

[0292] self.label = label

[0293] self.default_bias = default_bias

[0294] def rerank(chunks: List[Chunk], query: Query) -> List[Chunk]:

[0295] # Traverse each block and weight the blocks that match the user's domain knowledge

[0296] for chunk in chunks:

[0297] if chunk.label == query.label:

[0298] chunk.score *= query.default_bias # weighted processing

[0299] # Reorder the blocks by their scores (higher scores first)

[0300] chunks.sort(key=lambda x: x.score, reverse=True)

[0301] return chunks

[0302] ===========;

[0303] When it is identified that the user has professional knowledge in a related field, by focusing the answer on that field, the closeness of the answer to the question can be improved, thereby enhancing the user experience.

[0304] Therefore, in the process of interacting with users, for crop breeding issues, the large language model of some embodiments of the present invention utilizes a knowledge graph containing professional information combined with its reasoning ability to provide more accurate answers.

[0305] In order to construct a knowledge graph related to crop breeding, it is necessary to extract crop breeding related data from crop breeding related literature and extract triples from crop breeding related data. However, crop breeding related data has the following characteristics:

[0306] (1) Multimodal heterogeneity: data in multiple modalities, including gene sequencing data, phenotypic information, environmental data, and literature text data;

[0307] (2) Dynamic timing: multi-stage experimental data covering the entire growth cycle of crops (germination to maturity);

[0308] (3) Complex association: There is nonlinear interaction between genotype, phenotype and environment (G×E×P), involving more than 200 agronomic trait indicators;

[0309] (4) Domain specificity: includes crop-specific gene naming systems (such as Os01g0705600).

[0310] Due to the multidimensional characteristics of crop breeding-related data, existing regular / fuzzy search methods and natural language processing (NLP) methods cannot generate complete results when extracting triplets.

[0311] As an example, the triple extraction task is as follows:

[0312] ===========;

[0313] Bash

[0314] # Review report text

[0315] "Yangdao No. 6 is a conventional indica rice variety. It is short and strong in the seedling stage, grows well, has medium tillering ability, plant height of 110 cm, 1000-grain weight of 30 g, and a full growth period of about 156 days."

[0316] # Document description text

[0317] "OsPPKL1 increased the number of grains per panicle by inhibiting the expression of OsCKX2 in panicles (p<0.01)"

[0318] ===========;

[0319] For the regular / fuzzy search method, the results are as follows:

[0320] ===========;

[0321] Bash

[0322] # Review report text regularization results

[0323] [('Yangdao No. 6', 'Plant height', '110 cm'),

[0324] ('Yangdao No. 6', 'Thousand Grain Weight', '30g'},

[0325] ('Yangdao 6', 'Full growth period', '156 days')]

[0326] # Document description text regularization results

[0327] [('OsPPKL1', 'Inhibit', 'OsCKX2')]

[0328] ===========;

[0329] For the regular / fuzzy search method, the search process needs to take into account various situations, such as whether the variety name should contain the word "hao" (Yangdao 6hao / Yangdao 6), whether the units of various agronomic traits are abbreviations (cm, g, day). In addition, it can be seen from the results that the regular / fuzzy search method cannot capture unstructured descriptions such as "medium tillering" and completely loses the information of "increased number of grains per panicle" and significance (p value). Furthermore, the regular / fuzzy search method cannot automatically associate gene IDs and gene symbols in the text. For example, the gene OsCKX2 cannot be mapped to Os01g0197700 and Os01g0197700, so searching only for gene IDs will result in missed detections.

[0330] For the natural language processing method, the results are as follows:

[0331] ===========;

[0332] Python

[0333] # Review report text nlp results

[0334] [('Yangdao No. 6', 'Plant height', '110 cm'),

[0335] ('Yangdao No. 6', 'Thousand Grain Weight', '30g'},

[0336] ('Yangdao 6', 'Full growth period', '156 days'),

[0337] ("Yangdao No. 6", "Type", "Indica conventional rice"),

[0338] ("Yangdao No. 6", "Has characteristics", "Short and strong in seedling stage"),

[0339] ("Yangdao No. 6", "Has characteristics", "Medium tillering")]

[0340] # Document description text nlp results

[0341] [("OsPPKL1", "Negative regulation", "OsCKX2"), ("OsCKX2", "Effect", "Number of grains per ear"),("The above relationship", "Significance", "p<0.01")]

[0342] ===========;

[0343] Natural language processing methods rely on syntactic analysis and rule templates and can capture some unstructured relationships. However, such methods usually rely on predefined sets of entities and relationships and have strong prior constraints. Agronomic traits require labeled samples and the specifications rely on manual rule verification.

[0344] Therefore, some embodiments of the present invention propose to use a large language model to construct a knowledge graph.

[0345] In some embodiments, the large language model 130 may be configured to obtain documents related to crop breeding. The large language model 130 is configured to extract triples of a crop breeding dataset from the document. The large language model 130 is configured to build a knowledge graph based on the triples of the crop breeding dataset, where the triples are mapped to text blocks in the document.

[0346] For example, a search may be performed in a database in Chinese, English, or other languages ​​to obtain documents based on crop breeding related terms. Then, the large language model 130 may extract triples based on the prompts to construct a knowledge graph. The following is an example of a prompt:

[0347] ===========;

[0348] Plain Text

[0349] ## Role settings:

[0350] You are an agricultural informatics expert responsible for extracting structured knowledge from multi-source breeding data, combining knowledge of molecular biology, agronomic trait standards, and breeding practices.

[0351] ## Mission Objectives:

[0352] Build a high-quality knowledge graph of crop breeding data based on the provided text

[0353] ## Restrictions

[0354] 1. It is prohibited to use vague expressions such as "a gene". Unknown genes are marked as "UNK-GENE-[serial number]"

[0355] 2. Yield-related traits must indicate the measurement standards

[0356] 3. When the text mentions some unquantified conditions, it will be associated with the evaluation level, such as "drought conditions", it will automatically be associated with the drought level 4. ......

[0358] ## Thinking process:

[0359] 1. Understand the content of the text;

[0360] 2. Locate and build each set of associations in turn;

[0361] 3. When constructing each set of associations, the knowledge information implicit in the associations

[0362] 4. Will return standard json structure data

[0363] ## Text content:

[0364] {}

[0365] ===========;

[0366] For the large language model to extract triples, the results are as follows:

[0367] ===========;

[0368] Python

[0369] # Review report text LLM results [

[0371] ("Yangdao No. 6", "Classification", "Indica conventional rice"),

[0372] ("Yangdao No. 6", "plant height", "110 cm"),

[0373] ("Yangdao No. 6", "1000-grain weight", "30g"),

[0374] ("Yangdao No. 6", "Growing period", "156 days"),

[0375] ("Yangdao No. 6", "Performance traits", "Short and strong in the seedling stage"),

[0376] ("Yangdao 6", "tillering characteristics", "medium") ]

[0378] # Document description text LLM results

[0379] {"core_relation":

[0380] {"subject": "OsPPKL1",

[0381] "action": "Suppress",

[0382] "object": "OsCKX2",

[0383] "effect":

[0384] {"trait": "Number of grains per ear",

[0385] "direction": "Increase",

[0386] "tissue": "spike"},

[0387] "evidence":

[0388] {"p_value": 0.01,

[0389] "experiment_type": "Gene Expression Analysis"}

[0390] }

[0391] }

[0392] ===========;

[0393] By using a large language model to extract triplets and construct a knowledge graph, implicit knowledge can be automatically associated, the biological significance of regulatory relationships can be understood (for example, inhibition corresponds to negative regulation), and regulatory chains can be associated (for example, from OsPPKL1 to OsCKX2, and then to phenotype). Therefore, by leveraging the information extraction capabilities of a large language model, open mining of crop breeding knowledge can be achieved, breaking through the preset limitations of traditional methods on the "entity-relationship" category and improving the flexibility and coverage of knowledge acquisition.

[0394] The large language model 130 has the following capabilities when building a knowledge graph:

[0395] (1) Complex relationship modeling capability: capturing complex relationship descriptions in the literature;

[0396] (2) Cross-literature knowledge fusion: automatically associate equivalent descriptions in different literature (e.g., "thousand-grain weight" ≈ "TKW" ≈ "1000-grain weight", OsGSL1 corresponds to LOC_Os01g12345);

[0397] (3) Self-check of agronomic trait specifications: whether the traits mapped by the gene meet the agronomic trait specifications;

[0398] (4) Identification of experimental dynamic relationships: Recognize information such as plant gene mutation and overexpression during the entire experimental process to establish the dynamic relationship between genes and traits;

[0399] (5) Context-aware reasoning: Able to understand the interactive phenotypes of multiple factors such as gene names and crop growth environment in the text, such as discovering implicit relationships such as "Gene X affects yield through pathway Y under low nitrogen conditions."

[0400] Therefore, the large language model 130 of some embodiments of the present invention can perform targeted processing on crop breeding related data in the literature, so that the extracted triples have richer information.

[0401] In some embodiments, the triple may include a head entity and a tail entity, and the large language model 130 may be configured to extract a gene name or a variety name from the document as the head entity. In some embodiments, when the head entity is a gene name, the large language model 130 may be configured to extract a breeding history, a trait record, or a cultivation technique from the document as the tail entity. In some embodiments, when the head entity is a variety name, the large language model 130 may be configured to extract agronomic traits from the document as the tail entity.

[0402] When the "head entity" is defined as a gene or variety, the possibility of the large language model extracting other information is excluded. Then, the "tail entity" is also specified in detail. For example, when the "head entity" is a variety, the "tail entity" is specified to be limited aspects such as breeding history, trait records, and cultivation techniques. For another example, when the "head entity" is a variety, the "tail entity" is specified to be limited aspects such as agronomic traits (such as plant height, yield, flowering period, etc.), and the large language model is not allowed to extract irrelevant information. Therefore, the provisions for head and tail entities improve the accuracy of information extraction.

[0403] In the documents related to crop breeding obtained by the large language model, the data in the documents can be multimodal data, such as text data, gene data, experimental data, etc. The existing technology cannot dynamically associate text descriptions with structured omics data (such as gene expression, protein interaction network), which limits its application in tasks such as gene function annotation and breeding strategy generation. Some embodiments of the present invention use a large language model to extract triplets from multimodal data to achieve alignment and fusion of multimodal data.

[0404] Here are examples of multimodal data and the triplets that the large language model extracts from the examples of multimodal data:

[0405] Text data (language information consisting of words, sentences, or paragraphs):

[0406] Longjing 31 is a japonica rice variety with 11 leaves on the main stem, 92 cm plant height, 15.7 cm panicle length, about 86 grains per panicle, 26.3 g 1000-grain weight, and strong cold resistance. According to the analysis of the Rice and Product Quality Supervision and Inspection Center of the Ministry of Agriculture (Harbin), the rough rice rate is 81.1%~81.2%, the polished rice rate is 71.6%~71.8%, the chalky rice rate is 0.0%~2.0%, the chalkiness is 0.0%~0.1%, the amylose content (dry basis) is 16.89%~17.43%, the gel consistency is 70.5~71.0 mm, and the taste quality is 79~82 points. The inoculation identification results are: leaf blight level 3~5, panicle neck blight level 1~5; cold resistance identification results: the empty shell rate after treatment is 11.39%~14.10%. In the adapted area, the growth period from germination to maturity is about 130 days, and the active accumulated temperature needs to be ≥10℃ and about 2350℃.

[0407] ===========;

[0408] Plain Text

[0409] ("Longjing 31", "Variety", "Japonica Rice")

[0410] ("Longjing 31", "main stem", "11 leaves")

[0411] ("Longjing No. 31", "ear length", "15.7cm")

[0412] ("Longjing 31", "Number of grains per ear", "86 grains")

[0413] ("Longjing No. 31", "1000-grain weight", "26.3g")

[0414] ("Longjing No. 31", "Roughness", "81.1%~81.2%")

[0415] ("Longjing 31", "Rice polishing rate", "71.6%~71.8%")

[0416] ("Longjing No. 31", "Chalky white rice rate", "0.0%~2.0%")

[0417] ("Longjing No. 31", "Chalkiness", "0.0%~0.1%")

[0418] ("Longjing No. 31", "Amylose content", "16.89%~17.43%")

[0419] ("Longjing No. 31", "Glue consistency", "70.5~71.0mm")

[0420] ("Longjing No. 31", "Taste Quality", "79~82 points")

[0421] ("Longjing 31", "Leaf Plague", "Level 3~5")

[0422] ("Longjing 31", "Neck plague", "Level 1~5")

[0423] ("Longjing No. 31", "Processing empty shell rate", "11.39%~14.10%")

[0424] ("Longjing 31", "Number of days to grow", "130 days")

[0425] ("Longjing 31", "Active Accumulated Temperature", "2350℃")

[0426] ===========;

[0427] For example, in the prompt of the large language model, you can clearly ask the large language model to find out what the "head entity" is first, and then ask the large language model to automatically find the "tail entity" and "relationship" based on semantic understanding. Taking the above text data as an example, the prompt can include two parts. First, the large language model automatically recognizes that Longjing No. 31 is the head entity in the text based on semantic understanding, and then asks the large language model to find the related "tail entity" and "relationship".

[0428] The following are examples of data from other modalities and the corresponding extracted triples:

[0429] Genomic data (e.g., DNA, RNA sequence data, gene expression data, mutation information, and other biological information):

[0430] AGIS_Os01g000010.mRNA:

[0431] MSSAAGQDNGDTAGDYIKWMCGAGGRAGGAMANLQRGVGSLVRDIGDPCLNPSPVKGSKMLKPEKWHTCFDNDGKVIGFRKALKFIVLGGVDPTIRAEVWEFLLGCYALSSTSEYRRKLRAVRREKYQILVRQCQSMHPSIGTGELAYAVGSKLMDVRTMSKETHIAEEVSTSQQTSQNTAGSLVEDSDYGPGGAQQSQKRESCSKSAELVGFNVHNDTSLYDSSNFIVSSTEVNNCSKDSQDYNDMGEPRYDTETFDDYPSLPVTNFFSTDGVGSNGVDKNHCSFSVPEDRLRHRDERMHSFQINNNIDLIIESNSCSSDVFRASNSDSAIFHSDAYKQDRWLDDNGYNREVIDSLRISDAPEADFVDGTKSNSVVASKDRVSEWLWTLHRIVVDVVRTDSHLDFYGESRNMARMSDILAVYAWVDPSTGYCQGMSDLLSPFVVLYEDDADAFWCFEMLLRRMRENFQMEGPTGVMKQLQALWKIMEITDVELFEHLSTIGAESLHFAFRMLLVLFRRELSFEESLSMWEMMWAADFNEDVILHLEENCLEPLLVDMRNDLSCEVKEEHRVNSYTRRKSKSRKPHHRNGEMRVACNLGMKPNTRNPLCGLSGATIWARHQQMPHISTNVLAKNGDDDLPIFCVAAILVINRHKIIRETRSIDDAIKMFNDNMLKINVKRCVRMAIKLRKKYIYKLLKGGSE

[0432] ==========;

[0433] Plain Text

[0434] ("AGIS_Os01g000010", "amino acid length", "702 amino acids")

[0435] ==========;

[0436] Experimental data (data collected through experimental means, such as numerical data (e.g., measurement values), image data, etc.):

[0437] During the tillering stage of rice, the varieties Pusa-44 (phosphorus deficiency sensitive type) and NIL-23 (phosphorus deficiency tolerant type) were compared under sufficient phosphorus (16 ppm) and no phosphorus (0 ppm) conditions. The transcript expression of gene AGIS_Os10g033440 was upregulated in stems, leaves, panicles and flower organs, while there was a downregulation in the root.

[0438] ===========;

[0439] Plain Text

[0440] ("AGIS_Os10g033440", "tissue", "stem", "upregulated")

[0441] ("AGIS_Os10g033440", "tissue", "leaf", "upregulated")

[0442] ("AGIS_Os10g033440", "tissue", "spike", "up-regulated")

[0443] ("AGIS_Os10g033440", "tissue", "flower", "up-regulated")

[0444] ("AGIS_Os10g033440", "tissue", "root", "downregulated")

[0445] ===========;

[0446] Therefore, the large language model of some embodiments of the present invention can realize the extraction of triplets of multi-omics data from multimodal data (such as text data, genomic data, experimental data, etc.), thereby realizing deep fusion between different data forms, enabling researchers to process and analyze complex biological problems in a unified system.

[0447] Therefore, the large language model of some embodiments of the present invention solves the problems of data processing and model adaptability in crop breeding research by combining with knowledge graphs, which helps to improve research efficiency and quality in many aspects including breeding, genetic improvement and climate adaptability research.

[0448] In some embodiments, the large language model 130 may be trained by pre-training the large language model base on a general corpus and a crop breeding corpus to obtain an intermediate large language model.

[0449] As an example, the large language model base can be the Tongyi Qianwen 2.5-7B base model, or other suitable base models. The general corpus can include Chinese and English educational corpora and code mathematics special corpora. The crop breeding corpus can include documents related to crop breeding.

[0450] The crop breeding corpus can be extracted in the following ways. First, the documents are searched in databases in Chinese, English and other languages ​​based on crop breeding-related terms. Then, the documents are converted into text and unified in format, and regular expression rules are designed based on statistical analysis of the documents to filter out low-quality texts (such as redundant paragraphs). Next, the similarity of text pairs is calculated to remove duplicate content at the sentence level. Finally, the texts are classified by subject, and only those related to fields such as agriculture, biology and chemistry are retained. At the same time, the texts are scored to eliminate those with low scores.

[0451] In order to further improve the accuracy of the large language model in answering crop breeding questions, the intermediate large language model can be subjected to supervised learning based on the crop breeding training set to obtain a trained large language model.

[0452] Since the large language model has been trained with a special crop breeding training set, it can accurately understand and process professional terms and complex problems in the field of crop breeding, improve the knowledge depth, reasoning ability and generation quality of the large language model, and greatly improve the applicability of the large language model in crop breeding research.

[0453] In the training process of large language models, high-quality domain corpora are required, but the existing corpora have low coverage and missing multilingual data, which affects the model training effect. In addition, the corpora in the literature related to crop breeding usually have the following characteristics:

[0454] (1) Terminology-intensive: Contains a large number of professional terms specific to the field of crop breeding, such as gene names, phenotypic traits, etc.;

[0455] (2) Scarcity of structured data: It is difficult to find datasets on the Internet that can be directly used for large language model training, and obtaining high-quality annotated data is costly and time-consuming;

[0456] (3) Complex knowledge associations: involving interactions between multiple factors, such as genes and traits, environmental conditions and yield performance, etc.

[0457] (4) Long-tail knowledge distribution: Some rare breeding techniques or specific variety information appear very rarely in the corpus, but are crucial for professional applications.

[0458] Therefore, the performance of large language models for professional fields can be optimized by generating training sets (i.e., synthetic data) related to the field of crop breeding. Existing data synthesis methods include knowledge graph-based methods and LLM-based methods. Early knowledge graph-based data synthesis methods mainly rely on templates. These methods generate questions from knowledge graphs through predefined templates. Although they can ensure grammatical correctness, the generated questions are relatively rigid, repetitive, lack diversity, and require a lot of manual annotation, and have poor scalability. Subsequently, methods based on RNN (Recurrent Neural Network) appeared, but they still face the challenges of ensuring factual consistency and generating diverse texts. On the other hand, LLM-based methods have poor controllability and hallucination problems. Although there are subsequent data synthesis methods using seed sets combined with LLM, where the seed set is a set of carefully selected or designed initial input samples, usually containing specific tasks or problems, it is still time-consuming and laborious to manually construct the seed set.

[0459] Therefore, some embodiments of the present invention propose to use a question-answer extraction model to generate synthetic data.

[0460] In some embodiments, the system 100 may also include a question and answer extraction model 140 .

[0461] The knowledge graph retrieved by the large language model 130 may be a first knowledge graph. The question-answer extraction model 140 may be configured to extract a second knowledge graph from a crop breeding corpus. The crop breeding corpus includes documents related to crop breeding. The question-answer extraction model 140 may be configured to generate statements based on entities and relationships in the second knowledge graph. The question-answer extraction model 140 may be configured to use the large language model 130 to predict whether the statement is correct to generate a prediction result, and the prediction result indicates the confidence of the large language model 130 in the statement. The question-answer extraction model 140 may be configured to calculate the difference between the prediction result and the true value of the statement. The question-answer extraction model 140 may be configured to determine the statement as a knowledge blind spot of the large language model 130 in response to the difference being greater than an error threshold. The question-answer extraction model 140 may be configured to extract a subgraph in the second knowledge graph based on the knowledge blind spot to generate a crop breeding question-answer sample training set for training the large language model 130.

[0462] As an example, the question-answer extraction model 140 can be a general model, such as Qwen2.5-72B, which can generate multiple statements based on each edge connecting two entities in the second knowledge graph. The statement can be a rewrite of the correct statement of each edge, and the rewritten statement can be correct or wrong. For example, "Long Liangyou 1212 is demonstrated in the Lvfeng High-quality Rice Cooperative in Changting County to further promote the variety" generated by the question-answer extraction model 140 is a correct statement. Then, the large language model 130 to be trained can be used to predict "correct" or "wrong" for these statements, that is, to predict the probability of the large language model 130 answering "yes" or "no", so as to obtain the confidence of the large language model 130 for each statement. Then, the difference between the confidence and the true value can be calculated as the understanding error. If the understanding error is greater than the error threshold, it means that the difference between the confidence of the large language model 130 on these knowledge and the actual judgment accuracy is large, which means that these knowledge are the knowledge blind spots of the large language model 130. For example, the cross entropy can be calculated to measure the difference between the distribution of confidences predicted by the large language model 130 for multiple statements and the distribution of true confidences of multiple statements, which is the understanding error of the large language model 130 for these statements. Finally, the question-answer extraction model 140 can extract subgraphs in the second knowledge graph based on knowledge blind spots to generate a crop breeding question-answer sample training set.

[0463] As an example, a subgraph can be extracted in the following manner. For example, a k-hop subgraph extraction method can be used to extract a subgraph from the second knowledge graph with a certain node as the center according to a predefined number of jumps, wherein the composition and scale of the subgraph are adjusted in combination with knowledge blind spots. For example, the number of jumps can be controlled to ensure that the subgraph covers a predefined jump range. Alternatively, the total length of all entity and relationship descriptions in the subgraph can be calculated, and an upper limit can be set to maintain a balanced data distribution. Additionally or alternatively, edges with higher understanding errors can be preferentially selected when expanding the subgraph, thereby increasing potential information gain.

[0464] Additionally, the question-answer extraction model 140 can use different prompt templates to control the style of generating questions and answers, including basic atomic knowledge (such as a single factual statement), complex aggregated knowledge (involving comprehensive analysis of multiple factors), and multi-hop knowledge (requiring reasoning across multiple knowledge nodes).

[0465] The question and answer extraction model of some embodiments of the present invention generates a crop breeding question and answer sample training set by identifying knowledge blind spots and prioritizing the generation of high-value question and answer pairs with long-tail knowledge, thereby addressing the shortcomings of existing methods in terms of coverage, knowledge representation depth and data diversity, and improving the quality and pertinence of synthetic data.

[0466] Taking rice as an example, after the large language model is supervised by the crop breeding question-answering sample training set obtained in the above way, it is tested on multiple common benchmark datasets (CMMLU (Chinese Massive Multitask Language Understanding), GSM8K (Grade School Math 8K), BBH (Big Bench Hard), MMLU (Massive Multitask Language Understanding)) and the rice question-answering evaluation benchmark. The multiple-choice task is measured by accuracy, and the fill-in-the-blank question is evaluated by perplexity. Ablation experiments show that on the rice question-answering test set, the large language model can avoid catastrophic forgetting, and the cross-domain versatility and performance of rice tasks are significantly improved. The BBH indicator is 23.6% higher than the baseline.

[0467] In order to verify the performance of the large language model of some embodiments of the present invention in answering crop breeding questions, a crop breeding test set of different levels, including basic (e.g., literature retrieval), intermediate (e.g., experimental data analysis), and advanced (e.g., multi-omics integrated reasoning), covering research fields such as gene function, transcriptomics, proteomics, traditional breeding, and molecular breeding, was designed to simulate real research scenarios. The following are examples of different levels of question answering using rice as an example:

[0468] Basic Q&A example:

[0469] ===========;

[0470] Q: How does the disease resistance of Nanjing 46 compare to its female parent?

[0471] A(zh_cn): The disease resistance of Nanjing 46 is similar to that of its parent variety Wuxiangjing 14, both of which are resistant to leaf stripe disease.

[0472] A(en): The disease resistance of Nanjing 46 is similar to that of its maternal parent Wuxiangjing 14, both possessing the characteristic of resistance to stripe blight.

[0473] ==========;

[0474] Intermediate Q&A Example:

[0475] ==========;

[0476] Q: Pib-mediated defense mechanisms protect rice against Magnaportheoryzae. What role do jasmonic acid-related genes play in this response?

[0477] A: Jasmonic acid-related genes play a crucial role in the Pib-mediated defense mechanisms that protect rice against Magnaporthe oryzae byenhancing the plant's resistance through the jasmonic acid signaling pathway.

[0478] ==========;

[0479] Advanced Q&A Example:

[0480] ==========;

[0481] Q: What were the main proteomic changes observed in rice leaves afterthe rice straw return treatment, and how did these changes affect thephotosynthetic and metabolic processes?

[0482] A: The main proteomic changes observed in rice leaves after the ricestraw return (SR) treatment included increased expression of proteins involved in photosynthesis and metabolic processes. These changes positively affected the photosynthetic efficiency and metabolic activities in the riceleaves.

[0483] ===========;

[0484] Based on the crop breeding test set designed above, the large language model of some embodiments of the present invention has the following experimental effects:

[0485] (1) As evaluated by 326 agricultural experts, the large language model of some embodiments of the present invention scored 80.61 in high-level tasks, significantly higher than the general model (e.g., DeepSeek-R1 scored 52.37), and surpassed the baseline of agricultural undergraduates by 40%;

[0486] (2) Compared with traditional search engines, the response time of the large language model of some embodiments of the present invention is shortened by 90%, the accuracy of answering advanced questions is increased to 88.14%, and the retrieval efficiency is improved;

[0487] (3) After combining with the knowledge graph for crop breeding, the large language model of some embodiments of the present invention has an accuracy of 92.3% in gene function prediction tasks, which is 27% higher than the unenhanced version;

[0488] (4) The large language model of some embodiments of the present invention has a gene identifier parsing accuracy of more than 95%, and the false hallucination rate is reduced to 12% (compared to 35% for general models);

[0489] (5) In addition to being able to answer crop breeding questions, the large language models of some embodiments of the present invention retain the ability to perform general language understanding, reasoning, and mathematical tasks, achieving accuracy scores comparable to general models in a variety of general knowledge tasks.

[0490] The following are comparison results of the answers given by the large language model of some embodiments of the present invention and the existing models to the same crop breeding question.

[0491] (1) Existing models:

[0492] ===========;

[0493] query: How was Shanyou 63 cultivated?

[0494] response: Shanyou 63 is a hybrid rice combination bred by the Sanming Agricultural Science Institute in Fujian Province using Zhenshan 97A and the restorer line Minghui 63.

[0495] ===========;

[0496] (2) Large language model of some embodiments of the present invention:

[0497] ===========;

[0498] <khi>:

[0499] How was Shanyou 63 cultivated?

[0500] <seed11m-20250110 / > :

[0501] The development process of Shanyou 63

[0502] First, 29A is a sterile line of mid-late indica three-line hybrid rice, and its main purpose is to cultivate the three-line hybrid rice Yongyou 12. 29A was provided by Hu Fengcai, a researcher at the Plant Protection Research and Development Center of Zhejiang Academy of Agricultural Sciences, and was used to extract donor genomic DNA.

[0503] However, 29A not only served as the parent of 25355A, but also provided an important genetic basis for the cultivation of Shanyou 63. 25355A is a mid-season three-line sterile line, and its genetic basis is 29A, which further contributed to the cultivation of Shanyou 63.

[0504] Shanyou 63 is a three-line hybrid rice selected by Mr. Hu Yuqi, and has become one of the hybrid rice varieties with the largest planting area in Chinese history. Mr. Hu Yuqi selected an individual with good self-pollination from 25355A as the parent, and finally formulated Shanyou 63. Shanyou 63 has not only been widely promoted and planted in Jiangsu, but its excellent traits have been assisted by molecular marker technology in modern biotechnology, further enhancing its quality and yield.

[0505] In addition, 26151A, as the restorer line of Shanyou 63, provides important genetic support for it. Shanyou 63, as one of the main parents of C815S and its hybrid combinations, contributes significant yield advantages.

[0506] Therefore, the successful breeding of Shanyou 63 not only benefits from its excellent genetic foundation, but also depends on the careful selection of scientists and the application of modern biotechnology. Mr. Hu Yuqi's contribution is particularly outstanding. He selected the ideal individuals from 25355A and finally formulated Shanyou 63, making a huge contribution to the high and stable yield of Chinese rice.

[0507] ===========;

[0508] It can be seen from the above answers that the large language model of some embodiments of the present invention can generate more detailed, complete and professional answers.

[0509] It should be understood that although the crop in the above description is taken as rice, the scope of the present invention is not limited thereto. For example, the crop may also include wheat, potatoes, and the like.

[0510] Figure 3 is a schematic diagram of a processing flow of a large language model generating an answer to a user input according to some embodiments of the present invention.

[0511] At a first block 310, user input is received. In some embodiments, the large language model may receive a first user input, which may include a first question.

[0512] At the first determination block 320, it is determined whether the question is related to the field of crop breeding. In some embodiments, the nature of the first question in the first user input may be determined.

[0513] When it is determined that the first question is related to the field of crop breeding, a search is performed in the knowledge graph at the second block 330. In some embodiments, the large language model may use a predefined operator to search in the knowledge graph to determine a knowledge graph search result.

[0514] At the second judgment block 340, it is determined whether the credibility meets the requirements. In some embodiments, the large language model can determine whether the credibility of the knowledge graph retrieval result meets the requirements.

[0515] When it is determined that the credibility of the knowledge graph search result meets the requirement, the knowledge graph search result is returned at the third block 350. In some embodiments, the large language model may return the knowledge graph search result as the final search result.

[0516] Then, at the fourth block 360, the large language model generates an answer. In some embodiments, the large language model may generate an answer based on the first question and the final search result. The above process may be combined with the above Figure 1 The relevant processes described correspond.

[0517] In some embodiments, when it is determined that the credibility of the knowledge graph retrieval result does not meet the requirements (for example, when the judgment result at the second judgment box 340 is "no"), at the fifth box 370, an additional search is performed and the additional search results are returned. In some embodiments, the large language model may perform an additional search on the knowledge graph to determine the additional search results, and return the additional search results as the final search results. Then, the large language model may generate an answer based on the first question and the final search results. The above process can be combined with the above Figure 1 The relevant processes described correspond.

[0518] During the interaction process, if the user inputs a question related to the field of crop breeding, the large language model needs to generate an answer in combination with the retrieval of the knowledge graph, which increases the use of processing resources and prolongs the response time, thus affecting the user experience. Therefore, a balance needs to be struck between obtaining an ideal answer and a good user experience.

[0519] In some embodiments, the large language model may be configured to receive a second input from the user, the second input including a second question. The large language model may be configured to determine the relevance of the second question to the field of crop breeding. In response to the second question being irrelevant to the field of crop breeding (e.g., when the judgment result at the first judgment box 320 is "no"), the processing flow may proceed directly to the fourth box 360. In some embodiments, the large language model may be configured to generate an answer to the second question based on the second question without searching in the knowledge graph. For example, when the user inputs a topic related to weather or greetings, the large language model may directly give a reply.

[0520] Whether the question in the user input is related to the field of crop breeding can be judged in the following ways.

[0521] In some embodiments, the large language model may be configured to divide semantic slots into questions in user input. The large language model may extract specific concepts or key information from the natural language text input by the user as semantic slots. Semantic slots are structured semantic information that are usually related to the user's intent and are easy for machines to understand and process. If the semantic slot is not related to the field of crop breeding, it can be determined that the question in the user input is not related to the field of crop breeding.

[0522] In some embodiments, the large language model can be configured to approximately match the dense features of the question in the user input with the entity word list of the knowledge base of the large language model. Dense features refer to the representation after the question (query) is mapped to a dense vector through a certain encoding method, which can efficiently represent complex semantic information. For example, PQ (Product Quantization) and HNSW (Hierarchical Navigable Small World) can be used to speed up feature comparison. If the result of the approximate match is lower than a predetermined threshold, it can be determined that the question in the user input is not related to the field of crop breeding.

[0523] By filtering user input, the large language model can use its general capabilities to quickly answer questions that are not related to professional fields, making the general performance of the large language model better and saving processing time and resources related to the knowledge graph.

[0524] Therefore, the large language model of some embodiments of the present invention can select the source of the reply content according to the nature of the question, for example, it can come from the knowledge graph, it can also come from the knowledge of the large language model, or it can be a fusion of the knowledge graph and the large language model knowledge, so that it can be dynamically adjusted according to the user's question.

[0525] According to another aspect of the present invention, a method for crop breeding question answering is provided.

[0526] Figure 4 is a flow chart of a first method for crop breeding question answering according to some embodiments of the present invention. The first method may be Figure 1 The large language model 130 in the example calls the computing resources 110 for execution.

[0527] The first method may include step S101: receiving input from a user, the input including a crop breeding problem.

[0528] The first method may include step S102: using a predefined operator to search in a knowledge graph based on a crop breeding problem to determine a knowledge graph search result, the knowledge graph includes a crop breeding data set, and the predefined operator includes an operation of formal constraints required for searching the knowledge graph.

[0529] The first method may include step S103: determining the credibility of the knowledge graph retrieval results.

[0530] The first method may include step S104: selectively performing additional searches based on the confidence level to determine a final search result.

[0531] The first method may include step S105: generating an answer to the crop breeding question based on the crop breeding question and the final search result.

[0532] In some embodiments, a crop breeding dataset may include multi-omics data.

[0533] In some embodiments, the crop breeding dataset may be organized in a knowledge graph in the form of triples, wherein the triples include entities and relationships, and the entities of the triples of multi-omics data include at least one of the following: genes, transcriptomes, proteins, events, gene annotations, traits, and environmental factors.

[0534] Figure 5 is a flow chart of a first process associated with a first method for crop breeding question answering according to some embodiments of the present invention. The first process may be Figure 1 The large language model 130 in the call computing resource 110 is executed, and can be Figure 4 The specific implementation of step S102 in the first method in FIG. 1 is described below, but the scope of the present invention is not limited thereto.

[0535] The first process may include step S1021: decomposing the crop breeding problem into multiple sub-steps.

[0536] The first process may include step S1022: performing an operation of at least one operator in each of the multiple sub-steps to obtain an intermediate search result of each sub-step.

[0537] The first process may include step S1023: determining the knowledge graph retrieval result based on the intermediate retrieval result of the last sub-step among multiple sub-steps.

[0538] In some embodiments, the predefined operator may include at least one of a knowledge graph query, an arithmetic operation on data, a logical operation on data, and a result output.

[0539] In some embodiments, the predefined operators may be determined in the following manner: for each evaluation problem in a crop breeding test set, the evaluation problem is decomposed into multiple sub-steps using a large language model, each sub-step including an operator; it is determined whether the operator list of the large language model covers the operators of the decomposed sub-steps of all evaluation problems, the operator list including the predefined operators; and the operator list is selectively updated according to the result of the determination.

[0540] In some embodiments, the predefined operators may be determined in the following manner: for each evaluation problem in the crop breeding test set, decomposing the evaluation problem using a large language model; for evaluation problems that fail to be decomposed, selectively adjusting the expression of operators in the operator list.

[0541] Figure 6 is a flow chart of a second process associated with the first method for crop breeding question answering according to some embodiments of the present invention. The second process may be Figure 1 The large language model 130 in the call computing resource 110 is executed, and can be Figure 4 The specific implementation of step S104 in the first method in FIG. 1 is described below, but the scope of the present invention is not limited thereto.

[0542] The second process may include step S1041: determining whether the credibility meets the requirement.

[0543] The second process may include step S1042: in response to the credibility meeting the requirement, returning the knowledge graph retrieval result as the final retrieval result.

[0544] The second process may include step S1043: in response to the credibility not meeting the requirement, performing an additional search on the knowledge graph to determine a final search result.

[0545] In some embodiments, step S1043 may include the following steps: decomposing the crop breeding problem into a multi-level vocabulary, the multi-level vocabulary including a first-level vocabulary and a second-level vocabulary, the second-level vocabulary including the superordinate concepts of the first-level vocabulary; searching in the knowledge graph using the multi-level vocabulary to determine additional search results; and returning the additional search results as the final search results.

[0546] Figure 7 is a flow chart of a third process associated with the first method for crop breeding question answering according to some embodiments of the present invention. The third process may be Figure 1 The large language model 130 in the call computing resource 110 is executed and may be an additional step of the first process or the second process associated with the first method for crop breeding question answering, but the scope of the present invention is not limited thereto.

[0547] The third process may include step S1024: determining the complexity of the crop breeding problem.

[0548] The third process may include step S1025: in response to the complexity exceeding the complexity threshold, retrieving a predetermined number of relationships associated with the crop breeding problem in the knowledge graph. The predetermined number is based on the length of the longest context of the large language model and the length of the text block mapped to the crop breeding dataset. The predetermined number of relationships includes a predetermined number of relationships with the highest priority, wherein the priority of the relationships in the knowledge graph is sorted based on the weight of the relationship determined when the knowledge graph is established, and the weight of the relationship reflects the closeness between two entity nodes connected by the relationship in the knowledge graph.

[0549] Figure 8 is a flow chart of a second method for crop breeding question answering according to some embodiments of the present invention. The second method may be Figure 1 The large language model 130 in the example calls the computing resources 110 for execution.

[0550] The second method may include step S201: recording historical messages of a conversation between a user and a large language model, where the historical messages include questions raised by the user.

[0551] The second method may include step S202: detecting the relevance of the question raised by the user with multiple aspects of the field of crop breeding.

[0552] The second method may include step S203: in response to detecting that a threshold number of questions are related to a first aspect among the multiple aspects, increasing the weight of a text block associated with the first aspect in the final search result.

[0553] In some embodiments, the second method can be independent of the above combination Figure 4 In some embodiments, the second method may be combined with the above Figure 4 The first method described is performed.

[0554] Fig. 9 is a flow chart of a third method for crop breeding question answering according to some embodiments of the present invention. The third method may be Figure 1 The large language model 130 in the example calls the computing resources 110 for execution.

[0555] The third method may include step S301: receiving a second input from a user, where the second input includes a second question.

[0556] The third method may include step S302: determining the relevance of the second question to the field of crop breeding.

[0557] The third method may include step S303: in response to the second question being irrelevant to the field of crop breeding, generating an answer to the second question based on the second question.

[0558] In some embodiments, the large language model is configured to perform the following steps: divide the second question into semantic slots, and if the semantic slots are irrelevant to the field of crop breeding, determine that the second question is irrelevant to the field of crop breeding.

[0559] In some embodiments, the large language model is configured to perform the following steps: approximately match the dense features of the second question with the entity word list of the knowledge base of the large language model, and if the result of the approximate match is lower than a predetermined threshold, determine that the second question is not related to the field of crop breeding.

[0560] In some embodiments, the third method can be independent of the above combination Figure 4 In some embodiments, the third method may be combined with the above Figure 4 The first method described is performed.

[0561] In some embodiments, the third method can be independent of the above combination Figure 8 In some embodiments, the third method may be combined with the above Figure 8 The second method described is performed.

[0562] Fig.10 is a flow chart of a fourth method for crop breeding question answering according to some embodiments of the present invention. The fourth method may be Figure 1 The large language model 130 in the example calls the computing resources 110 for execution.

[0563] The fourth method may include step S401: obtaining literature related to crop breeding.

[0564] The fourth method may include step S402: extracting triples of a crop breeding data set from literature, where the triples include a head entity and a tail entity.

[0565] Step S402 may include step S4021: extracting gene names or variety names from literature as head entities.

[0566] Step S402 may include step S4022: when the head entity is a gene name, extracting breeding history, trait records or cultivation techniques from literature as the tail entity.

[0567] Step S402 may include step S4023: when the head entity is a variety name, extracting agronomic traits from the literature as the tail entity.

[0568] The fourth method may include step S403: constructing a knowledge graph based on triples of the crop breeding dataset, wherein the triples are mapped to text blocks in the literature.

[0569] In some embodiments, the fourth method can be independent of the above combination Figure 4 In some embodiments, the fourth method may be combined with the above Figure 4 The first method described is performed.

[0570] In some embodiments, the fourth method can be independent of the above combination Figure 8 In some embodiments, the fourth method may be combined with the above Figure 8 The second method described is performed.

[0571] In some embodiments, the fourth method can be independent of the above combination Fig. 9 In some embodiments, the fourth method may be combined with the above Fig. 9 The third method described is performed.

[0572] Fig.11 is a flowchart of a fifth method for generating a training dataset according to some embodiments of the present invention.

[0573] In some embodiments, the fifth method can be combined with the above Figure 4 For example, the fifth method can be performed by Figure 1 The question-answer extraction model 140 in the embodiment calls the computing resource 110 to execute. In some embodiments, the fifth method can be independent of the above combination Figure 4 The first method described is performed.

[0574] The fifth method may include step S501: extracting a second knowledge graph from a crop breeding corpus, where the crop breeding corpus includes documents related to crop breeding.

[0575] The fifth method may include step S502: generating statements based on entities and relationships in the second knowledge graph.

[0576] The fifth method may include step S503: using the large language model to predict whether the statement is correct to generate a prediction result, where the prediction result indicates the confidence of the large language model on the statement.

[0577] The fifth method may include step S504: calculating the difference between the predicted result and the true value of the statement.

[0578] The fifth method may include step S505: in response to the difference being greater than an error threshold, determining the statement as a knowledge blind spot of the large language model.

[0579] The fifth method may include step S506: extracting a subgraph in the second knowledge graph based on the knowledge blind spot to generate a crop breeding question and answer sample training set for training a large language model.

[0580] According to another aspect of the present invention, a computer-readable storage medium is provided.

[0581] Fig.12 is a block diagram of a computer-readable storage medium 1200 according to some embodiments of the present invention.

[0582] The computer readable storage medium 1200 stores a computer program 1250. When the computer program 1250 is executed by the processor, the computer program 1250 implements the above-mentioned Figure 4-Figure 11 The steps of each method or process are described.

[0583] According to another aspect of the present invention, a computer program product is provided.

[0584] Fig.13 is a block diagram of a computer program product 1300 according to some embodiments of the invention.

[0585] The computer program product 1300 may include a computer program 1250. When the computer program 1250 is executed by a processor, the computer program 1250 implements the above-mentioned Figure 4-Figure 11 The steps of each method or process are described.

[0586] Embodiments of the present invention have been described with reference to the accompanying drawings. The embodiments are illustrative rather than restrictive.< / khi>

Claims

1. A system for crop breeding question and answer, characterized in that: include: Computing resources; a storage unit configured to store a knowledge graph, wherein the knowledge graph includes a crop breeding dataset; as well as A large language model, wherein the large language model is configured to call the computing resource to: receiving input from a user, the input comprising a crop breeding problem; Using a predefined operator, searching the knowledge graph based on the crop breeding problem to determine a knowledge graph search result, wherein the predefined operator includes an operation of formal constraints required for searching the knowledge graph; Determining the credibility of the knowledge graph retrieval result; Based on the confidence level, selectively performing additional searches to determine final search results; as well as An answer to the crop breeding question is generated based on the crop breeding question and the final search results.

2. The system according to claim 1, characterized in that The large language model is configured to invoke the computing resource to: Decomposing the crop breeding problem into multiple sub-steps; Execute the operation of at least one operator in each of the plurality of sub-steps to obtain an intermediate search result of each sub-step; as well as The knowledge graph retrieval result is determined based on the intermediate retrieval result of the last sub-step among the multiple sub-steps.

3. The system according to claim 1, wherein: The predefined operator is determined in the following manner: For each evaluation problem in the crop breeding test set, the evaluation problem is decomposed into a plurality of sub-steps using the large language model, each sub-step including an operator; judging whether the operator list of the large language model covers the operators of the decomposed sub-steps of all evaluation problems, the operator list including the predefined operators; and selectively updating the operator list according to the result of the judgment; or For each evaluation problem in the crop breeding test set, the large language model is used to decompose the evaluation problem; for the evaluation problem that fails to be decomposed, the expression of the operator in the operator list is selectively adjusted.

4. The system according to claim 1, characterized in that The predefined operator includes at least one of a knowledge graph query, an arithmetic operation on data, a logical operation on data, and a result output.

5. The system according to claim 1, wherein: The large language model is configured to invoke the computing resource to: Determining whether the credibility meets the requirements; In response to the credibility satisfying the requirement, returning the knowledge graph search result as the final search result; In response to the credibility not satisfying the requirement, performing an additional search on the knowledge graph to determine the final search result.

6. The system according to claim 5, characterized in that In response to the credibility not satisfying the requirement, the large language model is configured to call the computing resource to: Decomposing the crop breeding problem into a multi-level vocabulary, wherein the multi-level vocabulary includes a first-level vocabulary and a second-level vocabulary, wherein the second-level vocabulary includes a hypernym of the first-level vocabulary; Searching the knowledge graph using the multi-level vocabulary to determine additional search results; as well as The additional search result is returned as the final search result.

7. The system according to claim 1, wherein: The large language model is configured to invoke the computing resource to: determining the complexity of the crop breeding problem; and In response to the complexity exceeding a complexity threshold, a predetermined number of relationships associated with the crop breeding problem are retrieved in the knowledge graph, the predetermined number being based on a length of a longest context of the large language model and a length of a text block mapped to the crop breeding dataset.

8. The system according to claim 7, characterized in that The predetermined number of relationships includes the predetermined number of relationships with the highest priority, wherein the priority of the relationships in the knowledge graph is sorted based on the weight of the relationship determined when establishing the knowledge graph, and the weight of the relationship reflects the closeness between two entity nodes connected by the relationship in the knowledge graph.

9. The system according to claim 1, characterized in that The input is a first input, and the large language model is configured to call the computing resource to: receiving a second input from the user, the second input comprising a second question; Determine the relevance of the second question to the field of crop breeding; as well as In response to the second question being unrelated to the field of crop breeding, an answer to the second question is generated based on the second question.

10. The system according to claim 9, wherein: The large language model is configured to invoke the computing resource to: dividing the second question into semantic slots, and if the semantic slots are irrelevant to the crop breeding field, determining that the second question is irrelevant to the crop breeding field; or Approximately match the dense features of the second question with the entity word list of the knowledge base of the large language model, and if the result of the approximate match is lower than a predetermined threshold, determine that the second question is not related to the field of crop breeding.

11. The system according to claim 1, characterized in that The large language model is configured to: Recording historical messages of the conversation between the user and the large language model, wherein the historical messages include questions raised by the user; detecting the relevance of the question raised by the user to various aspects of the field of crop breeding; as well as In response to detecting that a threshold number of questions are related to a first aspect of the plurality of aspects, a weight of a text block associated with the first aspect in the final search result is increased.

12. The system of claim 1, wherein: The crop breeding dataset includes multi-omics data.

13. The system of claim 12, wherein: The crop breeding dataset is organized in the knowledge graph in the form of triples, wherein the triples include entities and relationships, and the entities of the triples of the multi-omics data include at least one of the following: genes, transcriptomes, proteins, events, gene annotations, traits, and environmental factors.

14. The system of claim 1, wherein: The large language model is configured to: Access to literature related to crop breeding; extracting triplets of the crop breeding dataset from the literature; and The knowledge graph is constructed based on triplets of the crop breeding dataset, wherein the triplets are mapped to text blocks in the document.

15. The system of claim 14, wherein: The triple includes a head entity and a tail entity, and the large language model is configured to: Extracting gene names or variety names from the document as head entities; When the head entity is a gene name, breeding history, trait records or cultivation techniques are extracted from the literature as the tail entity; When the head entity is a variety name, agronomic traits are extracted from the literature as the tail entity.

16. The system of claim 1, wherein: The knowledge graph is a first knowledge graph, and the system further includes a question-answer extraction model, wherein the question-answer extraction model is configured to: extracting a second knowledge graph from a crop breeding corpus, wherein the crop breeding corpus includes documents related to crop breeding; generating statements based on entities and relationships in the second knowledge graph; Using the large language model to predict whether the statement is correct to generate a prediction result, wherein the prediction result indicates a confidence level of the large language model on the statement; calculating the difference between the predicted result and the true value of the statement; In response to the difference being greater than an error threshold, determining the statement as a knowledge blind spot of the large language model; as well as A subgraph is extracted from the second knowledge graph based on the knowledge blind spot to generate a crop breeding question and answer sample training set for training the large language model.

17. A method for crop breeding question-answering, characterized in that: include: The large language model calls computing resources to perform the following steps: S101: receiving input from a user, wherein the input includes a crop breeding problem; S102: searching in the knowledge graph based on the crop breeding problem using a predefined operator to determine a knowledge graph search result, wherein the knowledge graph includes a crop breeding data set, and the predefined operator includes an operation of formal constraints required for searching the knowledge graph; S103: Determine the credibility of the knowledge graph retrieval result; S104: Based on the credibility, selectively perform additional search to determine the final search result; as well as S105: Generate an answer to the crop breeding question based on the crop breeding question and the final search result.

18. The method according to claim 17, characterized in that The step S102 includes: S1021: Decomposing the crop breeding problem into multiple sub-steps; S1022: Execute the operation of at least one operator in each of the multiple sub-steps to obtain an intermediate search result of each sub-step; and S1023: Determine the knowledge graph retrieval result based on the intermediate retrieval result of the last sub-step among the multiple sub-steps.

19. The method according to claim 17, wherein: The predefined operator includes at least one of a knowledge graph query, an arithmetic operation on data, a logical operation on data, and a result output, and the predefined operator is determined in the following manner: For each evaluation problem in the crop breeding test set, the evaluation problem is decomposed into a plurality of sub-steps using the large language model, each sub-step including an operator; judging whether the operator list of the large language model covers the operators of the decomposed sub-steps of all evaluation problems, the operator list including the predefined operators; and selectively updating the operator list according to the result of the judgment; or For each evaluation problem in the crop breeding test set, the large language model is used to decompose the evaluation problem; for the evaluation problem that fails to be decomposed, the expression of the operator in the operator list is selectively adjusted.

20. The method of claim 17, wherein: The step S102 includes: S1024: Determine the complexity of the crop breeding problem; and S1025: In response to the complexity exceeding a complexity threshold, retrieving a predetermined number of relationships associated with the crop breeding problem in the knowledge graph, the predetermined number being based on the length of the longest context of the large language model and the length of the text block mapped to the crop breeding dataset, the predetermined number of relationships including the predetermined number of relationships with the highest priority, wherein the priority of the relationships in the knowledge graph is sorted based on the weight of the relationships determined when the knowledge graph is established, and the weight of the relationship reflects the closeness between two entity nodes connected by the relationship in the knowledge graph.

21. The method of claim 17, wherein: The large language model calls computing resources to perform the following steps: S201: Recording historical messages of the conversation between the user and the large language model, where the historical messages include questions raised by the user; S202: Detecting the relevance of the question raised by the user with multiple aspects of the crop breeding field; as well as S203: In response to detecting that a threshold number of questions are related to a first aspect among the multiple aspects, increasing the weight of the text block associated with the first aspect in the final search result.

22. The method of claim 17, wherein: The crop breeding dataset includes multi-omics data, which is organized in the knowledge graph in the form of triples, wherein the triples include entities and relationships, and the entities of the triples of the multi-omics data include at least one of the following: genes, transcriptomes, proteins, events, gene annotations, traits, and environmental factors.

23. The method of claim 17, wherein: The large language model is configured to: S401: Obtain literature related to crop breeding; S402: extracting a triple of the crop breeding data set from the document, wherein the triple includes a head entity and a tail entity, including: S4021: extracting a gene name or a variety name from the document as a head entity; S4022: when the head entity is a gene name, extracting breeding history, trait records or cultivation techniques from the literature as a tail entity; S4023: when the head entity is a variety name, extracting agronomic traits from the literature as a tail entity; S403: Constructing the knowledge graph based on the triples of the crop breeding dataset, wherein the triples are mapped to text blocks in the document.

24. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 17 to 23 are implemented.

25. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 17 to 23 are implemented.

Images