An intelligent question-answering method based on a large language model and a domain knowledge graph

Abstract

The application discloses an intelligent question-answering method based on a large language model and a domain knowledge graph, which comprises the following steps: obtaining target vertical domain data, constructing a target domain knowledge graph, and determining a first large language model; when a user's question text is received, a second large language model is determined, if it is determined that the user's question text is related to the target domain knowledge graph, the user's question text is input into the first large language model and the second large language model respectively, and a first query result and a second query result are obtained; a first answer result and a second answer result are obtained from the first query result and the second query result through the second large language model; if it is determined that the first answer result is reasonable according to the second answer result, a final answer result is output according to the first answer result and the second answer result through the second large language model. The application can effectively improve the output efficiency and accuracy of the answer result of the large language model in the vertical domain knowledge question-answering.

Description

Technical Field

[0001] This invention relates to the field of intelligent question answering technology, and in particular to an intelligent question answering method, system, terminal, and computer-readable storage medium based on a large language model and a domain knowledge graph. Background Technology

[0002] With the development of artificial intelligence, large language models have become a hot topic in general artificial intelligence. Large language models (LLMs) are a type of language model that has been pre-trained on massive amounts of data and has a very large number of parameters. They are capable of understanding and generating natural language to perform various tasks. Large language models are widely used due to their powerful natural language understanding and generation capabilities.

[0003] However, existing large language models are mainly good at answering questions in general domains, but they have limitations in answering questions in vertical domains. As a result, the accuracy of the response results output by large language models when answering questions in vertical domains is low, and the output efficiency is also affected, which cannot meet the needs of users.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide an intelligent question-answering method, system, terminal, and computer-readable storage medium based on a large language model and a domain knowledge graph. This invention aims to solve the problem that existing technologies using large language models for knowledge-based question answering in vertical domains suffer from low accuracy and low efficiency in outputting response results, failing to meet user needs.

[0006] To achieve the above objectives, this invention provides an intelligent question answering method based on a large language model and a domain knowledge graph. The intelligent question answering method based on a large language model and a domain knowledge graph includes the following steps:

[0007] Acquire target vertical domain data, construct a target domain knowledge graph based on the target vertical domain data, and determine the first major language model based on the target domain knowledge graph;

[0008] When a user's question text is received, a second major language model is determined. If the second major language model determines that the user's question text is related to the target domain knowledge graph, the user's question text is input into the first major language model and the second major language model respectively to obtain the first query result and the second query result.

[0009] The first query result is input into the second large language model, and the second large language model obtains the first response result and the second response result based on the first query result and the second query result.

[0010] If the first response result is deemed reasonable based on the second response result, then the final response result is output through the second large language model based on the first response result and the second response result.

[0011] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein acquiring target vertical domain data, constructing a target domain knowledge graph based on the target vertical domain data, and determining a first large language model based on the target domain knowledge graph specifically includes:

[0012] Acquire target vertical domain data, construct a preset ontology model based on the target vertical domain data, and perform knowledge extraction processing on the preset ontology model to obtain a target domain knowledge graph;

[0013] Obtain the target language corresponding to the target domain knowledge graph, determine the large language model based on the target language, and perform supervised training fine-tuning on the large language model to obtain the first large language model.

[0014] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein when a user's question text is received, a second large language model is determined; if the second large language model determines that the user's question text is related to the target domain knowledge graph, then the user's question text is input into the first large language model and the second large language model respectively to obtain a first query result and a second query result, specifically including:

[0015] When a user's question text is received, the knowledge background and ontology model of the target domain knowledge graph are obtained;

[0016] A second language model is determined, and the user question text, the knowledge background, and the ontology model are input into the second language model. The second language model is then used to determine whether the user question text is related to the target domain knowledge graph.

[0017] If so, the user's question text is input into the first large language model to obtain the first query result, and the second large language model is used to obtain the second query result based on the user's question text.

[0018] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein inputting the user's question text into the first large language model to obtain the first query result specifically includes:

[0019] The user's question text is input into the first large language model to obtain the entities and corresponding relationships in the user's question text;

[0020] A preset template is determined, and the entity and the relationship are converted into a first query statement based on the preset template;

[0021] Obtain the graph database corresponding to the target domain knowledge graph, and perform a search on the graph database according to the first query statement to obtain the first query result.

[0022] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein obtaining the second query result based on the user's question text using the second large language model specifically includes:

[0023] The second query statement is output based on the user's question text and the ontology model using the second language model.

[0024] The second query statement is used to perform a search in the graph database to obtain the second query result.

[0025] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein the step of inputting the first query result into the second large language model, and obtaining a first response result and a second response result through the second large language model based on the first query result and the second query result, specifically includes:

[0026] The first query result is input into the second large language model, and the second large language model obtains the first response result based on the first query result and the second query result.

[0027] The second response result is obtained by using the second language model based on the first response result, the user's question text, the first query result, and the second query result.

[0028] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein the step of outputting a final response result based on the first response result and the second response result using the second large language model if the first response result is deemed reasonable, specifically includes:

[0029] Based on the second response result, determine whether the first response result is reasonable. If so, obtain the preset response requirements, which include the response style and the level of detail in the response information.

[0030] The preset answer requirements and the rationality analysis process of the first answer result are input into the second large language model. The second large language model outputs the final answer result based on the preset answer requirements, the user's question text, the first answer result, the second answer result, and the rationality analysis process of the first answer result.

[0031] Furthermore, to achieve the above objectives, the present invention also provides an intelligent question-answering system based on a large language model and a domain knowledge graph, wherein the intelligent question-answering system based on a large language model and a domain knowledge graph includes:

[0032] The knowledge graph construction module is used to acquire target vertical domain data, construct a target domain knowledge graph based on the target vertical domain data, and determine the first major language model based on the target domain knowledge graph.

[0033] The relevance determination module is used to determine the second major language model when a user's question text is received. If the second major language model determines that the user's question text is related to the target domain knowledge graph, the user's question text is input into the first major language model and the second major language model respectively to obtain the first query result and the second query result.

[0034] The response result generation module is used to input the first query result into the second large language model, and obtain the first response result and the second response result through the second large language model based on the first query result and the second query result;

[0035] The rationality determination generation module is used to output the final response result based on the first response result and the second response result through the second large language model if the first response result is determined to be reasonable according to the second response result.

[0036] In this invention, target vertical domain data is acquired, a target domain knowledge graph is constructed based on the target vertical domain data, and a first large language model is determined based on the target domain knowledge graph. When a user's question text is received, a second large language model is determined. If the second large language model determines that the user's question text is related to the target domain knowledge graph, the user's question text is input into both the first and second large language models to obtain a first query result and a second query result. The first query result is input into the second large language model, and the second large language model obtains a first response result and a second response result based on the first and second query results. If the second response result determines that the first response result is reasonable, the second large language model outputs a final response result based on the first and second response results. This invention, by constructing a domain knowledge graph and determining a large language model, and combining the domain knowledge graph and the large language model to analyze and process user question text, can fully utilize the powerful natural language processing capabilities of the large language model and the high knowledge quality, high density, and high retrieval efficiency of the knowledge graph, effectively improving the efficiency, versatility, accuracy, and stability of knowledge-based question answering. Attached Figure Description

[0037] Figure 1 This is a flowchart of a preferred embodiment of the intelligent question answering method based on a large language model and domain knowledge graph of the present invention;

[0038] Figure 2 This is a structural diagram of a preferred embodiment of the intelligent question-answering system based on a large language model and domain knowledge graph of the present invention;

[0039] Figure 3 This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0041] With the development of artificial intelligence, large language models have become a hot topic in general artificial intelligence. Large language models (LLMs) are a type of language model that has been pre-trained on massive amounts of data and has a very large number of parameters. They are capable of understanding and generating natural language to perform various tasks. Large language models are widely used due to their powerful natural language understanding and generation capabilities.

[0042] While large language models can enable end-to-end dialogue question answering, direct question answering based on large language models cannot guarantee the accuracy of the answers. Furthermore, large language models excel primarily in general-domain knowledge question answering; when dealing with vertical domains, the effectiveness of dialogue question answering based directly on large language models further declines. Retrieval Augmented Generation (RAG) technology uses unstructured data such as text as an external knowledge base to overcome the accuracy problem of large language models. This technology mainly involves vectorizing the unstructured data in the knowledge base, retrieving relevant fragments through vector retrieval, ranking them, and inputting the top-ranked answers into the large language model to obtain the question answering results. This technology has the following problems: First, it can only answer questions where the answer is contained within a specific fragment, and cannot answer questions that require a holistic understanding; second, the unstructured data is not processed and contains a large amount of invalid information, causing interference and leading to inaccurate matching of relevant fragments during retrieval; third, retrieval efficiency is low when dealing with large amounts of unstructured data.

[0043] A knowledge graph is a structured semantic knowledge base characterized by highly structured knowledge, high knowledge density, and high quality. Existing knowledge graph-based question answering technologies mainly employ logical rule-based and embedding retrieval methods, which are generally only suitable for cases with simple graph structures and simple questions. They often perform poorly and have low accuracy when based on complex vertical domain knowledge graphs or when facing complex problems.

[0044] To address the issues of low accuracy, limited applicability, and low efficiency in current knowledge-based question-answering methods, this invention proposes an intelligent question-answering method that combines a large language model and a domain knowledge graph. This method fully leverages the powerful natural language processing capabilities of the large language model and the high-quality, high-density, and high-efficiency retrieval characteristics of the knowledge graph, effectively improving the efficiency, versatility, accuracy, and stability of knowledge-based question answering.

[0045] The intelligent question answering method based on a large language model and domain knowledge graph described in the preferred embodiment of the present invention, such as... Figure 1 As shown, the intelligent question answering method based on a large language model and domain knowledge graph includes the following steps:

[0046] Step S10: Obtain target vertical domain data, construct a target domain knowledge graph based on the target vertical domain data, and determine the first major language model based on the target domain knowledge graph.

[0047] Specifically, target vertical domain data is acquired, a preset ontology model is constructed based on the target vertical domain data, and knowledge extraction processing is performed on the preset ontology model to obtain a target domain knowledge graph; the target language corresponding to the target domain knowledge graph is acquired, a large language model is determined based on the target language, and the large language model is fine-tuned through supervised training to obtain a first large language model.

[0048] First, data needs to be collected to construct a domain knowledge graph (this invention does not limit the specific domain of the knowledge graph; any domain can be used to conduct question answering by constructing a knowledge graph). The process is as follows: collect vertical domain data (i.e., target vertical domain data in this invention), construct an ontology model of domain knowledge, extract knowledge based on the ontology model, and form a domain knowledge graph through processes such as knowledge fusion and quality assessment.

[0049] Then, the entity relation extraction large language model is trained. The process is as follows: based on the already constructed domain knowledge graph (i.e., the target domain knowledge graph in this invention), a large language model M for entity relation extraction is fine-tuned. a (This refers to the first major language model in this invention), which can identify entities and relationships in a domain knowledge graph from user questions. The specific implementation process is as follows: 1. Construct a training set (for example, the training set construction process can be: "Input: Who is A's brother? Output: [{"entity":{"name":"A","type":"historical figure"},"relation":"brother"}]"). Each data point in the dataset consists of input and output parts. The input part is the sentence posed by the user, and the output is the entities and relationships extracted from the user question, output in JSON format. 2. Select a large language model for fine-tuning. An open-source large language model with billions of parameters can be selected (such as chatglm-3, llama2, etc.). First, based on the primary language of the knowledge in the knowledge graph, if it's primarily Chinese, prioritize large language models trained mainly on domestic Chinese corpora (such as the Chatglm series models and the Tongyi Qianwen series models). If it's primarily English, consider large language models trained mainly on English corpora (such as the Llama series). Then, select a model with an appropriate number of parameters based on available training resources. 3. Choose a fine-tuning method (fine-tuning methods include LoRA, Prefix Tuning, P-Tuning, etc.). Select a suitable fine-tuning method based on the large language model used; generally, LoRA is recommended. 4. Using the constructed dataset, select the large language model for fine-tuning. Perform supervised training using the chosen fine-tuning method to obtain the fine-tuned large language model M. a .

[0050] It is understandable that, in the specific implementation process, this invention uses a green and low-carbon development knowledge graph as an example to illustrate the above technical steps: 1. Data collection and construction of a domain knowledge graph: Collect data related to green and low-carbon development (in this specific implementation process, this invention uses data related to green and low-carbon development as an example, that is, data related to green and low-carbon development is the target vertical domain data in this invention), construct a green and low-carbon development ontology model, extract knowledge based on the ontology model, and form a green and low-carbon knowledge graph through knowledge fusion and quality assessment processes. 2. Training a large language model for entity and relation extraction: Based on the constructed green development knowledge graph, construct a dataset, select the chatglm3 model, and use LoRA to train a fine-tuned large language model M for extracting entities and relations from user-submitted questions. a .

[0051] Step S20: When a user's question text is received, a second major language model is determined. If the second major language model determines that the user's question text is related to the target domain knowledge graph, the user's question text is input into the first major language model and the second major language model respectively to obtain the first query result and the second query result.

[0052] Specifically, when a user's question text is received, the knowledge background and ontology model of the target domain knowledge graph are obtained; a second major language model is determined, and the user's question text, the knowledge background, and the ontology model are input into the second major language model. The second major language model is then used to determine whether the user's question text is related to the target domain knowledge graph. If so, the user's question text is input into the first major language model to obtain the entities and corresponding relationships in the user's question text.

[0053] Furthermore, the user's question (i.e., the user's question text in this invention) is obtained, input into the large language model, and the user's intent is identified as relevant to the domain. The implementation process is as follows: obtain the user's question, add prompt words to the knowledge background of the domain knowledge graph and the ontology model of the domain knowledge graph, and input them into the general large language model M. b (i.e., the second major language model in this invention, the general large language model M) b A sufficiently powerful large language model can be selected (without fine-tuning) to analyze whether the user's question is relevant to the knowledge graph. If it is, proceed to the next step; otherwise, directly generate the answer "The question is irrelevant to the green development graph," ending the question-and-answer process. In this step, the prompts input to the large language model mainly consist of the following:

[0054] 1. Role: Reviewer who determines whether a question is relevant to the field.

[0055] 2. Task: Based on the large language model's own understanding of various domains and the knowledge background of those domains, determine whether the problem is relevant to the domain.

[0056] 3. Information: Domain knowledge background, knowledge graph ontology model, user questions.

[0057] The process of identifying entities and relationships in user questions is as follows: Extract the large language model M using the aforementioned fine-tuned entity and relationship model. a Identify entities and relationships in user questions, that is, input user questions into a finely tuned and trained large language model M. a The output is obtained to get the entities and relationships extracted from the problem by the model.

[0058] A preset template is determined, and the entities and relationships are converted into a first query statement based on the preset template; the graph database corresponding to the target domain knowledge graph is obtained, and the first query statement is used to perform retrieval and search processing in the graph database to obtain a first query result; a second query statement is output through the second language model based on the user's question text and the ontology model; the second query statement is used to perform retrieval and search processing in the graph database to obtain a second query result.

[0059] Furthermore, based on the identified entities and relationships, query statements are generated using templates. The process is as follows: based on the large language model M... a The entities and relationships extracted from user queries are converted into graph database query statements Q using a pre-defined template. a The default template is MATCH(n: 'identified entity type' {name: 'identified entity name'}) - [r: 'identified relation type' * 1.. N] - (m) RETURN n, r, m. Here, N is a default parameter representing the depth of the query results. It needs to be pre-set based on the complexity of the knowledge graph being constructed. If the knowledge structure is not complex, setting it to 1 is sufficient.

[0060] Furthermore, using the query statement generated from the identified entity relationships, relevant results are searched in the knowledge graph. The process is as follows: using query statement Q... a Retrieve the graph database stored in the domain knowledge graph and obtain the query results R. a (i.e., the first query result in this invention).

[0061] Furthermore, large language models are used to directly generate graph database query statements: prompt words are added to the question and knowledge graph ontology model information, and then input into the large language model M. b In the middle, let the large language model M b Based on a holistic approach to the problem, directly generate the query statement Q.b (where Q) a It is obtained based on a pre-defined template, Q b It is the large model M b The outputs are generated in different ways and from different perspectives, resulting in different query statements Q. a and Q b They might be different; large language model M b A more reasonable response can be obtained by comprehensively judging the two different query results. In this step, the prompt words of the input large language model are mainly composed as follows:

[0062] 1. Role: An expert in writing database query statements based on the problem.

[0063] 2. Task: Based on the user's question and the ontology model structure of the knowledge graph, consider from a holistic perspective what information needs to be searched to answer the question, and generate a query statement.

[0064] 3. Information: Knowledge graph ontology model, user questions.

[0065] Furthermore, using the query statement generated by the graph database based on the large language model, relevant results are retrieved. The process is as follows: using the query statement Q... b Retrieve the graph database stored in the knowledge graph and obtain the query results R. b (i.e., the second query result in this invention).

[0066] Understandably, in the specific implementation process, user questions can be obtained, input into a large language model, and the relevance of the user's intent to the domain can be identified. For example, obtaining user questions such as "What are the next level subcategories of farmland ecosystems?", and incorporating the question, background knowledge of green and low-carbon development, and an ontology model of the knowledge graph into prompt words, can be input into the large language model M. b In the middle, large language model M b Choose an open-source model. Large language model M b Based on the information in the prompt, determine whether the question is related to the field of green and low-carbon development. If so, proceed to the next step.

[0067] Identifying entities and relations in user questions: Inputting user questions into a finely tuned, trained large language model M a Large Language Model M a Output: "[{"entity":{"name":"farmland ecosystem","type":"first-level ecosystem classification"}","relation":"with subclasses"}]", which are the entities and relations extracted from the user's question.

[0068] Furthermore, based on the identified entities and relationships, query statements are generated using templates: based on the large language model M a The entities and relationships extracted from user queries are converted into graph database query statements Q using a pre-defined template. a The query statement Q generated according to the template a The expression is: "MATCH(n:'Ecosystem primary classification'{name:'Farmland ecosystem'})-[r:'With subclassification']-(m)RETURN n,r,m".

[0069] Using the identified entity relationships to generate a query statement, relevant results are searched in the knowledge graph, and the query statement Q is used. a Retrieve the graph database stored in the knowledge graph and obtain the query results R. aIt is: [<Record n = <Node element_id='5249' labels=frozenset({'Ecosystem First-level Classification'}) properties={'name': 'Farmland Ecosystem'}> r = <Relationship element_id='5254' nodes=(<Node element_id='5249' labels=frozenset({'Ecosystem First-level Classification'}) properties={'name': 'Farmland Ecosystem'}>, <Node element_id='5267' labels=frozenset({'Ecosystem Second-level Classification'}) properties={'name': 'Garden Plot'}) type='Has Sub-classification' properties={}> m = <Node element_id='5267' labels=frozenset({'Ecosystem Second-level Classification'}) properties={'name': 'Garden Plot'}>>, <Record n = <Node element_id='5249' labels=frozenset({'Ecosystem First-level Classification'}) properties={'name': 'Farmland Ecosystem'}> r = <Relationship element_id='5255' nodes=(<Node element_id='5249' labels=frozenset({'Ecosystem First-level Classification'}) properties={'name': 'Farmland Ecosystem'}, <Node element_id='5268' labels=frozenset({'Ecosystem Second-level Classification'}) properties={'name': 'Cultivated Land'}) type='Has Sub-classification' properties={}> m = <Node element_id='5268' labels=frozenset({'Ecosystem Second-level Classification'}) properties={'name': 'Cultivated Land'}>>].

[0070] Furthermore, directly generate a graph database query statement using a large language model: Add the question and the ontology model information of the knowledge graph to the prompt, and input it into the large language model M b so that the large language model M b directly generates a query statement Q from an overall perspective according to the question b . The large language model generates a query statement Q based on its understanding of the question bIt is: "MATCH(a: First-level classification of ecosystem {name: 'Farmland ecosystem'})-[: Has sub-classification]->(b: Second-level classification of ecosystem) RETURN b".

[0071] Further, use the graph database to query relevant results according to the query statement generated by the large language model: Use the query statement Q b Retrieve the graph database stored in the knowledge graph to obtain the query result R b It is: "[<Record b = <Node element_id = '5267' labels = frozenset({'Second-level classification of ecosystem'}) properties = {'name': 'Garden land'}>>, <Record b = <Node element_id = '5268' labels = frozenset({'Second-level classification of ecosystem'}) properties = {'name': 'Cultivated land'}>>]".

[0072] Step S30: Input the first query result into the second large language model, and obtain the first response result and the second response result through the second large language model according to the first query result and the second query result.

[0073] Specifically, input the first query result into the second large language model, and obtain the first response result through the second large language model according to the first query result and the second query result; obtain the second response result through the second large language model according to the first response result, the user's question text, the first query result, and the second query result.

[0074] Further, combine the results obtained from the two-way query, and use the large language model to generate a preliminary answer: Input the query results R a and R b into the large language model M b after adding prompt words, and let the large language model M b generate a preliminary answer to the user's question (i.e., the first response result in the present invention) according to these reference information. In this step, the prompt words input into the large language model are mainly composed of the following:

[0075] 1. Role: An assistant that generates an answer to the user's question.

[0076] 2. Task: According to the user's question, generate an answer to the user's information based on the results obtained from the query search as reference information.

[0077] 3. Information: The user's question, the results of the two-path search.

[0078] Furthermore, the reasonableness of the initially generated answer is reflected upon using a large language model: the initially generated answer, the user question, and the results of the two-way query R are analyzed. a and R b Added to the prompt words and input into the large language model M b In the middle, let the large language model M b Is the result of reflection reasonable (Large Language Model M)? b (Based on the thought chain, determine if the answer is the correct answer to the user's question). If not, return to the step of directly generating a graph database query statement using the large language model. In this step, the prompt words input into the large language model mainly consist of the following:

[0079] 1. Role: Reviewer who judges whether the answers to user questions are correct.

[0080] 2. Task: Based on the user's question, search results, and generated preliminary answers, confirm the correctness of the answer through step-by-step consideration.

[0081] 3. Information: User questions, preliminary generated answers, and results of two-way queries.

[0082] Understandably, by combining the results from both queries, a preliminary answer is generated using a large language model: the query results R... a and R b Add prompt words and input them into the large language model M. b In the middle, let the large language model M b Based on this reference information, a preliminary answer to the user's question is generated, which is: "garden and cultivated land" (i.e., the second response result in this invention).

[0083] Step S40: If the first response result is determined to be reasonable based on the second response result, the final response result is output through the second large language model based on the first response result and the second response result.

[0084] Specifically, based on the second response result, it is determined whether the first response result is reasonable. If so, a preset response requirement is obtained, wherein the preset response requirement includes the response style and the level of detail of the response information. The preset response requirement and the reasonableness analysis process of the first response result are input into the second large language model. The second large language model outputs the final response result based on the preset response requirement, the user's question text, the first response result, the second response result, and the reasonableness analysis process of the first response result.

[0085] Furthermore, the final answer is generated using a large language model: if the answer is confirmed to be correct by the large language model, the large language model M is used again. bBased on the required response style and level of detail, the final response is generated. In this step, the prompt words input to the large language model are mainly composed as follows:

[0086] 1. Role: The organizer of the response language.

[0087] 2. Task: Organize language according to specific requirements to transform the generated answer into a reasonable response. The response should include the answer to the question and the thought process behind that answer, which may include, but is not limited to: style, level of detail in the response, target audience, and prohibited items.

[0088] 3. Information: The user's question, the generated preliminary answer, and the thought process by which the large language model considers the answer reasonable.

[0089] Understandably, it's worth reflecting on the reasonableness of the initially generated answers using a large language model: combining the initially generated answers "garden and farmland," the user question, and the results of the two-way query R... a and R b Added to the prompt words and input into the large language model M b In the middle, let the large language model M b Reflect on whether the results are reasonable. Large Language Model M b After reflection, the generated result is: "The user's question is about the next level of classification in a farmland ecosystem. Based on the provided information, there are two entities in the farmland ecosystem with the relationship 'has subcategories': an entity with the attribute name 'cultivated land' and an entity with the attribute name 'orchard'. The preliminary answer is 'orchard and cultivated land,' which is consistent with the provided information. Therefore, the answer is correct." Based on the large language model M... b The output indicates that the initially generated answer is reasonable.

[0090] Furthermore, the final answer is generated using a large language model: based on the required answer style and level of detail, the user's question, the initially generated answer, the final answer, and the thought process (also input in text form) deemed reasonable by the large language model are added as prompt words and input into the large language model M. bAmong them, the final answer generated is: "Dear user, the question you asked is 'What are the next-level sub-classifications of the farmland ecosystem?' According to the retrieval of relevant knowledge, the next-level sub-classifications of the farmland ecosystem include cultivated land and garden land. The relevant reference information is: <Node element_id='5267' labels=frozenset({'Ecological system second-level classification'}) properties={'name': 'Garden land'}>>, <Record b=<Node element_id='5268' labels=frozenset({'Ecological system second-level classification'}) properties={'name': 'Cultivated land'}>" (which is the final response result in this invention).

[0091] Advantages of the present invention: The present invention proposes an intelligent question-answering method combining a large language model and a domain knowledge graph. By calling the large language model multiple times, the natural language understanding and generation capabilities of the large language model are utilized to strengthen the understanding of questions. Using the two-way retrieval and recall method, considering from both local and overall perspectives, highly relevant knowledge to the answer is efficiently retrieved from the domain knowledge graph, ensuring that the most accurate reference knowledge can be retrieved and answered efficiently for various complex questions, effectively improving the efficiency, generality, accuracy, and stability of knowledge question-answering.

[0092] Furthermore, as Figure 2 shown, based on the above intelligent question-answering method based on a large language model and a domain knowledge graph, the present invention also correspondingly provides an intelligent question-answering system based on a large language model and a domain knowledge graph. Among them, the intelligent question-answering system based on a large language model and a domain knowledge graph includes:

[0093] A knowledge graph construction module 51, configured to obtain target vertical domain data, construct a target domain knowledge graph according to the target vertical domain data, and determine a first large language model according to the target domain knowledge graph;

[0094] A relevance determination module 52, configured to determine a second large language model when receiving a user question text. If it is determined by the second large language model that the user question text is relevant to the target domain knowledge graph, the user question text is respectively input into the first large language model and the second large language model to obtain a first query result and a second query result;

[0095] A response result generation module 53, configured to input the first query result into the second large language model, and obtain a first response result and a second response result by the second large language model according to the first query result and the second query result;

[0096] The rationality determination generation module 54 is used to output the final response result based on the first response result and the second response result through the second large language model if the first response result is determined to be reasonable according to the second response result.

[0097] Furthermore, such as Figure 3 As shown, based on the above-mentioned intelligent question answering method and system based on large language models and domain knowledge graphs, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 3 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0098] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores an intelligent question-answering program 40 based on a large language model and domain knowledge graph. This intelligent question-answering program 40 can be executed by the processor 10, thereby implementing the intelligent question-answering method based on a large language model and domain knowledge graph in this application.

[0099] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the intelligent question answering method based on a large language model and domain knowledge graph.

[0100] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0101] In one embodiment, the steps of the intelligent question answering method based on a large language model and a domain knowledge graph are implemented when the processor 10 executes the intelligent question answering program 40 based on a large language model and a domain knowledge graph stored in the memory 20.

[0102] In summary, this invention provides an intelligent question-answering method, system, and terminal based on a large language model and a domain knowledge graph. The method includes: acquiring target vertical domain data; constructing a target domain knowledge graph based on the target vertical domain data; and determining a first large language model based on the target domain knowledge graph. When a user's question text is received, a second large language model is determined. If the second large language model determines that the user's question text is related to the target domain knowledge graph, the user's question text is input into the first large language model and the second large language model respectively to obtain a first query result and a second query result. The first query result is input into the second large language model, and the second large language model obtains a first response result and a second response result based on the first query result and the second query result. If the second response result determines that the first response result is reasonable, the second large language model outputs a final response result based on the first response result and the second response result. This invention constructs a domain knowledge graph and determines a large language model, and combines the domain knowledge graph and the large language model to analyze and process user query text. It can fully utilize the powerful natural language processing capabilities of the large language model and the high quality, high density, and high retrieval efficiency of the knowledge graph, effectively improving the efficiency, versatility, accuracy, and stability of knowledge question answering.

[0103] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0104] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0105] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An intelligent question-answering method based on a large language model and domain knowledge graph, characterized in that, The intelligent question answering method based on large language models and domain knowledge graphs includes: Acquire target vertical domain data, construct a preset ontology model based on the target vertical domain data, and perform knowledge extraction processing on the preset ontology model to obtain a target domain knowledge graph; Obtain the target language corresponding to the target domain knowledge graph, determine the large language model based on the target language, and perform supervised training fine-tuning on the large language model to obtain the first large language model; Based on the target domain knowledge graph, a large language model for entity relation extraction is fine-tuned to obtain the first large language model. The first large language model can identify entities and relations of the domain knowledge graph from user questions. When a user's question text is received, the knowledge background and ontology model of the target domain knowledge graph are obtained; A second language model is determined, and the user question text, the knowledge background, and the ontology model are input into the second language model. The second language model is then used to determine whether the user question text is related to the target domain knowledge graph. If so, the user's question text is input into the first large language model to obtain the first query result, and the second large language model is used to obtain the second query result based on the user's question text; The step of inputting the user's query text into the first large language model to obtain the first query result specifically includes: The user's question text is input into the first large language model to obtain the entities and corresponding relationships in the user's question text; A preset template is determined, and the entity and the relationship are converted into a first query statement based on the preset template; Obtain the graph database corresponding to the target domain knowledge graph, and perform a search and retrieval process in the graph database according to the first query statement to obtain the first query result; The first query result is input into the second large language model, and the second large language model obtains the first response result and the second response result based on the first query result and the second query result. If the first response result is deemed reasonable based on the second response result, then the final response result is output through the second large language model based on the first response result and the second response result.

2. The intelligent question answering method based on a large language model and domain knowledge graph according to claim 1, characterized in that, The process of obtaining the second query result based on the user's query text using the second large language model specifically includes: The second query statement is output based on the user's question text and the ontology model using the second language model. The second query statement is used to perform a search in the graph database to obtain the second query result.

3. The intelligent question answering method based on a large language model and domain knowledge graph according to claim 1, characterized in that, The step of inputting the first query result into the second large language model, and obtaining the first response result and the second response result through the second large language model based on the first query result and the second query result, specifically includes: The first query result is input into the second large language model, and the second large language model obtains the first response result based on the first query result and the second query result. The second response result is obtained by using the second language model based on the first response result, the user's question text, the first query result, and the second query result.

4. The intelligent question answering method based on a large language model and domain knowledge graph according to claim 1, characterized in that, If the first response result is deemed reasonable based on the second response result, then the final response result is output through the second large language model based on the first response result and the second response result, specifically including: Based on the second response result, determine whether the first response result is reasonable. If so, obtain the preset response requirements, which include the response style and the level of detail in the response information. The preset answer requirements and the rationality analysis process of the first answer result are input into the second large language model. The second large language model outputs the final answer result based on the preset answer requirements, the user's question text, the first answer result, the second answer result, and the rationality analysis process of the first answer result.

5. An intelligent question-answering system based on a large language model and a domain knowledge graph, characterized in that, The intelligent question answering system based on a large language model and a domain knowledge graph is applied to the intelligent question answering method based on a large language model and a domain knowledge graph as described in any one of claims 1-4, wherein the intelligent question answering system based on a large language model and a domain knowledge graph comprises: The knowledge graph construction module is used to acquire target vertical domain data, construct a target domain knowledge graph based on the target vertical domain data, and determine the first major language model based on the target domain knowledge graph. The relevance determination module is used to determine the second major language model when a user's question text is received. If the second major language model determines that the user's question text is related to the target domain knowledge graph, the user's question text is input into the first major language model and the second major language model respectively to obtain the first query result and the second query result. The response result generation module is used to input the first query result into the second large language model, and obtain the first response result and the second response result through the second large language model based on the first query result and the second query result; The rationality determination generation module is used to output the final response result based on the first response result and the second response result through the second large language model if the first response result is determined to be reasonable according to the second response result.

6. A terminal, characterized in that, The terminal includes: a memory, a processor, and an intelligent question-answering program based on a large language model and a domain knowledge graph, stored in the memory and executable on the processor. When the intelligent question-answering program based on a large language model and a domain knowledge graph is executed by the processor, it implements the steps of the intelligent question-answering method based on a large language model and a domain knowledge graph as described in any one of claims 1-3.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an intelligent question-answering program based on a large language model and a domain knowledge graph. When the intelligent question-answering program based on a large language model and a domain knowledge graph is executed by a processor, it implements the steps of the intelligent question-answering method based on a large language model and a domain knowledge graph as described in any one of claims 1-3.

Images