Intelligent question answering method based on large language model and domain knowledge graph

Abstract

The invention discloses an intelligent question and answer method based on a large language model and a domain knowledge graph, and the method comprises the steps: obtaining target vertical domain data, constructing a target domain knowledge graph, and determining a first large language model; when a user question text is received, a second large language model is determined, and if it is judged that the user question text is related to the target domain knowledge graph, the user question text is input into the first large language model and the second large language model, and a first query result and a second query result are obtained; obtaining a first response result and a second response result according to the first query result and the second query result through a second large language model; and if the first response result is judged to be reasonable according to the second response result, outputting a final response result according to the first response result and the second response result through a second large language model. The method can effectively improve the output efficiency and accuracy of response results of the large language model in the aspect of vertical domain knowledge questions and answers.

Description

Technical Field

[0001] The present invention relates to the field of intelligent question answering technology, and in particular to an intelligent question answering method, system, terminal and computer-readable storage medium based on a large language model and a domain knowledge graph. Background Art

[0002] With the development of artificial intelligence, large language models have become a hot topic in general artificial intelligence. Large language models (LLMs) are a type of language model that has been pre-trained with massive data and has a very large number of parameters. They can understand and generate natural language to perform various tasks. Large language models are widely used for their powerful natural language understanding and generation capabilities.

[0003] However, existing large language models are mainly good at general-domain knowledge questions and answers, and have limitations in vertical-domain knowledge questions and answers. As a result, the accuracy of the answers output by large language models when answering vertical-domain knowledge questions is low, and the output efficiency is also affected, which cannot meet user needs.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide an intelligent question-answering method, system, terminal and computer-readable storage medium based on a large language model and a domain knowledge graph, aiming to solve the problem in the prior art that the large language model has low accuracy and low output efficiency in outputting answer results when answering knowledge questions in vertical fields, and cannot meet user needs.

[0006] To achieve the above object, the present invention provides an intelligent question answering method based on a large language model and a domain knowledge graph, and the intelligent question answering method based on a large language model and a domain knowledge graph comprises the following steps:

[0007] Acquire target vertical field data, construct a target field knowledge graph based on the target vertical field data, and determine a first language model based on the target field knowledge graph;

[0008] When receiving a user question text, determining a second largest language model, if the second largest language model determines that the user question text is related to the target domain knowledge graph, then inputting the user question text into the first largest language model and the second largest language model respectively, to obtain a first query result and a second query result;

[0009] Inputting the first query result into the second largest language model, and obtaining a first answer result and a second answer result according to the first query result and the second query result through the second largest language model;

[0010] If the first response result is determined to be reasonable according to the second response result, the final response result is outputted according to the first response result and the second response result through the second large language model.

[0011] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein the step of acquiring target vertical domain data, constructing a target domain knowledge graph according to the target vertical domain data, and determining a first large language model according to the target domain knowledge graph, specifically includes:

[0012] Acquire target vertical field data, construct a preset ontology model based on the target vertical field data, and perform knowledge extraction processing on the preset ontology model to obtain a target field knowledge graph;

[0013] A target language corresponding to the target domain knowledge graph is obtained, a large language model is determined according to the target language, and the large language model is fine-tuned by supervised training to obtain a first large language model.

[0014] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein when receiving a user question text, determining a second large language model, if the second large language model determines that the user question text is related to the target domain knowledge graph, then inputting the user question text into the first large language model and the second large language model respectively, to obtain a first query result and a second query result, specifically includes:

[0015] When receiving a user's question text, obtaining the knowledge background and ontology model of the target domain knowledge graph;

[0016] Determine a second language model, input the user question text, the knowledge background and the ontology model into the second language model, and determine whether the user question text is related to the target domain knowledge graph through the second language model;

[0017] If so, the user question text is input into the first language model to obtain a first query result, and the second query result is obtained according to the user question text through the second language model.

[0018] Optionally, the intelligent question-answering method based on a large language model and a domain knowledge graph, wherein the step of inputting the user question text into the first large language model to obtain a first query result, specifically includes:

[0019] Inputting the user question text into the first language model to obtain entities in the user question text and corresponding relationships;

[0020] Determine a preset template, and convert the entity and the relationship into a first query statement according to the preset template;

[0021] Obtain a graph database corresponding to the target domain knowledge graph, and perform a search and search process in the graph database according to the first query statement to obtain a first query result.

[0022] Optionally, the intelligent question-answering method based on the large language model and the domain knowledge graph, wherein obtaining the second query result according to the user question text by using the second large language model specifically includes:

[0023] Outputting a second query statement according to the user question text and the ontology model through the second largest language model;

[0024] A search and lookup process is performed in the graph database according to the second query statement to obtain a second query result.

[0025] Optionally, the intelligent question-answering method based on the large language model and the domain knowledge graph, wherein the first query result is input into the second large language model, and the first answer result and the second answer result are obtained according to the first query result and the second query result by the second large language model, specifically includes:

[0026] Inputting the first query result into the second largest language model, and obtaining a first answer result according to the first query result and the second query result through the second largest language model;

[0027] A second answer result is obtained by using the second large language model according to the first answer result, the user question text, the first query result and the second query result.

[0028] Optionally, the intelligent question-answering method based on the large language model and the domain knowledge graph, wherein if the first answer result is determined to be reasonable according to the second answer result, then outputting a final answer result according to the first answer result and the second answer result through the second large language model, specifically includes:

[0029] Determine whether the first answer result is reasonable according to the second answer result, and if so, obtain preset answer requirements, wherein the preset answer requirements include answer style and the degree of detail of answer information;

[0030] The preset answer requirement and the rationality analysis process of the first answer result are input into the second large language model, and the final answer result is output by the second large language model according to the preset answer requirement, the user question text, the first answer result, the second answer result and the rationality analysis process of the first answer result.

[0031] In addition, to achieve the above-mentioned purpose, the present invention also provides an intelligent question-answering system based on a large language model and a domain knowledge graph, wherein the intelligent question-answering system based on a large language model and a domain knowledge graph comprises:

[0032] A knowledge graph construction module, used to obtain target vertical field data, construct a target field knowledge graph according to the target vertical field data, and determine a first language model according to the target field knowledge graph;

[0033] A relevance determination module, configured to determine a second language model when receiving a user question text, and if the user question text is determined to be relevant to the target domain knowledge graph by the second language model, input the user question text into the first language model and the second language model respectively to obtain a first query result and a second query result;

[0034] a response result generating module, configured to input the first query result into the second large language model, and obtain a first response result and a second response result according to the first query result and the second query result through the second large language model;

[0035] A rationality determination generation module is used to output a final response result based on the first response result and the second response result through the second large language model if the first response result is determined to be rational based on the second response result.

[0036] In the present invention, target vertical domain data is obtained, a target domain knowledge graph is constructed according to the target vertical domain data, and a first language model is determined according to the target domain knowledge graph; when a user question text is received, a second language model is determined, and if the user question text is determined to be related to the target domain knowledge graph by the second language model, the user question text is respectively input into the first language model and the second language model to obtain a first query result and a second query result; the first query result is input into the second language model, and a first answer result and a second answer result are obtained according to the first query result and the second query result by the second language model; if the first answer result is determined to be reasonable according to the second answer result, the final answer result is output according to the first answer result and the second answer result by the second language model. The present invention analyzes and processes the user question text by constructing a domain knowledge graph and determining a large language model, and combining the domain knowledge graph with the large language model, which can make full use of the powerful natural language processing capabilities of the large language model and the high knowledge quality, high density, and high retrieval efficiency of the knowledge graph, and effectively improve the efficiency, versatility, accuracy and stability of knowledge question and answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of a preferred embodiment of the intelligent question answering method based on a large language model and a domain knowledge graph of the present invention;

[0038] Figure 2 It is a structural diagram of a preferred embodiment of the intelligent question-answering system based on a large language model and a domain knowledge graph of the present invention;

[0039] Figure 3 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] With the development of artificial intelligence, large language models have become a hot topic in general artificial intelligence. Large language models (LLMs) are a type of language model that has been pre-trained with massive data and has a very large number of parameters. They can understand and generate natural language to perform various tasks. Large language models are widely used for their powerful natural language understanding and generation capabilities.

[0042] Using a large language model can achieve end-to-end dialogue question and answer, but the question and answer based directly on the large language model cannot guarantee the accuracy of the answer. In addition, the large language model is mainly good at general knowledge question and answer. When it comes to vertical fields, the dialogue question and answer results based directly on the large language model are further reduced. The retrieval augmented generation (RAG) technology uses unstructured data such as text as an external knowledge base to overcome the accuracy problem of the large language model. This technology mainly vectorizes the unstructured data of the knowledge base, obtains the relevant fragments that may be included through vector retrieval, sorts them, obtains the top-ranked answers and inputs them into the large language model to obtain the question and answer results. This technology has the following problems: First, it can only answer questions whose answers are contained in a certain fragment, and cannot answer questions that require a holistic understanding to answer; second, the unstructured data has not been sorted and contains a lot of invalid information to form interference. When searching, it will be impossible to accurately match the fragments related to the answer due to invalid information; third, the retrieval efficiency is low when the amount of unstructured data is large.

[0043] Knowledge Graph is a structured semantic knowledge base, characterized by highly structured knowledge, high knowledge density and high quality. Existing knowledge graph-based knowledge question-answering technologies mainly use methods based on logical rules and embedded retrieval, which are generally only applicable to situations where the graph structure is not complex and the questions are simple. When it is based on complex vertical knowledge graphs or facing complex questions, the effect is often very poor and the accuracy is low.

[0044] In order to solve the problems of low accuracy, limited use and low efficiency of various knowledge question and answer methods currently used, the present invention proposes an intelligent question and answer method that combines a large language model and a domain knowledge graph. It fully utilizes the powerful natural language processing capabilities of the large language model and the high knowledge quality, high density and high retrieval efficiency of the knowledge graph, and can effectively improve the efficiency, versatility, accuracy and stability of knowledge question and answer.

[0045] The intelligent question answering method based on a large language model and a domain knowledge graph described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the intelligent question answering method based on a large language model and a domain knowledge graph includes the following steps:

[0046] Step S10: Acquire target vertical field data, construct a target field knowledge graph based on the target vertical field data, and determine a first language model based on the target field knowledge graph.

[0047] Specifically, target vertical domain data is acquired, a preset ontology model is constructed according to the target vertical domain data, and knowledge extraction processing is performed on the preset ontology model to obtain a target domain knowledge graph; a target language corresponding to the target domain knowledge graph is acquired, a large language model is determined according to the target language, and supervised training fine-tuning processing is performed on the large language model to obtain a first large language model.

[0048] First, you need to collect data and build a domain knowledge graph (the present invention does not limit the specific field of the knowledge graph, and any field can be used to conduct question and answer by building a knowledge graph). The process is: collect vertical field data (that is, the target vertical field data in the present invention), build an ontology model of domain knowledge, extract knowledge based on the ontology model, and form a domain knowledge graph through processes such as knowledge fusion and quality assessment.

[0049] Then, the entity relationship extraction large language model is trained. The process is as follows: based on the constructed domain knowledge graph (i.e., the target domain knowledge graph in the present invention), a large language model M for entity relationship extraction is fine-tuned. a (That is, the first large language model in the present invention), this model can realize the identification of entities and relations in the domain knowledge graph from user questions. Among them, the specific implementation process is: 1. Construct a training set (for example, the construction process of the training set can be: "Input: Who is A's brother? Output: [{"entity":{"name":"A", "type":"historical figure"},"relation":"brother"}]"). Each data set consists of an input and an output part. The sentence of the input part is the question asked by the user, and the output is the entity and relationship extracted from the user's question, which is output in a standardized json format. 2. Select a large language model for fine-tuning. You can choose an open source large language model with a parameter scale of billions (such as chatglm-3, llama2, etc.). First, based on the main language of the knowledge in the knowledge graph, if it is mainly Chinese, give priority to using large language models trained mainly with domestic Chinese corpus (such as the chatglm series of models, the Tongyi Qianwen series of models); if it is mainly English, consider using large language models trained mainly with English corpus (such as the llama series, etc.), and then select a model with appropriate parameter amounts based on the training resource conditions. 3. Select a fine-tuning method (fine-tuning methods include LoRA, Prefix Tuning, P-Tuning, etc.), and select an appropriate fine-tuning method based on the large language model used. In general, it is recommended to use the LoRA method for fine-tuning. 4. Using the constructed data set, select the large language model for fine-tuning, and use the selected fine-tuning method to perform supervised training to obtain the fine-tuned large language model M a .

[0050] It can be understood that in the specific implementation process, the present invention takes the green and low-carbon development knowledge graph as an example to illustrate the above technical steps: 1. Collect data and build a domain knowledge graph: collect data related to green and low-carbon development (in this specific implementation process, the present invention is explained with data related to green and low-carbon development, that is, data related to green and low-carbon development is the target vertical domain data in the present invention), build a green and low-carbon development ontology model, extract knowledge based on the ontology model, and form a green and low-carbon knowledge graph through processes such as knowledge fusion and quality assessment. 2. Train a large language model for entity relationship extraction: Based on the green development knowledge graph that has been constructed, build a data set, select the chatglm3 model, and use the LoRA method to train to obtain a fine-tuned large language model M for extracting entities and relationships from questions raised by users. a .

[0051] Step S20: When a user question text is received, a second language model is determined. If the second language model is used to determine that the user question text is related to the target domain knowledge graph, the user question text is input into the first language model and the second language model respectively to obtain a first query result and a second query result.

[0052] Specifically, when a user question text is received, the knowledge background and ontology model of the target domain knowledge graph are obtained; a second largest language model is determined, the user question text, the knowledge background and the ontology model are input into the second largest language model, and the second largest language model is used to determine whether the user question text is related to the target domain knowledge graph; if so, the user question text is input into the first largest language model to obtain the entities in the user question text and the corresponding relationships.

[0053] Further, the user question (i.e., the user question text in the present invention) is obtained and input into the large language model to identify whether the user intention is related to the domain. The implementation process is as follows: the question asked by the user is obtained, the knowledge background of the domain knowledge graph and the ontology model of the domain knowledge graph are added to the prompt word, and the general large language model M is input into the large language model M. b (i.e., the second largest language model in the present invention, the universal large language model M b You can select a large language model with sufficient capabilities currently available (no fine-tuning is required) and let the large language model analyze whether the question asked by the user is related to the knowledge graph. If so, proceed to the next step; otherwise, directly generate the answer "the question asked is not related to the green development graph" and end the entire question-answering process. In this step, the prompt words input into the large language model are mainly composed of the following:

[0054] 1. Role: Reviewer who determines whether the question is relevant to the field.

[0055] 2. Task: Determine whether the question is related to the field based on the large language model’s own understanding of the field and the knowledge background of the field.

[0056] 3. Information: knowledge background of the domain, knowledge graph ontology model, and user questions.

[0057] Identify entities and relations in user questions. The process is as follows: Use the previously fine-tuned entity relation extraction large language model M a , identify entities and relations in user questions, that is, input user questions into the fine-tuned large language model M a , get its output, and get the entities and relations extracted by the model from the question.

[0058] Determine a preset template, and convert the entity and the relationship into a first query statement according to the preset template; obtain a graph database corresponding to the target domain knowledge graph, and perform a search and find process in the graph database according to the first query statement to obtain a first query result, and output a second query statement according to the user question text and the ontology model through the second large language model; perform a search and find process in the graph database according to the second query statement to obtain a second query result.

[0059] Furthermore, based on the identified entities and relations, query statements are generated using templates. The process is as follows: Based on the large language model M a The entities and relations extracted from the user questions are converted into graph database query statements Q using preset templates. a The preset template is MATCH(n:'identified entity type'{name:'identified entity name'})-[r:'identified relationship type'*1..N]-(m)RETURN n, r, m. N is a preset parameter, which indicates the depth of the query result. It needs to be preset according to the complexity of the knowledge graph to be constructed. If the knowledge structure is not complex, 1 can be used.

[0060] Furthermore, the query statement generated by the identified entity relationship is used to search for relevant results in the knowledge graph. The process is as follows: a Retrieve the graph database stored in the domain knowledge graph and obtain the query result R a (ie, the first query result in the present invention).

[0061] Furthermore, the large language model is used to directly generate graph database query statements: the question and the ontology model information of the knowledge graph are added to the prompt word and input into the large language model M b In the example above, let the large language model M b Based on the problem, think from a holistic perspective and directly generate query statements Qb (Among them, Q a It is obtained according to the set template, Q b It is a large model M b The output of the two queries is generated in different ways and from different perspectives. The resulting query statement Q a and Q b May be different, the large language model M b A more reasonable answer result can be obtained by comprehensively judging the two different query results). In this step, the prompt words input into the large language model are mainly composed of the following:

[0062] 1. Role: Expert in writing graph database query statements based on the problem.

[0063] 2. Task: Based on the questions raised by the user and the ontology model structure of the knowledge graph, think about what information needs to be searched for the answer to the question from a holistic perspective and generate query statements.

[0064] 3. Information: knowledge graph ontology model, user questions.

[0065] Furthermore, the query statement generated by the graph database based on the large language model is used to query related results. The process is as follows: b Retrieve the graph database stored in the knowledge graph and obtain the query result R b (ie, the second query result in the present invention).

[0066] It is understandable that in the specific implementation process, the user's question can be obtained and input into the large language model to identify whether the user's intention is related to the field. Get the question asked by the user, such as "What are the next-level subcategories of farmland ecosystems?", add the question, the knowledge background of the green and low-carbon development field, and the ontology model of the knowledge graph to the prompt word, and input it into the large language model M. b Medium, large language model M b Choose an open source model. Large language model M b Based on the information in the prompt word, determine whether the question is related to the field of green and low-carbon development. If so, proceed to the next step.

[0067] Identify entities and relations in user questions: Input user questions into the fine-tuned large language model M a , large language model M a Output: "[{"entity":{"name":"agricultural ecosystem", "type":"first-level classification of ecosystem"}, "relation":"has subclassification"}]", which is the entities and relations extracted from the user's question.

[0068] Furthermore, based on the identified entities and relations, query statements are generated using templates: Based on the large language model M a The entities and relations extracted from the user questions are converted into graph database query statements Q using preset templates. a , query statement Q generated according to the template a For: "MATCH(n:'First-level classification of ecosystem'{name:'Agricultural ecosystem'})-[r:'has subclassification']-(m)RETURN n, r, m".

[0069] Using the query statement generated by the identified entity relationship, search for relevant results in the knowledge graph, and use the query statement Q a Retrieve the graph database stored in the knowledge graph and obtain the query result R ais: [<Record n = <Node element_id='5249' labels=frozenset({'Ecosystem First-level Classification'}) properties={'name': 'Farmland Ecosystem'}> r = <Relationship element_id='5254' nodes=(<Node element_id='5249' labels=frozenset({'Ecosystem First-level Classification'}) properties={'name': 'Farmland Ecosystem'}>, <Node element_id='5267' labels=frozenset({'Ecosystem Second-level Classification'}) properties={'name': 'Garden Land'}) type='Has Sub-classification' properties={}> m = <Node element_id='5267' labels=frozenset({'Ecosystem Second-level Classification'}) properties={'name': 'Garden Land'}>>, <Record n = <Node element_id='5249' labels=frozenset({'Ecosystem First-level Classification'}) properties={'name': 'Farmland Ecosystem'}> r = <Relationship element_id='5255' nodes=(<Node element_id='5249' labels=frozenset({'Ecosystem First-level Classification'}) properties={'name': 'Farmland Ecosystem'}, <Node element_id='5268' labels=frozenset({'Ecosystem Second-level Classification'}) properties={'name': 'Cultivated Land'}) type='Has Sub-classification' properties={}> m = <Node element_id='5268' labels=frozenset({'Ecosystem Second-level Classification'}) properties={'name': 'Cultivated Land'}>>].

[0070] Furthermore, directly generate a graph database query statement using a large language model: Add the question and the ontology model information of the knowledge graph to the prompt, and input it into the large language model M b so that the large language model M b directly generates a query statement Q from an overall perspective according to the question b . Based on the understanding of the question, the large language model generates the query statement Q bIt is: "MATCH(a:Ecosystem First - level Classification{name:'Farmland Ecosystem'})-[:Has Sub - classification]->(b:Ecosystem Second - level Classification) RETURN b".

[0071] Further, use the graph database to query relevant results according to the query statement generated by the large - language model: Use the query statement Q b to retrieve the graph database stored in the knowledge graph and obtain the query result R b It is: "[<Record b=<Nodeelement_id='5267'labels=frozenset({'Ecosystem Second - level Classification'})properties={'name':'Garden Plot'}>>, <Record b=<Node element_id='5268'labels=frozenset({'Ecosystem Second - level Classification'})properties={'name':'Cultivated Land'}>>]".

[0072] Step S30: Input the first query result into the second large - language model, and obtain the first response result and the second response result through the second large - language model according to the first query result and the second query result.

[0073] Specifically, input the first query result into the second large - language model, and obtain the first response result through the second large - language model according to the first query result and the second query result; obtain the second response result through the second large - language model according to the first response result, the user's question text, the first query result, and the second query result.

[0074] Further, combine the results obtained from the two - path queries and use the large - language model to generate a preliminary answer: Input the query results R a and R b into the large - language model M b after adding prompt words, and let the large - language model M b generate a preliminary answer to the user's question (i.e., the first response result in the present invention) based on these reference information. In this step, the prompt words input into the large - language model are mainly composed of the following:[[]]

[0075] 1. Role: An assistant that generates an answer to the user's question.

[0076] 2. Task: Generate an answer to the user's information based on the results obtained from the query search as reference information according to the user's question.

[0077] 3. Information: The user's question, the results of the two - path searches.

[0078] Furthermore, we use the large language model to reflect on whether the initially generated answer is reasonable: we combine the initially generated answer, the user question, and the result of the two-way query R a and R b Add to the prompt word and input into the large language model M b In the example above, let the large language model M b Reflect on whether the results are reasonable (Large Language Model M b Based on the thought chain, determine whether the answer is the correct answer to the user's question). If it is unreasonable, return to the step of directly generating a graph database query statement using the large language model. In this step, the prompt words input into the large language model are mainly composed of the following:

[0079] 1. Role: Reviewer who determines whether the answer to the user's question is correct.

[0080] 2. Task: Based on the user's question, search results and generated preliminary answers, confirm whether the answer is correct by thinking step by step.

[0081] 3. Information: user questions, preliminary generated answers, and results of two-way queries.

[0082] It is understandable that the results obtained by combining the two queries are used to generate a preliminary answer using the large language model: a and R b Add prompt words and input them into the large language model M b In the example above, let the large language model M b A preliminary answer to the user's question is generated based on these reference information, and the generated answer is: "gardens and cultivated land" (ie, the second response result in the present invention).

[0083] Step S40: If the first response result is determined to be reasonable according to the second response result, a final response result is outputted according to the first response result and the second response result through the second large language model.

[0084] Specifically, whether the first answer result is reasonable is judged according to the second answer result. If so, a preset answer requirement is obtained, wherein the preset answer requirement includes the answer style and the degree of detail of the answer information; the preset answer requirement and the rationality analysis process of the first answer result are input into the second large language model, and the final answer result is output by the second large language model according to the preset answer requirement, the user question text, the first answer result, the second answer result and the rationality analysis process of the first answer result.

[0085] Further, the large language model is used to generate the final answer: When the large language model confirms that the answer is correct, the large language model M is used again b, based on the required answer style and the level of detail required for the information, the final answer is generated. In this step, the prompt words input into the large language model are mainly composed of the following:

[0086] 1. Role: Language organizer of the answer.

[0087] 2. Task: Organize the language according to specific requirements to convert the generated answers into reasonable replies. The replies need to include the answers to the questions and the thinking process behind the answers, which may include but are not limited to: style, level of detail of the reply, the target audience, and matters that need to be prohibited.

[0088] 3. Information: the user’s question, the initial answer generated, and the thinking process of the large language model to determine whether the answer is reasonable.

[0089] It is understandable that the large language model is used to reflect on whether the initially generated answer is reasonable: the initially generated answer "garden and cultivated land", the user question and the result R of the two-way query are combined. a and R b Add to the prompt word and input into the large language model M b In the example above, let the large language model M b Reflect on whether the results are reasonable. Large Language Model M b After reflection, the result is: "The user's question is to ask what are the next-level classifications of farmland ecosystems. According to the information provided, there are two entities with the farmland ecosystem relationship of "has sub-classification": the entity with the attribute name 'cultivated land' and the entity with the attribute name 'garden'. The preliminary answer is 'garden and cultivated land', which is consistent with the information provided, so the answer is correct." According to the large language model M b The output of , judges that the initially generated answer is reasonable.

[0090] Furthermore, the large language model is used to generate the final answer: based on the required answer style and the level of detail of the information, the user question, the initially generated answer, the final answer, and the thinking process (also input in text form) that the large language model considers the answer to be reasonable are added to the prompt word and input into the large language model M. bAmong them, the final answer generated is: "Dear user, the question you asked is 'What are the next-level sub-classifications of the farmland ecosystem?' According to the retrieval of relevant knowledge, the next-level sub-classifications of the farmland ecosystem include cultivated land and garden land. The relevant reference information is: <Node element_id='5267' labels=frozenset({'Ecosystem secondary classification'}) properties={'name': 'Garden land'}>>, <Record b=<Node element_id='5268' labels=frozenset({'Ecosystem secondary classification'}) properties={'name': 'Cultivated land'}>" (i.e., the final response result in the present invention).

[0091] Advantages of the present invention: The present invention provides an intelligent question-answering method combining a large language model and a domain knowledge graph. By calling the large language model multiple times, the natural language understanding and generation capabilities of the large language model are utilized to strengthen the understanding of the question. By means of two-way retrieval and recall, considering from both local and overall perspectives, knowledge highly relevant to the answer is efficiently retrieved from the domain knowledge graph, ensuring that the most accurate reference knowledge can be retrieved and answered efficiently for various complex questions, effectively improving the efficiency, generality, accuracy, and stability of knowledge question-answering.

[0092] Furthermore, as Figure 2 shown, based on the above intelligent question-answering method combining a large language model and a domain knowledge graph, the present invention also correspondingly provides an intelligent question-answering system based on a large language model and a domain knowledge graph. Among them, the intelligent question-answering system based on a large language model and a domain knowledge graph includes:

[0093] A knowledge graph construction module 51, configured to obtain target vertical domain data, construct a target domain knowledge graph according to the target vertical domain data, and determine a first large language model according to the target domain knowledge graph;

[0094] A relevance determination module 52, configured to determine a second large language model when receiving a user's question text. If it is determined by the second large language model that the user's question text is relevant to the target domain knowledge graph, the user's question text is respectively input into the first large language model and the second large language model to obtain a first query result and a second query result;

[0095] A response result generation module 53, configured to input the first query result into the second large language model, and obtain a first response result and a second response result by the second large language model according to the first query result and the second query result;

[0096] The rationality determination generating module 54 is used to output a final response result according to the first response result and the second response result through the second large language model if the first response result is determined to be rational according to the second response result.

[0097] Furthermore, if Figure 3 As shown, based on the above-mentioned intelligent question answering method and system based on a large language model and a domain knowledge graph, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 3 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0098] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, an intelligent question-answering program 40 based on a large language model and a domain knowledge graph is stored on the memory 20, and the intelligent question-answering program 40 based on a large language model and a domain knowledge graph can be executed by the processor 10, thereby realizing the intelligent question-answering method based on a large language model and a domain knowledge graph in the present application.

[0099] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the intelligent question-answering method based on a large language model and a domain knowledge graph.

[0100] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0101] In one embodiment, when the processor 10 executes the intelligent question answering program 40 based on the large language model and the domain knowledge graph in the memory 20, the steps of the intelligent question answering method based on the large language model and the domain knowledge graph are implemented.

[0102] In summary, the present invention provides an intelligent question-answering method, system and terminal based on a large language model and a domain knowledge graph. The method includes: acquiring target vertical domain data, constructing a target domain knowledge graph according to the target vertical domain data, and determining a first large language model according to the target domain knowledge graph; when receiving a user question text, determining a second large language model, if the user question text is determined to be related to the target domain knowledge graph through the second large language model, then inputting the user question text into the first large language model and the second large language model respectively to obtain a first query result and a second query result; inputting the first query result into the second large language model, and obtaining a first answer result and a second answer result according to the first query result and the second query result through the second large language model; if the first answer result is determined to be reasonable according to the second answer result, then outputting a final answer result according to the first answer result and the second answer result through the second large language model. The present invention constructs a domain knowledge graph and determines a large language model, and combines the domain knowledge graph and the large language model to analyze and process the user's question text. It can fully utilize the powerful natural language processing capabilities of the large language model and the high knowledge quality, high density, and high retrieval efficiency of the knowledge graph, effectively improving the efficiency, versatility, accuracy, and stability of knowledge question answering.

[0103] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0104] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0105] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An intelligent question answering method based on a large language model and a domain knowledge graph, characterized in that: The intelligent question answering method based on a large language model and a domain knowledge graph includes: Acquire target vertical field data, construct a target field knowledge graph based on the target vertical field data, and determine a first language model based on the target field knowledge graph; When receiving a user question text, determining a second largest language model, if the second largest language model determines that the user question text is related to the target domain knowledge graph, then inputting the user question text into the first largest language model and the second largest language model respectively, to obtain a first query result and a second query result; Inputting the first query result into the second largest language model, and obtaining a first answer result and a second answer result according to the first query result and the second query result through the second largest language model; If the first response result is determined to be reasonable according to the second response result, the final response result is outputted according to the first response result and the second response result through the second large language model.

2. The intelligent question answering method based on a large language model and a domain knowledge graph according to claim 1, characterized in that: The step of acquiring target vertical field data, constructing a target field knowledge graph according to the target vertical field data, and determining a first language model according to the target field knowledge graph specifically includes: Acquire target vertical field data, construct a preset ontology model based on the target vertical field data, and perform knowledge extraction processing on the preset ontology model to obtain a target field knowledge graph; A target language corresponding to the target domain knowledge graph is obtained, a large language model is determined according to the target language, and the large language model is fine-tuned by supervised training to obtain a first large language model.

3. The intelligent question answering method based on a large language model and a domain knowledge graph according to claim 1, characterized in that: When receiving a user question text, determining a second language model, if the second language model determines that the user question text is related to the target domain knowledge graph, inputting the user question text into the first language model and the second language model respectively, and obtaining a first query result and a second query result, specifically including: When receiving a user's question text, obtaining the knowledge background and ontology model of the target domain knowledge graph; Determine a second language model, input the user question text, the knowledge background and the ontology model into the second language model, and determine whether the user question text is related to the target domain knowledge graph through the second language model; If so, the user question text is input into the first language model to obtain a first query result, and the second query result is obtained according to the user question text through the second language model.

4. The intelligent question answering method based on a large language model and a domain knowledge graph according to claim 3, characterized in that: Inputting the user question text into the first language model to obtain a first query result specifically includes: Inputting the user question text into the first language model to obtain entities in the user question text and corresponding relationships; Determine a preset template, and convert the entity and the relationship into a first query statement according to the preset template; Obtain a graph database corresponding to the target domain knowledge graph, and perform a search and search process in the graph database according to the first query statement to obtain a first query result.

5. The intelligent question answering method based on a large language model and a domain knowledge graph according to claim 4, characterized in that: The obtaining of the second query result according to the user question text by using the second largest language model specifically includes: Outputting a second query statement according to the user question text and the ontology model through the second largest language model; A search and lookup process is performed in the graph database according to the second query statement to obtain a second query result.

6. The intelligent question answering method based on a large language model and a domain knowledge graph according to claim 1, characterized in that: The step of inputting the first query result into the second language model, and obtaining a first answer result and a second answer result according to the first query result and the second query result by using the second language model specifically includes: Inputting the first query result into the second largest language model, and obtaining a first answer result according to the first query result and the second query result through the second largest language model; A second answer result is obtained by using the second large language model according to the first answer result, the user question text, the first query result and the second query result.

7. The intelligent question answering method based on a large language model and a domain knowledge graph according to claim 1, characterized in that: If the first response result is determined to be reasonable according to the second response result, outputting a final response result according to the first response result and the second response result through the second large language model specifically includes: Determine whether the first answer result is reasonable according to the second answer result, and if so, obtain preset answer requirements, wherein the preset answer requirements include answer style and the degree of detail of answer information; The preset answer requirement and the rationality analysis process of the first answer result are input into the second large language model, and the final answer result is output by the second large language model according to the preset answer requirement, the user question text, the first answer result, the second answer result and the rationality analysis process of the first answer result.

8. An intelligent question-answering system based on a large language model and a domain knowledge graph, characterized in that: The intelligent question answering system based on a large language model and a domain knowledge graph includes: A knowledge graph construction module, used to obtain target vertical field data, construct a target field knowledge graph according to the target vertical field data, and determine a first language model according to the target field knowledge graph; A relevance determination module, configured to determine a second language model when receiving a user question text, and if the user question text is determined to be relevant to the target domain knowledge graph by the second language model, input the user question text into the first language model and the second language model respectively to obtain a first query result and a second query result; a response result generating module, configured to input the first query result into the second large language model, and obtain a first response result and a second response result according to the first query result and the second query result through the second large language model; A rationality determination generation module is used to output a final response result based on the first response result and the second response result through the second large language model if the first response result is determined to be rational based on the second response result.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and an intelligent question-answering program based on a large language model and a domain knowledge graph stored in the memory and executable on the processor. When the intelligent question-answering program based on a large language model and a domain knowledge graph is executed by the processor, the steps of the intelligent question-answering method based on a large language model and a domain knowledge graph as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an intelligent question-answering program based on a large language model and a domain knowledge graph. When the intelligent question-answering program based on a large language model and a domain knowledge graph is executed by a processor, the steps of the intelligent question-answering method based on a large language model and a domain knowledge graph as described in any one of claims 1 to 7 are implemented.

Images