Question and answer method and device based on large language model, electronic equipment and computer program product

Abstract

The invention discloses a question and answer method and device based on a large language model, electronic equipment and a computer program product. The method comprises the steps of obtaining a question text of a user; according to the question text and the first cue word, utilizing a large language model to generate enhanced information of the question text; performing retrieval matching on the enhanced information of the question text and a knowledge graph and a vector knowledge base of the target domain to obtain a matching result of the knowledge graph and a matching result of the vector knowledge base; and according to the enhancement information of the question text, the matching result of the knowledge graph and the matching result of the vector knowledge base, utilizing a large language model to generate answer content corresponding to the question text. According to the method, the professional property of inputting the question is optimized by utilizing the large language model, the structured knowledge information retrieved by the knowledge graph and the semantic knowledge information retrieved by the vector knowledge base are combined and input into the large language model at one time, the question and answer effect of the large language model is enhanced, and the professional property and accuracy of answering of the large language model are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent question answering technology, and in particular to a question answering method, device, electronic device, and computer program product based on a large language model. Background Art

[0002] There is a lot of business knowledge in the financial field. When facing customer questions, relying solely on manual customer service to answer takes up a lot of human resources, affects the resolution of more complex customer problems, and is not timely. With the rapid development of computer hardware and deep learning technology, intelligent question answering has received widespread attention. Its implementation not only saves human resources, but also improves customer experience and can quickly resolve customer needs.

[0003] Currently, most intelligent question-answering services use keyword matching technology to calculate the similarity between the text entered by the user and the text in the knowledge base, and sort the appropriate question-answer pairs based on the literal information. However, many semantically similar questions are difficult to solve. For example, some questions are semantically similar, but the language organization and literal expression are very different. Intelligent question-answering services cannot solve such scenarios yet.

[0004] There are studies on using knowledge graphs to integrate intelligent question answering. Although knowledge graphs can understand the semantic information of questions, given the ambiguity and flexibility of natural language, how to improve the accuracy of question answering for complex questions is a major difficulty.

[0005] In recent years, large language models (LLM) have made rapid breakthroughs in various natural language processing tasks. Large language models such as ChatGPT, GPT4, and ChatGLM can understand and answer complex questions according to input instructions through pre-training, fine-tuning and other technologies. However, these models have some inherent defects, such as poor ability to process Chinese and prone to hallucination answers that lack authenticity. Therefore, there are still many problems in applying them to question-and-answering in professional fields. Summary of the invention

[0006] The embodiments of the present application provide a question-answering method, device, electronic device, and computer program product based on a large language model to improve the question-answering effect of the large language model.

[0007] The present application embodiment adopts the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides a question-answering method based on a large language model, and the question-answering method based on a large language model includes:

[0009] Get the user's question text;

[0010] Generate enhanced information of the question text using a large language model according to the question text and the first prompt word;

[0011] Retrieve and match the enhanced information of the question text with the knowledge graph and the vector knowledge base of the target domain respectively, and obtain matching results of the knowledge graph and the vector knowledge base;

[0012] The large language model is used to generate answer content corresponding to the question text according to the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base.

[0013] Optionally, generating enhanced information of the question text using a large language model according to the question text and the first prompt word includes:

[0014] Using a preset information screening strategy to screen the question text to obtain a screened question text;

[0015] According to the screened question text and the first prompt word, enhanced information of the question text is generated using a large language model.

[0016] Optionally, the question text is screened by using a preset information screening strategy to obtain the screened question text, including:

[0017] Obtain a set of questions in the target domain;

[0018] According to the question text and the question set of the target field, the question text is classified by using a text classification model to obtain a category of the question text;

[0019] Calculating the text similarity between the question text and the question set in the target field using a text similarity algorithm according to the question text and the question set in the target field;

[0020] The question text is screened according to the category of the question text and the text similarity between the question text and the question set in the target field to obtain a screened question text.

[0021] Optionally, the enhanced information of the question text includes a query sentence and a triple corresponding to the question text, and the step of generating the enhanced information of the question text by using a large language model according to the filtered question text and the first prompt word includes:

[0022] Generate a query statement corresponding to the question text using the large language model according to the first prompt word corresponding to the screened question text and the query statement;

[0023] According to the query sentence corresponding to the question text, a triple corresponding to the question text is generated using a fine-tuned large language model, wherein the fine-tuned large language model is obtained by fine-tuning the large language model.

[0024] Optionally, the enhanced information of the question text includes a query statement and a triple corresponding to the question text, and the enhanced information of the question text is searched and matched with the knowledge graph and the vector knowledge base of the target field respectively to obtain the matching results of the knowledge graph and the matching results of the vector knowledge base, including:

[0025] Search and match the triples corresponding to the question text with the knowledge graph of the target domain to obtain the knowledge entity information corresponding to the question text;

[0026] The query sentence corresponding to the question text is searched and matched with the vector knowledge base to obtain the knowledge base text corresponding to the question text.

[0027] Optionally, searching and matching the query statement corresponding to the question text with the vector knowledge base to obtain the knowledge base text corresponding to the question text includes:

[0028] Using a keyword matching algorithm, the query statement corresponding to the question text is searched and matched with the vector knowledge base to obtain a first knowledge base text;

[0029] The query statement corresponding to the question text is searched and matched with the vector knowledge base using a semantic information matching algorithm to obtain a second knowledge base text.

[0030] Optionally, generating answer content corresponding to the question text using the large language model according to the enhanced information of the question text, the matching result of the knowledge graph, and the matching result of the vector knowledge base includes:

[0031] The knowledge entity information corresponding to the question text is used to form a second prompt word;

[0032] The first knowledge base text and the second knowledge base text form a third prompt word;

[0033] The large language model is used to generate answer content corresponding to the question text according to the query sentence corresponding to the question text and the second prompt word and the third prompt word.

[0034] In a second aspect, an embodiment of the present application further provides a question-answering device based on a large language model, wherein the question-answering device based on a large language model comprises:

[0035] An acquisition unit, used to acquire the user's question text;

[0036] A first generating unit, configured to generate enhanced information of the question text using a large language model according to the question text and the first prompt word;

[0037] A search and matching unit, used to search and match the enhanced information of the question text with the knowledge graph and the vector knowledge base of the target field respectively, to obtain matching results of the knowledge graph and the vector knowledge base;

[0038] The second generating unit is used to generate answer content corresponding to the question text by using the large language model according to the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base.

[0039] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0040] A processor; and a memory arranged to store computer executable instructions, which, when executed, cause the processor to perform any of the aforementioned question-answering methods based on a large language model.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements any of the aforementioned question-answering methods based on a large language model.

[0042] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: the question-answering method based on the large language model in the embodiments of the present application first obtains the user's question text; then, based on the question text and the first prompt word, the large language model is used to generate enhanced information of the question text; then, the enhanced information of the question text is searched and matched with the knowledge graph and the vector knowledge base of the target field respectively, to obtain the matching results of the knowledge graph and the matching results of the vector knowledge base; finally, based on the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base, the large language model is used to generate the answer content corresponding to the question text. The question-answering method based on the large language model in the embodiments of the present application uses the large language model to optimize the professionalism of the input questions, synchronously combines the retrieval results of the knowledge graph and the vector knowledge base, and inputs the retrieved structured knowledge information and semantic knowledge information into the large language model at one time, which enhances the question-answering effect of the large language model, alleviates the problem of inaccurate hallucination errors that are prone to occur in the question-answering of the large language model, and improves the professionalism and accuracy of the answers of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0044] Figure 1 A flowchart of a question-answering method based on a large language model in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a question-answering process based on a large language model in an embodiment of the present application;

[0046] Figure 3 This is a schematic diagram of an information screening process in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a question-answering framework based on a large language model in an embodiment of the present application;

[0048] Figure 5 This is a schematic diagram of a knowledge graph matching process in an embodiment of the present application;

[0049] Figure 6 This is a schematic diagram of a vector knowledge base matching process in an embodiment of the present application;

[0050] Figure 7 Schematic diagram of the structure of a question-answering device based on a large language model in an embodiment of the present application;

[0051] Figure 8 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0053] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0054] Intelligent question answering can automatically answer questions input by users. The general process is that users input questions, which are processed by the question answering device and the output results are given to users. Intelligent question answering is the intersection of information retrieval and natural language processing. It is widely used in the field of Internet finance. With the synchronous development of computer hardware and artificial intelligence, the research on intelligent question answering is becoming more and more popular.

[0055] At present, there are two main methods for implementing intelligent question answering:

[0056] The first is to build a question-answer knowledge base, generate the embedding vector of the question-answer pair through ES index and vector index, and perform vector retrieval and ES retrieval on the embedding vector of the user query and the index in the knowledge base to obtain similar questions with similar words and semantics, and form a candidate set. The similarity between similar questions and user queries in the candidate set is calculated based on the refined ranking scoring model, and the candidate set is refined according to the calculation results. The refined similar questions are merged to obtain one or more standard questions and the corresponding answers to the standard questions, and the answers are returned to the user according to the pre-configured requirements.

[0057] The second method is to build a knowledge base first, and use a single BERT text classification model to filter information for input questions. Then input the large language model to generate the first question and answer, and then input the large language model question and answer results into the knowledge graph, infer the structured knowledge information corresponding to the triples, and input the structured knowledge information into the large language model for question and answer to obtain the final answer result.

[0058] In the intelligent question-and-answer (Q&A) field of finance, there is a great deal of randomness in the questions input by customers. The existing solutions do not take into account the ambiguity of natural language semantics, and the Q&A results are not accurate enough. For example, the first method obtains the knowledge base text by matching the input question only by literal and semantic vectors, and obtains the answer result based on the model's precise ranking and calculation score. The answer is easily confused by irrelevant text, resulting in the language expression of the Q&A result being unnatural and prone to ambiguity. The second solution is to retrieve relevant text from the knowledge base for the input question, and use a large language model to obtain a secondary Q&A result. This method not only has low Q&A efficiency, but is also prone to hallucination errors in the Q&A results. It cannot avoid the influence of the large language model's own pre-trained corpus text, and does not make deep use of the structured reasoning ability of the knowledge graph, resulting in redundancy in the information Q&A results.

[0059] Based on this, the embodiment of the present application provides a question-answering method based on a large language model, such as Figure 1 As shown, a flow chart of a question-answering method based on a large language model in an embodiment of the present application is provided, and the question-answering method based on a large language model at least includes the following steps S110 to S140:

[0060] Step S110, obtaining the user's question text.

[0061] Combination Figure 2 , provides a schematic diagram of a question-answering process based on a large language model in an embodiment of the present application. When performing intelligent question-answering based on a large language model, it is necessary to first obtain the question input by the user, that is, the specific content that the system wants to answer. The question originally input by the user can be in text format or a voice, which is converted into a text format that is convenient for subsequent model processing through voice recognition.

[0062] Step S120: Generate enhanced information of the question text using a large language model according to the question text and the first prompt word.

[0063] After obtaining the user's question text content, use pre-trained large language models such as BERT, GPT, etc. to process the user's input question text. When inputting the user's question text into the large language model, it is necessary to introduce the designed first prompt words such as domain keywords, etc. These prompt words help the model better understand the question and generate enhanced information that is more relevant to the question, such as more specialized query statements, entity triple information related to the question, etc.

[0064] Taking the field of financial knowledge as an example, the first prompt word P1 can be designed as follows:

[0065] Now that you are an expert in {a certain field such as: financial knowledge}, please refer to the following data: {question text}, and generate a query statement that is more in line with the field of {a certain field such as: financial knowledge}.

[0066] Of course, the specific design of the first prompt word can be flexibly set by those skilled in the art according to the actual field requirements, and no specific limitation is made here.

[0067] Step S130, searching and matching the enhanced information of the question text with the knowledge graph and the vector knowledge base of the target field respectively, to obtain the matching results of the knowledge graph and the matching results of the vector knowledge base.

[0068] After obtaining the enhanced information of the question text, based on the enhanced information of the question text, search and match are performed in the knowledge graph of the target field, such as the financial field, and the vector knowledge base, respectively, so as to retrieve the knowledge text related to the question. The embodiment of the present application combines the knowledge graph search results and the vector knowledge base search results, integrates the two knowledge sources, improves the accuracy and comprehensiveness of the search results, and further enhances the question-answering ability of the large language model, avoiding the illusion error of the answer.

[0069] Step S140, using the large language model to generate answer content corresponding to the question text according to the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base.

[0070] The question-related knowledge text retrieved from the two knowledge sources in the previous steps is combined with the enhanced information of the question text and input into the large language model. The large language model integrates this information to output knowledge answers that are consistent with the expertise of the target field.

[0071] The question-answering method based on a large language model in the embodiment of the present application utilizes a large language model to optimize the professionalism of input questions, synchronously combines the retrieval results of the knowledge graph and the vector knowledge base, and inputs the retrieved structured knowledge information and semantic knowledge information into the large language model at one time, thereby enhancing the question-answering effect of the large language model, alleviating the problem of inaccurate hallucination errors that are prone to occur in question-answering with a large language model, and improving the professionalism and accuracy of the answers given by the large language model.

[0072] In some embodiments of the present application, generating enhanced information of the question text using a large language model based on the question text and the first prompt word includes: filtering the question text using a preset information filtering strategy to obtain a filtered question text; and generating enhanced information of the question text using a large language model based on the filtered question text and the first prompt word.

[0073] Taking into account that the questions input by users in actual scenarios may contain some information content that is irrelevant to the target field, the embodiments of the present application can also adopt certain information screening strategies to process the questions input by customers, filter out questions that are irrelevant to the target field knowledge, and finally input the question text related to the field expertise into the large language model for processing.

[0074] The embodiment of the present application pre-filters information in the question text that is not related to domain knowledge, thereby avoiding ambiguity caused by the answer of the large language model being confused by irrelevant text content, making the language expression of the question and answer results output by the large language model more natural.

[0075] In some embodiments of the present application, the method of filtering the question text using a preset information filtering strategy to obtain the filtered question text includes: obtaining a question set in a target field; classifying the question text using a text classification model based on the question text and the question set in the target field to obtain a category of the question text; calculating the text similarity between the question text and the question set in the target field using a text similarity algorithm based on the question text and the question set in the target field; filtering the question text according to the category of the question text and the text similarity between the question text and the question set in the target field to obtain the filtered question text.

[0076] like Figure 3 As shown, a schematic diagram of an information screening process in an embodiment of the present application is provided. On the one hand, the question text and the existing question data set in the financial field can be respectively sent to the text classification model and the text similarity algorithm, and the pre-trained text classification model and text similarity algorithm are combined to enhance the text information screening ability.

[0077] The above-mentioned text classification model can adopt the BERT model, for example. The BERT model is a natural language pre-training model that can be used as a text filter to filter the input text at the semantic level to limit the scope of subsequent large language model professional questions and answers. During the BERT training process, the text set of questions unrelated to finance and the existing financial question data are combined and input into the BERT model, and then the results of BERT are input into the fully connected layer of the model to obtain the classification probability of the text. Finally, according to the labels of the respective data sets, the parameters of the fully connected layer are updated. When used, the label classification probability of the question input text is obtained according to the trained BERT text classification model.

[0078] The above text similarity algorithm can adopt the BM25 algorithm, for example. BM25 is a text matching algorithm improved based on TF-IDF, which is used to calculate the relevance of text search. The overall calculation formula is as follows:

[0079]

[0080] Among them, Q is the question text, d is the document in the document library, and f i is the frequency of occurrence of the word in the document, q i is the frequency of occurrence of the word in the question text, dl is the length of document d, avgdl is the average length of all documents, k 1 and b are adjustment factors, which are generally set to 1.5 and 0.75 according to parameter optimization search. The biggest improvement of BM25 and TF-IDF is that it takes into account the influence of document length and similarity score. When using, input the question input text and existing financial question text data into the BM25 algorithm, and you can get the text similarity between the question input text and each existing financial question text. Finally, select the maximum text similarity for subsequent fusion calculations.

[0081] The label classification probability output by the above text classification model and the text similarity calculated by the text similarity algorithm are weighted averaged to obtain the overall text similarity score S. If S is greater than the preset score threshold, such as 0.5, it can be considered that the question input text is related to the financial field and the next question and answer process is carried out. Otherwise, it is considered irrelevant to the profession and the question and answer process is stopped.

[0082] Of course, the specific type of text classification model and text similarity algorithm to be adopted can be flexibly selected by those skilled in the art according to actual needs, and no specific limitation is made here.

[0083] The embodiment of the present application combines a text classification model with a text similarity algorithm to implement information screening, filters input questions irrelevant to professional issues, avoids the extreme nature of a single filtering algorithm, and improves the accuracy of information screening.

[0084] In some embodiments of the present application, the enhanced information of the question text includes query statements and triples corresponding to the question text, and the generating the enhanced information of the question text using a large language model based on the screened question text and the first prompt word includes: generating a query statement corresponding to the question text using the large language model based on the screened question text and the first prompt word corresponding to the query statement; generating a triple corresponding to the question text using a fine-tuned large language model based on the query statement corresponding to the question text, wherein the fine-tuned large language model is obtained by fine-tuning the large language model.

[0085] After obtaining the filtered question text in the above embodiment, the filtered question text is input into the large language model for enhancement processing. Figure 4 As shown, a schematic diagram of a question-answering framework based on a large language model in an embodiment of the present application is provided. The LangChain framework is used to manage the interaction with the large language model, link multiple components together, simplify the difficulty of using the large language model, and more conveniently inject knowledge into the large language model to generate the target result. The large language model selected in the embodiment of the present application can be, for example, the ChatGLM-3 model, which has good open source effects and reasoning performance, and uses the powerful judgment and reasoning capabilities of the large language model to generate specialized knowledge and extract triple information from the input text.

[0086] Specifically, we first use the large language model and the first prompt to generate a query statement that is more in line with the professionalism of the target field, enhancing the professionalism of the question to avoid the adverse effects of irrelevant text when the user enters the question, which is more reasonable than directly performing knowledge retrieval. Taking the financial knowledge field as an example, the first prompt P1 is:

[0087] Now that you are an expert in {a certain field such as: financial knowledge}, please refer to the following data: {question text}, and generate a query statement that is more in line with the field of {a certain field such as: financial knowledge}.

[0088] After generating specialized query statements, the large language model is further used to extract triple information corresponding to the question text, including "entity-relationship-entity" or "concept-attribute-attribute". Here, the LoRA fine-tuning method can be used to enhance the information extraction capability of the large language model, and more specialized query statements can be input into the large language model after information extraction enhancement, so as to extract triples of structured expressions.

[0089] In some embodiments of the present application, the enhanced information of the question text includes query statements and triples corresponding to the question text, and the enhanced information of the question text is retrieved and matched with the knowledge graph and the vector knowledge base of the target field respectively to obtain the matching results of the knowledge graph and the matching results of the vector knowledge base, including: retrieving and matching the triples corresponding to the question text with the knowledge graph of the target field to obtain the knowledge entity information corresponding to the question text; retrieving and matching the query statement corresponding to the question text with the vector knowledge base to obtain the knowledge base text corresponding to the question text.

[0090] like Figure 5 As shown, a schematic diagram of a knowledge graph matching process in an embodiment of the present application is provided. The triple information extracted in the above embodiment is matched with the knowledge graph in the financial field, and the relevant entity nodes are retrieved and represented in an easy-to-read manner as an input information for the subsequent large language model.

[0091] like Figure 6 As shown, a schematic diagram of a vector knowledge base matching process in an embodiment of the present application is provided. The query statement generated in the embodiment is input into a vector knowledge base in the financial field, which may be composed of different types of text files, and the knowledge base text related to the question is obtained through retrieval and matching.

[0092] In some embodiments of the present application, the searching and matching of the query statement corresponding to the question text with the vector knowledge base to obtain the knowledge base text corresponding to the question text includes: using a keyword matching algorithm to search and match the query statement corresponding to the question text with the vector knowledge base to obtain a first knowledge base text; using a semantic information matching algorithm to search and match the query statement corresponding to the question text with the vector knowledge base to obtain a second knowledge base text.

[0093] Continue to refer Figure 6 The embodiment of the present application can use two algorithms, namely a keyword matching algorithm and a semantic vector matching algorithm, to retrieve knowledge text related to the vector knowledge base, combining the advantages of the two matching algorithms to improve the accuracy of the retrieval results.

[0094] For example, the keyword matching algorithm can use the Word2Vec model to perform text matching at the word level, calculate the text similarity between two words, and obtain knowledge base related text that is more consistent with the literal information, that is, the first knowledge base text mentioned above. For example, the semantic information matching algorithm can use the SimCSE (Simple Contrastive Learning of SentenceEmbeddings, sentence embedding based on simple contrast learning) model to perform text matching at the sentence level, calculate the text similarity between two sentences, output the embedding text features of the text sentences through the pre-trained model, and then calculate the semantic feature similarity, and retrieve the knowledge base text with more similar semantic information, that is, the second knowledge base text mentioned above.

[0095] In some embodiments of the present application, generating answer content corresponding to the question text using the large language model based on the enhanced information of the question text, the matching results of the knowledge graph, and the matching results of the vector knowledge base includes: forming a second prompt word with the knowledge entity information corresponding to the question text; forming a third prompt word with the first knowledge base text and the second knowledge base text; and generating answer content corresponding to the question text using the large language model based on the query statement corresponding to the question text and the second prompt word and the third prompt word.

[0096] In order to make the structured data of the knowledge graph easy to read, the knowledge-related entity node information obtained by matching the above knowledge graph can be converted into the second prompt P2, and the second prompt P2 is used as the input of the large language model. The second prompt P2 is exemplified as follows:

[0097] What is {knowledge related entity node data}? Please explain the meaning of {knowledge related entity node data} in detail, which is more in line with human understanding.

[0098] The knowledge base text retrieved by the matching algorithm of the two vector knowledge bases in the above embodiment is input into the large language model in the form of the third prompt P3, combined with the second prompt P2 of the knowledge graph search result. The structure of the third prompt P3 is as follows:

[0099] Now that you are an expert in {a certain field such as: financial knowledge}, please refer to the following data: {text related to the knowledge base}, and combine {P2 prompts} to explain the easy-to-read text of the knowledge graph entity node data and other data, and answer the following questions: {specialized query statements}.

[0100] The embodiment of the present application synchronously combines the knowledge graph retrieval results and the vector knowledge base retrieval results, uses structured knowledge information and field semantic knowledge information to enhance the question-answering effect of the large language model, alleviates the hallucination error problem that is prone to occur in question-answering with a large language model, and improves the accuracy of question-answering.

[0101] The present application embodiment also provides a question-answering device 700 based on a large language model, such as Figure 7 As shown, a schematic diagram of the structure of a question-answering device based on a large language model in an embodiment of the present application is provided. The question-answering device based on a large language model 700 includes: an acquisition unit 710, a first generation unit 720, a retrieval and matching unit 730, and a second generation unit 740, wherein:

[0102] An acquisition unit 710 is used to acquire a question text of a user;

[0103] A first generating unit 720, configured to generate enhanced information of the question text using a large language model according to the question text and the first prompt word;

[0104] A search and matching unit 730 is used to search and match the enhanced information of the question text with the knowledge graph and the vector knowledge base of the target field respectively, to obtain matching results of the knowledge graph and the vector knowledge base;

[0105] The second generating unit 740 is used to generate answer content corresponding to the question text using the large language model according to the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base.

[0106] In some embodiments of the present application, the first generating unit 720 is specifically used to: filter the question text using a preset information filtering strategy to obtain a filtered question text; and generate enhanced information of the question text using a large language model based on the filtered question text and the first prompt word.

[0107] In some embodiments of the present application, the first generating unit 720 is specifically used to: obtain a set of questions in a target domain; classify the question text using a text classification model based on the question text and the set of questions in the target domain to obtain a category of the question text; calculate the text similarity between the question text and the set of questions in the target domain using a text similarity algorithm based on the question text and the set of questions in the target domain; filter the question text based on the category of the question text and the text similarity between the question text and the set of questions in the target domain to obtain a filtered question text.

[0108] In some embodiments of the present application, the enhanced information of the question text includes query statements and triples corresponding to the question text, and the first generation unit 720 is specifically used to: generate the query statement corresponding to the question text using the large language model according to the first prompt word corresponding to the filtered question text and the query statement; generate the triples corresponding to the question text using the fine-tuned large language model according to the query statement corresponding to the question text, and the fine-tuned large language model is obtained after fine-tuning the large language model.

[0109] In some embodiments of the present application, the enhanced information of the question text includes query statements and triples corresponding to the question text, and the retrieval and matching unit 730 is specifically used to: retrieve and match the triples corresponding to the question text with the knowledge graph of the target field to obtain the knowledge entity information corresponding to the question text; retrieve and match the query statement corresponding to the question text with the vector knowledge base to obtain the knowledge base text corresponding to the question text.

[0110] In some embodiments of the present application, the retrieval and matching unit 730 is specifically used to: use a keyword matching algorithm to retrieve and match the query statement corresponding to the question text with the vector knowledge base to obtain a first knowledge base text; use a semantic information matching algorithm to retrieve and match the query statement corresponding to the question text with the vector knowledge base to obtain a second knowledge base text.

[0111] In some embodiments of the present application, the second generation unit 740 is specifically used to: form a second prompt word from the knowledge entity information corresponding to the question text; form a third prompt word from the first knowledge base text and the second knowledge base text; and generate answer content corresponding to the question text using the large language model according to the query statement corresponding to the question text and the second prompt word and the third prompt word.

[0112] It can be understood that the above-mentioned question-answering device based on a large language model can implement the various steps of the question-answering method based on a large language model provided in the aforementioned embodiments. The relevant explanations about the question-answering method based on a large language model are applicable to the question-answering device based on a large language model and will not be repeated here.

[0113] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 8At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0114] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0115] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0116] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a question-answering device based on a large language model at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0117] Get the user's question text;

[0118] Generate enhanced information of the question text using a large language model according to the question text and the first prompt word;

[0119] Retrieve and match the enhanced information of the question text with the knowledge graph and the vector knowledge base of the target domain respectively, and obtain matching results of the knowledge graph and the vector knowledge base;

[0120] The large language model is used to generate answer content corresponding to the question text according to the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base.

[0121] The above application Figure 1The method performed by the question-answering device based on the large language model disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0122] The present application also provides a computer program product, including a computer program or an instruction, which is implemented when the computer program or the instruction is executed by a processor. Figure 1 The question-answering method based on a large language model in the illustrated embodiment is specifically used to perform:

[0123] Get the user's question text;

[0124] Generate enhanced information of the question text using a large language model according to the question text and the first prompt word;

[0125] Retrieve and match the enhanced information of the question text with the knowledge graph and the vector knowledge base of the target domain respectively, and obtain matching results of the knowledge graph and the vector knowledge base;

[0126] The large language model is used to generate answer content corresponding to the question text according to the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base.

[0127] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0132] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0133] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0134] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0135] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0136] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A question-answering method based on a large language model, characterized in that: The question answering method based on the large language model includes: Get the user's question text; Generate enhanced information of the question text using a large language model according to the question text and the first prompt word; Retrieve and match the enhanced information of the question text with the knowledge graph and the vector knowledge base of the target domain respectively, and obtain matching results of the knowledge graph and the vector knowledge base; The large language model is used to generate answer content corresponding to the question text according to the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base.

2. The question-answering method based on a large language model according to claim 1, characterized in that: The step of generating enhanced information of the question text by using a large language model according to the question text and the first prompt word includes: Using a preset information screening strategy to screen the question text to obtain a screened question text; According to the screened question text and the first prompt word, enhanced information of the question text is generated using a large language model.

3. The question-answering method based on a large language model according to claim 2, characterized in that: The question text is screened by using a preset information screening strategy to obtain the screened question text, which includes: Get a set of questions in the target domain; According to the question text and the question set of the target field, the question text is classified by using a text classification model to obtain a category of the question text; Calculating the text similarity between the question text and the question set in the target field using a text similarity algorithm according to the question text and the question set in the target field; The question text is screened according to the category of the question text and the text similarity between the question text and the question set in the target field to obtain the screened question text.

4. The question-answering method based on a large language model according to claim 2, characterized in that: The enhanced information of the question text includes a query sentence and a triple corresponding to the question text, and the enhanced information of the question text generated by using a large language model according to the filtered question text and the first prompt word includes: Generate a query statement corresponding to the question text using the large language model according to the first prompt word corresponding to the screened question text and the query statement; According to the query sentence corresponding to the question text, a triple corresponding to the question text is generated using a fine-tuned large language model, wherein the fine-tuned large language model is obtained by fine-tuning the large language model.

5. The question-answering method based on a large language model according to claim 1, characterized in that: The enhanced information of the question text includes query statements and triples corresponding to the question text. The enhanced information of the question text is searched and matched with the knowledge graph and the vector knowledge base of the target field respectively, and the matching results of the knowledge graph and the matching results of the vector knowledge base are obtained, including: Search and match the triples corresponding to the question text with the knowledge graph of the target domain to obtain the knowledge entity information corresponding to the question text; The query sentence corresponding to the question text is searched and matched with the vector knowledge base to obtain the knowledge base text corresponding to the question text.

6. The question-answering method based on a large language model according to claim 5, characterized in that: The query statement corresponding to the question text is searched and matched with the vector knowledge base to obtain the knowledge base text corresponding to the question text, including: Using a keyword matching algorithm, the query statement corresponding to the question text is searched and matched with the vector knowledge base to obtain a first knowledge base text; The query statement corresponding to the question text is searched and matched with the vector knowledge base using a semantic information matching algorithm to obtain a second knowledge base text.

7. The question-answering method based on a large language model according to claim 6, characterized in that: The step of generating the answer content corresponding to the question text by using the large language model according to the enhanced information of the question text, the matching result of the knowledge graph and the matching result of the vector knowledge base includes: The knowledge entity information corresponding to the question text is used to form a second prompt word; The first knowledge base text and the second knowledge base text form a third prompt word; The large language model is used to generate answer content corresponding to the question text according to the query sentence corresponding to the question text and the second prompt word and the third prompt word.

8. A question-answering device based on a large language model, characterized in that: The question-answering device based on the large language model includes: An acquisition unit, used to acquire the user's question text; A first generating unit, configured to generate enhanced information of the question text using a large language model according to the question text and the first prompt word; A search and matching unit, used to search and match the enhanced information of the question text with the knowledge graph and the vector knowledge base of the target field respectively, to obtain matching results of the knowledge graph and the vector knowledge base; The second generating unit is used to generate answer content corresponding to the question text by using the large language model according to the enhanced information of the question text, the matching results of the knowledge graph and the matching results of the vector knowledge base.

9. An electronic device, comprising: processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the question answering method based on a large language model according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the method of the question-answering device based on a large language model according to any one of claims 1 to 7 is implemented.

Images