Knowledge question-answering method and system based on large model and storage medium

Abstract

The invention provides a knowledge question and answer method and system based on a large model and a storage medium, and the method comprises the steps: inputting a user question into a retrieval enhancement generation retriever for text matching, and obtaining a candidate text; splicing the candidate text and the user questions to obtain a spliced question text, and inputting the spliced question text into a pre-trained large model for information extraction to obtain key information; carrying out reliability detection on the key information and the user questions; and if the reliability detection is qualified, determining a question-answer reply according to the key information, and performing question-answer feedback on the question-answer reply. According to the embodiment of the invention, the candidate text and the key information in the user questions can be effectively obtained by extracting the spliced text of the questions, so that the knowledge question-answering result is more focused, the data redundancy phenomenon is prevented, the knowledge question-answering efficiency is improved, and the reliability of the user questions is improved by detecting the reliability of the key information and the user questions. The reliability of the key information is improved, and then the accuracy of question and answer reply is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a knowledge question and answer method, system, and storage medium based on a large model. Background Art

[0002] With the development of artificial intelligence technology, more and more artificial intelligence systems are widely used, and the knowledge question and answer system is one of them. In specific implementation, a user inputs a question into the knowledge question and answer system, and the knowledge question and answer system retrieves relevant knowledge for the user's question, so as to feedback corresponding content to the user.

[0003] In the existing knowledge question and answer process, generally, a corresponding question and answer reply is directly output based on the matched candidate text, which leads to redundant reply content and reduces the knowledge question and answer efficiency. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a knowledge question and answer method, system, and storage medium based on a large model to solve the problem of low knowledge question and answer efficiency in the prior art.

[0005] The embodiments of the present invention are implemented as follows. A knowledge question and answer method based on a large model, the method includes: Obtain a user's question, and input the user's question into a retrieval augmented generation retriever for text matching to obtain candidate text; Concatenate the candidate text and the user's question to obtain a question concatenated text, and input the question concatenated text into a pre-trained large model for information extraction to obtain key information; Perform reliability detection on the key information and the user's question; If the reliability detection is qualified, determine a question and answer reply according to the key information, and perform question and answer feedback on the question and answer reply.

[0006] Preferably, inputting the question concatenated text into a pre-trained large model for information extraction to obtain key information includes: Extract features of the question concatenated text according to the pre-trained large model to obtain a hidden layer representation, and calculate a word generation probability distribution of the input vocabulary in the question concatenated text at each moment according to the hidden layer representation; Determine keyword vocabulary in the input vocabulary according to the word generation probability distribution, and combine the keyword vocabulary to obtain the key information.

[0007] Preferably, the formula for calculating the text generation probability of the input vocabulary in the question concatenated text at each moment according to the hidden layer representation includes: represents the probability distribution of the word generation, and are the training parameters of the large model, represents the hidden layer representation at the moment, is the number of model layers of the large model, is the set of the maximum probabilities in the probability distribution of the word generation at each moment.

[0008] Preferably, input the user's question into the retrieval-enhanced generation retriever for text matching to obtain candidate texts, including: Obtain a knowledge base, and segment the knowledge texts in the knowledge base according to the retrieval-enhanced generation retriever to obtain at least two segmented texts; Perform vector conversion on each of the segmented texts and the user's question to obtain each segmented vector and the question vector, and construct a vector index according to all the segmented vectors; the vector index is the set of all the segmented vectors; For each segmented vector in the vector index, calculate the similarity between the question vector and the segmented vector to obtain the vector similarity, and determine the candidate text according to the vector similarity.

[0009] Preferably, the formula for calculating the similarity between the question vector and the segmented vector to obtain the vector similarity includes: is the vector similarity, is the question vector, is the segmented vector.

[0010] Preferably, perform reliability detection on the key information and the user's question, including: Determine the target paragraph in the candidate text according to the key information, and determine the target similarity as the vector similarity corresponding to the target paragraph; Calculate the average value of the maximum generation probabilities in the probability distribution of the word generation at each moment to obtain the probability average value, and calculate the harmonic mean between the probability average value and the target similarity; If the harmonic mean is greater than or equal to the numerical threshold, it is determined that the reliability detection is qualified; If the harmonic mean is less than the numerical threshold, it is determined that the reliability detection is unqualified.

[0011] Preferably, the formula for calculating the harmonic mean between the probability average value and the target similarity includes: is the harmonic mean is the target similarity is the average probability; is the key information in the th

[0012] Another object of the embodiments of the present invention is to provide a knowledge question-answering system based on a large model, and the system includes: A text matching module, configured to obtain a user's question, input the user's question into a retrieval enhanced generation retriever for text matching, and obtain candidate text; An information extraction module, configured to splice the candidate text and the user's question to obtain a question-spliced text, and input the question-spliced text into a pre-trained large model for information extraction to obtain key information; A reliability detection module, configured to perform reliability detection on the key information and the user's question; A question-answering module, configured to, if the reliability detection is qualified, determine a question-answering response according to the key information, and perform question-answering feedback on the question-answering response.

[0013] Preferably, the information extraction module is further configured to: Extract features from the question-spliced text according to the pre-trained large model to obtain a hidden layer representation, and calculate a word generation probability distribution of the input vocabulary in the question-spliced text at each moment according to the hidden layer representation; Determine keyword vocabulary in the input vocabulary according to the word generation probability distribution, and combine the keyword vocabulary to obtain the key information.

[0014] In the embodiments of the present invention, by extracting the question-spliced text, the key information in the candidate text and the user's question can be effectively obtained, making the result of the knowledge question-answering more focused, preventing the phenomenon of data redundancy, improving the efficiency of the knowledge question-answering, and by performing reliability detection on the key information and the user's question, the reliability of the key information is improved, thereby ensuring the accuracy of the question-answering response. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flowchart of a knowledge question-answering method based on a large model provided by the first embodiment of the present invention.

[0016] Figure 2 is a schematic structural diagram of a knowledge question-answering system based on a large model provided by the second embodiment of the present invention.

[0017] Figure 3 It is a schematic structural diagram of a terminal device provided by the third embodiment of the present invention. Detailed implementation manners

[0018] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] In order to illustrate the technical solutions described in the present invention, specific embodiments will be used for illustration below.

[0020] Embodiment 1 Please refer to Figure 1 , which is a flowchart of a knowledge Q&A method based on a large model provided by the first embodiment of the present invention. The knowledge Q&A method based on a large model can be applied to any device or system. The knowledge Q&A method based on a large model includes the following steps: Step S10: Obtain a user's question, and input the user's question into a retrieval-augmented generation retriever for text matching to obtain candidate texts.

[0021] Among them, by inputting the user's question into a retrieval-augmented generation (RAG) retriever for text matching to obtain candidate texts that are semantically similar to the user's question, the number of candidate texts can be set according to requirements. For example, the number of candidate texts can be set to 5, 6, 10, etc.

[0022] Optionally, inputting the user's question into a retrieval-augmented generation retriever for text matching to obtain candidate texts includes: Obtain a knowledge base, and segment the knowledge texts in the knowledge base according to the retrieval-augmented generation retriever to obtain at least two segmented texts; among them, the content of the knowledge base can be set according to requirements; Perform vector conversion on each of the segmented texts and the user's question to obtain each segmented vector and question vector, and construct a vector index according to all the segmented vectors; the vector index is a set of all the segmented vectors; among them, segment the knowledge text according to paragraphs to obtain at least two segmented texts, and vectorize each segmented text to construct a vector index, , represents the th paragraph in the vector index, represents the th vector therein, that is, the vector index is a set of all segmented vectors. Represent the user's question in vector form, , Indicates the th query vector after the user query is vectorized.

[0023] For each segmented vector in the vector index, calculate the similarity between the query vector and the segmented vector to obtain a vector similarity, and determine the candidate text according to the vector similarity; wherein, calculate the similarity between the user query and the text in the knowledge base to obtain the k candidate texts with the highest similarity scores. The candidate texts selected in the above manner are closer to the user query and have higher accuracy.

[0024] Further, the formula for calculating the similarity between the query vector and the segmented vector to obtain the vector similarity includes: is the vector similarity, is the query vector, is the segmented vector. Through the above formula, the similarity between the query vector and the segmented vector can be quickly determined.

[0025] Step S20, splice the candidate text and the user query to obtain a query spliced text, and input the query spliced text into a pre-trained large model for information extraction to obtain key information.

[0026] Among them, splice the k candidate texts, , represents the text sequence of the kth candidate text. Add the spliced candidate text to the text template of the large model and splice it with the user query content to obtain a query spliced text , for example, the query spliced text is: According to the question input by the user, give the fragment related to the user input in the candidate paragraph, where the user input is , and the candidate fragment is .

[0027] Optionally, inputting the query spliced text into a pre-trained large model for information extraction to obtain key information includes: Extract features from the query spliced text according to the pre-trained large model to obtain a hidden layer representation, and calculate the word generation probability distribution of the input vocabulary in the query spliced text at each moment according to the hidden layer representation; wherein, obtain the hidden layer representation by feature encoding the query spliced text through the large model : Among them, represents word embedding processing; Complete the hidden layer representation according to the decoder in the large model and output the feature decoding of the characteristics; Among them, represents the output of the -th moment of the -th layer of the large model. The large model has a total of L layers. Therefore, the hidden layer representation finally output by the large model at the -th moment is , where

[0028] Determine the keyword vocabulary in the input vocabulary according to the word generation probability distribution, and combine the keyword vocabulary to obtain the key information; among them, determine the input vocabulary corresponding to the maximum probability in each time word generation probability distribution as the keyword vocabulary, and combine the determined keyword vocabulary to obtain the key information , is the key information in the -th keyword vocabulary.

[0029] In the above method, by calculating the word generation probability distribution of the input vocabulary in the question splicing text at each moment, and determining the keyword vocabulary in the input vocabulary based on the word generation probability distribution, the key information is determined. Therefore, the keyword vocabulary in different paragraphs can be dynamically identified, improving the accuracy of the determined key information.

[0030] Furthermore, the formula for calculating the text generation probability of the input vocabulary in the question splicing text at each moment according to the hidden layer representation includes: represents the word generation probability distribution, and are the training parameters of the large model, represents the hidden layer representation at the -th moment, is the model layer number of the large model,

[0031] Even further, before inputting the question splicing text into the pre-trained large model for information extraction to obtain the key information, it also includes: inputting the training text into the large model for information extraction to obtain the predicted information, determining the loss value of the large model according to the predicted information, and updating the parameters of the large model according to the loss value until the large model converges to obtain the pre-trained large model.

[0032] Step S30: Perform reliability detection on the key information and the user's question.

[0033] Among them, by performing reliability detection on the key information and the user's question, the reliability of the key information is improved, thereby ensuring the accuracy of the Q&A response.

[0034] Optionally, performing reliability detection on the key information and the user's question includes: Determine the target paragraph in the candidate text according to the key information, and determine the vector similarity corresponding to the target paragraph as the target similarity; among them, match the key information with the candidate text, determine the text paragraph obtained by the match as the target paragraph, and obtain the vector similarity corresponding to the target paragraph to obtain the target similarity; Calculate the average value of the maximum generation probabilities in the word generation probability distribution at each moment to obtain the probability average value, and calculate the harmonic mean between the probability average value and the target similarity; among them, use the probability values obtained during the calculation as the scores of the keyword vocabulary to find the average value of the maximum generation probabilities corresponding to each keyword vocabulary, which is the probability average value of the key information; If the harmonic mean is greater than or equal to the numerical threshold, it is determined that the reliability detection is qualified; If the harmonic mean is less than the numerical threshold, it is determined that the reliability detection is unqualified; Among them, the numerical threshold can be set according to requirements. If the harmonic mean is greater than or equal to , it is determined that the key information is reliable and can be used for subsequent response generation. If it is less than , the key information is unreliable.

[0035] In the above method, by performing reliability detection on the key information and the user's question, the reliability of the key information can be improved.

[0036] Furthermore, the formula used to calculate the harmonic mean between the probability average value and the target similarity includes: is the harmonic mean, is the target similarity, is the probability average value.

[0037] Step S40: If the reliability test is qualified, determine the Q&A response according to the key information, and perform Q&A feedback on the Q&A response.

[0038] Among them, when the key information is reliable, the key information is matched with the knowledge base to obtain the Q&A response. Different key information and the corresponding Q&A responses are stored in the knowledge base. Specifically: is the output of the large model, represents determining the Q&A response according to the key information, and rejor SUM(KI) represents rejecting the answer or generating a fragment summary based on the answer corresponding to the candidate text as the final output.

[0039] In this embodiment, by extracting the question splicing text, the key information in the candidate text and the user's question can be effectively obtained, making the result of the knowledge Q&A more focused, preventing data redundancy, improving the efficiency of knowledge Q&A. By performing reliability tests on the key information and the user's question, the reliability of the key information is improved, thereby ensuring the accuracy of the Q&A response and reducing the hallucination of using the large model RAG to answer.

[0040] Embodiment Two Please refer to Figure 2 , which is a schematic structural diagram of the knowledge Q&A system 100 based on a large model provided by the second embodiment of the present invention, including: The text matching module 10 is used to obtain the user's question and input the user's question into the retrieval augmented generation retriever for text matching to obtain candidate texts. Among them, by inputting the user's question into the retrieval augmented generation (Retrieval-Augmented Generation, RAG) retriever for text matching to obtain candidate texts that are semantically similar to the user's question, the number of candidate texts can be set according to requirements. For example, the number of candidate texts can be set to 5, 6, 10, etc.

[0041] The information extraction module 11 is used to splice the candidate text and the user's question to obtain a question splicing text, and input the question splicing text into the pre-trained large model for information extraction to obtain key information. Among them, k candidate texts are spliced, , represents the text sequence of the kth candidate text. The spliced candidate text is added to the text template of the large model and spliced with the user's question content to obtain a question splicing text .

[0042] The reliability detection module 12 is used to detect the reliability of the key information and the user's question; wherein, by detecting the reliability of the key information and the user's question, the reliability of the key information is improved, thereby ensuring the accuracy of the Q&A reply.

[0043] The Q&A module 13 is used to, if the reliability detection is qualified, determine the Q&A reply according to the key information and perform Q&A feedback on the Q&A reply; wherein, when the key information is reliable, the key information is matched with the knowledge base to obtain the Q&A reply, and the corresponding relationship between different key information and the corresponding Q&A reply is stored in the knowledge base.

[0044] Preferably, the information extraction module 11 is further used to: extract features from the question splicing text according to the pre-trained large model to obtain a hidden layer representation, and calculate the word generation probability distribution of the input vocabulary in the question splicing text at each moment according to the hidden layer representation; Determine the keyword vocabulary in the input vocabulary according to the word generation probability distribution, and combine the keyword vocabulary to obtain the key information.

[0045] Preferably, the formula for calculating the text generation probability of the input vocabulary in the question splicing text at each moment according to the hidden layer representation includes: represents the word generation probability distribution, and are the training parameters of the large model, represents the hidden layer representation at the moment, is the number of model layers of the large model,

[0046] Preferably, the text matching module 10 is further used to: obtain the knowledge base, and segment the knowledge text in the knowledge base according to the retrieval enhanced generator to obtain at least two segmented texts; Perform vector conversion on each of the segmented texts and the user's question to obtain each segmented vector and the question vector, and construct a vector index according to all the segmented vectors; the vector index is the set of all the segmented vectors; For each segmented vector in the vector index, calculate the similarity between the question vector and the segmented vector to obtain the vector similarity, and determine the candidate text according to the vector similarity.

[0047] Preferably, according to the vector index, the similarity between the query vector and the segmented vector is calculated, and the formula for obtaining the vector similarity includes: is the vector similarity, is the query vector, is the segmented vector.

[0048] Preferably, the reliability detection module 12 is further configured to: determine the target paragraph in the candidate text according to the key information, and determine the vector similarity corresponding to the target paragraph as the target similarity; Calculate the average value of the maximum generation probabilities in the word generation probability distribution at each moment to obtain the probability average value, and calculate the harmonic mean between the probability average value and the target similarity; If the harmonic mean is greater than or equal to the numerical threshold, it is determined that the reliability detection is qualified; If the harmonic mean is less than the numerical threshold, it is determined that the reliability detection is unqualified.

[0049] Preferably, the formula for calculating the harmonic mean between the probability average value and the target similarity includes: is the harmonic mean, is the target similarity, is the probability average value; is the key information in the th

[0050] keyword in the keyword vocabulary.

[0051] Embodiment III Figure 3 is a structural block diagram of a terminal device 2 provided in the third embodiment of the present application. As Figure 3As shown, the terminal device 2 of this embodiment includes: a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program for a large model-based knowledge Q&A method. When the processor 20 executes the computer program 22, the steps in each of the above embodiments of the various large model-based knowledge Q&A methods are implemented.

[0052] Exemplarily, the computer program 22 can be divided into one or more modules. The one or more modules are stored in the memory 21 and executed by the processor 20 to complete this application. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, the processor 20 and the memory 21.

[0053] The so-called processor 20 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0054] The memory 21 may be an internal storage unit of the terminal device 2, such as the hard disk or memory of the terminal device 2. The memory 21 may also be an external storage device of the terminal device 2, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 2. Further, the memory 21 may also include both the internal storage unit and the external storage device of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 may also be used to temporarily store data that has been output or is to be output.

[0055] In addition, each functional module in various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0056] If the integrated module is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Among them, the computer-readable storage medium may be non-volatile or volatile. Based on such an understanding, all or part of the processes in the above-mentioned embodiment methods of the present application can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0057] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A knowledge question answering method based on a large model, characterized in that: The method comprises: Obtaining a user question, and inputting the user question into a search enhancement generation retriever for text matching to obtain candidate texts; The candidate text and the user question are concatenated to obtain a concatenated question text, and the concatenated question text is input into a pre-trained large model to extract information to obtain key information; Performing reliability testing on the key information and the user's questions; If the reliability test is qualified, the question and answer response is determined based on the key information, and the question and answer response is fed back as question and answer feedback.

2. The knowledge question answering method based on a large model as claimed in claim 1, characterized in that: The question concatenation text is input into the pre-trained large model to extract information and obtain key information, including: Extracting features of the concatenated question text according to the pre-trained large model to obtain a hidden layer representation, and calculating the word generation probability distribution of the input vocabulary in the concatenated question text at each moment according to the hidden layer representation; The key words in the input vocabulary are determined according to the word generation probability distribution, and the key words are combined to obtain the key information.

3. The knowledge question answering method based on a large model as claimed in claim 2, characterized in that: The formula used to calculate the text generation probability of the input vocabulary in the question concatenation text at each moment according to the hidden layer representation includes: represents the probability distribution of word generation, and are the training parameters of the large model, express The hidden layer representation at time, is the number of model layers of the large model, It is the set of maximum probabilities in the probability distribution of word generation at each moment.

4. The knowledge question answering method based on a large model as claimed in claim 2, characterized in that: The user question is input into the search enhancement generation retriever for text matching to obtain candidate texts, including: Acquire a knowledge base, and segment the knowledge text in the knowledge base according to the search enhancement generation retriever to obtain at least two segmented texts; Performing vector conversion on each of the segmented texts and the user questions to obtain segmentation vectors and question vectors, and constructing a vector index based on all the segmentation vectors; the vector index is a collection of all the segmentation vectors; For each segmentation vector in the vector index, the similarity between the question vector and the segmentation vector is calculated to obtain vector similarity, and the candidate text is determined according to the vector similarity.

5. The knowledge question answering method based on a large model as claimed in claim 4, characterized in that: The similarity between the question vector and the segmentation vector is calculated, and the formula used to obtain the vector similarity includes: is the vector similarity, is the question vector, is the segmentation vector.

6. The knowledge question answering method based on a large model as claimed in claim 4, characterized in that: Performing reliability testing on the key information and the user's question, including: Determine a target paragraph in the candidate text according to the key information, and determine the vector similarity corresponding to the target paragraph as a target similarity; Calculate the average value of the maximum generation probability in the word generation probability distribution at each moment to obtain a probability average value, and calculate the harmonic mean between the probability average value and the target similarity; If the harmonic mean is greater than or equal to the numerical threshold, the reliability test is determined to be qualified; If the harmonic mean is smaller than the numerical threshold, it is determined that the reliability test fails.

7. The knowledge question answering method based on a large model as claimed in claim 6, characterized in that: The formula used to calculate the harmonic mean between the probability average and the target similarity includes: is the harmonic mean, is the target similarity, is the probability mean; The key information Middle The key words mentioned.

8. A knowledge question answering system based on a large model, characterized in that: The system comprises: A text matching module is used to obtain user questions and input the user questions into a search enhancement generation retriever for text matching to obtain candidate texts; An information extraction module is used to concatenate the candidate text and the user question to obtain a concatenated question text, and input the concatenated question text into a pre-trained large model to extract information to obtain key information; A reliability detection module, used to perform reliability detection on the key information and the user's question; The question-and-answer module is used to determine a question-and-answer reply based on the key information if the reliability test is qualified, and to provide question-and-answer feedback for the question-and-answer reply.

9. The knowledge question answering system based on a large model as claimed in claim 8, characterized in that: The information extraction module is also used for: Extracting features of the concatenated question text according to the pre-trained large model to obtain a hidden layer representation, and calculating the word generation probability distribution of the input vocabulary in the concatenated question text at each moment according to the hidden layer representation; The key words in the input vocabulary are determined according to the word generation probability distribution, and the key words are combined to obtain the key information.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images