Question answering method and device based on retrieval enhancement, computer equipment and storage medium

Abstract

The invention relates to a question answering method and device based on retrieval enhancement, computer equipment and a storage medium. The method comprises the following steps: obtaining a plurality of candidate documents retrieved based on user questions, and clustering the plurality of candidate documents to obtain a plurality of document clusters; constructing a plurality of candidate document subsets according to the document clusters; through a plurality of lightweight generation models, concurrently performing question and answer preliminary processing according to the candidate document subsets and the user questions to obtain a plurality of candidate question and answer responses and a confidence result of each candidate question and answer response; performing self-consistency and rationality verification on each candidate question and answer response in sequence through the large model to obtain a self-consistency verification result and a rationality verification result corresponding to each candidate question and answer response; and according to the confidence degree result, the self-consistency verification result and the rationality verification result of each candidate question and answer response, determining a final question and answer response corresponding to the user question. By adopting the method, the accuracy and the reliability of the large model question and answer service can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and particularly to a question answering method, device, computer device, and storage medium based on retrieval enhancement. Background Art

[0002] The emergence of large models provides more intelligent capabilities for human-computer dialogue, and more and more industry applications have emerged, providing generative question answering services for users.

[0003] However, although large models perform well in processing unstructured data, when generating content, they may sometimes generate non-existent facts out of thin air or confuse known information, affecting the accuracy and reliability of the answer results. Therefore, how to improve the accuracy and reliability of large model answer services has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a question answering method, device, computer device, computer-readable storage medium, and computer program product based on retrieval enhancement, which can improve the accuracy and reliability of large model question answering services.

[0005] In a first aspect, the present application provides a question answering method based on retrieval enhancement, including:

[0006] Obtaining a plurality of candidate documents retrieved based on a user question, and performing clustering processing on the plurality of candidate documents to obtain a plurality of document clustering clusters;

[0007] Constructing a plurality of candidate document subsets according to the plurality of document clustering clusters; wherein, the candidate documents included in the candidate document subset come from different document clustering clusters;

[0008] Through a plurality of lightweight generation models, concurrently performing preliminary question answering processing on the candidate document subset and the user question to obtain a plurality of candidate question and answer responses and the confidence result of each candidate question and answer response;

[0009] Verifying the self-consistency and rationality of each candidate question and answer response through a large model in sequence to obtain the self-consistency verification result and rationality verification result corresponding to each candidate question and answer response;

[0010] Determining the final question and answer response corresponding to the user question according to the confidence result, self-consistency verification result, and rationality verification result of each candidate question and answer response.

[0011] In a second aspect, the present application provides a question answering device based on retrieval enhancement, including:

[0012] An obtaining module, configured to obtain a plurality of candidate documents retrieved based on a user question;

[0013] A clustering module, which is used to cluster multiple candidate documents to obtain multiple document clusters;

[0014] A construction module, which is used to construct multiple candidate document subsets according to multiple document clusters; among them, the candidate documents included in the candidate document subset come from different document clusters;

[0015] A drafting module, which is used to perform preliminary question-and-answer processing on the candidate document subset and the user question concurrently through multiple lightweight generation models, to obtain multiple candidate question-and-answer responses and the confidence results of each candidate question-and-answer response;

[0016] A verification module, which is used to verify the self-consistency and rationality of each candidate question-and-answer response in turn through a large model, to obtain the self-consistency verification result and the rationality verification result corresponding to each candidate question-and-answer response;

[0017] An output module, which is used to determine the final question-and-answer response corresponding to the user question according to the confidence result, the self-consistency verification result, and the rationality verification result of each candidate question-and-answer response.

[0018] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps in the above method are implemented.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method are implemented.

[0020] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps in the above method are implemented.

[0021] The above question-and-answer method, device, computer device, computer-readable storage medium, and computer program product based on retrieval enhancement generate multiple candidate question-and-answer responses concurrently based on the user question and the candidate document subset through multiple lightweight generation models; verify the self-consistency and rationality of the multiple candidate question-and-answer responses through a large model, and then select the optimal question-and-answer response to feedback to the user. Compared with directly using a large model to generate question-and-answer responses, the accuracy and reliability of the question-and-answer responses can be effectively improved. Description of the Drawings

[0022] Figure 1 It is an application environment diagram of a question-and-answer method based on retrieval enhancement provided by an embodiment of the present application;

[0023] Figure 2 It is a flowchart of a question-and-answer method based on retrieval enhancement provided by an embodiment of the present application;

[0024] Figure 3 A logical schematic diagram of a question - answering method based on retrieval enhancement provided by an embodiment of the present application;

[0025] Figure 4 A structural block diagram of a question - answering device based on retrieval enhancement provided by an embodiment of the present application;

[0026] Figure 5 An internal structure diagram of a computer device provided by an embodiment of the present application;

[0027] Figure 6 Another internal structure diagram of a computer device provided by an embodiment of the present application;

[0028] Figure 7 An internal structure diagram of a computer - readable storage medium provided by an embodiment of the present application. Detailed implementation manners

[0029] In order to make the objectives, technical solutions, and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] The question - answering method based on retrieval enhancement provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 wherein, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed in the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in - vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head - mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0031] As shown in Figure 2 the embodiments of the present application provide a question - answering method based on retrieval enhancement, which is described by taking the method applied to the terminal 102 or the server 104 in Figure 1 as an example. It can be understood that the computer device can include at least one of the terminal and the server. The method includes the following steps:

[0032] S201. Obtain a plurality of candidate documents retrieved based on the user's question, and perform clustering processing on the plurality of candidate documents to obtain a plurality of document clustering clusters.

[0033] In the embodiments of the present application, the user question can be the question text input by the user or the question input by the user in the form of voice, and no specific limitation is made here. The user question can be input into a preset retrieval enhancement system, and the preset retrieval enhancement system retrieves relevant documents from the database according to the user question and uses them as candidate documents; among them, the preset retrieval enhancement system is constructed based on the RAG (Retrieval Augmented Generation) technology. The obtained candidate document set can be denoted as D = {d1, d2, d3, …, d n}

[0034] After obtaining the candidate documents, any clustering method can be used to cluster the multiple candidate documents to obtain multiple document clusters; each document cluster can include multiple candidate documents with similar themes or contents. Exemplarily, the clustering methods can include: partitioning-based K-Means clustering, density-based clustering, and hierarchical clustering.

[0035] S202. Construct multiple candidate document subsets according to the multiple document clusters; among them, the candidate documents included in the candidate document subsets come from different document clusters.

[0036] In the embodiments of the present invention, constructing multiple candidate document subsets according to the multiple document clusters includes: randomly extracting one candidate document from each document cluster through a data analysis tool to form a candidate document subset; repeating the above steps, multiple candidate document subsets can be obtained. It can be understood that by sampling candidate documents from different document clusters, the understanding of the diversity and complexity in the retrieval results can be improved.

[0037] S203. Through multiple lightweight generation models, concurrently perform preliminary question-and-answer processing on the candidate document subsets and the user question to obtain multiple candidate question-and-answer responses and the confidence results of each candidate question-and-answer response.

[0038] In the embodiments of the present application, the question-and-answer system based on retrieval enhancement integrates multiple lightweight generation models and a large model, where the lightweight generation models are responsible for preliminarily generating candidate question-and-answer responses according to the user question and the document retrieval results; the large model verifies the candidate question-and-answer responses and selects the final question-and-answer response to feedback to the user. That is, in the solution of the present application, the multiple lightweight generation models and the large model cooperate to provide question-and-answer services for users.

[0039] In the embodiments of the present application, a single candidate document subset and the user question can be combined to construct a text interaction instruction applicable to the lightweight generation model. Exemplarily, a constructed text interaction instruction is as follows: "Please answer according to the following content and provide the reasoning process for obtaining the answer:

[0040] Question: {Q};

[0041] Dependent on the context content:

[0042] Doc1: {Content of candidate document 1};

[0043] Doc2: {Content of candidate document 2};

[0044] ...”. Among them, Doc1, Doc2, etc. come from the same subset of candidate documents.

[0045] Because multiple subsets of candidate documents are constructed through step 202, for each subset of candidate documents and the user's question, a text interaction instruction can be constructed. Further, the multiple constructed text interaction instructions can be concurrently input into different lightweight generation models for preliminary question and answer processing. Each lightweight generation model can output a candidate question and answer response and the confidence result of the candidate question and answer response according to the respective received text interaction instructions. Among them, the candidate question and answer response at least includes the candidate answer corresponding to the user's question and the reasoning chain given by the lightweight generation model for the candidate answer; the confidence result of the candidate question and answer response can be used to measure the accuracy of the candidate question and answer response.

[0046] It can be understood that through parallel processing, the data processing time can be reduced and the overall question and answer efficiency can be improved.

[0047] S204. Use the large model to sequentially verify the self-consistency and rationality of each candidate question and answer response, and obtain the self-consistency verification result and rationality verification result corresponding to each candidate question and answer response.

[0048] In the embodiment of the present invention, self-consistency is used to measure the consistency of the candidate answer and the reasoning chain in the context of the candidate question and answer response; rationality is used to measure whether the reasoning chain in the candidate question and answer response is logically supported.

[0049] To implement the sequential verification of the self-consistency and rationality of each candidate question and answer response through the large model, it is necessary to construct a text interaction instruction applicable to the large model according to each candidate question and answer response and the user's question. Exemplarily, the constructed text interaction instruction is as follows: "Question: {Q};

[0050] Answer: {Candidate answer α j};

[0051] Reasoning chain: {Reasoning chain β j};

[0052] Do you think the reasoning chain can support the answer content? Answer correctly or wrongly and give reasons".

[0053] The constructed text interaction instructions can be input into the large model, and self-consistency and rationality verification can be performed based on the output of the large model to obtain the self-consistency verification results and rationality verification results corresponding to each candidate Q&A response respectively.

[0054] S205. Determine the final Q&A response corresponding to the user's question according to the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response.

[0055] In the embodiment of the present application, the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response can be multiplied, and the calculation result is used as the score of the candidate Q&A response; furthermore, according to the scores of each candidate Q&A response, the candidate Q&A response with the highest score can be used as the final Q&A response corresponding to the user's question, and the final Q&A response is output to the user.

[0056] It can be seen that in the embodiment of the present application, multiple candidate Q&A responses are concurrently generated based on the user's question and candidate documents by multiple lightweight generation models; the large model performs self-consistency and rationality verification on multiple candidate Q&A responses, and then selects the optimal Q&A response to feedback to the user. Compared with directly using the large model to generate Q&A responses, the accuracy and reliability of the Q&A responses can be effectively improved.

[0057] In some embodiments, clustering processing is performed on multiple candidate documents to obtain multiple document clustering clusters, including:

[0058] Input multiple candidate documents into a preset embedding model for processing, and output multiple candidate documents represented by high-dimensional vectors;

[0059] Through a preset clustering method, clustering processing is performed on multiple candidate documents represented by high-dimensional vectors to obtain multiple document clustering clusters.

[0060] Exemplarily, the obtained candidate document set is D = {d1, d2, d3, …, d n}; the preset embedding model is Instruction-Aware Embedding Model; input the candidate document set D into the preset embedding model, and the generated high-dimensional vector representation: Embedding(D) = {E(d1), E(d2), E(d3), …, E(d n )}; that is, E(d1), E(d2),..., E(d n ) are candidate documents represented by high-dimensional vectors respectively.

[0061] The preset clustering method is partitioning-based clustering or density-based clustering. Taking partitioning-based K-Means clustering as an example, clustering is performed as follows: K-Means(E(d1), E(d2), E(d3),..., E(d n )) = {c1, c2,…, c k}. c1, c2,…, c k respectively represent different document clustering clusters.

[0062] In this embodiment, candidate documents with similar themes or contents are aggregated through clustering, providing a basis for constructing a candidate document subset in the subsequent process.

[0063] In some embodiments, multiple candidate document subsets are constructed according to multiple document clustering clusters, including:

[0064] Construct candidate document subsets according to the following steps until the number of constructed candidate document subsets is equal to the preset value:

[0065] Randomly select one candidate document from each document clustering cluster, and construct a candidate document subset based on the multiple randomly selected candidate documents.

[0066] In this embodiment, by randomly selecting candidate documents from different document clustering clusters to form a candidate document subset, the themes or contents of the candidate documents in the candidate document subset are different. When generating answer content based on the different documents, the understanding of the diversity and complexity in the document retrieval results can be improved.

[0067] In some embodiments, through multiple lightweight generation models, question-answering preliminary processing is concurrently performed according to the candidate document subset and the user's question, and multiple candidate question-and-answer responses and the confidence result of each candidate question-and-answer response are obtained, including:

[0068] According to the user's question and each candidate document subset, construct multiple interactive text instructions applicable to the lightweight generation model;

[0069] Concurrently input the multiple text interactive instructions into different lightweight generation models for question-answering preliminary processing, and output multiple candidate question-and-answer responses and the confidence result of each candidate question-and-answer response.

[0070] Exemplarily, one constructed text interactive instruction is as follows: "Please answer according to the following content and provide the reasoning process for obtaining the answer:

[0071] Question: {Q};

[0072] Context-dependent content:

[0073] Doc1: {Content of candidate document 1};

[0074] Doc2: {Content of candidate document 2};

[0075] ……”. Among them, Doc1, Doc2, etc. come from the same subset of candidate documents.

[0076] After concurrently inputting multiple text interaction instructions into different lightweight generation models for preliminary question - answering processing, the output candidate question - answering responses include the candidate answers corresponding to the user questions and the reasoning chains for obtaining the candidate answers; the confidence results of the candidate question - answering responses include the confidence of the candidate answers and the confidence of the reasoning chains of the candidate answers. Among them, when determining the confidence of the candidate answers, the candidate answers can be split into multiple tokens, and according to the probabilities of generating each token, a product process is performed, and the product result is the confidence of the candidate answers. Similarly, when determining the confidence of the reasoning chains of the candidate answers, the reasoning chains of the candidate answers can be split into multiple tokens, and according to the probabilities of generating each token, a product process is performed, and the product result is the confidence of the reasoning chains of the candidate answers. The confidence result of the candidate question - answering response can be the sum value of the confidence of the candidate answer and the confidence of the reasoning chain of the candidate answer.

[0077] In this embodiment, by concurrently performing preliminary question - answering processing through multiple lightweight generation models, the efficiency of question - answering processing can be improved.

[0078] In some embodiments, through the large - model, the self - consistency verification is performed on each candidate question - answering response in turn, and the self - consistency verification results corresponding to each candidate question - answering response are obtained, including:

[0079] For any candidate question - answering response, according to the candidate question - answering response and the user question, a text interaction instruction applicable to the large - model is constructed;

[0080] Based on the text interaction instruction and the large - model, the output result is determined, and the output result is split into independent tokens;

[0081] For any token that makes up the output result, if the token belongs to the candidate answer in the candidate question - answering response, calculate the respective conditional probabilities of the other tokens before this token in the output result, and take the product of the respective conditional probabilities of the other tokens as the first - type score; where the conditional probability of a token refers to the probability of generating this token; or,

[0082] For any token that makes up the output result, if the token belongs to the reasoning chain in the candidate question - answering response, calculate the respective conditional probabilities of the other tokens before this token in the output result, and take the product of the respective conditional probabilities of the other tokens as the second - type score;

[0083] According to the first - type score and the second - type score, the self - consistency verification result corresponding to the candidate question - answering response is determined.

[0084] Exemplarily, the constructed text interaction instruction is as follows: "Question: {Q};

[0085] Answer: {Candidate Answer α j};

[0086] Inference Chain: {Inference Chain β j};

[0087] Do you think the inference chain can support the answer content? Answer correctly or incorrectly and give reasons."

[0088] Input the constructed text interaction instruction into the large model for processing, obtain the output result of the large model, split the output result of the large model into independent tokens, and calculate the self - consistency verification result ρ corresponding to the candidate Q&A response according to the following formula Self-contain :

[0089]

[0090] where t i represents a certain token in the output result, α represents the candidate answer; β represents the inference chain; t <i represents all the tokens in the sequence before the current token t i in a token sequence; is the first - type score mentioned above; represents the second - type score.

[0091] The product of the first - type score and the second - type score can be used as the self - consistency verification result corresponding to the candidate Q&A response.

[0092] In this embodiment, by calculating the self - consistency verification result corresponding to the candidate Q&A response, quantitative analysis of whether each candidate answer is consistent with the inference chain is realized, providing a basis for subsequent screening of candidate answers.

[0093] In some embodiments, the large model is used to perform a rationality verification on each candidate Q&A response in turn, and the rationality verification result corresponding to each candidate Q&A response is obtained, including:

[0094] For any candidate Q&A response, construct a text interaction instruction applicable to the large model according to the candidate Q&A response and the user's question;

[0095] Based on the text interaction instruction and the large model, determine the output result and split the output result into independent tokens;

[0096] If the output result includes a pre - determined target token, determine the conditional probability of each of the other tokens before the target token when the target token is generated;

[0097] Determine the rationality verification result corresponding to the candidate Q&A response according to the conditional probabilities of other tokens.

[0098] Exemplarily, the constructed text interaction instruction is as follows: "Question: {Q};

[0099] Answer: {Candidate answer α j};

[0100] Inference chain: {Inference chain β j};

[0101] Do you think the inference chain can support the answer content? Answer correct or wrong and give reasons".

[0102] Input the constructed text interaction instruction into the large model for processing, obtain the output result of the large model, split the output result of the large model into independent tokens, and calculate the rationality verification result ρ corresponding to the inference chain according to the following formula Self-reflection :

[0103]

[0104] Among them, "correct" is the above-mentioned target token. It can be understood that the target token being "correct" is just an example, which is determined according to the content in the constructed text interaction instruction. The meaning of the formula is: the product of the probabilities of the tokens related to "correct" output by the large model. On this basis, when determining the rationality verification result corresponding to the candidate Q&A response according to the conditional probabilities of other tokens, the product of the conditional probabilities of other tokens can be used as the rationality verification result corresponding to the candidate Q&A response.

[0105] In this embodiment, it is possible to verify whether the inference chain has logical support, providing a quantitative basis for subsequent screening of candidate answers.

[0106] In some embodiments, according to the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response, determine the final Q&A response corresponding to the user's question, including:

[0107] Score each candidate Q&A response according to the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response to obtain the score of each candidate question response;

[0108] Sort the candidate Q&A responses according to the scores of each candidate Q&A response to obtain a sorting result;

[0109] According to the sorting result, take the candidate Q&A response with the highest score as the final Q&A response corresponding to the user's question.

[0110] Specifically, for any candidate Q&A response, the product of the confidence result, self-consistency verification result, and rationality verification result of the candidate Q&A response can be used as the score of the candidate Q&A response. Furthermore, the final Q&A response corresponding to the user's question can be selected based on the score.

[0111] In this embodiment, determining the final Q&A result according to the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response can ensure the accuracy and reliability of the Q&A result fed back to the user.

[0112] In some embodiments, obtaining multiple candidate documents retrieved based on the user's question includes:

[0113] Performing document retrieval on the user's question through a preset enhanced retrieval system, and using the retrieved documents that match the user's question as candidate documents.

[0114] In this embodiment, the preset enhanced retrieval system cooperates with a large model and multiple lightweight generation models to implement the Q&A service together, which can ensure that the Q&A service is replied according to the documents that match the user's question, improving the accuracy and reliability of the Q&A service.

[0115] Furthermore, to describe the content of this application in detail, reference can be made to Figure 3 , which shows a logical schematic diagram of the retrieval-enhanced Q&A method. The retrieval-enhanced Q&A method includes the following logic:

[0116] The user inputs a Query (i.e., the user's question), which first enters the RAG retrieval system for retrieval of relevant documents, and multiple candidate documents can be obtained; through the K-Means clustering method, the multiple candidate documents are clustered to obtain multiple clustering clusters; a candidate document is randomly extracted from each clustering cluster to form a candidate document subset, and this process can be repeated to obtain multiple candidate document subsets; the multiple candidate document subsets and the user input Query are concurrently input into different lightweight generation models (i.e., M Drafter ); each lightweight generation model processes the candidate document subset and the user input Query and outputs a candidate Q&A response (including a candidate answer and an inference chain); the large model (M Verifier ) verifies each lightweight generation model, mainly through the three formulas in the figure. Among them, the first formula and the second formula can refer to the description of the above embodiments; the third formula (ρ Draft,j = P(β j |Q,d j1 ,d j2 ,…,d jk ) + P(α j |Q,d j1 ,d j2 ,…,djk , β j )) represents the confidence result of the candidate Q&A response. Among them, P(β j |Q, d j1 , d j2 , …, d jk ) is the confidence of the inference chain, and P(α j |Q, d j1 , d j2 , …, d jk , β j ) is the confidence of the candidate answer. Through three formulas, the score of each candidate Q&A response can be calculated; furthermore, based on the scores, the final Q&A response can be selected and fed back to the user.

[0117] It should be understood that although each step in the flowcharts involved in the above embodiments is shown in sequence according to the indication of the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same moment, but can be executed at different moments. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turns with at least a part of other steps or steps or stages in other steps.

[0118] Based on the same inventive concept, the embodiments of the present application also provide a Q&A device based on retrieval enhancement. The implementation solutions provided by this device for solving problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the Q&A device based on retrieval enhancement provided below can refer to the limitations on the Q&A method based on retrieval enhancement in the above text, and will not be repeated here.

[0119] As Figure 4 shown, the embodiments of the present application provide a Q&A device 400 based on retrieval enhancement, including:

[0120] An obtaining module 401, configured to obtain a plurality of candidate documents retrieved based on a user's question;

[0121] A clustering module 402, configured to perform clustering processing on the plurality of candidate documents to obtain a plurality of document clustering clusters;

[0122] A construction module 403, configured to construct a plurality of candidate document subsets according to the plurality of document clustering clusters; wherein, the candidate documents included in the candidate document subsets come from different document clustering clusters;

[0123] Drafting module 404 is used to perform preliminary Q&A processing on the candidate document subset and the user question concurrently through multiple lightweight generation models, obtaining multiple candidate Q&A responses and the confidence results of each candidate Q&A response;

[0124] Verification module 405 is used to verify the self-consistency and rationality of each candidate Q&A response in turn through a large model, obtaining the self-consistency verification result and the rationality verification result corresponding to each candidate Q&A response;

[0125] Output module 406 is used to determine the final Q&A response corresponding to the user question according to the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response.

[0126] In some embodiments, in terms of clustering multiple candidate documents to obtain multiple document clustering clusters, the clustering module 402 is specifically used for:

[0127] Input multiple candidate documents into a preset embedding model for processing, and output multiple candidate documents represented by high-dimensional vectors;

[0128] Perform clustering processing on multiple candidate documents represented by high-dimensional vectors through a preset clustering method to obtain multiple document clustering clusters; wherein, the preset clustering method is partitioning-based clustering or density-based clustering;

[0129] In terms of constructing multiple candidate document subsets according to multiple document clustering clusters, the construction module 403 is specifically used for:

[0130] Construct candidate document subsets according to the following steps until the number of constructed candidate document subsets is equal to the preset value:

[0131] Randomly select one candidate document from each document clustering cluster, and construct a candidate document subset based on the randomly selected multiple candidate documents.

[0132] In some embodiments, in terms of performing preliminary Q&A processing on the candidate document subset and the user question concurrently through multiple lightweight generation models to obtain multiple candidate Q&A responses and the confidence results of each candidate Q&A response, the drafting module 404 is specifically used for:

[0133] Construct multiple interactive text instructions applicable to the lightweight generation model according to the user question and each candidate document subset;

[0134] Input multiple text interactive instructions into different lightweight generation models concurrently for preliminary Q&A processing, and output multiple candidate Q&A responses and the confidence results of each candidate Q&A response;

[0135] Among them, the candidate Q&A response includes the candidate answer corresponding to the user's question and the reasoning chain for obtaining the candidate answer; the confidence result of the candidate Q&A response includes the confidence of the candidate answer and the confidence of the reasoning chain of the candidate answer.

[0136] In some embodiments, in terms of sequentially performing self-consistency verification on each candidate Q&A response through a large model to obtain the self-consistency verification result corresponding to each candidate Q&A response, the verification module 405 is specifically configured to:

[0137] For any candidate Q&A response, construct a text interaction instruction applicable to the large model according to the candidate Q&A response and the user's question;

[0138] Based on the text interaction instruction and the large model, determine the output result and split the output result into independent tokens;

[0139] For any token that makes up the output result, if the token belongs to the candidate answer in the candidate Q&A response, calculate the conditional probability of each of the other tokens before the token in the output result, and take the product of the conditional probabilities of each of the other tokens as the first type of score; or,

[0140] For any token that makes up the output result, if the token belongs to the reasoning chain in the candidate Q&A response, calculate the conditional probability of each of the other tokens before the token in the output result, and take the product of the conditional probabilities of each of the other tokens as the second type of score;

[0141] According to the first type of score and the second type of score, determine the self-consistency verification result corresponding to the candidate Q&A response.

[0142] In some embodiments, in terms of sequentially performing rationality verification on each candidate Q&A response through a large model to obtain the rationality verification result corresponding to each candidate Q&A response, the verification module 405 is specifically configured to:

[0143] For any candidate Q&A response, construct a text interaction instruction applicable to the large model according to the candidate Q&A response and the user's question;

[0144] Based on the text interaction instruction and the large model, determine the output result and split the output result into independent tokens;

[0145] If the output result includes a pre-determined target token, determine the conditional probability of each of the other tokens before the target token when generating the target token;

[0146] According to the conditional probabilities of each of the other tokens, determine the rationality verification result corresponding to the candidate Q&A response.

[0147] In some embodiments, in determining the final Q&A response corresponding to the user's question based on the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response, the output module 406 is specifically configured to:

[0148] Score each candidate Q&A response according to the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response to obtain the score of each candidate question response;

[0149] Sort the candidate Q&A responses according to the scores of each candidate Q&A response to obtain a sorting result;

[0150] According to the sorting result, use the candidate Q&A response with the highest score as the final Q&A response corresponding to the user's question.

[0151] In some embodiments, in obtaining multiple candidate documents retrieved based on the user's question, the obtaining module 401 is specifically configured to:

[0152] Perform document retrieval on the user's question through a preset enhanced retrieval system, and use the retrieved documents that match the user's question as candidate documents.

[0153] Each module in the above-mentioned Q&A device based on retrieval enhancement can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0154] In some embodiments, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to implementing the Q&A method based on retrieval enhancement. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements the steps in the above-mentioned Q&A method based on retrieval enhancement.

[0155] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structural diagram may be as shown in Figure 6 . The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the steps in the above-mentioned question-answering method based on retrieval enhancement. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen may be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0156] Those skilled in the art can understand that Figure 5 or Figure 6 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0157] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps in the above-mentioned method embodiments.

[0158] In some embodiments, as shown in Figure 7 , an internal structural diagram of a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it implements the steps in the above-mentioned method embodiments.

[0159] In some embodiments, a computer program product is provided. The computer program product includes a computer program which, when executed by a processor, implements the steps in the above method embodiments.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0161] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., and are not limited thereto.

[0162] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0163] The above embodiments only express several implementation manners of the present application, and their descriptions are relatively specific and detailed. However, it should not be construed as a limitation on the scope of the patent of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A retrieval-enhanced question answering method, characterized in that, Including: Obtain multiple candidate documents retrieved based on a user question, and perform clustering processing on the multiple candidate documents to obtain multiple document clustering clusters; Construct multiple candidate document subsets according to the multiple document clustering clusters; wherein, the candidate documents included in the candidate document subset come from different document clustering clusters; Through multiple lightweight generation models, concurrently perform preliminary question-and-answer processing based on the candidate document subset and the user question to obtain multiple candidate question-and-answer responses and the confidence result of each candidate question-and-answer response; Verify the self-consistency and rationality of each candidate question-and-answer response in sequence through a large model to obtain the self-consistency verification result and the rationality verification result corresponding to each candidate question-and-answer response; Determine the final question-and-answer response corresponding to the user question according to the confidence result, self-consistency verification result, and rationality verification result of each candidate question-and-answer response.

2. The method according to claim 1, wherein The performing clustering processing on the multiple candidate documents to obtain multiple document clustering clusters includes: Input the multiple candidate documents into a preset embedding model for processing, and output multiple candidate documents represented by high-dimensional vectors; Perform clustering processing on the multiple candidate documents represented by high-dimensional vectors through a preset clustering method to obtain multiple document clustering clusters; wherein, the preset clustering method is partitioning-based clustering or density-based clustering; The constructing multiple candidate document subsets according to the multiple document clustering clusters includes: Construct candidate document subsets according to the following steps until the number of the constructed candidate document subsets is equal to a preset value: Randomly extract one candidate document from each document clustering cluster, and construct one candidate document subset based on the multiple randomly extracted candidate documents.

3. The method according to claim 1, characterized in that, The performing preliminary question-and-answer processing on the candidate document subset and the user question concurrently through multiple lightweight generation models to obtain multiple candidate question-and-answer responses and the confidence result of each candidate question-and-answer response includes: Construct multiple interactive text instructions applicable to the lightweight generation model according to the user question and each candidate document subset; Concurrently input the multiple text interactive instructions into different lightweight generation models for preliminary question-and-answer processing, and output multiple candidate question-and-answer responses and the confidence result of each candidate question-and-answer response; Wherein, the candidate question-and-answer response includes the candidate answer corresponding to the user question and the reasoning chain for obtaining the candidate answer; the confidence result of the candidate question-and-answer response includes the confidence of the candidate answer and the confidence of the reasoning chain of the candidate answer.

4. The method according to claim 3, characterized in that The verifying the self-consistency of each candidate question-and-answer response in sequence through a large model to obtain the self-consistency verification result corresponding to each candidate question-and-answer response includes: For any candidate question-and-answer response, construct a text interactive instruction applicable to the large model according to the candidate question-and-answer response and the user question; Based on the text interactive instruction and the large model, determine the output result, and split the output result into independent tokens; For any token that makes up the output result, if the token belongs to the candidate answer in the candidate Q&A response, calculate the conditional probability of each of the other tokens in the output result that are before the token, and use the product of the conditional probabilities of each of the other tokens as the first type of score; or, For any token that makes up the output result, if the token belongs to the reasoning chain in the candidate Q&A response, calculate the conditional probability of each of the other tokens in the output result that are before the token, and use the product of the conditional probabilities of each of the other tokens as the second type of score; Determine the self-consistency verification result corresponding to the candidate Q&A response according to the first type of score and the second type of score.

5. The method according to claim 1, wherein The step of sequentially performing rationality verification on each candidate Q&A response through a large model to obtain the rationality verification result corresponding to each candidate Q&A response respectively includes: For any candidate Q&A response, construct a text interaction instruction applicable to the large model according to the candidate Q&A response and the user question Based on the text interaction instruction and the large model, determine the output result, and split the output result into independent tokens; If the output result includes a pre-determined target token, determine the conditional probability of each of the other tokens in the output result that are before the target token when generating the target token; Determine the rationality verification result corresponding to the candidate Q&A response according to the conditional probabilities of each of the other tokens.

6. The method according to claim 1, characterized in that, The step of determining the final Q&A response corresponding to the user question according to the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response includes: Score each candidate Q&A response according to the confidence result, self-consistency verification result, and rationality verification result of each candidate Q&A response to obtain the score of each candidate question response; Sort the candidate Q&A responses according to the scores of each candidate Q&A response to obtain a sorting result; According to the sorting result, use the candidate Q&A response with the highest score as the final Q&A response corresponding to the user question.

7. The method according to claim 1, wherein The step of obtaining multiple candidate documents retrieved based on the user question includes: Perform document retrieval on the user question through a preset enhanced retrieval system, and use the retrieved documents that match the user question as candidate documents.

8. A question answering device based on retrieval enhancement, characterized in that, including: An obtaining module, configured to obtain multiple candidate documents retrieved based on the user question; A clustering module, configured to perform clustering processing on the multiple candidate documents to obtain multiple document clustering clusters; A construction module, configured to construct multiple candidate document subsets according to the multiple document clustering clusters; wherein, the candidate documents included in the candidate document subset come from different document clustering clusters; A drafting module, configured to concurrently perform preliminary Q&A processing on the candidate document subset and the user question through multiple lightweight generation models to obtain multiple candidate Q&A responses and the confidence result of each candidate Q&A response; A verification module, configured to sequentially perform self-consistency and rationality verification on each of the candidate Q&A responses through a large model, to obtain a self-consistency verification result and a rationality verification result respectively corresponding to each of the candidate Q&A responses; An output module, configured to determine a final Q&A response corresponding to the user question according to the confidence result, the self-consistency verification result, and the rationality verification result of each of the candidate Q&A responses.

9. A computer device, the computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images