Context answer generation method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of computers, and discloses a context answer generation method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining a retrieval result according to a to-be-replied question and a retrieval model, the retrieval type model is used for obtaining first text information corresponding to the to-be-replied question in the corpus and generating a retrieval result containing the first text information; obtaining a generation result according to the to-be-replied question and the generative model; according to a preset evaluation index, generating a first evaluation score of the retrieval result and a second evaluation score of the generation result, if the first evaluation score is greater than the second evaluation score, taking the retrieval result as a target result, otherwise, taking the generation result as the target result; and inputting the target result into a preset generator, and generating a context answer corresponding to the to-be-replied question. The problems that retrieval limitation is large, a retrieval result easily comprises misleading information, and the accuracy of generated context answers is affected are solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a context answer generation method, device, computer equipment and storage medium. Background Art

[0002] The Retrieval-Augmented Generation (RAG) model combines language generation models and information retrieval technology. When the RAG model needs to generate text or answer questions, it will first retrieve relevant information from a large document collection, and then use the retrieved information to guide the generation of contextual answers, thereby improving the quality and accuracy of the prediction. However, if the retrieval results of the RAG model in the retrieval stage are inaccurate, the contextual answers generated in the subsequent generation stage will also be inaccurate. The current retrieval stage generally adopts a single retrieval method, and performs semantic retrieval in the document collection according to the retrieval requirements to obtain retrieval results. This cannot effectively improve the accuracy of the retrieval results and is prone to include misleading information.

[0003] Therefore, the relevant technology has the problem that the retrieval limitations are large, the retrieval results are prone to include misleading information, and the accuracy of the generated contextual answers is affected. Summary of the invention

[0004] In view of this, the present invention provides a context answer generation method, apparatus, computer device and storage medium to solve the problem that the retrieval is relatively limited and the retrieval results are prone to include misleading information, which affects the accuracy of the generated context answer.

[0005] In a first aspect, the present invention provides a context answer generation method, comprising:

[0006] Obtaining a search result according to the question to be answered and the search formula model, wherein the search formula model is used to obtain first text information corresponding to the question to be answered in the corpus and generate a search result including the first text information;

[0007] Obtaining a generation result according to the question to be answered and the generative model, wherein the generative model is used to generate second text information corresponding to the question to be answered and generate a generation result including the second text information;

[0008] According to the preset evaluation index, a first evaluation score of the search result and a second evaluation score of the generated result are generated, if the first evaluation score is greater than the second evaluation score, the search result is used as the target result, otherwise, the generated result is used as the target result, wherein the preset evaluation index is used to determine the probability that the search result and the generated result are the target result;

[0009] The target result is input into a preset generator to generate a context answer corresponding to the question to be answered, wherein the preset generator is used to construct and generate a context answer based on the target result.

[0010] The context answer generation method provided in this embodiment is designed with a retrieval model and a generative model in the retrieval stage. The retrieval results corresponding to the questions to be answered are obtained according to the retrieval model, and the generative results corresponding to the questions to be answered are generated according to the generative model. The retrieval results and the generative results are evaluated according to the preset evaluation indicators, so as to facilitate the acquisition of the optimal target results, and greatly improve the accuracy of the results in the retrieval stage. The context answer is generated according to the target result using the preset generator, and the generated context information has a high retrieval accuracy, and the generated results are more stable and accurate. The problem that the retrieval is relatively limited and the retrieval results are prone to include misleading information, which affects the accuracy of the generated context answer is solved.

[0011] In some optional implementations, the search results are obtained according to the question to be answered and the search formula model, including:

[0012] Input the question to be answered into the search model to obtain the search results;

[0013] The retrieval model is used to perform feature extraction operations on the questions to be answered, obtain question features, match the question features with the question and answer texts in the corpus, obtain intermediate text information containing a first preset number of question and answer texts, and determine a second preset number of target question and answer texts from the first preset number of question and answer texts in the intermediate text information according to preset indicators, use the target question and answer texts as the first text information, and generate a retrieval result containing the first text information, wherein the second preset number is less than or equal to the first preset number.

[0014] In this embodiment, the question to be answered is input into the retrieval model, and the retrieval model is used to determine the target question and answer text corresponding to the question to be answered from the corpus to obtain the retrieval result. The preset generator generates the context answer based on the retrieval result to improve the quality and accuracy of the context answer.

[0015] In some optional implementations, generating a first evaluation score of a search result and a second evaluation score of a result according to a preset evaluation index includes:

[0016] Determine relevant text information related to the question to be answered in the first text information of the search result, and determine a first quantity of the relevant text information;

[0017] Obtain relevant question and answer texts related to the question to be answered in the corpus, and determine a second number of relevant question and answer texts;

[0018] Dividing the first number by the second number to obtain the accuracy of the search result;

[0019] Dividing the first number by the second preset number to obtain a recall rate of the search result;

[0020] According to the precision, the recall and the preset formula, a harmonic mean corresponding to the search result is obtained, and a first evaluation score is obtained according to the harmonic mean;

[0021] The preset formula satisfies:

[0022]

[0023] Among them, F is the harmonic mean, α is the preset parameter, P is the precision, and R is the recall rate;

[0024] Determining the perplexity and entropy of the resulting second text message, and determining an average reply length of the second text message;

[0025] Determine the total number of words, the third number of unigrams, and the fourth number of bigrams in the second text information of the generated result;

[0026] Dividing the third number by the total number of words to obtain a first ratio of unigrams, dividing the fourth number by the total number of words to obtain a second ratio of bigrams, and obtaining a response diversity index of the generated result according to the first ratio and the second ratio;

[0027] A second evaluation score is obtained based on perplexity, entropy, response diversity index, average response length, and preset weights.

[0028] In this embodiment, the evaluation model is used to evaluate the retrieval results and the generated results according to the preset evaluation indicators to obtain the first evaluation score of the retrieval results and the second evaluation score of the generated results. The evaluation score facilitates the acquisition of the optimal target result, greatly improving the accuracy of the results in the retrieval stage.

[0029] In some optional implementations, generating a first evaluation score of a search result and a second evaluation score of a result according to a preset evaluation index includes:

[0030] Inputting the search result into an evaluation model to obtain a first evaluation score, wherein the evaluation model is used to determine the probability that the search result is a target result according to a preset evaluation index, and the evaluation model is obtained by training an initial evaluation model according to the first training sample, and the initial evaluation model is constructed according to the preset evaluation index;

[0031] The generated result is input into the evaluation model to obtain a second evaluation score, wherein the evaluation model is used to determine the probability that the generated result is the target result according to a preset evaluation indicator.

[0032] In this embodiment, an evaluation model is constructed using preset evaluation indicators, and the evaluation model is used to evaluate the search results and generated results according to the preset evaluation indicators, and the corresponding evaluation scores are output. The evaluation scores facilitate the acquisition of the optimal target results, greatly improving the accuracy of the results in the search stage.

[0033] In some optional implementations, before obtaining a generation result according to the question to be answered and the generative model, the method further includes:

[0034] Build an initial model according to the preset algorithm;

[0035] A second training sample is obtained, and the initial model is trained according to the second training sample to obtain a generative model.

[0036] In this embodiment, an initial model is constructed according to a preset algorithm; and the initial model is trained according to the second training sample to obtain a generative model. The generative model generates a generated result that matches the question to be answered, and the generated result can make up for the problem of missing information or stiff expression in the search results.

[0037] In some optional implementations, a generation result is obtained according to the question to be answered and the generative model, including:

[0038] Input the question to be answered into the generative model to obtain the generated result;

[0039] The generative model includes an encoder and a decoder. The encoder of the generative model is used to determine the semantic features of the question to be answered, and the decoder of the generative model is used to generate the second text information according to the semantic features and generate a generation result containing the second text information.

[0040] In some optional embodiments, the method further comprises:

[0041] Acquire a newly added evaluation indicator, wherein the newly added evaluation indicator includes: the accuracy of the first third preset number of first text information in the search results and / or the average accuracy of the search results, the third preset number being less than or equal to the second preset number;

[0042] Update the preset evaluation indicators according to the newly added evaluation indicators;

[0043] The average accuracy satisfies:

[0044]

[0045] Among them, MAP is the average accuracy, AP is the average precision of the first text information, Q R is the total number of first text messages, and q is the sequence number.

[0046] In this embodiment, new evaluation indicators are added to the preset evaluation indicators to achieve further accuracy of the evaluation indicators of the retrieval results and the generated results, further improve the reliability of the evaluation, and then determine more accurate target results, ultimately improving the accuracy of the constructed contextual answers.

[0047] In a second aspect, the present invention provides a context answer generating device, comprising:

[0048] A retrieval result obtaining module, used to obtain retrieval results according to the question to be answered and the retrieval formula model, wherein the retrieval formula model is used to obtain the first text information corresponding to the question to be answered in the corpus and generate a retrieval result containing the first text information;

[0049] A generation result obtaining module, used for obtaining a generation result according to the question to be answered and the generative model, wherein the generative model is used for generating second text information corresponding to the question to be answered and generating a generation result including the second text information;

[0050] An evaluation module, used to generate a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index, and if the first evaluation score is greater than the second evaluation score, the search result is used as the target result, otherwise, the generated result is used as the target result, wherein the preset evaluation index is used to determine the probability that the search result and the generated result are the target result;

[0051] A generation module is used to input the target result into a preset generator to generate a context answer corresponding to the question to be answered, wherein the preset generator is used to construct and generate the context answer based on the target result.

[0052] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the context answer generation method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0053] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the context answer generation method of the first aspect or any corresponding embodiment thereof.

[0054] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the context answer generation method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 is a flow chart of a context answer generation method according to an embodiment of the present invention;

[0057] Figure 2 is a flow chart of an efficient retrieval and generation collaborative optimization method according to an embodiment of the present invention;

[0058] Figure 3 is a structural block diagram of a context answer generating device according to an embodiment of the present invention;

[0059] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0061] The existing RAG model lacks coordinated optimization in the retrieval stage and the generation stage, resulting in limited overall performance of the RAG model. Especially in the retrieval stage, inaccurate retrieval results will lead to inaccurate context information generated in the subsequent generation stage. Existing retrieval is generally a single retrieval method, which makes the retrieval limited and easy to generate misleading information, such as: obtaining the user's retrieval needs; according to the retrieval needs, obtaining multiple retrieval results, where the retrieval results include multiple text information and multiple historical conversations; according to each retrieval result and retrieval requirement, obtaining the target semantic distance between each retrieval result and the retrieval requirement; sorting each target semantic distance to obtain an array of retrieval requirements, and byte processing each element in the array in turn to obtain the context information of the large language model. Although the above technical solution can reduce the crash of the large language model, it cannot effectively improve the accuracy of the retrieval results.

[0062] Based on the above content, an embodiment of the present invention provides a context answer generation method, in which a retrieval model and a generative model are designed in the retrieval stage, and retrieval model evaluation indicators and generative model evaluation indicators are respectively designed to evaluate the results obtained by the two, so as to facilitate obtaining the optimal result, greatly improve the accuracy of the retrieval result, and then the context answer is constructed and generated according to the optimal result through the generator. It is also possible to expand and add new evaluation indicators to the retrieval model evaluation indicators, so as to further improve the accuracy of the retrieval evaluation indicators, further improve the reliability of the evaluation results, and then determine more accurate retrieval results, and finally improve the accuracy of the constructed context answers. It solves the problem that the existing RAG model has a single retrieval method in the retrieval stage, which makes the retrieval limited and easy to generate misleading information. The generated context information has a higher retrieval accuracy, and the generated results are more stable and accurate.

[0063] According to an embodiment of the present invention, a context answer generation embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer device with data processing capabilities, such as a computer, a server, etc., and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0064] In this embodiment, a context answer generation method is provided. Figure 1 is a flowchart of a context answer generation method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0065] Step S101, obtaining a search result according to the question to be answered and a search formula model, wherein the search formula model is used to obtain first text information corresponding to the question to be answered in a corpus and generate a search result containing the first text information.

[0066] Specifically, this embodiment first constructs a dialogue corpus consisting of a large number of inter-answer pairs. The corpus can use an open domain or a closed domain.

[0067] The question to be answered is input into the search model, which extracts features from the input question to be answered, and matches a small number of question-answer pairs in the corpus as candidate sets based on the extracted features. Finally, the best answer, i.e., the first text information, is selected from the candidate set using appropriate indicators, and a search result containing the first text information is generated. The question to be answered can be, for example: sentiment analysis questions, text classification questions, language acceptability questions (whether the sentence is reasonable, whether the grammar is smooth), etc.

[0068] Step S102, obtaining a generation result according to the question to be answered and the generative model, wherein the generative model is used to generate second text information corresponding to the question to be answered and generate a generation result including the second text information.

[0069] Specifically, the BERT algorithm is used to build a model and the model is trained on a large scale to obtain a generative model. During the large-scale prediction training process, the generative model learns the patterns and characteristics of human conversations, so that the generative model can generate responses that match the subject, making up for the information missing or stiff expression problems that may exist in the retrieval-based responses.

[0070] The question to be answered is input into the generative model, and the generative model generates second text information corresponding to the question to be answered, and generates a generation result including the second text information.

[0071] Step S103, based on preset evaluation indicators, generate a first evaluation score for the retrieval result and a second evaluation score for the generated result. If the first evaluation score is greater than the second evaluation score, the retrieval result is used as the target result, otherwise, the generated result is used as the target result, wherein the preset evaluation indicators are used to determine the probability that the retrieval result and the generated result are the target results.

[0072] Specifically, the preset evaluation indicators include indicators for evaluating retrieval results and indicators for evaluating generation results, where indicators for evaluating retrieval results include, for example, precision (P), recall (R), the harmonic mean of recall and precision, F, etc. Indicators for evaluating generation results include, for example, perplexity, entropy, response diversity indicators, average response length, etc. The evaluation indicators of the retrieval model focus on the evaluation of recall and precision, while the evaluation indicators of the generation model focus on the length of the generated response, and it is believed that the model that generates long sentences is of relatively higher quality. Based on this, in the specific evaluation, the retrieval results of the retrieval model or the generation results of the generation model can be selected according to the characteristics of the required response content.

[0073] According to the preset evaluation index, the retrieval results of the retrieval model and the generation results of the generative model are evaluated to determine the similarity or quality of the retrieval results and the generation results with the best response, and then determine the probability that the retrieval results and the generation results are the target results, and generate a first evaluation score for the retrieval results and a second evaluation score for the generation results to reflect the evaluation results. The target result is determined based on the first evaluation score and the second evaluation score. If the first evaluation score is greater than the second evaluation score, it means that the retrieval result of the retrieval model is more effective, and the retrieval result is used as the target result. Otherwise, it means that the generation result of the generative model is more effective, and the generation result is used as the target result.

[0074] Step S104, inputting the target result into a preset generator to generate a context answer corresponding to the question to be answered, wherein the preset generator is used to construct and generate the context answer according to the target result.

[0075] Specifically, a preset generator is a system or model that can automatically generate outputs based on specific inputs (such as target results). The target result is input into the preset generator, which organizes all contextual information related to the question to be answered, and the generator constructs and generates the contextual answer corresponding to the question to be answered.

[0076] The context answer generation method provided in this embodiment is designed with a retrieval model and a generative model in the retrieval stage. The retrieval results corresponding to the questions to be answered are obtained according to the retrieval model, and the generative results corresponding to the questions to be answered are generated according to the generative model. The retrieval results and the generative results are evaluated according to the preset evaluation indicators, so as to facilitate the acquisition of the optimal target results, and greatly improve the accuracy of the results in the retrieval stage. The context answer is generated according to the target result using the preset generator, and the generated context information has a high retrieval accuracy, and the generated results are more stable and accurate. The problem that the retrieval is relatively limited and the retrieval results are prone to include misleading information, which affects the accuracy of the generated context answer is solved.

[0077] In some optional implementations, the search results are obtained according to the question to be answered and the search formula model, including:

[0078] Input the question to be answered into the search model to obtain the search results;

[0079] The retrieval model is used to perform feature extraction operations on the questions to be answered, obtain question features, match the question features with the question and answer texts in the corpus, obtain intermediate text information containing a first preset number of question and answer texts, and determine a second preset number of target question and answer texts from the first preset number of question and answer texts in the intermediate text information according to preset indicators, use the target question and answer texts as the first text information, and generate a retrieval result containing the first text information, wherein the second preset number is less than or equal to the first preset number.

[0080] Specifically, the question to be answered is input into the retrieval model, and the retrieval model performs feature extraction on the input question to be answered to extract the question features. The prediction library is composed of a large number of question and answer texts, such as dialogue information. The retrieval model matches the question features with the question and answer texts in the corpus, and determines the successfully matched text information, that is, the intermediate text information. The intermediate text information has a first preset number, and the first preset number represents a small amount, such as 5, 8, 10, etc. The specific value is set according to actual needs. The retrieval model can also use deep learning technology NLP (Natural Language Processing) to conduct question and answer dialogue training in the corpus.

[0081] Preset indicators include semantic similarity, relevance, fluency, information richness, etc. According to the preset indicators, a second preset number of target question and answer texts with the best preset indicators are determined from the first preset number of question and answer texts in the intermediate text information, and the target question and answer texts are used as the first text information. The second preset number is a value less than or equal to the first preset number.

[0082] In this embodiment, the question to be answered is input into the retrieval model, and the retrieval model is used to determine the target question and answer text corresponding to the question to be answered from the corpus to obtain the retrieval result. The preset generator generates the context answer based on the retrieval result to improve the quality and accuracy of the context answer.

[0083] In some optional implementations, generating a first evaluation score of a search result and a second evaluation score of a result according to a preset evaluation index includes:

[0084] Determine relevant text information related to the question to be answered in the first text information of the search result, and determine a first quantity of the relevant text information;

[0085] Obtain relevant question and answer texts related to the question to be answered in the corpus, and determine a second number of relevant question and answer texts;

[0086] Dividing the first number by the second number to obtain the accuracy of the search result;

[0087] Dividing the first number by the second preset number to obtain a recall rate of the search result;

[0088] According to the precision, the recall and the preset formula, a harmonic mean corresponding to the search result is obtained, and a first evaluation score is obtained according to the harmonic mean;

[0089] The preset formula satisfies:

[0090]

[0091] Among them, F is the harmonic mean, α is the preset parameter, P is the precision, and R is the recall rate;

[0092] Determining the perplexity and entropy of the resulting second text message, and determining an average reply length of the second text message;

[0093] Determine the total number of words, the third number of unigrams, and the fourth number of bigrams in the second text information of the generated result;

[0094] Dividing the third number by the total number of words to obtain a first ratio of unigrams, dividing the fourth number by the total number of words to obtain a second ratio of bigrams, and obtaining a response diversity index of the generated result according to the first ratio and the second ratio;

[0095] A second evaluation score is obtained based on perplexity, entropy, response diversity index, average response length, and preset weights.

[0096] Specifically, the relevant text information related to the question to be answered in the first text information of the search result is determined, and the first number of the relevant text information is determined. When determining the relevant text information, a threshold can be set, and the first text information with a score higher than the threshold is determined as the relevant text information related to the question to be answered. By counting the first number of relevant text information with a score higher than the threshold, the number of relevant replies retrieved by the system can be obtained. Similarly, the question and answer texts in the corpus with a score higher than the threshold are determined as the relevant question and answer texts related to the question to be answered, and the second number of the relevant question and answer texts is determined, and the total number of all relevant replies in the system can be obtained.

[0097] Precision (abbreviated as P) is used to check what proportion of the responses given by the system are truly relevant. P = relevant responses retrieved by the system / total number of all relevant responses of the system, that is, P = first number / second number. Therefore, the first number is divided by the second number to get the accuracy P of the retrieval result.

[0098] According to the above embodiment, there are a second preset number of first text messages in the search results, so the total number of all the replies retrieved by the system is equal to the second preset number. The recall rate (Recall, abbreviated as R) is used to check the system's ability to find all relevant replies, R = the number of relevant replies retrieved by the system / the total number of all the replies retrieved by the system, that is, R = the first number / the second preset number, so the recall rate R of the search result is obtained by dividing the first number by the second preset number.

[0099] Input the accuracy rate P and the recall rate R into the above preset formula to calculate the harmonic mean F corresponding to the retrieval result. The F value is the harmonic mean of the recall rate and the accuracy rate, which is used to comprehensively evaluate the performance of the retrieval system. The higher the F value, the better the retrieval system performs in terms of both the accuracy rate and the recall rate. In addition, the recall rate examines the ability of the system to find all relevant responses, while the accuracy rate examines the ability of the system to find the correct responses. The two complement each other and comprehensively reflect the system performance from two different aspects. When a = 1, it means that the system achieves a better balance between the precision rate and the recall rate, that is, the F1 value, which is the harmonic mean of the recall rate and the accuracy rate, as shown in formula (1).

[0100]

[0101] Obtain the first evaluation score according to the harmonic mean. For example: directly use the harmonic mean as the first evaluation score, adjust the harmonic mean according to actual needs as the first evaluation score, use the ratio of the harmonic mean to the harmonic mean of the artificially generated standard result as the first evaluation score, etc.

[0102] In a language model, perplexity is usually used to measure the probability of a sentence appearing and is often used in dialogue generation evaluation to evaluate the language quality of the generated response sentences. The basic idea is that the higher the quality of the generated response language, the smaller the perplexity, the closer it is to normal human speech, and the better the model. The disadvantage of the perplexity metric is that it cannot evaluate the relevance of the response to the previous context in a dialogue. Entropy can be used to measure the information content of the generated response. The average length is used to measure the dialogue generation effect, and it is considered that a model that generates long sentences has a relatively higher quality.

[0103] Determine the perplexity and entropy of the second text information of the generated result, and determine the average response length of the second text information. The calculation steps of perplexity can include: calculating the conditional probability. For each word (x_t) in the second text information, calculate its conditional probability (p(x_t|x_{<t})) given the prefix (x_{<t}). Take the logarithm of the conditional probability of each word to obtain the log probability. Calculate the average of the log probabilities of all words as the average log probability. The final perplexity is: exp(average log probability). The calculation steps of entropy can include: for each word in the second text information, calculate its conditional probability (p(x_n|x_{<n})) given the prefix (x_{<n}). For each word (x_n), calculate the entropy of its conditional probability distribution.

[0104] Unigrams refer to a collection of single words or symbols. In language models, unigrams are usually used to count the frequency of each word in a text. Bigrams refer to a combination of two consecutive words or symbols. In language models, bigrams are usually used to capture the order relationship between words and calculate the probability of word pairs. Determine the total number of words, the third number of unigrams, and the fourth number of bigrams in the second text information of the generated result.

[0105] The third number is divided by the total number of words to obtain the first proportion of unigrams (Distinct-1), and the fourth number is divided by the total number of words to obtain the second proportion of bigrams (Distinct-2). According to the first proportion and the second proportion, the reply diversity index of the generated result is obtained, the reply diversity index Distinct-1&2. To address the problem of universal replies in dialogue systems, the diversity of replies is measured by calculating the proportion of unigrams (the richness of vocabulary in generated replies) and bigrams (the influence of word order and phrases on reply diversity) in the generated replies.

[0106] The second evaluation score is obtained according to the perplexity, entropy, reply diversity index, average reply length and preset weights. For example, the preset weights of perplexity, entropy, reply diversity index and average reply length are 0.2, 0.2, 0.3 and 0.3 respectively. The preset weights can be set according to actual needs, but the sum of the preset weights of each item must be 1. In the process of determining the second evaluation score, the perplexity, entropy, reply diversity index and average reply length of the standard result generated by the user can be obtained first, and the calculated perplexity, entropy, reply diversity index and average reply length are divided by the perplexity, entropy, reply diversity index and average reply length of the standard result respectively, and the four results are multiplied by the corresponding preset weights, and the final result is used as the second evaluation score.

[0107] In this embodiment, the evaluation model is used to evaluate the retrieval results and the generated results according to the preset evaluation indicators to obtain the first evaluation score of the retrieval results and the second evaluation score of the generated results. The evaluation score facilitates the acquisition of the optimal target result, greatly improving the accuracy of the results in the retrieval stage.

[0108] In some optional implementations, generating a first evaluation score of a search result and a second evaluation score of a result according to a preset evaluation index includes:

[0109] Inputting the search result into an evaluation model to obtain a first evaluation score, wherein the evaluation model is used to determine the probability that the search result is a target result according to a preset evaluation index, and the evaluation model is obtained by training an initial evaluation model according to the first training sample, and the initial evaluation model is constructed according to the preset evaluation index;

[0110] The generated result is input into the evaluation model to obtain a second evaluation score, wherein the evaluation model is used to determine the probability that the generated result is the target result according to a preset evaluation indicator.

[0111] Specifically, this embodiment uses a neural network to construct an initial evaluation model according to the above-mentioned preset evaluation indicators. The neural network is, for example, a recurrent neural network (RNN). The recurrent neural network (RNN) can be used for text quality evaluation. The initial evaluation model is trained according to the first training sample to obtain an evaluation model. The first training sample includes: an artificially generated standard result and a corresponding evaluation score; an erroneous result and a corresponding evaluation score. The step of creating an evaluation model according to the preset evaluation indicator may include: collecting the first training sample, preprocessing the first training sample, including word segmentation, removing stop words, converting to lowercase, etc. Feature extraction, converting the text into a numerical form, and each word is represented by a vector. A common method is to use word embeddings, or the encoder of the RNN itself can be used to extract features. Model construction, using one or more RNN layers to process the input word vector sequence, RNN can capture the sequential information and contextual dependency in the text. The type of RNN can be selected from standard RNN, LSTM (long short-term memory) or GRU (gated recurrent unit), etc. 4. Training model, using the labeled first training sample to train the RNN model. The goal is to adjust the model parameters to minimize the difference between the predicted quality and the actual quality. Choose an appropriate loss function, such as mean squared error (MSE), cross entropy loss, etc.

[0112] The retrieval results and the generated results are input into the evaluation model, and the evaluation model evaluates the retrieval results and the generated results according to the preset evaluation indicators, and outputs the first evaluation score of the retrieval results and the second evaluation score of the generated results. The evaluation score reflects the similarity or quality of the result with the best result.

[0113] In this embodiment, an evaluation model is constructed using preset evaluation indicators, and the evaluation model is used to evaluate the search results and generated results according to the preset evaluation indicators, and the corresponding evaluation scores are output. The evaluation scores facilitate the acquisition of the optimal target results, greatly improving the accuracy of the results in the search stage.

[0114] In some optional implementations, before obtaining a generation result according to the question to be answered and the generative model, the method further includes:

[0115] Build an initial model according to the preset algorithm;

[0116] A second training sample is obtained, and the initial model is trained according to the second training sample to obtain a generative model.

[0117] Specifically, the preset algorithm is, for example, the BERT algorithm, which is a deep learning technology based on the Transformer architecture. The BERT algorithm is used to build the initial model. The second training sample is obtained, for example, a large-scale corpus, such as large-scale unlabeled text data collected including encyclopedias, news, web pages, questions and answers, conversations, etc. The initial model is trained on a large-scale corpus based on the second training sample, so that the model learns the patterns and characteristics of human conversations, so that the model can generate results that match the questions to be answered by itself, and make up for the problems of missing information or stiff expressions that may exist in retrieval-based replies;

[0118] In addition, the steps of creating a generative model may include: data preparation, collecting the second training sample, and preprocessing it, including word segmentation, removing stop words, adding special tags (such as [CLS], [SEP]), etc. Use a deep learning framework (such as TensorFlow, PyTorch) to build the BERT architecture as the initial model, including components such as a multi-layer Transformer encoder, a self-attention mechanism, and a positional encoding. Perform masked language model (MLM) and next sentence prediction (NSP) tasks on the second training sample to pre-train the initial model, and adjust the model parameters through back propagation to enable it to learn a general language representation. Fine-tune the initial model, add a task-specific output layer based on the pre-trained BERT model for a specific NLP task (text generation), and then fine-tune it on the corresponding small-scale annotated dataset to optimize the model's adaptability to specific tasks.

[0119] In this embodiment, an initial model is constructed according to a preset algorithm; and the initial model is trained according to the second training sample to obtain a generative model. The generative model generates a generated result that matches the question to be answered, and the generated result can make up for the problem of missing information or stiff expression in the search results.

[0120] In some optional implementations, a generation result is obtained according to the question to be answered and the generative model, including:

[0121] Input the question to be answered into the generative model to obtain the generated result;

[0122] The generative model includes an encoder and a decoder. The encoder of the generative model is used to determine the semantic features of the question to be answered, and the decoder of the generative model is used to generate the second text information according to the semantic features and generate a generation result containing the second text information.

[0123] Specifically, the generative model of this embodiment includes an encoder and a decoder.

[0124] After the question to be answered is input into the generative model, the encoder of the generative model will learn the semantic features of the question to be answered. Then, the decoder of the generative model is used to generate the second text information according to the semantic features and generate a generation result including the second text information.

[0125] In some optional embodiments, the method further comprises:

[0126] Acquire a newly added evaluation indicator, wherein the newly added evaluation indicator includes: the accuracy of the first third preset number of first text information in the search results and / or the average accuracy of the search results, the third preset number being less than or equal to the second preset number;

[0127] Update the preset evaluation indicators according to the newly added evaluation indicators;

[0128] The average accuracy satisfies:

[0129]

[0130] Among them, MAP is the average accuracy, AP is the average precision of the first text information, Q R is the total number of first text messages, and q is the sequence number.

[0131] Specifically, as the scale of the test set increases and the understanding of the test results deepens, this embodiment can add new evaluation indicators that can more accurately reflect system performance to the preset evaluation indicators. The process includes: obtaining the new evaluation indicators and updating the preset evaluation indicators according to the new evaluation indicators.

[0132] New evaluation indicators may include: the accuracy of the first third preset number of first text information in the search results, the average accuracy of the search results, the accuracy of the first 10 replies in a single result, etc.

[0133] The third preset number is a value less than or equal to the second preset number W. The accuracy of the first third preset number of first text information in the search results indicates how many of the first W first text information are related to the question to be answered, that is, the accuracy of the first W positions of the search results.

[0134] According to the above average accuracy formula, it can be known that AP needs to be calculated to calculate the average accuracy. The method for calculating AP is to directly perform numerical integration on the PR curve (precision-recall curve).

[0135] In this embodiment, new evaluation indicators are added to the preset evaluation indicators to achieve further accuracy of the evaluation indicators of the retrieval results and the generated results, further improve the reliability of the evaluation, and then determine more accurate target results, ultimately improving the accuracy of the constructed contextual answers.

[0136] In some optional implementations, an efficient retrieval and generation collaborative optimization method is provided, which can solve the same technical problems as steps S101 to S104 and produce the same technical effects, such as Figure 2 As shown, the method includes: step S1, constructing a retrieval model. Step S2, constructing a generative model. Step S3, constructing a model evaluation index. Step S4, evaluating the retrieval and generative model algorithms. Step S5, constructing a generated context answer.

[0137] Specifically, the search model and the generative model are used in parallel to evaluate the response design algorithms of the search model and the generative model. If the search response is effective, the search response is used as the target result, otherwise a generative response is returned as the target result. Based on the final target result, the context answer is constructed and generated through the generator. It has higher search accuracy, and the generated results are more stable and accurate.

[0138] In this embodiment, by designing a retrieval model and a generative model in the retrieval stage, and also designing retrieval model evaluation indicators and generative model evaluation indicators respectively for evaluating the retrieval results of the two, it is convenient to obtain the optimal result, and then the construction and generation of context answers are realized through the generator, which greatly improves the accuracy of the retrieval results. It solves the problem that the existing RAG model has a single retrieval method in the retrieval stage, which makes the retrieval limited and easy to generate misleading information.

[0139] In some optional implementations, the specific process of the retrieval model obtaining the retrieval results corresponding to the question to be answered may also include steps A1 to A3.

[0140] Step A1: input the question to be answered into the first search module, and search in the pre-stored database to obtain a first set number of samples to be selected.

[0141] Specifically, after obtaining the question to be answered, based on the question to be answered, a shared index is constructed in a pre-stored database using a pre-built first retriever, and based on the shared index, a first set number of samples to be selected are retrieved from the pre-stored database. The first set number can be 100 or 150. In this stage, the first retriever needs to retrieve k samples to be selected from a large-scale external data set. The classic BM25, BERT or SimCSE retriever is adopted, and the Faiss open source framework is used to build a large-scale index and MIPS is used for retrieval. It should be noted that the first retriever is shared on all tasks. Since the scale of the first retriever used in this stage is relatively small, higher efficiency can be achieved.

[0142] Step A2: input the samples to be selected and the questions to be answered into the second search module, sort the samples to be selected in response to the task prompt, and obtain a second set number of search samples.

[0143] Specifically, after obtaining the first set number of samples to be selected, the second retriever is used to rearrange the k samples retrieved in the previous stage, and d samples are taken out as the final retrieval samples. In this stage, task-specific prompts and pre-built language models (LLMs) such as OPT13b are used to perform task-specific retrieval. After obtaining the first set number of samples to be selected, the task prompt is obtained first. For the k samples to be selected and the question (query) in the question to be answered, the prompt method is first used for transformation. A variety of tasks can be converted into language model tasks through prompts. The prompt method can unify different forms of NKI tasks and obtain task-specific representations. For example: the question to be answered is whether this sentence has positive or negative emotions? Then the pre-built language model will first convert it into an input-output task that needs to complete the sentence, and the output prompt annotation answer is negative. After calculating the sample similarity of each prompt-annotated sample to be selected, the prompt-annotated sample to be selected is rearranged in a preset order according to the sample similarity, and the first second set number of prompt-annotated samples to be selected are selected as retrieval samples. The preset order can be from large to small.

[0144] Step A3, input the search sample into the pre-stored answer generation module to obtain the search result.

[0145] Specifically, after obtaining the search sample, the search sample is input into a pre-stored answer generation module to read the search sample, obtain the reading result and generate the search result according to the reading result. In some embodiments, the answer generation module includes a reader / generator, and specifically, FiD (Fusion in Decoder) can be used as a generator.

[0146] In this implementation, in the first stage, a first retriever that is common to all tasks is used to build a shared index and extract samples to be selected; in the second stage, a prompt-guided pre-trained language model is used to rearrange the samples to be selected and select retrieval samples; finally, the retrieval samples will be used as input to the answer generation module, and the answer generation module will generate retrieval results. This two-stage method can achieve better performance in various NKI tasks while saving considerable computational overhead.

[0147] In this embodiment, a context answer generation device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0148] This embodiment provides a context answer generation device, such as Figure 3 As shown, including:

[0149] A search result obtaining module 301 is used to obtain a search result according to the question to be answered and a search formula model, wherein the search formula model is used to obtain the first text information corresponding to the question to be answered in the corpus and generate a search result containing the first text information;

[0150] A generation result obtaining module 302 is used to obtain a generation result according to the question to be answered and the generative model, wherein the generative model is used to generate the second text information corresponding to the question to be answered and generate a generation result including the second text information;

[0151] An evaluation module 303 is used to generate a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index. If the first evaluation score is greater than the second evaluation score, the search result is used as the target result; otherwise, the generated result is used as the target result, wherein the preset evaluation index is used to determine the probability that the search result and the generated result are the target result;

[0152] The generation module 304 is used to input the target result into a preset generator to generate a context answer corresponding to the question to be answered, wherein the preset generator is used to construct and generate the context answer according to the target result.

[0153] In some optional implementations, the search result obtaining module 301 obtains the search results according to the question to be answered and the search formula model, including:

[0154] Input the question to be answered into the search model to obtain the search results;

[0155] The retrieval model is used to perform feature extraction operations on the questions to be answered, obtain question features, match the question features with the question and answer texts in the corpus, obtain intermediate text information containing a first preset number of question and answer texts, and determine a second preset number of target question and answer texts from the first preset number of question and answer texts in the intermediate text information according to preset indicators, use the target question and answer texts as the first text information, and generate a retrieval result containing the first text information, wherein the second preset number is less than or equal to the first preset number.

[0156] In some optional implementations, the evaluation module 303 generates a first evaluation score for the search result and a second evaluation score for the generated result according to a preset evaluation index, including:

[0157] Determine relevant text information related to the question to be answered in the first text information of the search result, and determine a first quantity of the relevant text information;

[0158] Obtain relevant question and answer texts related to the question to be answered in the corpus, and determine a second number of relevant question and answer texts;

[0159] Dividing the first number by the second number to obtain the accuracy of the search result;

[0160] Dividing the first number by the second preset number to obtain a recall rate of the search result;

[0161] According to the precision, the recall and the preset formula, a harmonic mean corresponding to the search result is obtained, and a first evaluation score is obtained according to the harmonic mean;

[0162] The preset formula satisfies:

[0163]

[0164] Among them, F is the harmonic mean, α is the preset parameter, P is the precision, and R is the recall rate;

[0165] Determining the perplexity and entropy of the resulting second text message, and determining an average reply length of the second text message;

[0166] Determine the total number of words, the third number of unigrams, and the fourth number of bigrams in the second text information of the generated result;

[0167] Dividing the third number by the total number of words to obtain a first ratio of unigrams, dividing the fourth number by the total number of words to obtain a second ratio of bigrams, and obtaining a response diversity index of the generated result according to the first ratio and the second ratio;

[0168] A second evaluation score is obtained based on perplexity, entropy, response diversity index, average response length, and preset weights.

[0169] In some optional implementations, the evaluation module 303 generates a first evaluation score for the search result and a second evaluation score for the generated result according to a preset evaluation index, including:

[0170] Inputting the search result into an evaluation model to obtain a first evaluation score, wherein the evaluation model is used to determine the probability that the search result is a target result according to a preset evaluation index, and the evaluation model is obtained by training an initial evaluation model according to the first training sample, and the initial evaluation model is constructed according to the preset evaluation index;

[0171] The generated result is input into the evaluation model to obtain a second evaluation score, wherein the evaluation model is used to determine the probability that the generated result is the target result according to a preset evaluation indicator.

[0172] In some optional implementations, before obtaining a generation result according to the question to be answered and the generative model, the device is used to:

[0173] Build an initial model according to the preset algorithm;

[0174] A second training sample is obtained, and the initial model is trained according to the second training sample to obtain a generative model.

[0175] In some optional implementations, the generation result obtaining module 302 obtains the generation result according to the question to be answered and the generation model, including:

[0176] Input the question to be answered into the generative model to obtain the generated result;

[0177] The generative model includes an encoder and a decoder. The encoder of the generative model is used to determine the semantic features of the question to be answered, and the decoder of the generative model is used to generate the second text information according to the semantic features and generate a generation result containing the second text information.

[0178] In some optional embodiments, the device is also used for:

[0179] Acquire a newly added evaluation indicator, wherein the newly added evaluation indicator includes: the accuracy of the first third preset number of first text information in the search results and / or the average accuracy of the search results, the third preset number being less than or equal to the second preset number;

[0180] Update the preset evaluation indicators according to the newly added evaluation indicators;

[0181] The average accuracy satisfies:

[0182]

[0183] Among them, MAP is the average accuracy, AP is the average precision of the first text information, Q R is the total number of first text messages, and q is the sequence number.

[0184] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0185] The context answer generation device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0186] The embodiment of the present invention also provides a computer device having the above Figure 3 The context answer generating device shown.

[0187] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0188] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include an integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0189] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0190] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0191] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0192] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0193] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0194] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0195] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined in this application.

Claims

1. A context answer generation method, characterized in that: The method comprises: Obtaining a search result according to the question to be answered and a search formula model, wherein the search formula model is used to obtain first text information corresponding to the question to be answered in a corpus and generate the search result including the first text information; Obtaining a generation result according to the question to be answered and the generative model, wherein the generative model is used to generate second text information corresponding to the question to be answered and generate the generation result including the second text information; According to a preset evaluation index, a first evaluation score of the retrieval result and a second evaluation score of the generated result are generated, if the first evaluation score is greater than the second evaluation score, the retrieval result is used as the target result, otherwise, the generated result is used as the target result, wherein the preset evaluation index is used to determine the probability that the retrieval result and the generated result are the target result; The target result is input into a preset generator to generate a context answer corresponding to the question to be answered, wherein the preset generator is used to construct and generate the context answer based on the target result.

2. The method according to claim 1, characterized in that The step of obtaining the search results according to the question to be answered and the search model includes: Inputting the question to be answered into the search model to obtain the search result; The retrieval model is used to perform feature extraction operations on the question to be answered to obtain question features, match the question features with the question and answer texts in the corpus, obtain intermediate text information containing a first preset number of the question and answer texts, and determine a second preset number of target question and answer texts from the first preset number of question and answer texts in the intermediate text information according to preset indicators, use the target question and answer text as the first text information, and generate the retrieval result containing the first text information, wherein the second preset number is less than or equal to the first preset number.

3. The method according to claim 2, characterized in that Generating a first evaluation score of the search result according to the preset evaluation index includes: Determine relevant text information related to the question to be answered in the first text information of the search result, and determine a first quantity of the relevant text information; Obtaining relevant question and answer texts related to the question to be answered in the corpus, and determining a second number of the relevant question and answer texts; Dividing the first number by the second number to obtain the accuracy of the search result; Dividing the first number by the second preset number to obtain a recall rate of the search result; Obtaining a harmonic mean corresponding to the search result according to the precision, the recall and a preset formula, and obtaining the first evaluation score according to the harmonic mean; The preset formula satisfies: Wherein, F is the harmonic mean, α is a preset parameter, P is the precision, and R is the recall rate.

4. The method according to claim 1, characterized in that Generating a second evaluation score of the generated result according to the preset evaluation index includes: Determining the perplexity and entropy of the second text information of the generated result, and determining the average reply length of the second text information; Determining the total number of words, the third number of unigrams, and the fourth number of bigrams in the second text information of the generated result; Dividing the third number by the total number of words to obtain a first ratio of the unigrams, dividing the fourth number by the total number of words to obtain a second ratio of the bigrams, and obtaining a response diversity index of the generated result according to the first ratio and the second ratio; The second evaluation score is obtained according to the perplexity, the entropy, the response diversity index, the average response length and the preset weight.

5. The method according to claim 1, characterized in that The step of generating a first evaluation score of the search result and a second evaluation score of the generated result according to a preset evaluation index comprises: Inputting the search result into an evaluation model to obtain the first evaluation score, wherein the evaluation model is used to determine the probability that the search result is the target result according to the preset evaluation index, and the evaluation model is obtained by training an initial evaluation model according to the first training sample, and the initial evaluation model is constructed according to the preset evaluation index; The generated result is input into the evaluation model to obtain the second evaluation score, wherein the evaluation model is used to determine the probability that the generated result is the target result according to the preset evaluation index.

6. The method according to claim 1, characterized in that The step of obtaining a generation result according to the question to be answered and the generative model includes: Inputting the question to be answered into the generative model to obtain the generated result; The generative model includes an encoder and a decoder, the encoder of the generative model is used to determine the semantic features of the question to be answered, and the decoder of the generative model is used to generate the second text information according to the semantic features and generate the generation result containing the second text information.

7. The method according to claim 1, characterized in that The method further comprises: Acquire a newly added evaluation indicator, wherein the newly added evaluation indicator includes: the accuracy of the first third preset number of first text information in the search results and / or the average accuracy of the search results, the third preset number being less than or equal to the second preset number; Update the preset evaluation indicator according to the newly added evaluation indicator; The average accuracy satisfies: Wherein, MAP is the average accuracy, AP is the average precision of the first text information, Q R is the total number of the first text information, and q is the sequence number.

8. A context answer generation device, characterized in that: The device comprises: A retrieval result obtaining module, used to obtain retrieval results according to the question to be answered and the retrieval formula model, wherein the retrieval formula model is used to obtain the first text information corresponding to the question to be answered in the corpus and generate the retrieval result containing the first text information; A generation result obtaining module, used for obtaining a generation result according to the question to be answered and the generative model, wherein the generative model is used for generating the second text information corresponding to the question to be answered and generating the generation result including the second text information; An evaluation module, used to generate a first evaluation score of the retrieval result and a second evaluation score of the generated result according to a preset evaluation index, and if the first evaluation score is greater than the second evaluation score, the retrieval result is used as the target result, otherwise, the generated result is used as the target result, wherein the preset evaluation index is used to determine the probability that the retrieval result and the generated result are the target result; A generation module is used to input the target result into a preset generator to generate a context answer corresponding to the question to be answered, wherein the preset generator is used to construct and generate the context answer based on the target result.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the context answer generation method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the context answer generation method according to any one of claims 1 to 7.

Images