A retrieval enhancement method, system and application for multi-turn conversational question answering

Abstract

The application belongs to the technical field of intelligent dialogue and provides a retrieval enhancement method, system and application for multi-round dialogue question answering, which constructs a quadruple data set containing consulting questions, documents, answers and rewritten retrieval texts, combines dialogue state caching and hybrid retrieval strategies, uses a Rewriter model to generate rewritten texts mainly oriented to retrieval tasks, dynamically mines new difficult examples according to the rewritten texts, combines a generated loss function and a multi-positive sample comparison loss function to construct a joint loss function, and jointly optimizes the Rewriter model. The application proposes a multi-positive sample comparison loss function to fully utilize the supervision signals in the multi-round data set and improve the understanding ability of the rewriting model for the context, and proposes a dynamic difficult example mining method to dynamically mine difficult examples in real time according to the rewriting results during the training process, so as to solve the problem that the rewritten query cannot be optimized in real time according to the retrieval feedback.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent dialogue, and particularly relates to a retrieval enhancement method and system for multi-turn conversational question answering and application thereof. BACKGROUND

[0002] With the breakthrough progress of large language models (LLMs), the development of conversational question answering systems (ConvQA) centered on generative AI has been greatly promoted, and has shown great potential in intelligent customer service, virtual assistants and other fields. However, the inherent knowledge limitations of LLMs (such as the lack of knowledge timeliness and hallucination problems) have seriously restricted their reliability and accuracy in key task scenarios. To overcome this bottleneck, the retrieval-augmented generation (RAG) architecture has become the mainstream paradigm, which is based on using retrieval technology to obtain relevant document fragments from external knowledge bases in real time to provide supplementary contextual information for LLMs, thereby improving the accuracy, timeliness and traceability of answers.

[0003] Although the RAG architecture has significant advantages in theory, when it is actually applied to multi-turn conversational question answering scenarios, the existing technical system still has systematic defects in the retrieval link, which seriously restricts the improvement of the final question answering performance. These defects are specifically manifested in the following three interrelated aspects:

[0004] 1. Fuzzy query failure: the traditional retrieval mechanism relying on keyword matching and embedding models is sensitive to inaccurate terms and ambiguous expressions, and the retrieval recall rate drops sharply;

[0005] Existing dialogue rewriting techniques usually aim to generate fluent, natural and human expression habit-compliant questions. However, this ignores the core requirement of retrieval-augmented question answering: the retriever (usually an embedding model) needs to find a document fragment that can answer the original dialogue context. Fluent rewriting does not necessarily generate the most query representation friendly to the embedding model, which may cause the retrieval results to be misaligned with the actual information needs of the dialogue context. The mainstream rewriting models (such as Seq2Seq or fine-tuned LLM) are generally optimized to generate "fluent", "natural" or "human expression habit-compliant" standard questions (e.g., maximizing the likelihood probability or BLEU score). This optimization objective ignores the core requirement of retrieval-augmented question answering - finding a document that can answer the question. Generating a fluent question for humans does not mean generating a question that is most easily found by the downstream embedding model / retriever. This leads to:

[0006] (a) The generated query may contain redundant information or expressions deviating from the core intent, misleading the embedding model.

[0007] (b) Failure to fully exploit the characteristics of the embedding model (e.g., sensitivity to specific vocabulary, sentence patterns, or semantic structures).

[0008] (c) Final result: suboptimal search recall (Recall@K) and precision (Precision@K), with a large number of relevant documents not effectively recalled due to the "semantic gap" between query representation and document representation in the embedding space.

[0009] 2. Weak contextual understanding: unable to effectively utilize key information in the dialogue history, leading to search results deviating from user intent;

[0010] Current mainstream contrastive learning methods (such as InfoNCE Loss) are usually based on a triplet (anchor Query, single positive sample Document+, multiple negative samples Document-). In the dialogue scenario, the positive sample or answer source related to the query may not be just one, which leads to the inability of single positive sample contrastive learning to fully learn the differences between multiple positive samples and negative samples. This single positive sample setting fails to fully utilize the rich supervision signals contained in multi-round dialogue:

[0011] (a) Ignoring the potential multiple relevant expressions in the dialogue history (such as the user's previous questions, different expressions of the same entity mentioned in different rounds).

[0012] (b) Final result: the semantic representation learned by the model is not robust and generalizable, with poor adaptability to the user's diverse questioning methods, synonymous substitutions in context, and reference changes, limiting the ability to accurately depict the core intent of the dialogue in the embedding space.

[0013] 3. Poor dynamic adaptability: unable to optimize the rewritten query in real time based on search feedback;

[0014] The adoption of static difficult example mining strategies restricts the improvement of model discrimination, and the selection of negative samples in training often uses random sampling or intra-batch negative samples. These strategies are difficult to identify the most challenging negative samples (hard negative examples) for the current model in real time and efficiently. In the complex semantic environment of multi-round dialogue:

[0015] (a) Random sampling contains a large number of simple negative examples, contributing limitedly to the improvement of model capabilities.

[0016] (b) Offline static mining strategies lag behind model training dynamics and cannot provide adaptive difficulty challenges as the model's capabilities evolve. Specifically, the pre-fixed negative sample set cannot match the model's constantly improving semantic discrimination ability during training, resulting in the initial screening of "difficult examples" gradually degrading into "simple negative examples" after the model's ability improves, which cannot provide effective training pressure, thereby restricting the model's ability to evolve in distinguishing subtle semantic differences.

[0017] (c) It is difficult to accurately capture confusing items that are semantically close but actually irrelevant (for example, documents discussing similar topics but answering different specific questions).

[0018] (d) The final result: the model's ability to distinguish subtle semantic differences is insufficient, and when faced with the ambiguity, ambiguity, and highly relevant interference items commonly found in real conversations, it is easy to retrieve documents that are semantically close but actually irrelevant (false positives), which severely affects retrieval accuracy.

[0019] It is worth noting that even if the underlying LLM has excellent reasoning ability, the quality of its output will still be significantly constrained if the retrieval module cannot accurately locate relevant knowledge sources. Therefore, a method that combines context analysis, semantic understanding, and dynamic retrieval enhancement is needed to improve the performance of multi-turn dialogue systems. SUMMARY

[0020] The purpose of the present application is to overcome the existing defects and provide a retrieval enhancement method, system and application for multi-turn dialogue question answering.

[0021] To solve the above technical problems, the present application provides the following technical solutions:

[0022] The first purpose of the present application is to provide a retrieval enhancement method for multi-turn dialogue question answering, comprising:

[0023] (1) Data set construction: constructing a data constructor that can simulate the dialogue question answering process, taking domain knowledge documents as the data source, and outputting a data set including four-tuple data, the four-tuple data including consultation questions, relevant documents, answers, and rewritten retrieval texts;

[0024] (2) Dialogue state management: cache the historical dialogue data between the user and the question answering system, set the maximum cache length N, and only store the last N rounds of user input, without storing the question answering system response results;

[0025] (3) Initialization retrieval: based on the current dialogue input and the historical dialogue splicing result, a hybrid retrieval strategy integrating keyword retrieval and vector retrieval is used to obtain the first retrieval result;

[0026] (4) Retrieval-oriented query rewriting: concatenate the current dialogue input, historical dialogue and the first retrieval result as a text, input it into the Rewriter model for rewriting, and generate a rewritten text mainly oriented to the retrieval task;

[0027] (5) Generation loss calculation: based on the rewritten text and the retrieval text with target rewriting in the data set, a generation loss function is constructed to calculate the generation loss of the Rewriter model;

[0028] (6) Dynamic difficult example mining: vector retrieval is performed on the rewritten text, and the recalled document segments are filtered. The document segments related to the query text in the data set are used as positive samples, and the document segments unrelated to the query text in the data set are used as negative samples. The negative samples with high relevance and not belonging to the positive samples are retained;

[0029] (7) Document retrieval loss calculation: based on the rewritten text, the positive samples and the dynamically mined negative samples, a multi-positive sample comparison loss function is constructed to calculate the document retrieval loss of the Rewriter model;

[0030] (8) Joint training: a joint loss function is constructed by combining the generation loss function and the multi-positive sample comparison loss function, and the Rewriter model is jointly optimized.

[0031] Further, the specific steps of constructing the data set include:

[0032] Through regular means, the paragraphs of the domain knowledge document are segmented, and the segments with low information content are removed;

[0033] The segmented document segments are used as contexts, a seed topic is generated by using a large model, and based on the document segments and the seed topic, a multi-round dialogue scenario is simulated;

[0034] The large model generates a non-professional expression consultation question according to the document segments and the seed topic, and selects relevant document segments that can answer the consultation question;

[0035] Based on the generated consultation question and the relevant document segments, the large model generates a correct reply, and rewrites the consultation question into a text with a composite retrieval expression. The rewriting standard is that the relevant document segments can be retrieved by using the rewritten text;

[0036] The large model continues to generate a new consultation question according to the consultation question of the last round. The new consultation question needs to have a contextual dependency with the last round, and can only have a topic change at most once. The new consultation question is subjected to document segment selection, answer generation and text rewriting to generate initial four-tuple data;

[0037] The large model is used to evaluate the data quality of the initial four-tuple data, filter out the data below the evaluation threshold, and form a final four-tuple data set.

[0038] Further, in the data quality evaluation of the initial quadruple data using the large model, the evaluation benchmark includes whether the scene of the multiple rounds of dialogue conforms to the daily dialogue scene, whether the rewritten retrieval text is related to the corresponding document segment, and whether the answer is in the document segment.

[0039] Further, the initialization retrieval specifically includes:

[0040] In the first round of dialogue, the dialogue history is empty, and the mixed retrieval strategy is used by default for the user's input, that is, keyword retrieval and vector retrieval are effective at the same time, extensive retrieval is performed on the relevant documents, Top-K documents are obtained by keyword retrieval, and Top-K documents are obtained by vector retrieval based on a similarity threshold for preliminary screening, and then the union of the keyword retrieval results and the vector retrieval results is taken as the first retrieval result.

[0041] In the multiple rounds of dialogue, the historical dialogue is spliced, and the spliced dialogue is taken as the input of the first retrieval, and the retrieval strategy is the same as the first round.

[0042] Further, the specific steps of the dynamic difficult example mining include:

[0043] The rewritten retrieval text is vector encoded, and the encoding model uses a BERT model to encode the features of the input text to obtain a feature vector that can represent the semantic information of the text.

[0044] The inner product calculation is performed on the document segment vector library to obtain the relevance score of the retrieval text and the document segment vector library in the document segment vector library, and the document segments below the threshold are filtered out, and only the highly similar document segments are retained.

[0045] Based on the relevant segments in the data set, the recalled document segments are filtered to filter out the positive samples and retain the text segments with high relevance and not belonging to the positive samples.

[0046] Further, the loss function is represented as:

[0047]

[0048] wherein, represents a cross-entropy loss function, is the rewritten query text, is the target rewritten retrieval text in the data set.

[0049] Further, the multi-positive sample contrast loss function is represented as:

[0050]

[0051] denotes a similarity calculation function, and the calculation formula is as follows:

[0052]

[0053] wherein, denotes a batch size, denotes a number of positive samples corresponding to the query text, denotes a number of negative samples corresponding to the query text, denotes the query text, denotes a document segment in the data set, denotes a positive sample, denotes a negative sample.

[0054] Further, the joint loss function formula is as follows:

[0055]

[0056] wherein, is a generation loss, is a retrieval loss, takes a value of 0.3, takes a value of 0.7.

[0057] Another object of the present application is to provide a retrieval enhancement system for multi-round dialogue question answering, comprising:

[0058] A data set construction module is configured to construct a data constructor capable of simulating a dialogue question answering process, take a domain knowledge document as a data source, and output a data set including four-tuple data, wherein the four-tuple data includes a consultation question, a related document, an answer, and a rewritten retrieval text.

[0059] A dialogue state management module is configured to cache historical dialogue data between a user and a question answering system, set a maximum cache length N, and only store the latest N rounds of user input, without storing a question answering system response result.

[0060] An initialization retrieval module is configured to obtain a first retrieval result by using a hybrid retrieval strategy integrating keyword retrieval and vector retrieval based on a current dialogue input and a historical dialogue splicing result.

[0061] A retrieval-oriented query rewriting module is configured to splice the current dialogue input, the historical dialogue, and the first retrieval result into a text, input the text into a Rewriter model for rewriting, and generate a rewritten text mainly oriented to a retrieval task.

[0062] A generation loss calculation module is configured to construct a generation loss function based on the rewritten text and a target rewritten retrieval text in the data set, and calculate a generation loss of the Rewriter model.

[0063] a dynamic difficult case mining module, configured to perform vector retrieval on the rewritten text, filter the recalled document segments, and retain negative samples that are highly relevant and do not belong to positive samples, with the positive samples being document segments in the data set that are relevant to the query text, and the negative samples being document segments in the data set that are not relevant to the query text;

[0064] a document retrieval loss calculation module, configured to construct a multi-positive sample contrast loss function based on the rewritten text, the positive samples and the dynamically mined negative samples, and calculate a document retrieval loss of the Rewriter model;

[0065] a joint training module, configured to construct a joint loss function by combining the generation loss function and the multi-positive sample contrast loss function, and jointly optimize the Rewriter model.

[0066] Another object of the present application is to provide a multi-round dialogue question answering system configured with the retrieval enhancement method for multi-round dialogue question answering provided by the first object of the present application, which is configured to perform the following steps:

[0067] In the first round, the user inputs a consultation question into the question answering system, the dialogue manager records the current user input and splices the historical dialogue of the context to obtain a consultation question with context, performs two queries simultaneously for document generalization recall according to the user's consultation question, namely keyword query and vector query, and takes the union of the two recall results after removing duplicates; the user's consultation question and the document segments after removing duplicates are input into the Rewriter model, the Rewriter model generates a rewritten consultation question by reasoning on the input, performs vector retrieval on the rewritten consultation question to obtain accurate documents, and then inputs the recalled documents and the user's original consultation question into the instruction constructor, which generates a question answering task instruction, and the large model performs natural language reasoning on the user's consultation question and the retrieved documents according to the task instruction to obtain the final answer;

[0068] In the n-th round (n>2), the user inputs a new consultation question into the question and answer system, the dialogue manager records the current user input, splices the historical dialogue of the context, obtains the consultation question with the context, makes two queries at the same time according to the consultation question with the context to perform document generalization recall, which are keyword query and vector query respectively, takes the union of the two recall results to remove duplicates, and takes the consultation question with the context and the document segment after removing duplicates as the input of the Rewriter model, the Rewriter model generates a rewritten question for the current round by reasoning on the input, performs vector retrieval on the rewritten consultation question to obtain accurate documents, and then sends the recalled documents and the consultation question with the context to the instruction constructor, the instruction constructor generates a question and answer task instruction, and the large model performs natural language reasoning on the user consultation question and the retrieved documents in the current round according to the task instruction to obtain the final answer.

[0069] Compared with the prior art, the beneficial effects possessed by the technical scheme are:

[0070] The rewriting model is combined with the retrieval task, the retrieval task is mainly oriented, the task is generated secondarily, a retrieval-friendly rewriting model is proposed, the rewriting model takes "maximizing retrieval relevance" as the target (instead of human fluency), the generated query can accurately match the semantic understanding characteristics of the embedding model, can filter irrelevant expressions in the dialogue history, eliminate redundant interference and focus on core information needs; adapt to sensitive features embedded, automatically generate sensitive words / sentences of the embedded model (such as highlighting entity relationships and preserving reference structures), and optimize query vector representation; the Recall@K and Precision@K indexes are significantly improved, and the missing detection rate caused by "semantic gap" is greatly reduced.

[0071] The application proposes a multi-positive sample contrast loss function to fully utilize the supervision signals in the multi-round data set, enhances the robustness and generalization ability of semantic representation, and improves the understanding ability of the rewriting model to the context. By comparing multiple positive samples (such as different expressions of user questions and synonymous expressions in the history) at the same time, the rewriting model can:

[0072] Capture semantic diversity: learn to map different expressions of the same intent (such as entity aliases and reference variants) to a similar embedding space;

[0073] Strengthen the consistency of intent: Figure One Resist context interference (such as irrelevant topic drift), and accurately depict the core intent of the dialogue;

[0074] Effect embodiment: adaptively improve the diversity of user questions (change the way to say, change the reference), and the retrieval accuracy is more stable in noisy dialogue.

[0075] The application proposes a dynamic difficult example mining method, which makes real-time dynamic mining according to the rewriting results in the training process, improves the distinguishing ability of the model to subtle semantic differences, and solves the problem of being unable to optimize the rewritten query in real time according to the search feedback. By real-time screening of negative samples (such as documents with similar topics but irrelevant answers) that are "most difficult to distinguish by the current model" in training, the following is achieved:

[0076] Precise confrontation confusion items: targeted learning to distinguish negative samples with similar semantics (such as "Apple financial report" vs. "apple planting technology");

[0077] Adaptive training difficulty: dynamically adjust the complexity of negative samples according to the model's ability to avoid ineffective training of simple negative examples;

[0078] Effect: significantly reduce the false positive rate (retrieve documents with semantic relevance but irrelevant answers), especially improve the Top-K ranking accuracy of high relevance documents. BRIEF DESCRIPTION OF DRAWINGS

[0079] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, which together with the embodiments of the application, is used to explain the application, and does not constitute a limitation on the application. In the drawings:

[0080] Figure 1 is a flowchart of the retrieval enhancement method for multi-turn conversational question answering provided by the embodiments of the application;

[0081] Figure 2 is a schematic diagram of the retrieval enhancement method for multi-turn conversational question answering provided by the embodiments of the application;

[0082] Figure 3 is a method flowchart of the data set construction provided by the embodiments of the application;

[0083] Figure 4 is a method flowchart of the dynamic difficult example mining provided by the embodiments of the application;

[0084] Figure 5 is a schematic diagram of the multi-turn conversational question answering system provided by the embodiments of the application. DETAILED DESCRIPTION

[0085] The preferred embodiments of the application are described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the application, and not to limit the application.

[0086] Embodiment 1:

[0087] As shown in Figure 1 is one embodiment of the retrieval enhancement method for multi-turn conversational question answering provided by the application, which specifically includes the following steps:

[0088] S1: Dataset Construction: Construct a data builder that can simulate a dialogue question-and-answer process, using domain knowledge documents as the data source, and outputting a dataset including four-tuple data, which includes the question, related documents, answer and rewritten search text;

[0089] S2: Dialogue State Management: Cache the historical dialogue data between the user and the question-and-answer system, set the maximum cache length N, and only store the most recent N rounds of user input, without storing the response results of the question-and-answer system;

[0090] S3: Initial Search: Based on the current dialogue input and the results of historical dialogue concatenation, a hybrid search strategy integrating keyword search and vector search is used to obtain the initial search results;

[0091] S4: Retrieval-oriented query rewriting: The current dialogue input, historical dialogue, and initial retrieval results are concatenated into text, which is then input into the Rewriter model for rewriting, generating rewritten text that is primarily oriented towards the retrieval task;

[0092] S5: Generation loss calculation: Construct a generation loss function based on the rewritten text and the retrieved text of the target rewritten text in the dataset, and calculate the generation loss of the Rewriter model;

[0093] S6: Dynamic hard example mining: Perform vector retrieval on the rewritten text, filter the recalled document fragments, take the document fragments in the dataset that are related to the query text as positive samples, take the document fragments in the dataset that are not related to the query text as negative samples, and retain the negative samples that are highly relevant but do not belong to the positive samples.

[0094] S7: Document retrieval loss calculation: Construct a multi-positive sample contrast loss function based on rewritten text, positive samples, and dynamically mined negative samples, and calculate the document retrieval loss of the Rewriter model;

[0095] S8: Joint Training: Combine the generation loss function and the multi-positive-sample comparison loss function to construct a joint loss function and jointly optimize the Rewriter model.

[0096] Specifically, such as Figure 2 As shown, the dialogue state management module is responsible for recording the dialogue between the user and the question-and-answer system. To prevent the dialogue history from becoming too long, a maximum cache length N is typically set, storing only the most recent N rounds of dialogue. Specifically, in this embodiment, the dialogue state management module does not store the results of the question-and-answer system's responses; it is set to empty by default to prevent abnormal responses from interfering with historical dialogues.

[0097] The initialization retrieval module is responsible for the first retrieval of the current input of the user. When it is a single round of dialogue (the first dialogue), the dialogue history is empty, and the mixed retrieval strategy is used by default, i.e., the keyword retrieval and the vector retrieval are effective at the same time, and the relevant documents are retrieved extensively. In the embodiment, the keyword retrieval takes the top-K documents, the vector retrieval takes the top-K documents after the preliminary screening based on the similarity threshold, and then the union of the keyword retrieval result and the vector retrieval result is taken as the first retrieval result. When it is a multi-round dialogue, the dialogue history is not empty, and the dialogue history needs to be merged. The merging strategy is to splice the dialogue history, i.e., the spliced result is "the first round of dialogue input, the second round of dialogue input,..., the Nth round of dialogue input, the current dialogue input", and then the spliced dialogue is taken as the input of the first retrieval, and the retrieval strategy is the same as that of the single round.

[0098] The Rewriter model generates the result according to the current dialogue input, the historical dialogue and the retrieval result. In the embodiment, the Rewriter model is designed based on the Decoder-only architecture. The input of the model is a text. Specifically, the current dialogue input, the historical dialogue and the retrieval result are spliced into a text as the input of the model, and the output of the model is also a text, i.e., the rewritten text.

[0099] The generation loss adopts the cross-entropy loss function, minimizes the gap between the rewritten text and the rewriting target (the ideal rewritten text), and uses the rewriting target as a supervision signal to realize the training of the model.

[0100] The difficult case mining is to use the output result of the Rewriter model to perform new document recall. Different from the first retrieval, only the vector retrieval is used, and the top-M is taken as the final retrieval result.

[0101] The document retrieval loss adopts the multi-positive sample contrast loss function, compares the target positive sample with all other samples (including other positive samples and negative samples), encourages the model to learn to distinguish the query from all other samples, including distinguishing different positive samples. This helps to learn more fine and discriminative representations.

[0102] Finally, the joint loss function (i.e., the generation loss function and the multi-positive sample contrast loss function) is used to train the Rewriter model.

[0103] As a preferred, the specific steps of the data set construction in the embodiment of the application include:

[0104] The paragraphs of the domain knowledge document are segmented by the regular means, and the segments with low information content are removed;

[0105] The segmented document segment is taken as a context to generate a seed topic by the large model, and based on the document segment and the seed topic, a multi-round dialogue scenario is simulated;

[0106] The large model generates a non-professionalized expression consulting question according to the document segment and the seed topic, and selects a related document segment that can answer the consulting question;

[0107] Based on the generated consulting question and the related document segment, the large model generates a correct reply, and rewrites the consulting question into a composite retrieval expression text, and the rewriting standard is that the related document segment can be retrieved by using the rewritten text;

[0108] The large model continues to generate a new consulting question according to the consulting question of the last round, the new consulting question needs to have a contextual dependence relationship with the last round, and can only have a topic change at most once, and performs document segment selection, answer generation and text rewriting on the new consulting question to generate initial four-tuple data;

[0109] The large model is used to perform data quality evaluation on the initial four-tuple data, and filter out data below the evaluation threshold to form a final four-tuple data set.

[0110] Specifically, a conventional question and answer system is difficult to collect high-quality data that can be used for model training, and the embodiment uses an open source generation model, such as DeepSeek-v3, to design a data constructor that can simulate a dialogue question and answer process, and the data source is a field knowledge document, and the output is a four-tuple of (consulting question, related document, answer, rewritten retrieval text). As shown in Figure 3 The specific construction process is as follows:

[0111] a. The knowledge document is usually an article with more text, and when performing related knowledge retrieval, the document is usually not directly retrieved, but a document segment. In this paper, the document is segmented by regular means, and the segments with low information content (such as continuous spaces, continuous line breaks, pure numbers, and short segments) are removed.

[0112] b. The segmented segment is taken as a context to let the large model generate a seed topic, and based on the document and the topic, a multi-round dialogue scenario is simulated.

[0113] c. The large model generates a consulting question according to the document and the topic, and the consulting question is usually a non-professionalized expression, such as semantic ambiguity, implicit reference and colloquialization. While generating the consulting question, the model also gives a related segment that can answer the consulting question.

[0114] d. Based on the generated consultation questions and related document fragments, the large model needs to give the correct reply, and rewrite the consultation question into a more complex retrieval expression consultation question (the standard is to retrieve the relevant document fragments using the rewritten text).

[0115] e. The large model needs to continue to generate consultation questions based on the previous round of consultation questions, the current consultation question needs to have a contextual dependence on the previous round, and at most only one topic change (can be unrelated to the previous consultation question), and make document fragment selection, answer generation and text rewriting for the new consultation question.

[0116] f. The large model needs to play the role of data quality evaluation, scoring the generated four-tuple data, scoring criteria are whether the multi-round dialogue scene meets the daily dialogue scene, whether the rewritten retrieval text is related to the corresponding document fragment, and whether the answer is in the document fragment. Three aspects of evaluation. Filter out data below the evaluation threshold to form the final four-tuple data set.

[0117] In order to let the rewriting model (Rewriter model) distinguish between relevant documents (positive samples) and irrelevant documents (negative samples) with query text, a contrast learning method is usually used, that is, maximizing consistency with positive samples while minimizing consistency with negative samples. During training, most negative samples belong to simple negative samples (i.e. the model can easily distinguish them through learning), which is limited to improving the discrimination of the model. In this embodiment, a dynamic hard example mining method is proposed, that is, during training, new hard examples are dynamically mined according to the rewritten text. As shown in Figure 4 The specific process is as follows:

[0118] a. Vector encoding is performed on the rewritten retrieval text, and the encoding model uses the BERT model to encode the input text to obtain a feature vector that can represent the semantic information of the text.

[0119] b. Perform inner product calculation on the document fragment vector library to get the relevance score of the retrieval text and the document fragment library, and filter out document fragments below the threshold, only keeping high-similarity document fragments.

[0120] c. Based on the relevant fragments (positive samples) in the data set, filter the recalled document fragments, filter out positive samples, and only keep text fragments with high relevance and not belonging to positive samples (hard examples).

[0121] Contrastive learning is usually a query text, a positive sample and multiple negative samples. Such a contrastive loss function is not suitable for the training target in this paper. In this paper, the input of the objective function is a query text, multiple positive samples and multiple negative samples. Therefore, a contrastive loss function suitable for multiple positive samples is proposed in this paper, and the specific formula is as follows:

[0122]

[0123] represents a similarity calculation function, and the calculation formula is as follows:

[0124]

[0125] wherein, represents the size of a batch, represents the number of positive samples corresponding to the query text, represents the number of negative samples corresponding to the query text, represents a query text, represents a document segment in a data set, represents a positive sample (a document segment related to the query text), represents a negative sample (a document segment unrelated to the query text).

[0126] The technical solutions of the embodiments of the application will be further described below in combination with specific examples.

[0127] In data construction, the number of rounds of the dialogue ranges from 3 to 5; in data quality evaluation, the scoring range is [1, 5], and data with an evaluation result less than or equal to 2 will be discarded.

[0128] The Rewriter model architecture is Qwen2.5-1.5B.

[0129] In the training process, the maximum cache length N of the dialogue management module is set to 5, and in the management of dialogue context, only the input query of the user is stored, and the reply of the system is not stored. The specific storage form is as follows: [first round input, first round input + second round input, first round input + second round input + third round input,...].

[0130] Based on the current input, hybrid retrieval is performed. First, the relevant documents are retrieved using keywords, and the number of retrieved document segments is TOP-K, and K is set to 10, that is, the top 10 document segments in the retrieval result are taken. Then, vector retrieval is performed, and the top 10 document segments are also taken. Finally, the results of the two retrieval methods are merged, and the duplicate document segments are removed.

[0131] The rewriting model rewrites the query text based on the current input of the dialogue and the retrieved document segments, and the rewritten query text is denoted as The retrieval text in the data set is denoted as The generation loss calculation is performed on the target rewritten retrieval text (denoted as

[0132]

[0133] wherein, represents a cross-entropy loss function.

[0134] The document fragments are mined based on the vector retrieval method for the rewritten text, the maximum number of retrieved documents is set to 20, the minimum number of document fragments is 1, and the similarity threshold is 0.6. Specifically, the top-20 document fragments are recalled first, and then filtered based on the threshold. The document fragments below the threshold are discarded. If the remaining documents after filtering are empty, only the top-1 document fragment is retained.

[0135] Based on the rewritten text, the positive samples and the dynamically mined negative samples, the document retrieval loss calculation is performed, and the calculation formula is as follows:

[0136]

[0137] wherein, represents the size of a batch, represents the number of positive samples corresponding to the query text, represents the number of negative samples corresponding to the query text, represents the query text, represents the positive sample (the document fragment related to the query text), represents the negative sample (the document fragment unrelated to the query text), represents a similarity calculation function, and the calculation formula is as follows:

[0138]

[0139] Based on the generation loss and the retrieval loss, the final target function is constructed, and the formula is as follows:

[0140]

[0141] wherein, takes a value of 0.3, takes a value of 0.7.

[0142] Embodiment 2:

[0143] The embodiment of the application provides a retrieval enhancement system for multi-round dialogue question answering, which comprises:

[0144] A data set construction module is configured to construct a data constructor capable of simulating a dialogue question and answer process, take a domain knowledge document as a data source, and output a data set including four-tuple data, the four-tuple data including a consultation question, a related document, an answer, and a rewritten retrieval text;

[0145] A dialogue state management module is configured to cache historical dialogue data of a user and a question and answer system, set a maximum cache length N, and only store the latest N rounds of user input, without storing a question and answer system response result;

[0146] An initialization retrieval module is configured to splice a current dialogue input and historical dialogue based on a splicing result, and acquire a first retrieval result by using a hybrid retrieval strategy integrating keyword retrieval and vector retrieval;

[0147] A retrieval-oriented query rewriting module is configured to splice the current dialogue input, the historical dialogue, and the first retrieval result into a text, input the text into a Rewriter model for rewriting, and generate a rewritten text mainly oriented to a retrieval task;

[0148] A generation loss calculation module is configured to construct a generation loss function based on the rewritten text and a retrieval text targeted for rewriting in the data set, and calculate a generation loss of the Rewriter model;

[0149] A dynamic difficult case mining module is configured to perform vector retrieval on the rewritten text, filter recalled document segments, take document segments related to a query text in the data set as positive samples, take document segments unrelated to the query text in the data set as negative samples, and retain negative samples that are highly relevant and do not belong to the positive samples;

[0150] A document retrieval loss calculation module is configured to construct a multi-positive sample comparison loss function based on the rewritten text, the positive samples, and the dynamically mined negative samples, and calculate a document retrieval loss of the Rewriter model;

[0151] A joint training module is configured to construct a joint loss function by combining the generation loss function and the multi-positive sample comparison loss function, and jointly optimize the Rewriter model.

[0152] Embodiment 3:

[0153] As shown in Figure 5 , the present embodiment provides a multi-round dialogue question and answer system, which is configured with the retrieval enhancement method for multi-round dialogue question and answer provided in Embodiment 1 of the present application, and is configured to perform the following steps:

[0154] In the first round, the user inputs a consultation question into the question and answer system, and the dialogue manager records the current user input and splices the context of the historical dialogue to obtain a consultation question with context. Since there is no context dialogue history in the first round of dialogue, the output of the dialogue manager is still the original user consultation question. According to the user's consultation question, two queries are made simultaneously for document generalization recall, which are keyword query and vector query, respectively. The top 10 ranked documents are taken by default, and the union set of the two recall results is de-duplicated. The user's consultation question and the de-duplicated document segments are taken together as the input of the Rewriter model. The Rewriter model generates a rewritten consultation question by reasoning on the input. Since the Rewriter model is mainly oriented towards the retrieval task in the training process, the rewritten consultation question of the Rewriter model only needs to do vector retrieval to obtain accurate documents (the top 5 documents are taken by default). Then, the recalled documents and the user's original consultation question are sent to the instruction constructor, which generates a question and answer task instruction. The large model performs natural language reasoning on the user's consultation question and the retrieved documents according to the task instruction to obtain the final answer.

[0155] In the nth round (n≥2), the user inputs a new consultation question into the question and answer system, and the dialogue manager records the current user input and splices the context of the historical dialogue to obtain a consultation question with context. According to the consultation question with context, two queries are made simultaneously for document generalization recall, which are keyword query and vector query, respectively. The top 10 ranked documents are taken by default, and the union set of the two recall results is de-duplicated. The consultation question with context and the de-duplicated document segments are taken together as the input of the Rewriter model. The Rewriter model generates a rewritten question for the current round by reasoning on the input. Since the Rewriter model is mainly oriented towards the retrieval task in the training process, the rewritten consultation question of the Rewriter model only needs to do vector retrieval to obtain accurate documents (the top 5 documents are taken by default). Then, the recalled documents and the consultation question with context are sent to the instruction constructor, which generates a question and answer task instruction. The large model performs natural language reasoning on the user's consultation question and the retrieved documents according to the task instruction to obtain the final answer.

[0156] It should be understood that, although the steps in the flowcharts of the embodiments of the present application are shown in a certain order according to the arrows, the steps are not necessarily executed in the order of the arrows. Unless otherwise specified in the present application, the execution of the steps is not strictly limited in order, and the steps can be executed in other orders. Moreover, at least some of the steps in the embodiments can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the sub-steps or stages is not necessarily sequential, but can be alternated or interleaved with at least part of other steps or sub-steps or stages of other steps.

[0157] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0158] Finally, it should be noted that: the above only describes the preferred embodiments of the present application, and is not used to limit the present application, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A retrieval enhancement method for multi-turn conversational question answering, characterized in that, The method comprises: (1) Data set construction: constructing a data constructor that can simulate the dialogue question and answer process, taking domain knowledge documents as data sources, and outputting a data set comprising four-tuple data, the four-tuple data comprising a consultation question, a related document, an answer, and a rewritten retrieval text; (2) Dialogue state management: caching historical dialogue data between the user and the question and answer system, setting a maximum cache length N, and only storing the latest N rounds of user input, without storing the question and answer system response results; (3) Initialization retrieval: based on the current dialogue input and the spliced results of the historical dialogue, using a hybrid retrieval strategy integrating keyword retrieval and vector retrieval to obtain the first retrieval result; (4) Retrieval-oriented query rewriting: splicing the current dialogue input, historical dialogue and the first retrieval result into a text, inputting it into the Rewriter model for rewriting, and generating a rewritten text mainly oriented to retrieval tasks; (5) Generation loss calculation: based on the rewritten text and the target rewritten retrieval text in the data set, a generation loss function is constructed to calculate the generation loss of the Rewriter model; (6) Dynamic difficult example mining: vector retrieval is performed on the rewritten text, and the recalled document segments are filtered, taking the document segments related to the query text in the data set as positive samples, and taking the document segments unrelated to the query text in the data set as negative samples, and retaining negative samples that are highly relevant and do not belong to positive samples; (7) Document retrieval loss calculation: based on the rewritten text, the positive samples and the dynamically mined negative samples, a multi-positive sample comparison loss function is constructed to calculate the document retrieval loss of the Rewriter model; The multi-positive sample comparison loss function is represented as: ； represents a similarity calculation function, and the calculation formula is as follows: ； wherein, denotes batch size, denotes the number of positive samples corresponding to the query text, denotes the number of negative samples corresponding to the query text, denotes the query text, denotes a document snippet in the dataset, denotes a positive sample, denotes a negative sample; (8) Joint training: a joint loss function is constructed by combining the generation loss function and the multi-positive sample comparison loss function, and the Rewriter model is optimized jointly.

2. The retrieval enhancement method for multi-turn conversational question answering of claim 1, wherein, The specific steps of the data set construction comprise: By regular means, the paragraphs of the domain knowledge documents are segmented, and the segments with low information content are removed; The segmented document segments are used as context to generate a seed topic using a large model, and based on the document segments and the seed topic, a multi-round dialogue scenario is simulated; The large model generates a consultation question in non-professional language based on the document segments and the seed topic, and selects relevant document segments that can answer the consultation question; Based on the generated consultation question and the relevant document segments, the large model generates a correct reply, and rewrites the consultation question into a text with composite retrieval expression, and the rewriting standard is that the relevant document segments can be retrieved using the rewritten text; The large model generates a new consultation question based on the previous consultation question, the new consultation question needs to have a contextual dependency with the previous round, and can only have a topic change at most once, and performs document segment selection, answer generation and text rewriting on the new consultation question to generate initial four-tuple data; The large model performs data quality evaluation on the initial four-tuple data, filters out data below the evaluation threshold, and forms a final four-tuple data set.

3. The retrieval enhancement method for multi-turn conversational question answering of claim 2, wherein, In the data quality evaluation of the initial quadruple data by using the large model, the evaluation benchmark includes whether the scene of the multiple rounds of dialogues conforms to the daily dialogue scene, whether the rewritten retrieval text is related to the corresponding document segment, and whether the answer is in the document segment.

4. The retrieval enhancement method for multi-turn conversational question answering of claim 1, wherein, The initialization retrieval specifically includes: In the first round of dialogue, the dialogue history is empty, and the mixed retrieval strategy is used by default for the input of the user, that is, the keyword retrieval and the vector retrieval are effective at the same time, the relevant documents are extensively retrieved, the Top-K documents are obtained by keyword retrieval, the Top-K documents are obtained by the vector retrieval based on the similarity threshold for preliminary screening, and then the union of the keyword retrieval result and the vector retrieval result is taken as the first retrieval result; In the multiple rounds of dialogues, the historical dialogues are spliced, and the spliced dialogues are taken as the input of the first retrieval, and the retrieval strategy is the same as the first round.

5. The retrieval enhancement method for multi-turn conversational question answering of claim 1, wherein, The specific steps of the dynamic difficult example mining include: The rewritten retrieval text is vector encoded, the encoding model adopts the BERT model, the input text is feature encoded, and a feature vector representing the semantic information of the text is obtained; The inner product calculation is performed on the document segment vector library to obtain the relevance score of the retrieval text and the document segment vector library, and the document segments below the threshold are filtered out, and only the high-similarity document segments are retained; Based on the relevant segments in the data set, the recalled document segments are filtered to filter out the positive samples and retain the text segments with high relevance and not belonging to the positive samples.

6. The retrieval enhancement method for multi-turn conversational question answering of claim 1, wherein, The generation loss function is represented as: ； wherein, represents a cross-entropy loss function, is the rewritten query text, is the target rewritten search text in the data set.

7. The retrieval enhancement method for multi-turn conversational question answering of claim 1, wherein, The joint loss function formula is as follows: ； wherein, is a generation loss, is a retrieval loss, takes the value 0.3, takes the value 0.

7.

8. A retrieval enhancement system for multi-turn conversational question answering, characterized in that, The system includes: A data set construction module is configured to construct a data constructor capable of simulating a dialogue question and answer process, take a domain knowledge document as a data source, and output a data set including quadruple data, the quadruple data including a consultation question, a relevant document, an answer, and a rewritten retrieval text; A dialogue state management module is configured to cache historical dialogue data of a user and a question and answer system, set a maximum cache length N, and only store the latest N rounds of user input, without storing the response results of the question and answer system; An initialization retrieval module is configured to use a keyword retrieval and vector retrieval integrated mixed retrieval strategy to obtain a first retrieval result based on the current dialogue input and the historical dialogue splicing result; A retrieval-oriented query rewriting module is configured to splice the current dialogue input, the historical dialogue, and the first retrieval result into a text, input the text into a Rewriter model for rewriting, and generate a rewritten text mainly oriented to the retrieval task; A generation loss calculation module is configured to construct a generation loss function based on the rewritten text and the target rewritten retrieval text in the data set, and calculate the generation loss of the Rewriter model; A dynamic difficult example mining module is configured to perform vector retrieval on the rewritten text, filter the recalled document segments, take the document segments related to the query text in the data set as positive samples, take the document segments unrelated to the query text in the data set as negative samples, and retain the negative samples with high relevance and not belonging to the positive samples. The document retrieval loss calculation module is configured to construct a multi-positive sample comparison loss function based on the rewritten text, the positive sample, and the dynamically mined negative sample, and calculate a document retrieval loss of the Rewriter model. The multi-positive sample comparison loss function is expressed as: ； represents a similarity calculation function, and the calculation formula is as follows: ； wherein, denotes batch size, denotes the number of positive samples corresponding to the query text, denotes the number of negative samples corresponding to the query text, denotes the query text, denotes a document snippet in the dataset, denotes a positive sample, denotes a negative sample; The joint training module is configured to construct a joint loss function by combining the generation loss function and the multi-positive sample comparison loss function, and jointly optimize the Rewriter model.

9. A multi-turn conversational question answering system, characterized in that, The retrieval enhancement method for multi-turn conversational question answering is configured according to any one of claims 1 to 7, and the multi-turn conversational question answering system is configured to perform the following steps: In the first round, the user inputs a consultation question into the question answering system, the dialogue manager records the current user input, splices the historical dialogue of the context, obtains a consultation question with context, performs two queries at the same time for document generalization recall according to the user's consultation question, that is, keyword query and vector query, takes the union of the two recall results and removes duplicates; the user's consultation question and the de-duplicated document segments are input into the Rewriter model together, the Rewriter model generates a rewritten consultation question by reasoning on the input, performs vector retrieval on the rewritten consultation question to obtain accurate documents, and then inputs the recalled documents and the user's original consultation question into the instruction constructor, which generates a question answering task instruction, and the large model performs natural language reasoning on the user's consultation question and the retrieved documents according to the task instruction to obtain the final answer. In the n-th round (n≥2), the user inputs a new consultation question into the question answering system, the dialogue manager records the current user input, splices the historical dialogue of the context, obtains a consultation question with context, performs two queries at the same time for document generalization recall according to the consultation question with context, that is, keyword query and vector query, takes the union of the two recall results and removes duplicates, and inputs the consultation question with context and the de-duplicated document segments into the Rewriter model together, the Rewriter model generates a rewritten question for the current round by reasoning on the input, performs vector retrieval on the rewritten consultation question to obtain accurate documents, and then inputs the recalled documents and the consultation question with context into the instruction constructor, which generates a question answering task instruction, and the large model performs natural language reasoning on the user's consultation question and the retrieved documents according to the task instruction to obtain the final answer.

Images