Retrieval enhancement method and system for multi-round dialogue type questions and answers and application

Abstract

The invention belongs to the technical field of intelligent dialogues, and provides a retrieval enhancement method and system for multi-round dialogue type questions and answers and application. A quaternion data set containing consultation questions, documents, answers and rewriting retrieval texts is constructed, and a retrieval result is obtained by combining dialogue state caching and a mixed retrieval strategy. The method comprises the following steps: generating a rewritten text taking a retrieval task as a main guide by utilizing a Rewriter model, dynamically mining a new difficult case according to the rewritten text, constructing a joint loss function by combining a generation loss function and a multi-positive sample comparison loss function, and jointly optimizing the Rewriter model. According to the method, a multi-positive sample comparison loss function is provided to make full use of supervision signals in a multi-round data set, and the ability of a rewriting model to understand the context is improved; a dynamic hard case mining method is provided, and dynamic mining is performed in real time according to a rewriting result in a training process, so that the problem that rewriting query cannot be optimized in real time according to retrieval feedback is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent dialogue, and particularly relates to a retrieval enhancement method and system for multi-turn conversational question answering and application thereof. BACKGROUND

[0002] With the breakthrough progress of large language models (LLMs), the development of conversational question answering systems (ConvQA) centered on generative AI has been greatly promoted, and has shown great potential in intelligent customer service, virtual assistants and other fields. However, the inherent knowledge limitations of LLMs (such as the lack of knowledge timeliness and hallucination problems) have seriously restricted their reliability and accuracy in key task scenarios. To overcome this bottleneck, the retrieval-augmented generation (RAG) architecture has become the mainstream paradigm, which is based on using retrieval technology to obtain relevant document fragments from external knowledge bases in real time to provide supplementary contextual information for LLMs, thereby improving the accuracy, timeliness and traceability of answers.

[0003] Although the RAG architecture has significant advantages in theory, when it is actually applied to multi-turn conversational question answering scenarios, the existing technical system still has systematic defects in the retrieval link, which seriously restricts the improvement of the final question answering performance. These defects are specifically manifested in the following three interrelated aspects:

[0004] 1. Fuzzy query failure: the traditional retrieval mechanism relying on keyword matching and embedding models is sensitive to inaccurate terms and ambiguous expressions, and the retrieval recall rate drops sharply;

[0005] Existing dialogue rewriting techniques usually aim to generate fluent, natural and human expression habit-compliant questions. However, this ignores the core requirement of retrieval-augmented question answering: the retriever (usually an embedding model) needs to find a document fragment that can answer the original dialogue context. Fluent rewriting does not necessarily generate the most query representation friendly to the embedding model, which may cause the retrieval results to be misaligned with the actual information needs of the dialogue context. The mainstream rewriting models (such as Seq2Seq or fine-tuned LLM) are generally optimized to generate "fluent", "natural" or "human expression habit-compliant" standard questions (e.g., maximizing the likelihood probability or BLEU score). This optimization objective ignores the core requirement of retrieval-augmented question answering - finding a document that can answer the question. Generating a fluent question for humans does not mean generating a question that is most easily found by the downstream embedding model / retriever. This leads to:

[0006] (a) The generated query may contain redundant information or expressions deviating from the core intent, misleading the embedding model.

[0007] (b) Failure to fully exploit the characteristics of the embedding model (e.g., sensitivity to specific vocabulary, sentence patterns, or semantic structures).

[0008] (c) Final result: Unfavorable search recall rate (Recall@K) and precision (Precision@K), a large number of relevant documents cannot be effectively recalled due to the "semantic gap" between query representation and document representation in the embedding space.

[0009] 2. Weak contextual understanding: unable to effectively utilize key information in the dialogue history, leading to search results deviating from user intent;

[0010] Current mainstream contrast learning methods (such as InfoNCE Loss) are usually based on a three-tuple (anchor Query, single positive sample Document+, multiple negative samples Document-) contrast loss. In the dialogue scenario, the positive sample or answer source related to the query may not be just one, which leads to the inability of single positive sample contrast learning to fully learn the differences between multiple positive samples and negative samples. This single positive sample setting fails to fully utilize the rich supervision signals contained in multi-round dialogue:

[0011] (a) Ignoring the potential multiple relevant expressions in the dialogue history (such as the user's previous questions, different expressions of the same entity mentioned in different rounds).

[0012] (b) Final result: The semantic representation learned by the model is not robust and generalizable, and it is not adaptable to the user's diverse questioning methods, synonyms in context, and changes in reference, limiting the ability to accurately depict the core intent of the dialogue in the embedding space.

[0013] 3. Poor dynamic adaptability: unable to optimize the rewritten query in real time according to the search feedback;

[0014] The adoption of static difficult example mining strategies restricts the improvement of model discrimination, and the selection of negative samples in training often uses random sampling or intra-batch negative samples. These strategies are difficult to identify the most challenging negative samples (hard negative examples) for the current model in real time and efficiently. In the complex semantic environment of multi-round dialogue:

[0015] (a) Random sampling contains a large number of simple negative examples, which has limited contribution to the improvement of model capabilities.

[0016] (b) Offline static mining strategies lag behind the dynamics of model training and cannot provide challenges to adapt to the evolving capabilities of the model. Specifically, the pre-fixed set of negative samples is difficult to match the constantly improving semantic discrimination ability of the model during training. As a result, the "difficult examples" selected in the early stage gradually degenerate into "simple negative examples" after the model's capabilities improve, and cannot continuously provide effective training pressure, thus restricting the evolution of the model's ability to discriminate subtle semantic differences.

[0017] (c) It is difficult to accurately capture semantically similar but actually unrelated confounding items (e.g., documents that discuss similar topics but answer different specific questions).

[0018] (d) Final result: The model is not good at distinguishing subtle semantic differences. When faced with ambiguity, vagueness and highly related interference items common in real dialogue, it is easy to retrieve documents that are semantically similar but actually irrelevant (false positive), which seriously affects the retrieval accuracy.

[0019] It is worth noting that even if the underlying LLM possesses superior reasoning capabilities, its output quality will still be significantly limited if the retrieval module cannot accurately locate relevant knowledge sources. Therefore, a method combining contextual analysis, semantic understanding, and dynamic retrieval enhancement is needed to improve the performance of multi-turn dialogue systems. Summary of the Invention

[0020] The purpose of this invention is to overcome the shortcomings of existing methods and provide a retrieval enhancement method, system, and application for multi-turn conversational question answering.

[0021] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0022] The first objective of this invention is to provide a retrieval enhancement method for multi-turn conversational question answering, comprising:

[0023] (1) Dataset construction: Construct a data builder that can simulate the dialogue question-and-answer process, using domain knowledge documents as the data source, and output a dataset including four-tuple data, which includes the question, related documents, answer and rewritten search text;

[0024] (2) Dialogue state management: The historical dialogue data between the user and the question-and-answer system is cached, and the maximum cache length N is set. Only the most recent N rounds of user input are stored, and the response results of the question-and-answer system are not stored.

[0025] (3) Initial retrieval: Based on the current dialogue input and the results of historical dialogue splicing, a hybrid retrieval strategy integrating keyword retrieval and vector retrieval is adopted to obtain the initial retrieval results;

[0026] (4) Search-oriented query rewriting: The current dialogue input, historical dialogue and the first search result are concatenated into text and input into the Rewriter model for rewriting to generate rewritten text with the search task as the main orientation;

[0027] (5) Calculation of generation loss: Construct a generation loss function based on the rewritten text and the target rewritten retrieval text in the dataset, and calculate the generation loss of the Rewriter model;

[0028] (6) Dynamic hard example mining: Perform vector retrieval on the rewritten text, filter the recalled document fragments, take the document fragments in the dataset that are related to the query text as positive samples, take the document fragments in the dataset that are not related to the query text as negative samples, and retain the negative samples that are highly relevant and do not belong to the positive samples.

[0029] (7) Calculation of document retrieval loss: Construct a multi-positive sample contrast loss function based on rewritten text, positive samples and dynamically mined negative samples, and calculate the document retrieval loss of the Rewriter model;

[0030] (8) Joint training: Combine the generation loss function and the multi-positive sample comparison loss function to construct a joint loss function and jointly optimize the Rewriter model.

[0031] Furthermore, the specific steps for constructing the dataset include:

[0032] By using regular expressions, the domain knowledge document is segmented into paragraphs, and segments with low information content are removed.

[0033] Using the segmented document fragments as context, a seed topic is generated using a large model, and multi-turn dialogue scenarios are simulated based on the document fragments and the seed topic.

[0034] The large model generates non-technical consultation questions based on document fragments and seed topics, and selects relevant document fragments that can answer these consultation questions.

[0035] Based on the generated consultation questions and related document fragments, the large model generates correct responses and rewrites the consultation questions into text of composite search expressions. The rewriting standard is that the rewritten text can be used to retrieve related document fragments.

[0036] The large model continues to generate new consultation questions based on the consultation questions from the previous round. The new consultation questions must have a contextual dependency relationship with the previous round, and there can be at most one topic change. The model also performs document fragment selection, answer generation, and text rewriting on the new consultation questions to generate the initial four-tuple data.

[0037] The initial quadruple dataset is evaluated using a large model, and data below the evaluation threshold is filtered out to form the final quadruple dataset.

[0038] Furthermore, in the process of using a large model to evaluate the data quality of the initial quadruple data, the evaluation criteria include whether the multi-turn dialogue scenario conforms to the daily dialogue scenario, whether the rewritten search text is related to the corresponding document fragment, and whether the answer is in the document fragment.

[0039] Furthermore, the initial retrieval specifically includes:

[0040] In the first round of dialogue, the dialogue history is empty. By default, a hybrid search strategy is used for the user's input, that is, keyword search and vector search are effective at the same time. A broad search is performed on relevant documents. Keyword search retrieves the top-K documents, and vector search first performs preliminary filtering based on similarity threshold. Then, the top-K documents are retrieved from the filtered results. Finally, the union of the keyword search results and the vector search results is taken as the first search result.

[0041] In multi-round dialogues, historical dialogues are spliced ​​together, and the spliced ​​dialogue is used as the input for the first retrieval, with the retrieval strategy being the same as in the first round.

[0042] Furthermore, the specific steps of the dynamic hard example mining include:

[0043] The rewritten search text is vector-encoded using the BERT model. The input text is feature-encoded to obtain feature vectors that can represent the semantic information of the text.

[0044] The inner product of the document fragment vector library is calculated to obtain the relevance score between the searched text and the document fragment vector library, and document fragments below the threshold are filtered out, leaving only highly similar document fragments.

[0045] Based on relevant fragments in the dataset, the recalled document fragments are filtered out, positive samples are filtered out, and text fragments that are highly relevant and do not belong to positive samples are retained.

[0046] Furthermore, the generation loss function is expressed as:

[0047]

[0048] in, Represents the cross-entropy loss function. The rewritten query text, The search text is rewritten for the target in the dataset.

[0049] Furthermore, the multi-positive-sample contrast loss function is expressed as:

[0050]

[0051] The similarity calculation function is shown below:

[0052]

[0053] in, Indicates batch size, This indicates the number of positive samples corresponding to the query text. This indicates the number of negative samples corresponding to the query text. Indicates the query text. Represents document fragments in the dataset. Indicates a positive sample. This indicates a negative sample.

[0054] Furthermore, the formula for the joint loss function is as follows:

[0055]

[0056] in, To generate loss, To retrieve loss, The value is 0.3. The value is 0.7.

[0057] Another object of the present invention is to provide a retrieval enhancement system for multi-turn conversational question answering, comprising:

[0058] The dataset construction module is used to build a data builder that can simulate a dialogue question-and-answer process. It uses domain knowledge documents as the data source and outputs a dataset that includes four-tuple data, which includes the question, related documents, answer and rewritten search text.

[0059] The dialogue state management module is used to cache the historical dialogue data between the user and the question-and-answer system. It sets a maximum cache length N and only stores the most recent N rounds of user input, without storing the response results of the question-and-answer system.

[0060] The initial retrieval module is used to obtain the first retrieval results based on the current dialogue input and the concatenation results of historical dialogues, using a hybrid retrieval strategy that integrates keyword retrieval and vector retrieval.

[0061] The retrieval-oriented query rewriting module is used to concatenate the current dialogue input, historical dialogue, and initial retrieval results into text, which is then input into the Rewriter model for rewriting, generating rewritten text that is primarily oriented towards the retrieval task.

[0062] The generation loss calculation module is used to construct a generation loss function based on the rewritten text and the target rewritten retrieval text in the dataset, and to calculate the generation loss of the Rewriter model;

[0063] The dynamic hard example mining module is used to perform vector retrieval on the rewritten text, filter the recalled document fragments, take the document fragments in the dataset that are related to the query text as positive samples, take the document fragments in the dataset that are not related to the query text as negative samples, and retain the negative samples that are highly relevant but do not belong to the positive samples.

[0064] The document retrieval loss calculation module is used to construct a multi-positive sample contrast loss function based on rewritten text, positive samples, and dynamically mined negative samples, and to calculate the document retrieval loss of the Rewriter model.

[0065] The joint training module is used to combine the generation loss function and the multi-positive-sample contrast loss function to construct a joint loss function and jointly optimize the Rewriter model.

[0066] Another object of the present invention is to provide a multi-turn conversational question answering system, wherein the multi-turn conversational question answering system is configured with the retrieval enhancement method for multi-turn conversational question answering provided in the first object of the present invention, and the multi-turn conversational question answering system is configured to perform the following steps:

[0067] In the first round, a user enters a question into the question-answering system. The dialogue manager records the current user input and concatenates it with the historical dialogues in context to obtain a question with context. Based on the user's question, two queries are performed simultaneously for document generalization and retrieval: keyword query and vector query. The union of the two retrieval results is then used to remove duplicates. The user's question and the deduplicated document fragments are used as input to the Rewriter model. The Rewriter model infers from the input to generate a rewritten question. Vector retrieval is then performed on the rewritten question to obtain the precise document. The retrieved document and the user's original question are then fed into the instruction constructor, which generates a question-answering task instruction. The main model performs natural language inference on the user's question and the retrieved document according to the task instruction to obtain the final answer.

[0068] In the nth round (n≥2), the user enters a new question into the question-answering system. The dialogue manager records the current user input and concatenates the historical dialogues with context to obtain a context-rich question. Based on this context-rich question, two queries are performed simultaneously for document generalization and recall: keyword query and vector query. The union of the two recall results is used to remove duplicates. The context-rich question and the deduplicated document fragments are then used as input to the Rewriter model. The Rewriter model infers from the input to generate a rewritten question for the current round. Vector retrieval is then performed on the rewritten question to obtain the precise document. The recalled document and the context-rich question are then fed into the instruction constructor, which generates a question-answering task instruction. Based on the task instruction, the main model performs natural language inference on the user's question for the current round and the retrieved document to obtain the final answer.

[0069] In combination with the above technical solutions, the beneficial effects of the present invention compared with the prior art are as follows:

[0070] This invention combines a rewriting model with a retrieval task, prioritizing the retrieval task and secondary to the generation task. It proposes a retrieval-friendly rewriting model that aims to maximize retrieval relevance (rather than human fluency). The generated queries accurately match the semantic understanding characteristics of the embedded model, filtering irrelevant expressions from the dialogue history to eliminate redundant interference and focus on core information needs. It adapts to embedded-sensitive features, proactively generating vocabulary / sentence structures sensitive to the embedded model (e.g., highlighting entity relationships and preserving referential structures), and optimizing query vector representation. The Recall@K and Precision@K metrics are significantly improved, and the false negative rate caused by the "semantic gap" is greatly reduced.

[0071] This invention proposes a multi-positive-sample contrastive loss function to fully utilize the supervision signals in multi-round datasets, enhancing the robustness and generalization ability of semantic representation and improving the rewriting model's ability to understand context. By simultaneously comparing multiple positive samples (such as different expressions of user questions or synonyms in history), the rewriting model can:

[0072] Capturing semantic diversity: Learning to map different expressions of the same intent (such as entity aliases and referential variants) to similar embedding spaces;

[0073] Strengthen intention Figure One Consistency: Resisting contextual interference (such as irrelevant topic shifts) and accurately depicting the core intent of the conversation;

[0074] Results: Improved adaptability to the diversity of user questions (rephrasing, changes in pronouns), and more stable retrieval accuracy in noisy conversations.

[0075] This invention proposes a dynamic hard example mining method. During training, it dynamically mines hard examples in real time based on the rewriting results, improving the model's ability to distinguish subtle semantic differences and addressing the problem of not being able to optimize rewritten queries in real time based on retrieval feedback. This is achieved by filtering negative samples (such as documents with similar topics but irrelevant answers) that are "most difficult for the current model to distinguish" in real time during training.

[0076] Precisely combat confounding terms: Targeted learning to distinguish semantically similar negative samples (e.g., "Apple's financial report" vs. "Apple planting technology");

[0077] Adaptive training difficulty: The complexity of negative samples is dynamically adjusted according to the model's capabilities to avoid ineffective training with simple negative examples;

[0078] Results: Significantly reduced false positive rate (retrieving documents that are semantically relevant but irrelevant to the answer), especially improving the Top-K ranking accuracy of highly relevant documents. Attached Figure Description

[0079] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0080] Figure 1 This is a flowchart of a retrieval enhancement method for multi-turn conversational question answering provided in an embodiment of the present invention;

[0081] Figure 2 This is a schematic diagram of the retrieval enhancement method for multi-turn conversational question answering provided in an embodiment of the present invention;

[0082] Figure 3 This is a flowchart of the method for constructing a dataset provided in an embodiment of the present invention;

[0083] Figure 4 This is a flowchart of the dynamic hard example mining method provided in the embodiments of the present invention;

[0084] Figure 5 This is a schematic diagram of the multi-turn conversational question-and-answer system provided in an embodiment of the present invention. Detailed Implementation

[0085] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0086] Example 1:

[0087] like Figure 1 The image shows an embodiment of the retrieval enhancement method for multi-turn conversational question answering provided by the present invention, which specifically includes the following steps:

[0088] S1: Dataset Construction: Construct a data builder that can simulate a dialogue question-and-answer process, using domain knowledge documents as the data source, and outputting a dataset including four-tuple data, which includes the question, related documents, answer and rewritten search text;

[0089] S2: Dialogue State Management: Cache the historical dialogue data between the user and the question-and-answer system, set the maximum cache length N, and only store the most recent N rounds of user input, without storing the response results of the question-and-answer system;

[0090] S3: Initial Search: Based on the current dialogue input and the results of historical dialogue concatenation, a hybrid search strategy integrating keyword search and vector search is used to obtain the initial search results;

[0091] S4: Retrieval-oriented query rewriting: The current dialogue input, historical dialogue, and initial retrieval results are concatenated into text, which is then input into the Rewriter model for rewriting, generating rewritten text that is primarily oriented towards the retrieval task;

[0092] S5: Generation loss calculation: Construct a generation loss function based on the rewritten text and the retrieved text of the target rewritten text in the dataset, and calculate the generation loss of the Rewriter model;

[0093] S6: Dynamic hard example mining: Perform vector retrieval on the rewritten text, filter the recalled document fragments, take the document fragments in the dataset that are related to the query text as positive samples, take the document fragments in the dataset that are not related to the query text as negative samples, and retain the negative samples that are highly relevant but do not belong to the positive samples.

[0094] S7: Document retrieval loss calculation: Construct a multi-positive sample contrast loss function based on rewritten text, positive samples, and dynamically mined negative samples, and calculate the document retrieval loss of the Rewriter model;

[0095] S8: Joint Training: Combine the generation loss function and the multi-positive-sample comparison loss function to construct a joint loss function and jointly optimize the Rewriter model.

[0096] Specifically, such as Figure 2 As shown, the dialogue state management module is responsible for recording the dialogue between the user and the question-and-answer system. To prevent the dialogue history from becoming too long, a maximum cache length N is typically set, storing only the most recent N rounds of dialogue. Specifically, in this embodiment, the dialogue state management module does not store the results of the question-and-answer system's responses; it is set to empty by default to prevent abnormal responses from interfering with historical dialogues.

[0097] The initial retrieval module is responsible for the first retrieval based on the user's current input. In a single-turn dialogue (the first dialogue), the dialogue history is empty, and a hybrid retrieval strategy is used by default, meaning keyword retrieval and vector retrieval are both active simultaneously, performing a broad search of relevant documents. In this embodiment, keyword retrieval retrieves the top-K documents, and vector retrieval first performs preliminary filtering based on a similarity threshold. Then, the top-K documents from the filtered results are selected, and the union of the keyword retrieval results and vector retrieval results is used as the initial retrieval result. In multi-turn dialogues, the dialogue history is not empty, and the dialogue history needs to be merged. The merging strategy is to concatenate the dialogue history, i.e., the concatenated result is "first-turn dialogue input, second-turn dialogue input, ..., Nth-turn dialogue input, current dialogue input". The concatenated dialogue is then used as the input for the first retrieval, and the retrieval strategy is the same as in the single-turn dialogue.

[0098] The Rewriter model performs a generation task based on the current dialogue input, historical dialogues, and search results. The generated result is the rewritten result. In this embodiment, the Rewriter model is designed based on a Decoder-only architecture. The model's input is a text string. Specifically, the current dialogue input, historical dialogues, and search results are concatenated into a single text string, which is then used as the model's input. The model's output is also a text string, i.e., the rewritten text.

[0099] The generation loss uses the cross-entropy loss function to minimize the gap between the rewritten text and the rewritten target (the ideal rewritten text), and the rewritten target is used as a supervision signal to train the model.

[0100] Hard example mining utilizes the output of the Rewriter model to recall new documents. Unlike the initial retrieval, it only uses vector retrieval and takes the Top-M as the final retrieval results.

[0101] The document retrieval loss employs a multi-positive-sample contrastive loss function, comparing the target positive sample with all other samples (including other positive and negative samples). This encourages the model to learn to distinguish the query from all other samples, including different positive samples. This helps to learn a more refined and discriminative representation.

[0102] Finally, a joint loss function (i.e., the generation loss function and the multiple positive sample contrast loss function) is used to train the Rewriter model.

[0103] Preferably, the specific steps for constructing the dataset in this embodiment of the invention include:

[0104] By using regular expressions, the domain knowledge document is segmented into paragraphs, and segments with low information content are removed.

[0105] Using the segmented document fragments as context, a seed topic is generated using a large model, and multi-turn dialogue scenarios are simulated based on the document fragments and the seed topic.

[0106] The large model generates non-technical consultation questions based on document fragments and seed topics, and selects relevant document fragments that can answer these consultation questions.

[0107] Based on the generated consultation questions and related document fragments, the large model generates correct responses and rewrites the consultation questions into text of composite search expressions. The rewriting standard is that the rewritten text can be used to retrieve related document fragments.

[0108] The large model continues to generate new consultation questions based on the consultation questions from the previous round. The new consultation questions must have a contextual dependency relationship with the previous round, and there can be at most one topic change. The model also performs document fragment selection, answer generation, and text rewriting on the new consultation questions to generate the initial four-tuple data.

[0109] The initial quadruple dataset is evaluated using a large model, and data below the evaluation threshold is filtered out to form the final quadruple dataset.

[0110] Specifically, conventional question-answering systems struggle to collect high-quality data suitable for model training. This embodiment utilizes open-source generative models, such as DeepSeek-v3, to design a data builder that simulates a dialogue-based question-answering process. It uses domain knowledge documents as the data source and outputs a quadruple (question, related documents, answer, and rewritten search text). Figure 3 As shown, the specific construction process is as follows:

[0111] a. Knowledge documents are usually long articles, and when searching for relevant knowledge, the search usually doesn't go directly to the entire document, but rather to document fragments. This article uses regular expressions to segment the document into paragraphs, removing fragments with low information content (such as consecutive spaces, consecutive line breaks, pure numbers, and short segments).

[0112] b. Using the segmented fragments as context, the large model generates a seed topic and simulates a multi-turn dialogue scenario based on the document and topic.

[0113] c. The large model generates a consultation question based on the document and topic. This consultation question is usually a non-technical expression, such as semantic ambiguity, implicit reference, and colloquialism. Simultaneously, the model also provides relevant fragments that can answer this consultation question.

[0114] d. Based on the generated consultation questions and related document fragments, the large model needs to provide the correct response and rewrite the consultation questions into a more complex search expression (the standard is that the rewritten text can be used to retrieve related document fragments).

[0115] e. The large model needs to continue generating consultation questions based on the consultation questions from the previous round. The current consultation question needs to have a contextual dependency relationship with the previous round, and there can be at most one topic change (which can be unrelated to the previous consultation question). The model also needs to perform document fragment selection, answer generation, and text rewriting for the new consultation question.

[0116] f. The large model needs to play the role of data quality assessment, scoring the generated quadruple data. The scoring criteria are whether the multi-turn dialogue scenario conforms to everyday dialogue scenarios, whether the rewritten search text is relevant to the corresponding document fragment, and whether the answer is in the document fragment. Data below the evaluation threshold is filtered out to form the final quadruple dataset.

[0117] To enable the rewriter model to distinguish between documents relevant to the query text (positive samples) and irrelevant documents (negative samples), contrastive learning is typically employed, maximizing consistency with positive samples while minimizing consistency with negative samples. During training, most negative samples are simple (easily distinguishable by the model), which has limited impact on improving the model's discriminative power. This embodiment proposes a dynamic hard example mining method, which dynamically mines new hard examples based on the rewritten text during training. Figure 4 As shown, the specific process is as follows:

[0118] a. The rewritten search text is vector-encoded using the BERT model. The input text is feature-encoded to obtain feature vectors that can represent the semantic information of the text.

[0119] b. Calculate the inner product of the document fragment vector library to obtain the relevance score between the retrieved text and the document fragment library, and filter out document fragments below the threshold, keeping only highly similar document fragments.

[0120] c. Based on the relevant fragments (positive samples) in the dataset, filter the recalled document fragments, filter out the positive samples, and keep only the text fragments that are highly relevant and do not belong to the positive samples (difficult examples).

[0121] Contrastive learning typically involves a query text, a positive sample, and multiple negative samples. Such a contrastive loss function is unsuitable for the training objective presented in this paper. In this paper, the objective function takes a query text, multiple positive samples, and multiple negative samples as input. Therefore, this paper proposes a contrastive loss function suitable for multiple positive samples, as shown in the following formula:

[0122]

[0123] The similarity calculation function is shown below:

[0124]

[0125] in, Indicates the size of a batch. This indicates the number of positive samples corresponding to the query text. This indicates the number of negative samples corresponding to the query text. Indicates the query text. Represents document fragments in the dataset. This indicates a positive sample (a document fragment related to the query text). This indicates a negative sample (a document fragment that is not related to the query text).

[0126] The technical solutions of the embodiments of the present invention will be further described below with reference to specific examples.

[0127] In data construction, the number of dialogue rounds ranges from 3 to 5; the scoring range for data quality assessment is [1, 5], and data with an assessment result of less than or equal to 2 will be discarded.

[0128] The Rewriter model architecture is Qwen2.5-1.5B.

[0129] During training, the maximum cache length N of the dialogue management module is set to 5. When managing the dialogue context, it is only responsible for storing user input queries and does not store system responses. The specific storage format is as follows: [first round input, first round input + second round input, first round input + second round input + third round input,...].

[0130] Based on the current input, a hybrid search is performed. First, relevant documents are searched using keywords, and the number of document fragments retrieved is TOP-K, where K is set to 10, meaning the top 10 document fragments in the search results are selected. Then, vector search is used, again selecting the top 10 document fragments. Finally, the results of the two search methods are merged, and duplicate document fragments are removed.

[0131] The rewriting model rewrites the query text based on the input of the current dialogue and the retrieved document fragments, resulting in the rewritten query text, denoted as . The retrieved text rewritten from the target in the dataset (denoted as...) The generation loss is calculated using the following formula:

[0132]

[0133] in, This represents the cross-entropy loss function.

[0134] The rewritten text is used to perform hard instance mining on document fragments based on vector retrieval methods. The maximum number of documents retrieved is set to 20, the minimum number of document fragments is 1, and the similarity threshold is 0.6. Specifically, the document fragments are first recalled according to the top-20, and then filtered based on the threshold. Those below the threshold are discarded. If the remaining documents after filtering are empty, that is, only the top-1 document fragments are kept.

[0135] Based on the rewritten text, positive samples, and dynamically mined negative samples, the document retrieval loss is calculated using the following formula:

[0136]

[0137] in, Indicates the size of a batch. This indicates the number of positive samples corresponding to the query text. This indicates the number of negative samples corresponding to the query text. Indicates the query text. This indicates a positive sample (a document fragment related to the query text). This indicates a negative sample (a document fragment that is not related to the query text). The similarity calculation function is shown below:

[0138]

[0139] Based on the generation loss and retrieval loss, the final objective function is constructed as follows:

[0140]

[0141] in, The value is 0.3. The value is 0.7.

[0142] Example 2:

[0143] This invention provides a retrieval enhancement system for multi-turn conversational question answering, comprising:

[0144] The dataset construction module is used to build a data builder that can simulate a dialogue question-and-answer process. It uses domain knowledge documents as the data source and outputs a dataset that includes four-tuple data, which includes the question, related documents, answer and rewritten search text.

[0145] The dialogue state management module is used to cache the historical dialogue data between the user and the question-and-answer system. It sets a maximum cache length N and only stores the most recent N rounds of user input, without storing the response results of the question-and-answer system.

[0146] The initial retrieval module is used to obtain the first retrieval results based on the current dialogue input and the concatenation results of historical dialogues, using a hybrid retrieval strategy that integrates keyword retrieval and vector retrieval.

[0147] The retrieval-oriented query rewriting module is used to concatenate the current dialogue input, historical dialogue, and initial retrieval results into text, which is then input into the Rewriter model for rewriting, generating rewritten text that is primarily oriented towards the retrieval task.

[0148] The generation loss calculation module is used to construct a generation loss function based on the rewritten text and the target rewritten retrieval text in the dataset, and to calculate the generation loss of the Rewriter model;

[0149] The dynamic hard example mining module is used to perform vector retrieval on the rewritten text, filter the recalled document fragments, take the document fragments in the dataset that are related to the query text as positive samples, take the document fragments in the dataset that are not related to the query text as negative samples, and retain the negative samples that are highly relevant but do not belong to the positive samples.

[0150] The document retrieval loss calculation module is used to construct a multi-positive sample contrast loss function based on rewritten text, positive samples, and dynamically mined negative samples, and to calculate the document retrieval loss of the Rewriter model.

[0151] The joint training module is used to combine the generation loss function and the multi-positive-sample contrast loss function to construct a joint loss function and jointly optimize the Rewriter model.

[0152] Example 3:

[0153] like Figure 5 As shown, this embodiment of the invention provides a multi-turn conversational question-answering system. The multi-turn conversational question-answering system is configured with the retrieval enhancement method for multi-turn conversational question answering provided in Embodiment 1 of the invention. The multi-turn conversational question-answering system is configured to perform the following steps:

[0154] In the first round, the user enters the question-answering system with a question. The dialogue manager records the current user input and concatenates it with the context of the historical dialogue to obtain a context-rich question. Since it's the first round of dialogue and there's no context history, the dialogue manager's output is still the original user question. Two queries are performed simultaneously based on the user's question for document generalization and recall: keyword query and vector query. By default, the top 10 ranked documents are retrieved from each query. The union of the two recall results is then used to remove duplicates. The user's question and the deduplicated document fragments are used as input to the Rewriter model. The Rewriter model infers from the input to generate a rewritten question. Since the Rewriter model is primarily retrieval-oriented during training, the rewritten question only requires vector retrieval to obtain accurate documents (by default, the top 5 documents). The retrieved documents and the original user question are then fed into the instruction constructor, which generates a question-answering task instruction. The main model performs natural language inference on the user's question and the retrieved documents according to the task instruction to obtain the final answer.

[0155] In the nth round (n≥2), the user enters a new question into the question-answering system. The dialogue manager records the current user input and concatenates it with the historical dialogues in context to obtain a question with context. Based on this contextual question, two queries are performed simultaneously for document generalization and recall: keyword query and vector query. By default, the top 10 ranked documents are retrieved from each query. The union of the two recall results is then used to remove duplicates. The contextual question and the deduplicated document fragments are used as input to the Rewriter model. The Rewriter model infers from the input to generate a rewritten question for the current round. Since the Rewriter model is primarily retrieval-oriented during training, the rewritten question only requires vector retrieval to obtain accurate documents (by default, the top 5 documents). The retrieved documents and the contextual question are then fed into the instruction constructor, which generates a question-answering task instruction. The main model performs natural language inference on the user's question and the retrieved documents based on the task instruction to obtain the final answer.

[0156] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in the present invention, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0157] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0158] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A retrieval enhancement method for multi-turn conversational question answering, characterized in that, The method includes: (1) Dataset construction: Construct a data builder that can simulate the dialogue question-and-answer process, using domain knowledge documents as the data source, and output a dataset including four-tuple data, which includes the question, related documents, answer and rewritten search text; (2) Dialogue state management: The historical dialogue data between the user and the question-and-answer system is cached, and the maximum cache length N is set. Only the most recent N rounds of user input are stored, and the response results of the question-and-answer system are not stored. (3) Initial retrieval: Based on the current dialogue input and the results of historical dialogue splicing, a hybrid retrieval strategy integrating keyword retrieval and vector retrieval is adopted to obtain the initial retrieval results; (4) Search-oriented query rewriting: The current dialogue input, historical dialogue and the first search result are concatenated into text and input into the Rewriter model for rewriting to generate rewritten text with the search task as the main orientation; (5) Calculation of generation loss: Construct a generation loss function based on the rewritten text and the target rewritten retrieval text in the dataset, and calculate the generation loss of the Rewriter model; (6) Dynamic hard example mining: Perform vector retrieval on the rewritten text, filter the recalled document fragments, take the document fragments in the dataset that are related to the query text as positive samples, take the document fragments in the dataset that are not related to the query text as negative samples, and retain the negative samples that are highly relevant and do not belong to the positive samples. (7) Calculation of document retrieval loss: Construct a multi-positive sample contrast loss function based on rewritten text, positive samples and dynamically mined negative samples, and calculate the document retrieval loss of the Rewriter model; (8) Joint training: Combine the generation loss function and the multi-positive sample comparison loss function to construct a joint loss function and jointly optimize the Rewriter model.

2. The retrieval enhancement method for multi-turn conversational question answering according to claim 1, characterized in that, The specific steps for constructing the dataset include: By using regular expressions, the domain knowledge document is segmented into paragraphs, and segments with low information content are removed. Using the segmented document fragments as context, a seed topic is generated using a large model, and multi-turn dialogue scenarios are simulated based on the document fragments and the seed topic. The large model generates non-technical consultation questions based on document fragments and seed topics, and selects relevant document fragments that can answer these consultation questions. Based on the generated consultation questions and related document fragments, the large model generates correct responses and rewrites the consultation questions into text of composite search expressions. The rewriting standard is that the rewritten text can be used to retrieve related document fragments. The large model continues to generate new consultation questions based on the consultation questions from the previous round. The new consultation questions must have a contextual dependency relationship with the previous round, and there can be at most one topic change. The model also performs document fragment selection, answer generation, and text rewriting on the new consultation questions to generate the initial four-tuple data. The initial quadruple dataset is evaluated using a large model, and data below the evaluation threshold is filtered out to form the final quadruple dataset.

3. The retrieval enhancement method for multi-turn conversational question answering according to claim 2, characterized in that, In the process of using a large model to evaluate the data quality of the initial quadruple data, the evaluation criteria include whether the multi-turn dialogue scenario conforms to the daily dialogue scenario, whether the rewritten search text is related to the corresponding document fragment, and whether the answer is in the document fragment.

4. The retrieval enhancement method for multi-turn conversational question answering according to claim 1, characterized in that, The initial retrieval specifically includes: In the first round of dialogue, the dialogue history is empty. By default, a hybrid search strategy is used for the user's input, that is, keyword search and vector search are effective at the same time. A broad search is performed on relevant documents. Keyword search retrieves the top-K documents, and vector search first performs preliminary filtering based on similarity threshold. Then, the top-K documents are retrieved from the filtered results. Finally, the union of the keyword search results and the vector search results is taken as the first search result. In multi-round dialogues, historical dialogues are spliced ​​together, and the spliced ​​dialogue is used as the input for the first retrieval, with the retrieval strategy being the same as in the first round.

5. The retrieval enhancement method for multi-turn conversational question answering according to claim 1, characterized in that, The specific steps of dynamic hard example mining include: The rewritten search text is vector-encoded using the BERT model. The input text is feature-encoded to obtain feature vectors that can represent the semantic information of the text. The inner product of the document fragment vector library is calculated to obtain the relevance score between the searched text and the document fragment vector library, and document fragments below the threshold are filtered out, leaving only highly similar document fragments. Based on relevant fragments in the dataset, the recalled document fragments are filtered out, positive samples are filtered out, and text fragments that are highly relevant and do not belong to positive samples are retained.

6. The retrieval enhancement method for multi-turn conversational question answering according to claim 1, characterized in that, The generation loss function is expressed as: ； in, Represents the cross-entropy loss function. The rewritten query text, The search text is rewritten for the target in the dataset.

7. The retrieval enhancement method for multi-turn conversational question answering according to claim 1, characterized in that, The multi-positive-sample contrast loss function is expressed as follows: ； The similarity calculation function is shown below: ； in, Indicates batch size, This indicates the number of positive samples corresponding to the query text. This indicates the number of negative samples corresponding to the query text. Indicates the query text. Represents document fragments in the dataset. Indicates a positive sample. This indicates a negative sample.

8. The retrieval enhancement method for multi-turn conversational question answering according to claim 1, characterized in that, The formula for the joint loss function is as follows: ； in, To generate loss, To retrieve loss, The value is 0.

3. The value is 0.

7.

9. A retrieval enhancement system for multi-turn conversational question answering, characterized in that, The system includes: The dataset construction module is used to build a data builder that can simulate a dialogue question-and-answer process. It uses domain knowledge documents as the data source and outputs a dataset that includes four-tuple data, which includes the question, related documents, answer and rewritten search text. The dialogue state management module is used to cache the historical dialogue data between the user and the question-and-answer system. It sets a maximum cache length N and only stores the most recent N rounds of user input, without storing the response results of the question-and-answer system. The initial retrieval module is used to obtain the first retrieval results based on the current dialogue input and the concatenation results of historical dialogues, using a hybrid retrieval strategy that integrates keyword retrieval and vector retrieval. The retrieval-oriented query rewriting module is used to concatenate the current dialogue input, historical dialogue, and initial retrieval results into text, which is then input into the Rewriter model for rewriting, generating rewritten text that is primarily oriented towards the retrieval task. The generation loss calculation module is used to construct a generation loss function based on the rewritten text and the target rewritten retrieval text in the dataset, and to calculate the generation loss of the Rewriter model; The dynamic hard example mining module is used to perform vector retrieval on the rewritten text, filter the recalled document fragments, take the document fragments in the dataset that are related to the query text as positive samples, take the document fragments in the dataset that are not related to the query text as negative samples, and retain the negative samples that are highly relevant but do not belong to the positive samples. The document retrieval loss calculation module is used to construct a multi-positive sample contrast loss function based on rewritten text, positive samples, and dynamically mined negative samples, and to calculate the document retrieval loss of the Rewriter model. The joint training module is used to combine the generation loss function and the multi-positive-sample contrast loss function to construct a joint loss function and jointly optimize the Rewriter model.

10. A multi-turn conversational question-and-answer system, characterized in that, The system is configured with the retrieval enhancement method for multi-turn conversational question answering as described in any one of claims 1 to 8, wherein the multi-turn conversational question answering system is configured to perform the following steps: In the first round, a user enters a question into the question-answering system. The dialogue manager records the current user input and concatenates it with the historical dialogues in context to obtain a question with context. Based on the user's question, two queries are performed simultaneously for document generalization and retrieval: keyword query and vector query. The union of the two retrieval results is then used to remove duplicates. The user's question and the deduplicated document fragments are used as input to the Rewriter model. The Rewriter model infers from the input to generate a rewritten question. Vector retrieval is then performed on the rewritten question to obtain the precise document. The retrieved document and the user's original question are then fed into the instruction constructor, which generates a question-answering task instruction. The main model performs natural language inference on the user's question and the retrieved document according to the task instruction to obtain the final answer. In the nth round (n≥2), the user enters a new question into the question-answering system. The dialogue manager records the current user input and concatenates the historical dialogues with context to obtain a context-rich question. Based on this context-rich question, two queries are performed simultaneously for document generalization and recall: keyword query and vector query. The union of the two recall results is used to remove duplicates. The context-rich question and the deduplicated document fragments are then used as input to the Rewriter model. The Rewriter model infers from the input to generate a rewritten question for the current round. Vector retrieval is then performed on the rewritten question to obtain the precise document. The recalled document and the context-rich question are then fed into the instruction constructor, which generates a question-answering task instruction. Based on the task instruction, the main model performs natural language inference on the user's question for the current round and the retrieved document to obtain the final answer.

Images