Data processing method for improving correlation of retrieval enhancement generated text
By performing sentence-level decomposition and correlation scoring model evaluation on the retrieved text, unnecessary information is eliminated, and the problem of poor relevance of the search data is solved, which significantly improves the accuracy of the question answering.
Patent Information
- Application Number
- CN202411792968.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-08
- Publication Date
- 2025-03-21
AI Technical Summary
The existing search enhancement generation technology has the problem of poor relevance of search data when used, which leads to unsatisfactory output of the generative model and lacks effective reference text data cleaning strategies, which is inefficient.
By decomposing the retrieved text at the sentence level, using the association scoring model to evaluate the correlation degree of each sentence, clearing unnecessary information, and filtering out search reference sentences with correlation values greater than 0.5. The simplified reference text content is used to improve the accuracy of question answering.
It effectively improves the accuracy of answers to questions, and improves the relevance of generated text by clearing redundant and irrelevant information.
Smart Images
Figure CN119669423A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of data processing, and in particular relates to a data processing method for improving the relevance of retrieval-enhanced generated text. Background Art
[0002] Retrieval-Augmented Generation (RAG) is a technique that combines information retrieval and generative models to improve the performance and accuracy of generative models. RAG enhances the generation process by retrieving relevant information before generating text. It is particularly suitable for tasks that require the combination of external knowledge, such as question-answering systems, dialogue generation, and content creation. However, RAG still has the disadvantage of poor relevance of retrieved data when it is currently used. For example, the quality of the retrieved text directly affects the accuracy and relevance of the generated results. If the retrieval system fails to find sufficiently relevant or accurate information, the generative model may produce unsatisfactory output. Therefore, building an efficient and accurate retrieval system is key. In addition, there is redundancy in the relevance of the retrieved text content to the question. Some retrieved texts are not relevant to the answer to a specific question. The retrieved information lacks an effective cleaning method associated with the question, and some may even mislead the large language model to generate incorrect answers. How to effectively simplify the retrieved information at the data level is very important. At present, there is a lack of effective reference text data cleaning strategies in the industry. Manual cleaning and annotation are usually used in different problem areas, resulting in low efficiency. Summary of the invention
[0003] In order to solve the above problems, this application designs a data processing method to improve the relevance of retrieval-enhanced generated text. The retrieved text is decomposed into sentence levels, the relevance of each sentence is evaluated using a relevance scoring model, and unnecessary information is removed. The streamlined reference text content can effectively improve the accuracy of answering questions.
[0004] A data processing method for improving the relevance of retrieval enhancement generated text comprises the following steps:
[0005] Step S1, inputting the input question text into the vector model to obtain question text vector data;
[0006] Step S2: search the knowledge base according to the input question text to obtain the knowledge base text;
[0007] Step S3, segmenting the knowledge base text into sentences to obtain retrieval reference sentences, and inputting the retrieval reference sentences into the vector model to obtain sentence text vector data;
[0008] Step S4: construct a correlation evaluation model and perform training;
[0009] Step S5: input the sentence text vector data and the question text vector data into the relevance evaluation model, perform relevance value calculation, retain the retrieval reference sentences with relevance values greater than 0.5, remove the remaining retrieval reference sentences, and screen out the simplified knowledge base text;
[0010] Step S6: Integrate the simplified knowledge base text selected in step S5 and the input question text, and input them into the large language model.
[0011] Preferably, the method for constructing the relevance evaluation model includes:
[0012] Step S401, perform knowledge base search on the training question text to obtain the training knowledge base text;
[0013] Step S402: Segment the training knowledge base text into sentences to obtain training retrieval reference sentences, and input the training retrieval reference sentences into a vector model to obtain training sentence text vector data;
[0014] Step S403, inputting the training question text into the vector model to obtain training question text vector data;
[0015] Step S404: input the training question text vector data and the training sentence text vector data into the cross attention network model for fusion to obtain a fused feature vector;
[0016] Step S405: input the fused feature vector into a feedforward neural network to obtain a correlation value.
[0017] Preferably, the training method of the relevance evaluation model includes:
[0018] Step S411, constructing a training data set by manual screening, wherein the training data set includes training questions and corresponding answer retrieval reference texts, i.e., corresponding answer knowledge base texts;
[0019] Step S412, decomposing the response search reference text into multiple sentences;
[0020] Step S413: Combine a single training question with multiple sentences one by one, and input them into the large language model as prompt word texts respectively; record whether the selected sentences can provide effective support for the output answer through manual review, and if so, mark the corresponding sentence as 1, otherwise mark it as -1;
[0021] Step S414, and so on, similarly process each combination of training questions, and then use the batch training method to pass the training question text and the corresponding sentence text into the relevance evaluation model, and finally obtain the relevance score q of each sentence. The relevance score q and the manually annotated relevance score q' together constitute the mean square error loss function.
[0022] L(q,q′)=E[(qq′) 2 ],
[0023] Among them, E represents the expectation of the data set;
[0024] Step S415: Finally, the AdamW optimizer is used to perform iterative optimization and update the correlation evaluation model parameters.
[0025] Preferably, the feedforward neural network comprises two linear layers.
[0026] Preferably, the cross-attention network model is defined as:
[0027]
[0028] Where Q = W q *I, K = W k *P,V=W v *P;
[0029] W q is the query parameter matrix, W k is the key parameter matrix, W v is the value parameter matrix, d is the size of the feature vector dimension, Q is the query vector, K is the key vector, and V is the value vector.
[0030] The advantages and effects of this application are as follows:
[0031] The present application designs a data processing method for improving the relevance of retrieval enhanced generated text, including: inputting the input question text into a vector model to obtain question text vector data; performing knowledge base retrieval according to the input question text to obtain the knowledge base text; performing sentence segmentation on the knowledge base text to obtain retrieval reference sentences, and inputting them into the vector model to obtain sentence text vector data; constructing a relevance evaluation model and training it; inputting the sentence text vector data and the question text vector data into the relevance evaluation model, performing relevance numerical calculation, retaining the retrieval reference sentences with relevance values greater than 0.5, removing the remaining retrieval reference sentences, and screening out the streamlined knowledge base text; integrating the screened streamlined knowledge base text and the input question text, and inputting them into the large language model. The present application can use the relevance scoring model to remove unnecessary knowledge base text information and effectively improve the accuracy of question answers.
[0032] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application so that it can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings as follows.
[0033] Based on the detailed description of the specific embodiments of the present application in combination with the accompanying drawings below, those skilled in the art will become more aware of the above and other objects, advantages and features of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.
[0035] Figure 1 A flowchart of a data processing method designed for this application to improve the relevance of retrieval-enhanced generated text;
[0036] Figure 2 A method for constructing a relevance assessment model designed for this application. DETAILED DESCRIPTION
[0037] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, rather than all of the embodiments. In the following description, specific details such as specific configuration and components are provided only to help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, for clarity and brevity, the description of known functions and structures is omitted in the embodiment.
[0038] It should be understood that the references to "one embodiment" or "this embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "one embodiment" or "this embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0039] In addition, the present application may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplicity and clarity, and does not in itself indicate the relationship between the various embodiments and / or settings discussed.
[0040] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" in this article describes another type of association object relationship, indicating that there can be two relationships. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship.
[0041] The term "at least one" in this article is merely a description of the association relationship of associated objects, indicating that there may be three relationships. For example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.
[0042] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusions.
[0043] Example 1
[0044] At present, the mainstream question retrieval method in the industry is to use a vector model to process the input question text and knowledge base information to obtain the corresponding vector data. Then, the vector of the input question text is matched with the knowledge base vector in the future for similarity, and a set of knowledge base information with the highest similarity is selected as reference information for reference input for subsequent question answers. However, this processing method is not flexible enough when it is necessary to summarize the correlation between knowledge base information, and it is necessary to construct prompt word engineering text for different question descriptions. In addition, not all texts in the retrieved reference content have a positive feedback effect on the target question, and there may be redundant or even misleading erroneous information.
[0045] Please refer to Figure 1 This embodiment designs a data processing method for improving the relevance of retrieval enhancement generated text, including the following steps:
[0046] Step S1, inputting the input question text into the vector model to obtain question text vector data;
[0047] Step S2: search the knowledge base according to the input question text to obtain the knowledge base text;
[0048] Step S3, segmenting the knowledge base text into sentences to obtain retrieval reference sentences, and inputting the retrieval reference sentences into the vector model to obtain sentence text vector data;
[0049] Step S4: construct a correlation evaluation model and perform training;
[0050] Step S5: input the sentence text vector data and the question text vector data into the relevance evaluation model, perform relevance value calculation, retain the retrieval reference sentences with relevance values greater than 0.5, remove the remaining retrieval reference sentences, and screen out the simplified knowledge base text;
[0051] Step S6: Integrate the simplified knowledge base text selected in step S5 and the input question text, and input them into the large language model.
[0052] Furthermore, in order to remove unnecessary knowledge base text information, this embodiment designs a relevance evaluation model.
[0053] Please refer to Figure 2 , the construction method of the relevance evaluation model includes:
[0054] Step S401, perform knowledge base search on the training question text to obtain the training knowledge base text;
[0055] Step S402: Segment the training knowledge base text into sentences to obtain training retrieval reference sentences, and input the training retrieval reference sentences into a vector model to obtain training sentence text vector data;
[0056] Furthermore, the text content of the training knowledge base is split into N parts according to the granularity of the sentence level, where N represents the number of training retrieval reference sentences; P represents the text vector data of the training sentences;
[0057] Step S403, input the training question text into the vector model to obtain training question text vector data; I represents the training question text vector data;
[0058] Step S404: input the training question text vector data and the training sentence text vector data into the cross attention network model for fusion to obtain a fused feature vector;
[0059] Step S405: input the fused feature vector into a feedforward neural network to obtain a correlation value.
[0060] Furthermore, the training method of the relevance evaluation model includes:
[0061] Step S411, constructing a training data set by manual screening, wherein the training data set includes training questions and corresponding answer retrieval reference texts, i.e., corresponding answer knowledge base texts;
[0062] Step S412, decomposing the response search reference text into multiple sentences;
[0063] Step S413: Combine a single training question with multiple sentences one by one, and input them into the large language model as prompt word texts respectively; record whether the selected sentences can provide effective support for the output answer through manual review, and if so, mark the corresponding sentence as 1, otherwise mark it as -1;
[0064] Step S414, and so on, similarly process each combination of training questions, and then use the batch training method to pass the training question text and the corresponding sentence text into the relevance evaluation model, and finally obtain the relevance score q of each sentence. The relevance score q and the manually annotated relevance score q' together constitute the mean square error loss function.
[0065] L(q,q′)=E[(qq′) 2 ],
[0066] Among them, E represents the expectation of the data set;
[0067] Step S415: Finally, the AdamW optimizer is used to perform iterative optimization and update the correlation evaluation model parameters.
[0068] Furthermore, the feedforward neural network includes two linear layers.
[0069] Furthermore, the cross-attention network model is defined as:
[0070]
[0071] Where Q = W q *I, K = W k *P,V=W v *P;
[0072] W q is the query parameter matrix, W k is the key parameter matrix, W v is the value parameter matrix, d is the size of the feature vector dimension, Q is the query vector, K is the key vector, and V is the value vector.
[0073] The present application designs a data processing method for improving the relevance of retrieval enhanced generated text, including: inputting the input question text into a vector model to obtain question text vector data; performing knowledge base retrieval according to the input question text to obtain the knowledge base text; performing sentence segmentation on the knowledge base text to obtain retrieval reference sentences, and inputting them into the vector model to obtain sentence text vector data; constructing a relevance evaluation model and training it; inputting the sentence text vector data and the question text vector data into the relevance evaluation model, performing relevance numerical calculation, retaining the retrieval reference sentences with relevance values greater than 0.5, removing the remaining retrieval reference sentences, and screening out the streamlined knowledge base text; integrating the screened streamlined knowledge base text and the input question text, and inputting them into the large language model. The present application can use the relevance scoring model to remove unnecessary knowledge base text information and effectively improve the accuracy of question answers.
[0074] The above description is only the preferred embodiment of the present invention, which does not limit the protection scope of the present invention. For those skilled in the art, the present invention can be modified and varied in various ways. Within the spirit and principle of the present invention, any change, modification, replacement, integration and parameter change of these embodiments by conventional substitution or capable of achieving the same function without departing from the principle and spirit of the present invention shall fall within the protection scope of the present invention.
Claims
1. A data processing method for improving the relevance of retrieval-enhanced generated text, characterized in that: The following steps are involved: Step S1, inputting the input question text into the vector model to obtain question text vector data; Step S2: search the knowledge base according to the input question text to obtain the knowledge base text; Step S3, segmenting the knowledge base text into sentences to obtain retrieval reference sentences, and inputting the retrieval reference sentences into the vector model to obtain sentence text vector data; Step S4: constructing a correlation evaluation model and performing training; Step S5: input the sentence text vector data and the question text vector data into the relevance evaluation model, perform relevance value calculation, retain the retrieval reference sentences with relevance values greater than 0.5, remove the remaining retrieval reference sentences, and screen out the simplified knowledge base text; Step S6: Integrate the simplified knowledge base text selected in step S5 and the input question text, and input them into the large language model.
2. A data processing method for improving the relevance of retrieval enhancement generated text according to claim 1, characterized in that: The method for constructing the relevance evaluation model includes: Step S401, perform knowledge base search on the training question text to obtain the training knowledge base text; Step S402: Segment the training knowledge base text into sentences to obtain training retrieval reference sentences, and input the training retrieval reference sentences into a vector model to obtain training sentence text vector data; Step S403, inputting the training question text into the vector model to obtain training question text vector data; Step S404: input the training question text vector data and the training sentence text vector data into the cross attention network model for fusion to obtain a fused feature vector; Step S405: input the fused feature vector into a feedforward neural network to obtain a correlation value.
3. A data processing method for improving the relevance of retrieval enhancement generated text according to claim 2, characterized in that: The training method of the relevance evaluation model includes: Step S411, constructing a training data set by manual screening, wherein the training data set includes training questions and corresponding answer retrieval reference texts, i.e., corresponding answer knowledge base texts; Step S412, decomposing the response search reference text into multiple sentences; Step S413: Combine a single training question with multiple sentences one by one, and input them into the large language model as prompt word texts respectively; record whether the selected sentences can provide effective support for the output answer through manual review, and if so, mark the corresponding sentence as 1, otherwise mark it as -1; Step S414, and so on, similarly process each combination of training questions, and then use the batch training method to pass the training question text and the corresponding sentence text into the relevance evaluation model, and finally obtain the relevance score q of each sentence. The relevance score q and the manually annotated relevance score q' together constitute the mean square error loss function. L(q,q′)=E[(q-q′) 2 ] Among them, E represents the expectation of the data set; Step S415: Finally, the AdamW optimizer is used to perform iterative optimization and update the correlation evaluation model parameters.
4. A data processing method for improving the relevance of retrieval enhancement generated text according to claim 2, characterized in that: The feedforward neural network contains two linear layers.
5. A data processing method for improving the relevance of retrieval enhancement generated text according to claim 2, characterized in that: The cross-attention network model is defined as: where Q = W q *I, K = W k *P, V = W v *P; W q is the query parameter matrix, W k is the key parameter matrix, W v is the value parameter matrix, d is the size of the feature vector dimension, Q is the query vector, K is the key vector, and V is the value vector; P represents the training sentence text vector data; I represents the training question text vector data.