RAG performance improvement method based on process optimization

Abstract

The invention discloses an RAG performance improvement method based on process optimization. The method comprises the following steps: S1, preprocessing text data; s2, performing hierarchical text segmentation, obtaining a document, storing the text by utilizing a database after the document is segmented, converting the text into vectors, and storing the vectors into a vector knowledge base; s3, querying and rewriting to obtain a rewritten question; s4, combining parent document search with mixed retrieval, and retrieving related contexts for the rewritten questions; s5, performing rearrangement and denoising to obtain a final context; and S6, generating an answer, and sending the final context into a large language model for understanding questions and answers. According to the method, the links of text preprocessing, text partitioning, text embedding, query optimization, retrieval, retrieval post-processing, generation and the like are sequentially optimized according to the data flowing direction, the semantic retrieval precision is improved, and the performance of RAG in the aspects of text retrieval precision, answer generation quality, user interaction and the like can be remarkably improved; and the flexibility and the practicability of the system are enhanced, so that the system is more excellent in practical application.

Description

Technical Field

[0001] The present invention relates to the technical field of language processing models, and in particular to a RAG performance improvement method based on process optimization. Background Art

[0002] Although large language models (LLMs) have achieved remarkable success, they still face some important limitations, especially in specific domains or knowledge-intensive tasks, and in particular when dealing with queries that are beyond their training data or require current information, which may produce so-called "hallucinations". To address these problems, Retrieval-Augmented Generation (RAG) retrieves relevant document fragments from external knowledge bases through semantic similarity calculation to enhance the performance of LLMs. By referring to external knowledge, RAG effectively reduces the problem of generating factually incorrect content. The integration of RAG and LLMs has been widely used, making it a key technology to promote the development of chatbots and enhance the applicability of LLMs in practical applications.

[0003] The general RAG application process mainly includes two stages:

[0004] (1) Data preparation stage: (A) Data extraction → (B) Chunking → (C) Embedding → (D) Data storage;

[0005] (2) Retrieval generation phase: (A) Question vectorization → (B) Match data according to question query → (C) Obtain index data → (D) Inject data into prompt → (E) LLM generates answer.

[0006] At present, although RAG technology has been applied in many products and has surpassed traditional LLM in performance, there is still great potential for improvement in the quality and accuracy of its text generation. For example, the general RAG process has the following problems in actual application:

[0007] (1) Retrieval problem: There is often a conflict between precision and recall in the retrieval stage, which ultimately leads to inconsistent recall or irrelevant chunks and the loss of key information.

[0008] (2) Generation difficulties: When generating responses, the model may face the problem of hallucination, that is, the content it generates is not supported by the retrieved context. This stage may also suffer from output irrelevance, toxicity, or bias, which affects the quality and reliability of the response.

[0009] (3) Enhancement barriers: Integrating retrieved information with different tasks sometimes produces incoherent or disconnected outputs. When retrieving similar information from multiple sources, the process may also encounter redundancy, resulting in duplicate responses. Faced with complex problems, a single retrieval based on the original query may not be sufficient to obtain sufficient contextual information.

[0010] Therefore, the processing flow of RAG technology can still be optimized and improved in many aspects to improve the performance of RAG. Summary of the invention

[0011] In view of the above problems, the present invention provides a RAG performance improvement method based on process optimization, comprising the following steps:

[0012] S1, text data preprocessing, extracting the text information of the document and removing unnecessary information;

[0013] S2, hierarchical text segmentation, obtains documents, divides the documents and stores the text in the database, converts the text into vectors and stores them in the vector knowledge base;

[0014] S3, query rewriting, obtaining the user question, rewriting the user question in light of the context, and obtaining the rewritten question;

[0015] S4, combining parent document search with hybrid retrieval, combining hybrid retrieval and parent document retrieval to retrieve relevant context for the rewritten question;

[0016] S5, rearrangement and denoising, integrating, filtering, and rearranging the retrieved context to obtain the final context;

[0017] S6, generates an answer and sends the final context to a large language model for question and answer understanding.

[0018] As a further illustration of the present invention, in step S1, the text data preprocessing includes removing irrelevant text / documents, identifying and correcting errors, and replacing enhanced semantics.

[0019] Furthermore, in step S2, the original text data is segmented in two levels, including using a parent document segmenter to divide the text into large document blocks, and using a child document segmenter to further subdivide the parent document to form smaller text segments.

[0020] Furthermore, both the parent document segmenter and the child document segmenter employ recursive character-by-character segmentation, where recursive chunking uses a set of separators to divide the input text into smaller chunks in a hierarchical and iterative manner; if an initial attempt to segment the text does not generate chunks of the desired size or structure, it recursively calls itself on the generated chunks using different separators or criteria until the desired chunk size or structure is achieved.

[0021] Furthermore, in step S2, a database is used to store text indexes and vector indexes.

[0022] Furthermore, the vector index converts text into vectors using an Embedding model.

[0023] Furthermore, in step S4, keyword retrieval and vector retrieval are used to construct a retrieval module of the RAG system, and N sub-document blocks are respectively recalled through keyword retrieval and vector retrieval.

[0024] Furthermore, in step S5, repeated sub-document blocks are first filtered out, and then the filtered sub-document blocks are rearranged to remove completely irrelevant sub-document blocks.

[0025] Furthermore, in step S6, the rearranged sub-document block id is taken to the corresponding parent text block, combined with the user question, and input into the large language model, and the user question is answered using the large language model.

[0026] Furthermore, if the user's question is unclear, the user will be asked follow-up questions, and the number of questions is limited to 3.

[0027] Beneficial effects of the present invention:

[0028] The present invention optimizes the links of text preprocessing, text segmentation, text embedding, query optimization, retrieval, post-retrieval processing and generation in sequence according to the direction of data flow, and adopts one or more strategies to improve them, thereby improving the accuracy of semantic retrieval, and can significantly improve the performance of RAG, especially in terms of text retrieval accuracy, answer generation quality and user interaction. It not only optimizes the various functions of the RAG system, but also enhances the flexibility and practicality of the system, making it perform well in practical applications, including the following aspects:

[0029] 1. System integration: Effectively integrate multiple links such as text preprocessing, hierarchical text segmentation, hybrid retrieval mechanism, query rewriting, result re-ranking and answer generation to form an efficient and accurate information retrieval and generation system.

[0030] 2. Process optimization: Specific optimization plans are proposed for each technical link, including but not limited to denoising, semantic enhancement, hybrid retrieval strategies, etc., to improve data quality, accuracy and relevance of information retrieval, and the quality of RAG answer results.

[0031] 3. Improved user experience: A follow-up question mechanism was introduced to improve the user experience during the interaction process and ensure that users can get satisfactory answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1The present invention is a flowchart of a RAG performance improvement method based on process optimization. DETAILED DESCRIPTION

[0033] The following is a detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0034] In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", "first", "second", etc., indicating directions or positions or sequential relationships, are based on the directions or positions or sequential relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.

[0035] The invention discloses a RAG performance improvement method based on process optimization, characterized in that it comprises the following steps:

[0036] S1, text data preprocessing, extracting text information from documents and removing unnecessary information. Specifically, in this embodiment, in the raw data processing stage, the data is cleaned and standardized to improve data quality and enhance the relevance and accuracy of subsequent retrieval. In specific applications, the preprocessing of text data includes the following operations:

[0037] (1) Remove irrelevant text / documents: All documents irrelevant to the LLM (Large Language Model) answer should be deleted and noisy data should be cleaned. This includes removing special characters, stop words (such as "the", "a" and other common English words) and HTML tags.

[0038] (2) Identify and correct errors: This step involves identifying and correcting spelling errors, grammatical errors, etc. in the text.

[0039] (3) Enhanced semantic replacement: Replacing pronouns with actual nouns in a text block can significantly enhance semantic understanding during the retrieval process and help improve the accuracy and relevance of information retrieval.

[0040] S2, hierarchical text segmentation, obtain documents, divide the documents and store the text in the database, convert the text into vectors and store them in the vector knowledge base. Specifically, in this embodiment, the segmentation method of the original text data is to perform two-level segmentation to capture the document hierarchy more carefully, using a parent document segmenter and a child document segmenter respectively. The chunck_size of the parent document segmenter is longer, and the context of each fragment is retained as much as possible; the chunck_size of the child document segmenter is shorter, and smaller documents can improve the recall rate. First, use the parent document segmenter to divide the text into large document blocks; then, use the child document segmenter to further subdivide the parent document to form smaller text fragments. This balances the two conflicting requirements of smaller text blocks matching the user's questions and larger document blocks retaining enough content. Both the child document segmenter and the parent document segmenter use recursive character segmentation. The recursive block division is as follows:

[0041] Recursive chunking divides the input text into smaller chunks in a hierarchical and iterative manner using a set of delimiters (typically ["\n\n","\n","",""]). If the initial attempt to chunk the text does not produce chunks of the desired size or structure, the method recursively calls itself on the resulting chunks, using different delimiters or criteria, until the desired chunk size or structure is achieved.

[0042] The segmented documents need to be stored, and a database can be used to store text indexes and vector indexes. In this embodiment, the database for storing text indexes is an Elasticsearch database, and the vector index uses an Embedding model (such as a bge-m3 model) to convert text into vectors.

[0043] S3, query rewriting, obtain the user's question, rewrite the user's question in context, and obtain the rewritten question. In this step, the LLM's ability is used to rephrase the user's query and try again. The LLM can be prompted to rewrite the user's question to align the query and document semantics. For example, when the user's question contains ambiguous references or is related to the context, by rewriting the question, we can retrieve information more effectively. The prompt is:

[0044]

[0045] Specific example: User question 1: I want to change my data package. The large language model answers: We have a variety of data packages to choose from, including basic packages, family packages, and unlimited data packages. User question 2: Which one is more cost-effective? If question 2 is directly searched in the subsequent knowledge base, the word "that" cannot clearly refer to "basic package, family package, and unlimited data package", so the user question needs to be rewritten. You can feed the historical context into the large language model, write a prompt, and let the large language model generate a new question. The specific example is as follows:

[0046]

[0047] The new question obtained by feeding it into the large language model is: "Among the basic package, family package and unlimited data package, which package has a relatively higher cost-effectiveness?", which achieves question rewriting and eliminates the reference relationship.

[0048] S4, parent document search is combined with hybrid retrieval. The rewritten question is retrieved for relevant context by combining hybrid retrieval and parent document retrieval. The retrieval accuracy is improved by combining parent document search and hybrid retrieval. When searching at the parent document level, traditional retrieval methods (such as BM25, etc.) and embedded vector retrieval methods are combined to achieve higher retrieval accuracy. Through this hybrid retrieval mechanism, user questions are matched with sub-documents, and then the main document to which the sub-document belongs is sent to LLM. Shorter sub-documents can be matched more accurately, and longer parent documents can retain more contextual information, thereby improving the quality of RAG answer results. The specific operations are:

[0049] Parent document search: Step 2: Hierarchical text segmentation first divides the document into a larger parent document, and then divides the parent document into two levels of shorter child documents. The user question will be matched with the child document, and then the parent document to which the child document belongs will be sent to LLM.

[0050] Hybrid retrieval: Hybrid retrieval achieves the synergy of different retrieval technologies by mixing multiple retrieval methods, thereby maximizing the fact recall rate. The present invention adopts the method of keyword retrieval (such as BM25) + vector retrieval (such as bge-m3) to construct the retrieval module of the RAG system. The keyword retrieval and vector retrieval recall N answers respectively.

[0051] S5, re-ranking and denoising, integrates, filters, and re-ranks the retrieved context to obtain the final context. The re-ranking model performs a more in-depth relevance evaluation and ranking of the initial search results to ensure that the results finally displayed to the user are more in line with their query intent. Specifically, for the document results obtained from the initial search (the above-mentioned BM25 and bge-m3 mixed search), first filter out duplicate sub-document blocks, and then use the bge-reranker-v2-m3 model to re-rank the results, remove completely irrelevant sub-document blocks, and reduce noise.

[0052] S6, generate an answer, and send the final context to the large language model for question understanding and answering. Specifically, the id of the rearranged sub-document block is taken to the corresponding parent text block, combined with the user's question, and input into the large language model (such as Qwen2.5-14B), and the user's question is answered using the capabilities of the large language model Qwen2.5-14B.

[0053] In the preferred implementation, one of the situations considered is to add a follow-up question mechanism when the RAG system cannot give an answer to the user's unclear query. Specifically, add "If the user's question cannot be answered based on the background knowledge, the user will be asked follow-up questions based on the background knowledge content, and the number of questions is limited to 3" in the prompt, and the user gradually clarifies his or her question in multiple rounds of guidance, so that he or she can get a suitable answer.

[0054] Through the above optimization method, the present invention can significantly improve the performance of RAG, especially in terms of text retrieval accuracy, answer generation quality and user interaction.

[0055] The above only describes the preferred embodiments of the present invention, but it cannot be understood as limiting the claims. The present invention is not limited to the above embodiments, and its specific structure is allowed to be changed. In short, all changes made within the protection scope of the independent claims of the present invention are within the protection scope of the present invention.

Claims

1. A RAG performance improvement method based on process optimization, characterized in that: The following steps are involved: S1, text data preprocessing, extracting the text information of the document and removing unnecessary information; S2, hierarchical text segmentation, obtains documents, divides the documents and stores the text in the database, converts the text into vectors and stores them in the vector knowledge base; S3, query rewriting, obtaining the user question, rewriting the user question in light of the context, and obtaining the rewritten question; S4, combining parent document search with hybrid retrieval, combining hybrid retrieval and parent document retrieval to retrieve relevant context for the rewritten question; S5, rearrangement and denoising, integrating, filtering, and rearranging the retrieved context to obtain the final context; S6, generates an answer and sends the final context to a large language model for question and answer understanding.

2. The RAG performance improvement method based on process optimization according to claim 1 is characterized in that: In step S1, text data preprocessing includes removing irrelevant text / documents, identifying and correcting errors, and replacing them to enhance semantics.

3. The RAG performance improvement method based on process optimization according to claim 1 is characterized in that: In step S2, the original text data is segmented in two levels, including using a parent document segmenter to divide the text into large document blocks, and using a child document segmenter to further subdivide the parent document to form smaller text segments.

4. The RAG performance improvement method based on process optimization according to claim 3 is characterized in that: Both the parent document segmenter and the child document segmenter employ recursive character-by-character segmentation, where recursive chunking uses a set of separators to divide the input text into smaller chunks in a hierarchical and iterative manner; if the initial attempt to segment the text does not produce chunks of the desired size or structure, it recursively calls itself on the generated chunks using different separators or criteria until the desired chunk size or structure is achieved.

5. The RAG performance improvement method based on process optimization according to claim 1 is characterized in that: In step S2, a database is used to store text indexes and vector indexes.

6. The RAG performance improvement method based on process optimization according to claim 5 is characterized in that: The vector index converts text into vectors using the Embedding model.

7. The RAG performance improvement method based on process optimization according to claim 1 is characterized in that: In the step S4, keyword retrieval and vector retrieval are used to construct a retrieval module of the RAG system, and N sub-document blocks are respectively recalled through keyword retrieval and vector retrieval.

8. The RAG performance improvement method based on process optimization according to claim 1 is characterized in that: In step S5, repeated sub-document blocks are first filtered out, and then the filtered sub-document blocks are rearranged to remove completely irrelevant sub-document blocks.

9. The RAG performance improvement method based on process optimization according to claim 1 is characterized in that: In step S6, the rearranged sub-document block id is obtained into the corresponding parent text block, combined with the user question, and input into the large language model, and the user question is answered using the large language model.

10. The RAG performance improvement method based on process optimization according to claim 1 is characterized in that: If the user's question is unclear, ask the user follow-up questions, limiting the number of questions to 3 or less.

Images