Method and device for improving retrieval step in RAG through context retrieval
By combining context embedding and context BM25 technology, the search steps of RAG systems are improved, and the shortcomings of traditional RAG systems in context information processing and precise matching are solved, achieving more efficient and accurate information retrieval.
Patent Information
- Application Number
- CN202510046408.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-13
- Publication Date
- 2025-04-22
AI Technical Summary
Traditional RAG systems remove context information when encoding information, resulting in the inability to fully understand the meaning and context dependence of the information block, and when processing queries containing unique identifiers or technical terms, relevant information cannot be effectively retrieved, which increases computing cost and delay.
The search steps of the RAG system are improved by combining context embedding and context BM25 technology. Specific steps include knowledge base preprocessing, creating TF-IDF encoding and semantic embedding, combining BM25 and semantic embedding techniques, context embedding and context BM25, prompt cache and rearrangement Reranking.
It significantly improves the accuracy of retrieving relevant information from large knowledge bases, reduces the cost and delay of handling large knowledge bases, improves efficiency, and demonstrates the effectiveness of the technology in various knowledge areas.
Smart Images
Figure CN119862264A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and specifically provides a method and device for improving a retrieval step in a RAG through context retrieval. Background Art
[0002] Information retrieval (IR) has always been a core research area in the field of artificial intelligence, especially in the development of natural language processing (NLP) technology. With the explosive growth of the Internet and digital information, users have an increasing demand for fast and accurate access to required information. Traditional information retrieval systems mainly rely on keyword matching and statistical methods such as TF-IDF and BM25, which work well in some cases, but have limitations in processing complex queries and understanding user intent.
[0003] To address these issues, the Retrieval Augmented Generation (RAG) system was developed. By combining the advantages of information retrieval and sequence generation, the RAG system aims to retrieve the most relevant information to the user's query from a large knowledge base and combine it with the user input to generate a richer and more accurate response. The core idea of RAG is to decompose the knowledge base into smaller text blocks, and then use an embedding model to convert these blocks into vector embeddings, which are stored in a vector database for search and retrieval by semantic similarity.
[0004] Although RAG systems have made some progress in processing large-scale knowledge bases and complex queries, they still face some challenges in practical applications. First, traditional RAG systems often remove contextual information when encoding information, which results in the system being unable to fully understand the true meaning and contextual dependencies of information blocks, and thus being unable to accurately retrieve relevant information from the knowledge base. For example, in a financial information collection, an information block may contain "the company's revenue increased by 3% over the previous quarter", but if the specific company and time period to which the information block belongs are not provided, the system cannot determine which company or time period the growth refers to. Second, when processing queries containing unique identifiers or technical terms, traditional RAG systems often rely on the semantic understanding capabilities of embedding models, while ignoring the importance of exact matching. This may lead to the omission of key information in some cases, because the embedding model may not capture all the precise semantic relationships. For example, when processing scientific papers or technical documents, precise terms and concepts are crucial to understanding the content of the document, and traditional RAG systems may not be able to effectively retrieve these key information. In addition, as the size of the knowledge base grows, the cost and efficiency issues of traditional RAG systems in processing large knowledge bases are becoming increasingly prominent. Traditional RAG systems need to generate and process a large number of vector embeddings for each query, which not only increases the computational cost but also prolongs the response time. In practical applications, especially in scenarios that require real-time response, such delays and costs are unacceptable.
[0005] To address the above problems, researchers have proposed a variety of improvement methods. Some methods try to enhance retrieval by adding general document summaries, but experimental results show that this method has limited benefits. Other methods, such as hypothetical document embedding (HyDE) and summary-based indexing, although attractive in theory, are not ideal in practical applications. Unlike contextual retrieval methods, these methods do not make full use of the contextual information of the block to improve the accuracy of retrieval. Summary of the invention
[0006] The present invention aims at the above-mentioned deficiencies of the prior art and provides a method for improving the retrieval step in RAG through context retrieval with strong practicability.
[0007] A further technical task of the present invention is to provide a device that is reasonably designed, safe and applicable for improving the retrieval step in RAG through context retrieval.
[0008] The technical solution adopted by the present invention to solve its technical problem is:
[0009] A method to improve the retrieval step in RAG by contextual retrieval, combining contextual embedding and contextual BM25 techniques, has the following steps:
[0010] S1, knowledge base preprocessing;
[0011] S2, create TF-IDF encoding and semantic embedding;
[0012] S3, combining BM25 and semantic embedding technology;
[0013] S4, context embedding;
[0014] S5, Context BM25;
[0015] S6, prompt cache;
[0016] S7. Reranking.
[0017] Furthermore, in step S1, in the RAG system, the knowledge base needs to be preprocessed first, including decomposing the knowledge base into small text blocks, which are then used to generate vector embeddings and stored in a vector database.
[0018] Furthermore, in step S2, for each text block, TF-IDF encoding and semantic embedding are created. TF-IDF measures the importance of words and forms the basis of BM25; semantic embedding captures the semantic information of the text block through the embedding model.
[0019] Further, in step S3, BM25 exact matching is used to find the top blocks, and embedding is used to find the top blocks based on semantic similarity. Then, the results of these two methods are combined through ranking fusion technology and duplicates are removed.
[0020] Furthermore, in step S4, a chunk-specific explanatory context is appended to each chunk before embedding to preserve the contextual information of the chunk.
[0021] Furthermore, in step S5, for queries applicable to queries containing unique identifiers or technical terms, the context information of the blocks is also considered when creating the BM25 index.
[0022] Furthermore, in step S6, using the hint cache function, there is no need to pass in a reference document for each block, but rather the document is loaded into the cache once and then the previously cached content is referenced.
[0023] Furthermore, in step S7, in the last step, context retrieval and re-ranking techniques are combined to obtain more performance improvements.
[0024] An apparatus for improving a retrieval step in a RAG by contextual retrieval, comprising: at least one memory and at least one processor;
[0025] The at least one memory is used to store a machine-readable program;
[0026] The at least one processor is configured to call the machine-readable program to execute a method for improving a search step in a RAG through contextual search.
[0027] Compared with the prior art, the method and device for improving the retrieval step in RAG by context retrieval of the present invention has the following outstanding beneficial effects:
[0028] The present invention significantly improves the accuracy of retrieving relevant information from large knowledge bases through context embedding and context BM25. Through prompt caching and reordering technology, this technology reduces the cost and delay of processing large knowledge bases and improves efficiency. This technology can be extended to huge knowledge bases, far beyond the scope that a single prompt can adapt to, and has good scalability. By combining BM25 and embedding models, this technology balances precise term matching with broader semantic understanding, providing more comprehensive and accurate retrieval results. Experiments were conducted in various knowledge fields (code bases, novels, ArXiv papers, scientific papers), embedding models, retrieval strategies and evaluation indicators, and the experimental results proved the effectiveness of this technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0030] Attached Figure 1 is a flowchart of a method for improving the retrieval step in RAG through contextual retrieval. DETAILED DESCRIPTION
[0031] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] A best embodiment is given below:
[0033] like Figure 1 As shown, a method for improving the retrieval step in RAG by context retrieval in this embodiment has the following steps:
[0034] S1, knowledge base preprocessing;
[0035] In the RAG system, the knowledge base needs to be preprocessed first, including breaking the knowledge base into smaller text blocks, usually no more than a few hundred tokens. These blocks are then used to generate vector embeddings and stored in a vector database for semantic similarity search.
[0036] S2, create TF-IDF encoding and semantic embedding;
[0037] For each text chunk, we create TF-IDF encoding and semantic embedding. TF-IDF (Term Frequency-Inverse Document Frequency) measures the importance of words and forms the basis of BM25. Semantic embedding captures the semantic information of the text chunk through an embedding model.
[0038] S3, combining BM25 and semantic embedding technology;
[0039] BM25 is used to find the top blocks based on exact matching, while embedding is used to find the top blocks based on semantic similarity. The results of these two methods are then combined through a ranking fusion technique and duplicates are removed.
[0040] S4, context embedding;
[0041] Before embedding, a chunk-specific explanatory context is appended to each chunk to preserve the chunk’s contextual information. This step solves the problem of context loss in traditional RAG systems and improves retrieval accuracy.
[0042] S5, Context BM25;
[0043] When creating a BM25 index, the contextual information of the block is also taken into account. This approach is particularly suitable for queries containing unique identifiers or technical terms, improving the ability to accurately match.
[0044] original_chunk="The company's revenue grew by 3%over the previous quarter."
[0045] contextualized_chunk="This chunk is from an SEC filing on ACME corp's performance in Q2 2023; the previous quarter's revenue was $314million. The company's revenue grew by 3%over the previous quarter."
[0046] S6, Prompt Caching;
[0047] To reduce the cost of context retrieval, we use a special hint cache feature. This way, instead of passing in a reference document for each chunk, the document is loaded into the cache once and then references the previously cached content.
[0048] S7, Reranking;
[0049] In the last step, context retrieval and re-ranking techniques are combined to obtain more performance improvements. Re-ranking ensures that only the most relevant chunks are passed to the model, providing better responses and reducing cost and latency.
[0050] By using the above method, the present invention can cache frequently used prompts between API calls, reducing latency by more than 2 times and reducing costs by up to 90%, which is obviously faster and more cost-effective. As the knowledge base grows, contextual retrieval will play a greater role.
[0051] The context embedding generator is responsible for generating chunk-specific explanatory context for each chunk of text. This is achieved by using the Claude model and specific prompts that ask the model to provide concise, chunk-specific context.
[0052] The prompt cache system is responsible for caching frequently used prompts, reducing latency and costs between API calls. This system allows developers to cache prompts between API calls, significantly improving the efficiency of context retrieval.
[0053] The reranking model is responsible for scoring and filtering the large number of chunks returned by the initial retrieval, ensuring that only the most relevant chunks are passed to the model. This approach improves the model’s efficiency and response quality by filtering out irrelevant chunks of information.
[0054] Based on the above method, a device for improving the retrieval step in RAG through context retrieval in this embodiment includes: at least one memory and at least one processor;
[0055] The at least one memory is used to store a machine-readable program;
[0056] The at least one processor is configured to call the machine-readable program to execute a method for improving a search step in a RAG through contextual search.
[0057] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any technical solutions that conform to the above-mentioned specific implementations of the present invention and any appropriate changes or substitutions made by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.
[0058] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A method for improving the retrieval step in RAG by contextual retrieval, characterized in that: Combining context embedding and context BM25 technology, the steps are as follows: S1, knowledge base preprocessing; S2, create TF-IDF encoding and semantic embedding; S3, combining BM25 and semantic embedding technology; S4, context embedding; S5, Context BM25; S6, prompt cache; S7. Reranking.
2. A method for improving the retrieval step in RAG by contextual retrieval according to claim 1, characterized in that: In step S1, in the RAG system, the knowledge base first needs to be preprocessed, including decomposing the knowledge base into small text blocks, which are then used to generate vector embeddings and stored in a vector database.
3. A method for improving the retrieval step in RAG by contextual retrieval according to claim 2, characterized in that: In step S2, for each text block, TF-IDF encoding and semantic embedding are created. TF-IDF measures the importance of words and forms the basis of BM25; semantic embedding captures the semantic information of the text block through the embedding model.
4. A method for improving the retrieval step in RAG by contextual retrieval according to claim 3, characterized in that: In step S3, BM25 exact matching is used to find the top blocks, while embedding is used to find the top blocks based on semantic similarity. Then, the results of these two methods are combined through ranking fusion technology and duplicates are removed.
5. A method for improving the retrieval step in RAG by contextual retrieval according to claim 4, characterized in that: In step S4, a chunk-specific explanatory context is appended to each chunk before embedding to preserve the contextual information of the chunk.
6. A method for improving the retrieval step in RAG by contextual retrieval according to claim 5, characterized in that: In step S5, for queries applicable to queries containing unique identifiers or technical terms, the contextual information of the blocks is also considered when creating the BM25 index.
7. A method for improving the retrieval step in RAG by contextual retrieval according to claim 6, characterized in that: In step S6, using the hint cache function, there is no need to pass in a reference document for each block, but the document is loaded into the cache once and then the previously cached content is referenced.
8. A method for improving the retrieval step in RAG by contextual retrieval according to claim 7, characterized in that: In step S7, in the last step, context retrieval and re-ranking techniques are combined to obtain more performance improvement.
9. A device for improving the retrieval step in RAG by contextual retrieval, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 8.