RAG Document Chunk Retrieval Using Synthetic Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) struggle to provide accurate responses to queries related to information outside their training dataset, such as enterprise-specific knowledge, necessitating a more efficient and automated method for generating responses using retrieval augmented generation (RAG) systems.
Innovation Solution
A system that subdivides documents into chunks, generates synthetic information using a synthetic generation LLM, creates vectors with an embedding model, and uses an information retrieval system to identify relevant chunks, followed by a re-ranker LLM to rank and select a subset of chunks for generating a response using a generation LLM.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If LLM is trained on vast amount of data from various fields, then LLM can generate human-like text based on recognized patterns, but LLM cannot provide accurate answers to queries related to specific domain or enterprise internal knowledge
Solution Approach 1:
The patent introduces an information retrieval system as an intermediary between the LLM and enterprise documents. The system retrieves relevant document chunks based on the query, then provides these retrieved contents to the LLM as additional context. This mediator approach allows the LLM to access enterprise-specific information without retraining, resolving the contradiction between versatility and reliability on domain-specific knowledge.
2Reliability
If enterprise has internal and external documents containing information to resolve customer queries, then accurate responses can be generated, but it is cumbersome for agents to locate relevant information manually
Solution Approach 1:
The system enables automated information retrieval where the computer itself searches and retrieves relevant document chunks based on the query, without requiring human agents to manually search through enterprise documents. The retrieval system automatically processes queries, ranks document chunks by relevance, and provides the information needed to generate accurate responses, eliminating the time loss associated with manual information location.
3Ease of manufacture
If RAG system retrieves information from external knowledge base and provides it to LLM, then LLM output is enhanced without re-training, but system complexity increases
Solution Approach 1:
The patent segments the system into distinct functional modules: an information retrieval system that processes queries and searches document corpora, an embedding model that converts text to vectors, a ranking system that orders retrieved chunks by relevance, and the LLM that generates final responses. This segmentation allows each component to be independently optimized and updated, making the system easier to manufacture and maintain despite its complexity. The modular architecture enables knowledge updates by simply adding or modifying documents in the corpus without affecting other system components.
Data Source
AI summary
Systems and methods for retrieving information from a corpus of documents that is relevant to a query. The method comprising: subdividing each document in the corpus of documents into one or more chunks; using a synthetic generation LLM to generate at least one piece of synthetic information related to each chunk; generating, using an embedding model a plurality of vectors for each chunk, the plurality of vectors for a chunk comprising a vector generated from the chunk, and a vector generated from each of the at least one piece of synthetic information related to that chunk; and using an information retrieval system to identify, from the plurality of vectors for each chunk, a set of chunks that are relevant to a query.