Retrieval augmented generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models struggle with hallucinations and lack of responsiveness to recent or private information due to their training data limitations, making it difficult to generate accurate responses.
Innovation Solution
Augmenting prompts with relevant information retrieved from a vector database using embeddings, allowing the model to consider context beyond its training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large language models are trained with extensive data to improve accuracy and reduce hallucinations, then response accuracy improves, but training cost and time increase significantly
Solution Approach 1:
The system performs preliminary actions by pre-computing and storing embeddings of external knowledge sources in a vector database before queries are received. This allows the model to quickly retrieve relevant information without performing time-consuming training or real-time external data processing during the actual response generation.
Solution Approach 2:
The patent introduces an intermediary vector database that mediates between the language model and external knowledge sources. The database stores pre-computed embeddings as intermediaries, allowing the model to access external information efficiently through similarity search without direct integration with external systems during inference.
2Ease of manufacture
If models are trained offline with static data, then training process is simplified, but the model becomes unaware of recent or private data
Solution Approach 1:
The system transforms the static training approach into a dynamic one by maintaining an external knowledge base that can be continuously updated with new information. The model itself remains static, but its access to information becomes dynamic through the vector database that can incorporate recent and private data without retraining.
Solution Approach 2:
The vector database serves as an intermediary layer that bridges the static model with dynamic external knowledge. It mediates between the model's fixed parameters and the need for current information, allowing the system to adapt to recent data while keeping the model training process simple and offline.
3Productivity
If the model generates responses without external context, then response generation is faster, but hallucinations increase and relevancy decreases
Solution Approach 1:
The system performs preliminary retrieval of relevant embeddings from the vector database before generating responses. By pre-fetching and storing contextual information, the model can quickly access needed data during inference without performing complex real-time searches, thus maintaining speed while improving accuracy.
Solution Approach 2:
The vector database acts as an intermediary that provides contextual information to the model during response generation. This intermediary layer enables the model to access relevant external knowledge efficiently, reducing hallucinations and improving relevancy while maintaining fast response times through optimized similarity search.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques retrieval augmented generation of language model responses using an embedding database. Embeddings for data is stored in an embedding database. When a prompt related to the data is received, relevant embeddings may be retrieved from the database and used generate an augmented prompt based on the initial prompt and the retrieved embeddings from the database. The augmented prompt can be input into a machine learning model. Although the model may be unaware of the data from which the embeddings of the embedding database were generated, the augmented prompt enables the model to use the data to improve breadth and depth of responses.