In-Memory Data Grid Vector Database for Scalable RAG Ingestion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data analytics systems face challenges in efficiently storing and searching large numbers of dense vector embeddings to support generative artificial intelligence and retrieval-augmented generation processes, particularly in handling large volumes of data from diverse document sources.
Innovation Solution
Utilizing an in-memory data grid as a vector database with linearly-scalable data ingestion, enabling parallel processing of content ingestion and vector similarity searches, and supporting various document sources like HTTP URLs and cloud storage services.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional database systems are used to store and search vector embeddings, then data persistence and scalability are achieved, but search speed and processing efficiency deteriorate
Solution Approach 1:
The system segments data into two categories: frequently accessed vector embeddings stored in memory for fast search, and less frequently accessed data stored on disk for persistence. This segmentation allows the system to achieve both fast search speeds and data persistence by placing different data types in appropriate storage media.
Solution Approach 2:
The patent introduces an intermediary mechanism (memory-mapped files or hybrid storage layer) that bridges between in-memory storage and disk storage. This intermediary allows vector embeddings to be quickly accessed from memory while maintaining the ability to persist data to disk, resolving the contradiction between speed and reliability.
2Productivity
If more computing hardware resources are allocated for parallel processing, then processing efficiency and throughput improve, but system complexity and resource management difficulty increase
Solution Approach 1:
The patent creates a universal data grid system that can perform multiple functions (storage, computation, indexing, searching) using the same underlying infrastructure. This multi-functionality allows the system to efficiently utilize hardware resources without requiring separate specialized systems for each function, thereby reducing overall system complexity while maintaining high productivity.
Solution Approach 2:
The data grid implementation provides self-service capabilities for resource management, including automatic data distribution, load balancing, and fault tolerance. This self-service approach reduces the complexity of manual resource management while enabling efficient parallel processing across multiple hardware resources.
3Productivity
If vector embeddings are stored in memory for fast access, then search efficiency improves, but memory consumption and cost increase
Solution Approach 1:
The patent applies local quality by storing only the most frequently accessed vector embeddings in memory, while less frequently accessed embeddings are stored on disk. This selective approach ensures that memory resources are used efficiently for data that provides the most value in terms of search efficiency, thereby improving search performance without unnecessarily increasing memory consumption.
Solution Approach 2:
The system implements partial action by loading a subset of vector embeddings into memory based on access patterns and query requirements, rather than loading all embeddings. This allows the system to achieve good search efficiency for common queries while keeping memory consumption manageable by selectively loading only necessary data into fast storage.
Data Source
AI summary
In accordance with an embodiment, described herein are systems and methods for use of an in-memory data grid as a vector database, with linearly-scalable data ingestion, for use in generative artificial intelligence (AI), data visualization, or other applications that include the use of a large language model (LLM) or a retrieval-augmented generation (RAG) process. In accordance with an embodiment, the in-memory data grid provides functionality to represent content as document chunks containing text, embedding, and metadata, which allows the system to support a variety of RAG framework integrations in a consistent manner. To further support the use of RAG processes, the system can support document ingestion via various types of document sources, such as the use of HTTP URLs that allow retrieval of documents using HTTP GET calls; or, for example in cloud environments, the use of object storage and/or other cloud provider storage services as appropriate.