Data Storage Device Caching for Vector-Queried AI Reference Files
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) face delays in accessing reference files due to the high storage requirements of vector embeddings, which are typically stored in lower-performance memory, leading to inefficiencies in response times.
Innovation Solution
Implementing a system where a Data Storage Device (DSD) or host caches reference files in a faster access memory by prefetched vector embeddings, using Approximate Nearest Neighbor (ANN) searches to identify and store relevant reference files for quicker access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If reference files are stored in lower-performance memory to reduce storage costs, then storage cost is reduced, but access speed deteriorates
Solution Approach 1:
The system performs preliminary actions by identifying reference files that are likely to be needed based on vector embedding queries, then prefetches these files into high-performance memory before they are actually requested. This advance preparation ensures fast access when the files are needed while avoiding the cost of keeping all reference files in high-performance memory continuously.
Solution Approach 2:
The data storage device autonomously monitors its own operation, detecting when vector embeddings are queried and automatically determining which reference files should be prefetched. The system serves itself by identifying its own needs and taking appropriate action without external intervention, optimizing the balance between storage cost and access speed dynamically.
2Speed
If all reference files are kept in high-performance memory, then access speed is improved, but storage cost increases
Solution Approach 1:
Instead of uniformly storing all reference files in high-performance memory, the system applies local quality by selectively placing only those reference files that are currently needed or likely to be needed into high-performance memory. The majority of reference files remain in lower-performance storage, achieving fast access where necessary while minimizing overall storage cost.
Solution Approach 2:
The system performs preliminary identification of which reference files will be needed based on ongoing vector embedding queries, then prefetches only those specific files into high-performance memory in advance. This selective preliminary action optimizes the distribution of reference files across storage tiers based on actual usage patterns.
3Quantity of substance
If reference files are stored in lower-performance memory, then storage capacity is optimized, but response time deteriorates
Solution Approach 1:
The system detects vector embedding queries and automatically prefetches the corresponding reference files into high-performance memory before they are requested by the AI model. This preliminary action eliminates the time delay that would otherwise occur when files are retrieved from lower-performance storage, while maintaining optimized storage capacity by not permanently residing files in fast memory.
Solution Approach 2:
The system uses feedback from vector embedding queries to dynamically adjust which reference files are prefetch ed into high-performance memory. By monitoring query patterns and using this feedback to guide prefetching decisions, the system optimizes response time for actual usage scenarios while maintaining efficient storage capacity utilization.
Data Source
AI summary
A Data Storage Device (DSD) includes a first memory storing reference files used to derive vector embeddings in a vector database. A query vector embedding is received from a host and one or more vector embeddings similar to the query vector embedding are identified in the vector database. One or more reference files from which the one or more vector embeddings were derived are identified and stored in a second memory for faster access. In another aspect, a query vector embedding is received by a host that identifies one or more vector embeddings that are similar to the query vector embedding and retrieves the one or more vector embeddings from a DSD to provide to an Artificial Intelligence (AI) model. One or more reference files are identified from which the one or more vector embeddings were derived and are prefetched from the DSD for storage in a host memory.