Optimized Embedding Search with Batched Embedding Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing embedding-based search methods for finding similar data records are inefficient in terms of computing resources and time, especially for 'find similar' or 'more like this' searches, as frequent embedding generation and model updates are costly and time-consuming.
Innovation Solution
A computer-implemented method that generates embeddings for new or updated data records in a batch process infrequently and caches them, allowing for optimized 'find similar' or 'more like this' searches by using cached embeddings or generating them on demand, and optionally performing a keyword search in parallel to expedite results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If frequent embedding generation and model updates are performed for each new or updated data record, then search accuracy for finding similar data records is improved, but computing resource consumption and time cost increase significantly
Solution Approach 1:
The patent applies preliminary action by performing batch embedding generation for new or updated data records before they are added to the collection. Embeddings are generated in advance during batch processing cycles, so when similarity search is needed, the embeddings are already available in the model, eliminating the need for real-time embedding generation during search operations.
Solution Approach 2:
The patent implements periodic action by scheduling batch processing at intervals rather than continuously updating the model with every new or updated data record. The system periodically processes batches of new or updated records, generates their embeddings, and updates the model in these scheduled cycles, reducing overall computing resource consumption while maintaining search functionality.
2Reliability
If frequent embedding generation and model updates are performed for each new or updated data record, then the model remains up-to-date with current data, but processing time and operational cost increase
Solution Approach 1:
The system performs preliminary action by generating embeddings for new or updated data records in advance during batch processing, before the search operation occurs. This ensures the model is updated with current data embeddings prior to being queried, maintaining reliability without requiring real-time processing during search.
Solution Approach 2:
The patent applies dynamics by making the embedding generation frequency adaptive rather than static. The system dynamically adjusts between batch processing mode (for new/updated records) and direct search mode (for existing records), optimizing the balance between model currency and processing time based on the specific search scenario.
3Measurement precision
If embeddings are generated on demand for each search query involving new or updated data records, then search results remain accurate, but response time deteriorates
Solution Approach 1:
The patent resolves this contradiction by performing preliminary action - generating embeddings for new or updated data records in advance during batch processing cycles. When a search query is received, if the data record was part of a recent batch, its embedding is already available in the model, allowing immediate similarity search without on-demand embedding generation, thus maintaining both accuracy and fast response time.
Data Source
AI summary
Systems and methods are disclosed for an embedding search that are optimized, e.g., for a “find similar” or “more like this” search. In one embodiment, a computer-implemented method comprises receiving a request to find data records in a collection of data records that are similar to a particular data record, determining that the particular data record is not included in a model comprising data indicative of similar data records among the collection based on embeddings of data records in the collection, and, responsive thereto, generating at least one embedding of the particular data record and searching the model for a subset of the collection that are similar to the particular data record, based on the at least one embedding. The method further comprises providing a response to the request comprising at least some of the similar data records.