Versioned Embedding Documents for Atomic Real-Time Recommendations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search engines and recommendation systems face challenges in managing multiple model embeddings efficiently, as updating a model requires updating multiple documents, leading to lengthy operations and lack of atomicity in inference processes, especially when model-agnostic metadata updates are involved.
Innovation Solution
Implement versioning techniques to manage ML models by isolating and updating configuration documents, using internal and external versions to ensure atomic swaps and separate model-specific from model-agnostic data, enabling efficient model updates without downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If model embeddings are stored in separate documents for storage and retrieval, then retrieval operations can be performed efficiently, but updating a model involves updating multiple documents which is a lengthy operation
Solution Approach 1:
The patent segments the update operation into two independent parts: (1) generating new version documents with incremented version identifiers, and (2) atomically swapping the model pointer to point to the new version. This segmentation allows retrieval operations to continue using the old version while updates proceed in the background, eliminating the bottleneck of updating multiple documents sequentially.
Solution Approach 2:
The patent introduces a version identifier field as an intermediary mechanism that decouples the retrieval process from the update process. Retrieval operations query documents based on the current model pointer's version identifier, while updates create new documents with incremented version identifiers. This intermediary version field allows both operations to proceed independently without interfering with each other, resolving the contradiction between fast retrieval and efficient updates.
2Reliability
If multiple documents are updated to reflect a model change, then the model embedding is kept consistent across the system, but the update operation becomes non-atomic and lengthy
Solution Approach 1:
The patent performs preliminary actions by pre-generating all new version documents with incremented version identifiers before executing the atomic swap operation. This preliminary generation ensures that all necessary documents are ready and consistent, and the actual model update is reduced to a single atomic pointer swap, thereby maintaining model consistency while dramatically improving update throughput.
Solution Approach 2:
Instead of updating multiple existing documents in place, the patent creates copies of the model documents with incremented version identifiers. These copied documents contain the updated embeddings, and the system atomically swaps the model pointer to reference the new copied version. This copying approach maintains consistency by ensuring the new version is fully prepared before activation, while improving productivity by avoiding lengthy in-place updates.
3Reliability
If versioning is implemented to manage model updates, then atomic swaps and separation of model-specific and model-agnostic data are achieved, but system complexity increases
Solution Approach 1:
The patent applies local quality by adding versioning capabilities only where needed - specifically in the model document structure and the model pointer management - rather than redesigning the entire system. The version identifier field is added to documents locally, and the atomic swap mechanism is implemented locally in the model management layer. This localized approach achieves update atomicity and data separation while minimizing overall system complexity.
Data Source
AI summary
The disclosed embodiments describe techniques for isolating and managing models via versioning. In one embodiment, a method is disclosed comprising reading a configuration document associated with a first model; incrementing an internal write version of the configuration document; storing the internal write version in the configuration document; generating documents belonging to a second model such that respective document identifiers of the documents include a next external version a first field and the incremented internal write version in a second field; and uploading the documents to a serving system, causing the serving system to replace the first model with the second model.