AI Feature Cache Subsystem for Reducing Retraining Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The retraining of artificial intelligence/machine learning models is hindered by high latency due to the time-consuming processes of full feature extraction and data augmentation, especially for shallow models, and existing approaches waste resources by duplicating data and inefficient caching methods.
Innovation Solution
A caching subsystem that stores deterministic augmentations and extracted features in a key-value storage, allowing for quick retrieval and reuse, reducing the need for repeated data augmentation and feature extraction processes by caching only modified data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full feature extraction and data augmentation are performed during each retraining, then model training completeness is ensured, but retraining latency increases significantly
Solution Approach 1:
The system performs feature extraction and data augmentation in advance before model training, storing the results in a cache. During retraining, pre-extracted features and pre-augmented data are retrieved from cache, eliminating the need to repeat these computationally intensive operations. This preliminary action significantly reduces retraining latency while ensuring model training completeness by having all necessary processed data ready beforehand.
Solution Approach 2:
The caching mechanism selectively caches only the specific features and data augmentations that are actually needed for training, rather than computing and storing all possible variations. The system identifies which training examples require augmentation and which features are relevant, then caches only those specific items, optimizing resource utilization and reducing unnecessary computation during retraining.
2Adaptability or versatility
If data augmentation is performed repeatedly for each retraining, then data diversity is maintained, but computational resources are wasted
Solution Approach 1:
Data augmentation is performed once in advance and the augmented data is cached for reuse across multiple retraining operations. The system generates diverse augmented training examples beforehand, storing them in a cache that can be efficiently retrieved during retraining. This maintains data diversity while avoiding repeated computational expenditure on the same augmentation operations.
Solution Approach 2:
The system creates copies of original training data through augmentation operations, storing multiple augmented versions in the cache. These copied and augmented versions maintain the diversity needed for robust model training, while the copying process is performed once beforehand rather than repeatedly during each retraining cycle, significantly reducing computational resource waste.
3Reliability
If feature extraction is performed on all training data during retraining, then feature completeness is ensured, but processing time increases
Solution Approach 1:
Feature extraction is performed in advance on all training data before retraining begins, with results stored in a cache. During retraining, the system retrieves pre-extracted features from cache rather than re-extracting them, ensuring feature completeness is maintained while dramatically improving processing speed. The preliminary feature extraction ensures all necessary features are captured beforehand.
Solution Approach 2:
The system extracts and separates the feature extraction operation from the retraining process, isolating it as a preliminary step whose results are cached and reused. This extraction of the feature extraction step allows it to be performed once comprehensively, then the extracted features are taken out and stored for efficient retrieval during subsequent retraining operations, improving overall processing productivity.
Data Source
AI summary
An augmentation and feature caching subsystem is described for training AI/ML models. In one particular aspect, a method is provided that includes receiving data comprising training examples, one or more augmentation configuration hyperparameters and one or more feature extraction configuration hyperparameters; generating a first key based on one of the training examples and the one or more augmentation configuration hyperparameters; searching a first key-value storage based on the first key; obtaining one or more augmentations based on the search of the first key-value storage; applying the obtained one or more augmentations to the training examples to result in augmented training examples; generating a second key based on one of the augmented training examples and the one or more feature extraction configuration hyperparameters; searching a second key-value storage based on the second key; obtaining one or more features based on the search of the second key-value storage.