Auto-Tiered Storage for ML Training Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual object storage systems face inefficiencies in data access and processing for intensive applications like machine learning, due to high latency and overhead, which underutilizes GPU computing capacity and lacks effective multi-tenancy and data management.
Innovation Solution
Implementing an auto-tiering method that uses a 'hot' and 'cold' local storage approach, where a server fetches data from a virtual object storage service, stores it on local devices, and synchronizes modifications to optimize data access, allowing high-performance storage for frequently accessed data and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data objects are retrieved from virtual object storage service via API, then data manageability and multi-tenancy are provided, but data access latency is high and processing time is increased
Solution Approach 1:
The system performs preliminary actions by caching data objects from the virtual object storage service into local storage before they are actually needed for processing. This advance preparation eliminates the latency of retrieving data from remote storage at the moment of processing, while the caching mechanism maintains data manageability and multi-tenancy through proper cache invalidation and synchronization protocols.
2Speed
If data is stored on high-performance storage devices for low-latency access, then data access speed is improved, but storage cost increases
Solution Approach 1:
The system applies local quality by differentiating storage locations based on data access patterns. Frequently accessed data objects are cached in high-performance local storage to achieve low-latency access, while less frequently accessed data remains in the virtual object storage service. This selective approach ensures that expensive high-performance storage is used only where necessary, optimizing the balance between access speed and storage cost.
3Productivity
If GPU computing capacity is maximized, then processing throughput is improved, but data transfer overhead from CPU increases
Solution Approach 1:
The system introduces a caching layer as an intermediary between the virtual object storage service and the processing units. This cache stores data objects locally, reducing the frequency and volume of data transfers between CPU and GPU. By having data readily available in local storage, the system minimizes transfer overhead and energy consumption while maintaining high GPU utilization for actual processing tasks.
Data Source
AI summary
Method and system for training a machine learning model based on a training dataset formed by data objects distributed across a virtual object storage service. The method comprises fetching from the virtual object storage service, the training dataset; copying the fetched training dataset on a first local storage device and maintaining a list of modifications executed on the training dataset that occurred on the virtual object storage service. The method comprises, upon receiving a request to initiate training of the machine learning model, generating a synchronized training dataset mirroring the training dataset stored in the virtual object storage service; storing the synchronized training dataset in a second local storage device; and fetching training data from the synchronized training dataset stored in the second local storage device as the training of the machine learning model is executed.