Deep Learning Cluster Data Access via Candidate Cache Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning training systems face performance overhead and expense due to remote data access from storage area networks or cloud object storage, with limited compute cluster storage capacity preventing full data caching.
Innovation Solution
A method and system for determining optimal data access by selecting candidate cache locations, fetching mini-batches, collecting job completion times, and choosing the best cache location based on these times to minimize data transfer latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is cached in compute cluster storage, then data access speed is improved, but storage space is limited and cannot accommodate full dataset
Solution Approach 1:
The patent segments the large dataset into smaller chunks or blocks that can be individually cached in the compute cluster's limited storage. Instead of attempting to cache the entire dataset, the system divides data into manageable segments that fit within available storage resources while still providing accelerated access to frequently used portions.
Solution Approach 2:
The patent implements local quality by creating compute-node-specific cache locations tailored to individual node requirements. Each compute node has its own optimized cache space configured according to its specific workload characteristics, allowing localized data residency that maximizes access speed for each node while efficiently utilizing total cluster storage capacity.
2Volume of stationary object
If data is accessed from remote storage services, then storage capacity is preserved, but network latency and access time increase
Solution Approach 1:
The patent applies preliminary action by pre-fetching and caching data blocks into the compute cluster before they are actually needed for training operations. The system proactively loads data segments into local cache storage in advance, so that when compute nodes need to access the data, it is already available locally rather than requiring remote retrieval, thereby eliminating network latency.
Solution Approach 2:
The patent introduces an intermediary caching layer between the remote storage services and the compute nodes. This intermediate cache storage acts as a mediator that holds copies of frequently accessed data, allowing compute nodes to retrieve data from the local cache rather than directly from remote storage, thus reducing network traffic and access time while preserving the ability to access the full dataset in remote storage.
3Productivity
If more data is cached in compute cluster, then training efficiency is improved, but storage resource constraints are violated
Solution Approach 1:
The patent implements dynamics by making the cache configuration adaptive and adjustable rather than fixed. The system dynamically optimizes the amount of data cached based on actual training workload patterns, data access frequencies, and available storage resources. This allows the cache size to flexibly adjust to maximize training efficiency while respecting storage constraints, rather than using a static cache configuration.
Solution Approach 2:
The patent applies parameter changes by modifying cache-related parameters such as cache size, data block sizes, and caching thresholds to optimize the balance between training efficiency and storage usage. The system adjusts these parameters based on observed performance metrics and resource availability, enabling it to achieve high training efficiency within the given storage capacity limits.
Data Source
AI summary
A computer-implemented method, a computer program product, and a computer system for determining optimal data access for deep learning applications on a cluster. A server determines candidate cache locations for one or more compute nodes in the cluster. The server fetches a mini-batch of a dataset located at a remote storage service into the candidate cache locations. The server collects information about time periods of completing a job on the one or more nodes, where the job is executed against fetched mini-batch at the candidate cache locations and the mini-batch at the remote storage location. The server selects, from the candidate cache locations and the remote storage location, a cache location. The server fetches the data of the dataset from the remote storage service to the cache location, and the one or more nodes execute the job against fetched data of the dataset at the cache location.