Distributed Shared Memory Cache for Deep Learning I/O Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cluster computing technologies face an I/O performance bottleneck due to repeated access of data files from cache established in hard disk space, limiting computing performance in deep learning applications.
Innovation Solution
A dynamic distributed shared memory cache space is built using a distributed shared memory deployment and data file access management module, which overrides Dataset APIs like TensorFlow and PyTorch, utilizing RDMA technology and Gluster File System to create a RAM disk across computing nodes, allowing direct access to data files from memory rather than traditional hard disks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If cache is established in hard disk space for storing data files in cluster computing, then data storage capacity is improved, but data access speed deteriorates due to repeated I/O operations
Solution Approach 1:
The patent replaces the mechanical hard disk I/O system with a memory-based caching system. By establishing a cache in the memory space of each computing node rather than in hard disk space, the system eliminates repeated I/O operations and achieves fast data access while maintaining adequate storage capacity through the memory cache mechanism.
2Speed
If distributed shared memory cache space is built using memory instead of hard disk, then data access speed is improved, but storage capacity is reduced
Solution Approach 1:
The patent merges the memory resources of multiple computing nodes into a distributed shared memory cache space. By combining the cache capabilities of individual node memories across the cluster, the system achieves both high-speed access (inherent to memory) and expanded effective storage capacity (through aggregation of multiple nodes' memory resources).
3Device complexity
If traditional hard disk cache is used in cluster computing, then system complexity is kept simple, but I/O performance is limited due to repeated access operations
Solution Approach 1:
The patent segments the cache functionality from the hard disk subsystem and implements it as a separate memory-based distributed caching layer. This segmentation allows the system to maintain simple hard disk storage while adding high-performance memory caching capabilities, thereby improving I/O performance without significantly increasing overall system complexity.
Data Source
AI summary
Disclosed are an on-demand shared data caching method, a computer program, and a computer readable medium applicable for distributed deep learning computing. The method includes a step of dynamically building a distributed shared memory cache space, in which a distributed shared memory deployment and data file access management module is added to a deep learning framework to build the distributed shared memory cache space by a memory set of a multiple of computing nodes of a cluster computer; and a distributed deep learning computing step, in which the computing node overrides a Dataset API of the deep learning framework to execute the distributed deep learning computing. When reading a data file, if the data file exists in the distributed shared memory cache space, then it will be accessed directly, or else it will be obtained from an original specified directory location and stored in the distributed shared memory cache space.