Multi-Module Scheduling via Shared Memory for Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing multi-module scheduling scheme in deep learning faces inconsistencies in data input and significant delays in data transmission between computing modules, leading to computation errors and reduced system throughput.
Innovation Solution
A method and system for multi-module scheduling that utilizes a master process to control computation logic through a directed computation graph, ensuring that computing modules are called only when all input data is complete, and employs shared storage spaces for efficient data communication, reducing the need for serialization and deserialization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a socket communication mechanism is used between processes for data transmission, then inter-machine communication can be provided and processes can be distributed across different machines, but data inconsistency occurs due to transmission delays and computation errors increase
Solution Approach 1:
The system pre-allocates shared memory spaces for data exchange between processes before actual computation begins. This preliminary setup of communication channels ensures that data can be transmitted without serialization delays, maintaining consistency across distributed processes while enabling inter-machine communication.
2Adaptability or versatility
If data is transmitted between processes using a communication mechanism, then module distribution is enabled, but significant delays occur due to serialization, deserialization, compression and decompression operations
Solution Approach 1:
The patent extracts the data transmission mechanism from the standard socket communication protocol and replaces it with direct shared memory access. By removing the serialization, deserialization, compression and decompression steps from the communication path, the system achieves significant reduction in communication delay while maintaining module distribution capability.
Solution Approach 2:
The system introduces shared memory spaces as intermediaries between distributed processes. Instead of directly transmitting data through socket mechanisms with multiple processing steps, processes write to and read from shared memory blocks, which act as efficient intermediaries that eliminate serialization overhead and reduce communication latency.
3Productivity
If more computing modules are added to process deep learning tasks, then processing capability is enhanced, but the number of computation errors increases due to data inconsistency from messaging delays
Solution Approach 1:
The patent merges the data transmission and storage functions by using shared memory spaces that serve both purposes simultaneously. This consolidation eliminates the separate messaging layer that causes delays and data inconsistency, allowing multiple computing modules to access consistent data directly from shared memory, thereby maintaining computation accuracy while enhancing processing capability.
Data Source
AI summary
The present disclosure provides a method, an apparatus and a system for multi-module scheduling, capable of solving the problem associated with inconsistency in data inputted to a computing module in the multi-module scheduling technique in the related art. The method includes: reading, by a master process, a pre-stored configuration file storing a directed computation graph; initializing, by the master process, states of all the nodes and connecting edges in the directed computation graph initially in computation in a current computing period; determining a node to be called based on the computation direction in the directed computation graph and the states of the nodes, the node to be called comprising a node having all of its input edges in a complete state; transmitting, to the computing module in the slave process corresponding to the node to be called, a call request of Remote Process Call (RPC) to execute the computing module; updating the state of the node and the state of each output edge of the node upon receiving a response to the call request; and proceeding with a next computing period upon determining that the states of all the nodes have been updated.