Distributed Neural Network Execution Across Low-Latency Cluster Resources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Executing large neural networks on a single physical resource leads to increased execution time due to repeated reconfigurations of data, which consumes computational resources and is inefficient.
Innovation Solution
Distribute the execution of a neural network across multiple physical resources on different computing devices within a cluster, determining these resources based on communication latency to minimize data transfer delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a large neural network is executed on a single physical resource, then the execution can be completed on one device, but the execution time increases due to repeated data reconfigurations
Solution Approach 1:
The neural network execution is divided into multiple parts, with each part assigned to a different physical resource. The system identifies split points in the computational graph and distributes layers or operations across multiple devices, eliminating the need for repeated data reconfigurations on a single resource and reducing overall execution time.
Solution Approach 2:
The system transitions from a single-device execution model to a multi-device distributed execution model, adding the dimension of spatial distribution across multiple physical resources. This dimensional change allows parallel execution of different neural network parts, overcoming the time penalty of sequential processing on one device.
2Productivity
If data is repeatedly reconfigured on a single physical resource, then the neural network can be executed sequentially, but computational resources are consumed inefficiently
Solution Approach 1:
By segmenting the neural network into distributed parts across multiple physical resources, the system eliminates repeated data reconfigurations that occur when processing the entire network sequentially on one device. Each physical resource processes its assigned part once, improving computational efficiency and reducing energy consumption.
Solution Approach 2:
The system combines multiple physical resources into a coordinated distributed execution environment, where each resource contributes to processing different parts of the neural network. This merging of resources allows parallel processing without the overhead of repeated reconfigurations, enhancing both productivity and energy efficiency.
3Speed
If the neural network is distributed across multiple physical resources, then execution speed increases, but communication latency between devices must be managed
Solution Approach 1:
The system strategically segments the neural network at split points that minimize communication requirements. By carefully selecting where to divide the computational graph, the system reduces the frequency and volume of data transfers between physical resources, thereby minimizing communication latency while maintaining execution speed advantages.
Solution Approach 2:
The system performs preliminary analysis of the computational graph to identify optimal split points before distribution. This preliminary action allows the system to anticipate and minimize communication requirements, arranging the distribution strategy in advance to reduce latency during actual execution.
Data Source
AI summary
Example techniques for causing execution of neural networks are described. A neural network includes a first part and a second part. A determination is made that a first physical resource in a first computing device is to execute the first part and that a second physical resource in a second computing device is to execute the second part. The determination is based on a latency in communication between the first physical resource and the second physical resource. The first computing device and the second computing device are part of a cluster.