Neural Network Layer Processing with Distributed Partial Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing methods face bottlenecks and inefficiencies due to reliance on centralized computing systems, particularly when handling extensive datasets and deeper network layers, leading to prolonged training times and suboptimal resource utilization.
Innovation Solution
A method and apparatus for distributed parallel processing of neural network layers that dynamically utilize available computing resources by partitioning input graphs into partial graphs, which are processed in parallel across multiple devices, and combining results to accelerate computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If centralized computing systems are used for neural network processing, then system simplicity is maintained, but processing speed and scalability deteriorate due to bottlenecks
Solution Approach 1:
The patent divides the centralized neural network processing system into multiple distributed computing nodes. Each node processes a portion of the neural network layers independently, eliminating the single-point bottleneck. The input graph is partitioned into multiple sub-graphs that are distributed across different computing resources, enabling parallel processing and significantly improving processing speed.
Solution Approach 2:
The patent transitions from a single-dimensional centralized processing architecture to a multi-dimensional distributed architecture. By introducing spatial distribution across multiple computing nodes and organizing neural network layers in a distributed manner, the system achieves scalability and improved performance without proportionally increasing overall system complexity.
2Productivity
If distributed computing resources are utilized, then processing efficiency is improved, but resource coordination complexity increases
Solution Approach 1:
The patent performs preliminary actions by pre-partitioning the input graph into sub-graphs and pre-assigning them to available computing resources before processing begins. This preliminary organization eliminates the need for complex real-time coordination during processing, as the distribution strategy is determined in advance based on resource availability information.
Solution Approach 2:
The patent introduces a coordinator component that acts as an intermediary between computing resources and the neural network processing task. This coordinator manages resource allocation, monitors availability, and handles the partitioning logic, thereby simplifying the coordination complexity by centralizing management functions while maintaining distributed execution.
3Loss of time
If graph partitioning is performed for parallel processing, then computation time is reduced, but communication overhead between nodes increases
Solution Approach 1:
The patent applies local quality by keeping related graph nodes and edges localized within the same computing node whenever possible. The partitioning strategy preserves spatial locality of data, minimizing the need for inter-node communication. Each computing node processes its assigned sub-graph with minimal external interactions, thereby reducing communication overhead while maintaining computation speed benefits.
Data Source
AI summary
A method and apparatus for distributed parallel processing for a layer of a neural network are disclosed. The method includes identifying an available resource among a plurality of computing resources, generating a partial graph from an input graph based on the available resource, performing, using the available resource, a neural network operation to the partial graph to obtain an updated partial graph, and generating an output graph based on the updated partial graph.