Neural Network Partitioning Layer for Accelerator Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As neural networks grow in size, the increasing processing power required for deep learning accelerators leads to limitations in semiconductor chip size and power consumption, resulting in slow communication speeds due to large intermediate data transfer between accelerators, which degrades overall performance.
Innovation Solution
A device and method for partitioning a neural network by interposing a partitioning layer, using an autoencoder structure with an encoding and decoding layer, to reduce the size of intermediate data transferred between accelerators, allowing for efficient division and processing of neural network partitions without requiring additional hardware or software for data compression and decompression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If the size of neural network increases to improve service quality, then processing power of accelerator increases, but size of semiconductor chip increases beyond limits
Solution Approach 1:
The patent divides a large neural network into multiple smaller neural network partitions that can be processed by different accelerators. This segmentation allows the system to handle complex neural networks without requiring a single large accelerator chip, thus avoiding chip size limitations while maintaining high processing power through distributed computation across multiple smaller chips.
2Area of stationary object
If neural network is partitioned into multiple accelerators, then chip size limitation is avoided, but communication speed between accelerators decreases due to large intermediate data transfer
Solution Approach 1:
The patent introduces a partitioning layer as an intermediary component between neural network partitions. This partitioning layer includes an encoder that compresses intermediate data before transmission and a decoder that reconstructs it at the receiving accelerator. This intermediary compression mechanism significantly reduces the amount of data transferred between accelerators, thereby improving communication speed while maintaining the benefits of distributed processing.
3Speed
If data compression is used to reduce intermediate data size, then communication speed improves, but additional software and hardware for compression and decompression is required
Solution Approach 1:
The patent implements the compression and decompression functionality directly within the neural network architecture itself. The encoder and decoder are integrated as part of the partitioning layer in the neural network model, allowing existing accelerators to perform compression/decompression using their standard neural network processing capabilities without requiring specialized compression hardware or software. This universal approach leverages the existing multi-functional nature of neural network processors.
Data Source
AI summary
A device for partitioning an input neural network includes an interposing circuit configured to determine a partitioning position to at which the input neural network is to be partitioned, to interpose a partitioning layer in the input neural network at the partitioning position; and to output and entire neural network that is obtained by interposing the partitioning layer in the input neural network, a training circuit configured to train the entire neural network; and a partitioning circuit configured to divide the entire neural network into a plurality of neural network partitions by partition the partitioning layer. The input neural network includes a plurality of layers.