Neural Core Array Data Distribution via Programmable Weight Paths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for parallel processing among arrays of neural cores face inefficiencies in data distribution and computation, particularly in distributing and accumulating partial sums across neural cores to compute output activation tensors efficiently.
Innovation Solution
Each neural core in the array computes partial sums by traversing the weight tensor according to a programmable path, communicating these partial sums to adjacent cores via an on-chip network, and accumulating them to generate the final output activation tensor, allowing for efficient parallel processing and alignment of input and output layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If partial sums are communicated between adjacent neural cores via network, then computation accuracy is improved, but communication overhead and processing time increase
Solution Approach 1:
The weight tensor is divided into multiple segments that are distributed across different neural cores. Each core processes a specific segment and computes partial sums independently, then communicates only the necessary partial results to adjacent cores. This segmentation enables parallel processing while minimizing communication overhead by reducing the volume of data that needs to be transmitted between cores.
Solution Approach 2:
Input activations are pre-distributed to the appropriate neural cores before the computation begins. Each core receives and stores the input activations it needs for its segment of the weight tensor, so that when computation starts, no time is wasted on data transfer during the processing phase. This preliminary distribution optimizes the workflow by separating data preparation from computation and communication.
2Productivity
If weight tensor is distributed across neural cores, then parallel processing capability is improved, but data distribution complexity increases
Solution Approach 1:
The weight tensor is segmented into discrete blocks that map directly to individual neural cores or groups of cores. This segmentation creates a regular, predictable distribution pattern that simplifies the data placement logic. Each core is responsible for a specific segment, making it easy to manage and update weight distributions without requiring complex coordination between cores.
Solution Approach 2:
Each neural core is assigned a specific portion of the weight tensor that it processes locally, rather than requiring access to the entire weight tensor. This local quality approach means each core has specialized data relevant to its function, reducing memory bandwidth requirements and simplifying the data distribution scheme. The system achieves parallel processing by giving each core its own localized data subset.
3Adaptability or versatility
If programmable path is used to traverse weight tensor, then computation flexibility is improved, but control complexity increases
Solution Approach 1:
The system uses programmable paths that can be dynamically configured to traverse the weight tensor in different patterns depending on the computational requirements. These paths allow the system to adapt to different neural network architectures and operations by changing the traversal sequence without requiring physical reconfiguration of the hardware. The dynamic nature of the paths provides flexibility while maintaining relatively simple control logic through software or firmware configuration.
Data Source
AI summary
Parallel processing among arrays of physical neural cores is provided. An array of neural cores is adapted to compute, in parallel, an output activation tensor of a neural network layer. A network is operatively connected to each of the neural cores. The output activation tensor is distributed across the neural cores. An input activation tensor is distributed across the neural cores. A weight tensor is distributed across the neural cores. Each neural core's computation comprises multiplying elements of a portion of the input activation tensor at that core with elements of a portion of the weight tensor at that core, and storing the summed products in a partial sum corresponding to an element of the output activation tensor. Each element of the output activation tensor is computed by accumulating all of the partial sums corresponding to that element via the network. The partial sums for each element of the output activation tensor are computed in a sequence of steps whose order is described by tracing a path through the weight tensor that visits every weight tensor element that contributes to any partial sum.