Deep Learning Accelerator Virtual Channel Task Activation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning technologies face challenges in achieving improvements in accuracy, performance, and energy efficiency for neural network training and inference tasks.
Innovation Solution
The implementation of a deep learning accelerator system that utilizes a fabric of processing elements interconnected via a 2D mesh, enabling wavelet-based dataflow and instruction processing, with routers managing communication through virtual channels and data structure descriptors for efficient task activation and computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional deep learning processing is used with external storage access, then data storage capacity is sufficient, but access latency increases and performance decreases
Solution Approach 1:
The system segments the deep learning processing into distinct phases (forward propagation, backward propagation, weight update) and assigns them to different processing elements. Each processing element has dedicated local storage for its specific computational tasks, eliminating the need for external storage access during critical computation phases and thereby reducing access latency.
Solution Approach 2:
The patent introduces a new dimensional organization of storage by creating a hierarchical memory structure with local registers, local memory, and external storage. This multi-dimensional storage architecture allows frequently accessed data to be cached in faster local storage, reducing the time penalty associated with external storage access while maintaining sufficient storage capacity through the hierarchical structure.
2Productivity
If sequential processing of dataflow graph layers is used, then resource requirements are reduced, but execution time increases and throughput decreases
Solution Approach 1:
The system divides the neural network computation into segmented stages (input layer processing, hidden layer processing, output layer processing) with dedicated processing elements for each stage. This segmentation enables concurrent execution of different layers while maintaining manageable complexity through modular architecture, where each processing element handles a specific segment of the computation.
Solution Approach 2:
The patent implements dynamic task allocation and scheduling mechanisms that adaptively assign computational tasks to available processing elements based on current system state and resource availability. This dynamic approach allows the system to maximize parallel execution of dataflow graph layers while maintaining flexibility to handle varying computational loads, thereby increasing throughput without requiring a static over-provisioning of resources.
3Productivity
If more processing elements are added to increase parallelism, then computational performance improves, but energy consumption and system complexity increase
Solution Approach 1:
The system implements partial parallelism by activating only the necessary number of processing elements based on the current computational workload and dataflow graph structure. Rather than maintaining full parallel execution capacity at all times, the system dynamically adjusts the number of active processing elements to match the actual computational requirements, thereby achieving good performance for specific tasks while reducing energy consumption when full parallelism is not needed.
Solution Approach 2:
The patent assigns different functional characteristics to different processing elements based on their specific roles in the dataflow graph processing. Each processing element is optimized for its specific computational task (e.g., multiplication-heavy elements for convolution layers, addition-heavy elements for activation functions), allowing the system to achieve high computational performance with specialized, energy-efficient processing units rather than requiring all elements to perform all functions at maximum capacity.
Data Source
AI summary
Techniques in advanced deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements performs flow-based computations on wavelets of data. Each processing element has a compute element and a routing element. Each router enables communication via wavelets with at least nearest neighbors in a 2D mesh. Routing is controlled by virtual channel specifiers in each wavelet and routing configuration information in each router. Execution of an activate instruction or completion of a fabric vector operation activates one of the virtual channels. A virtual channel is selected from a pool comprising previously activated virtual channels and virtual channels associated with previously received wavelets. A task corresponding to the selected virtual channel is activated by executing instructions corresponding to the selected virtual channel.