Neural Network Accelerator Placement Using Wavelet Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning technologies face challenges in achieving improvements in accuracy, performance, and energy efficiency.
Innovation Solution
An array of processing elements with routers and compute elements in a neural network accelerator performs flow-based computations on wavelets, utilizing a software stack to determine optimized placement and configuration based on neural network descriptions, enabling efficient communication and computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional deep learning technologies are used, then implementation is simpler, but accuracy, performance, and energy efficiency are insufficient
Solution Approach 1:
The system segments the deep learning processing into multiple processing elements arranged in a 2D mesh array, where each element handles specific computations. This segmentation enables parallel processing of neural network operations, improving accuracy and performance while managing complexity through modular architecture.
Solution Approach 2:
The patent introduces a two-dimensional mesh array dimension for processing elements, adding spatial organization to the computational architecture. This dimensional structure enables efficient data flow patterns and communication pathways that improve processing performance and energy efficiency compared to conventional flat architectures.
2Productivity
If more processing elements are added to improve performance, then neural network processing capability increases, but energy consumption increases
Solution Approach 1:
The patent merges computation and communication functions into integrated processing elements within the mesh array. Each processing element combines computational units with communication interfaces, reducing the energy overhead of separate communication infrastructure and enabling efficient parallel processing that scales performance without linearly increasing energy consumption.
Solution Approach 2:
The mesh array architecture enables continuous data flow and computation through interconnected processing elements. Data moves continuously through the mesh without bottlenecks, keeping all processing elements actively engaged in useful computation, thereby maximizing performance per unit of energy consumed.
3Productivity
If optimized placement is implemented, then resource allocation efficiency improves, but system configuration complexity increases
Solution Approach 1:
The system performs preliminary placement optimization during the configuration phase, where the software stack pre-determines the optimal assignment of neural network operations to processing elements based on the network description. This preliminary action establishes an optimized mapping that improves resource allocation efficiency during execution without requiring complex real-time decisions.
Solution Approach 2:
The patent introduces a software stack as an intermediary between the neural network description and the hardware configuration. This software layer automatically performs placement optimization, translating high-level network descriptions into optimized hardware mappings, thereby improving resource allocation while shielding users from configuration complexity.
Data Source
AI summary
Techniques in optimized placement for efficiency for accelerated deep learning provide improvements in one or more of accuracy, performance, and energy efficiency. An array of processing elements comprising a portion of a neural network accelerator performs flow-based computations on wavelets of data. Each processing element comprises a compute element to execute programmed instructions using the data and a router to route the wavelets. The routing is in accordance with virtual channel specifiers of the wavelets and controlled by routing configuration information of the router. A software stack determines optimized placement based on a description of a neural network. The determined placement is used to configure the routers including usage of the respective colors. The determined placement is used to configure the compute elements including the respective programmed instructions each is configured to execute.