Schedule-Aware Tensor Distribution Module for DNN Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural network (DNN) accelerators face challenges due to their fixed-schedule tensor data distribution modules, which lead to suboptimal energy efficiency and increased bank conflicts during data transfer between layers.
Innovation Solution
A flexible schedule-aware tensor data distribution module (FSAD) is introduced, which enables dynamic scheduling and dataflow management. This module includes a load finite state machine (Load FSM) and a drain finite state machine (Drain FSM) to optimize tensor data movement across layers, reducing bank conflicts and enhancing energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a fixed-schedule tensor data distribution module is used, then the device complexity is reduced, but the energy efficiency deteriorates and bank conflicts increase
Solution Approach 1:
The patent implements a schedule-aware tensor data distribution module that dynamically adjusts data distribution strategies based on the specific computational schedule being executed. Instead of using a fixed distribution pattern, the system adapts the tensor data layout and distribution to match the temporal and spatial requirements of different neural network layers, thereby reducing bank conflicts and improving energy efficiency without excessive complexity
Solution Approach 2:
The system changes key parameters of the data distribution module based on schedule requirements. It modifies parameters such as data tiling size, distribution pattern, and memory access timing to optimize for each specific layer's computational needs. This parameter adaptation allows the system to achieve better energy efficiency by aligning data movement with computational schedules
2Device complexity
If a fixed-schedule tensor data distribution module is used, then the device complexity is reduced, but the productivity deteriorates due to bank conflicts
Solution Approach 1:
The schedule-aware distribution module dynamically configures data distribution patterns to match the execution schedule of each neural network layer. By adapting the data layout and distribution timing to the specific computational requirements, the system eliminates bank conflicts that would otherwise stall processing, thereby maintaining high productivity without requiring overly complex hardware
3Use of energy by moving object
If dynamic scheduling is implemented, then the energy efficiency improves, but the device complexity increases
Solution Approach 1:
The system achieves dynamic scheduling by changing parameters within the existing data distribution module rather than adding entirely new hardware components. It adjusts distribution patterns, tiling parameters, and memory access timing based on schedule information, thereby improving energy efficiency through software-controlled parameter adaptation rather than complex hardware reconfiguration
4Reliability
If schedule-aware data distribution is implemented, then bank conflicts are reduced, but the device complexity increases
Solution Approach 1:
The schedule-aware distribution module performs preliminary data distribution and layout optimization before data access occurs. By pre-organizing tensor data according to the known computational schedule, the system eliminates bank conflicts during actual processing without requiring complex runtime reconfiguration mechanisms
Data Source
AI summary
Methods and systems include a neural network system that includes a neural network accelerator. The neural network accelerator includes multiple processing engines coupled together to perform arithmetic operations in support of an inference performed using the deep neural network system. The neural network accelerator also includes a schedule-aware tensor data distribution circuitry or software that is configured to load tensor data into the multiple processing engines in a load phase, extract output data from the multiple processing engines in an extraction phase, reorganize the extracted output data, and store the reorganized extracted output data to memory.