DNN Accelerator Memory Hierarchy for Flexible Dataflow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep neural network accelerators face inefficiencies in energy consumption and performance due to biased hardware configurations that prioritize specific dataflows, lacking flexibility in hardware mapping and dataflow, which limits their versatility across various workload configurations and hardware specifications.
Innovation Solution
A control method and accelerator design that dynamically calculate offsets for data tiles based on hardware mapping and dataflow, allowing for flexible data processing between memory levels without an interconnection network, optimizing energy efficiency and performance by supporting various dataflows and hardware mappings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a reconfigurable interconnection network is incorporated to provide flexible scheduling options, then adaptability is improved, but device complexity and hardware overhead increase significantly
Solution Approach 1:
The patent removes the complex reconfigurable interconnection network from the accelerator architecture entirely. Instead, it achieves flexible dataflow support through software-based compilation and memory hierarchy design, extracting the unnecessary hardware complexity while preserving the essential flexibility through alternative means.
Solution Approach 2:
The patent implements a universal memory hierarchy structure that can handle multiple dataflow types (row-major, column-major, interleaved) without requiring specialized interconnection hardware. The same memory controllers and buffers serve multiple purposes across different dataflow configurations, eliminating the need for dedicated reconfigurable networks.
2Productivity
If the accelerator size is expanded to improve performance, then productivity is improved, but the relative contribution of NoC overhead increases
Solution Approach 1:
By removing the NoC entirely, the patent eliminates the overhead that would otherwise scale with accelerator size. The memory hierarchy is designed to work efficiently at any scale without proportionally increasing interconnection complexity.
3Productivity
If heuristic design strategy is used to optimize computation for specific workloads, then productivity is improved for target workloads, but adaptability deteriorates
Solution Approach 1:
The patent implements dynamic dataflow selection where the optimal dataflow type (row-major, column-major, or interleaved) is automatically chosen based on the specific workload characteristics and hardware configuration. This dynamic adaptation allows the system to optimize for each workload while maintaining versatility across different configurations.
Solution Approach 2:
The patent changes the dataflow parameters (access pattern, memory layout) based on workload requirements rather than fixing them in hardware. The compilation system adjusts these parameters dynamically to match the specific computational kernel and hardware resources available.
Data Source
AI summary
Provided is a control method of a deep neural network (DNN) accelerator for optimized data processing. The control method includes, based on a dataflow and a hardware mapping value of neural network data allocated to a first-level memory, calculating a plurality of offsets representing start components of a plurality of data tiles of the neural network data, based on receiving an update request for the neural network data from a second-level memory, identifying a data type of an update data tile corresponding to the received update request among the plurality of data tiles, identifying one or more components of the update data tile, based on the data type of the update data tile and an offset of the update data tile among the calculated plurality of offsets, and updating neural network data of the identified one or more components between the first-level memory and the second-level memory.