Configurable Stacked Architecture for DNN Softmax and Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Deep Neural Network (DNN) accelerators face performance limitations when executing softmax and normalization operations due to inefficient execution of multiple chained instructions and high data dependency, leading to low hardware utilization and increased computational complexity.
Innovation Solution
A configurable stacked architecture with programmable and fixed-function datapaths that exploit pipeline and SIMD parallelism, allowing direct data passing between execution units without memory access, and optimizing the quantity and order of execution units for specific DNN operations like softmax and layer normalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing DNN accelerators use custom-tailored architecture to match common DNN patterns, then performance and cost-efficiency are maximized, but performance is limited when performing compute for NLP operations like softmax and normalization
Solution Approach 1:
The patent implements a universal accelerator architecture that can handle both common DNN patterns (convolution, fully-connected layers) and NLP-specific operations (softmax, normalization, non-linearity). The unified execution unit and configurable datapath allow the same hardware to perform multiple different operations, achieving both high performance and broad adaptability without requiring separate specialized accelerators
2Reliability
If existing accelerators execute softmax and normalization operations using multiple chained instructions, then the operations can be completed, but hardware utilization becomes low and computational complexity increases
Solution Approach 1:
The patent merges multiple chained instructions (add, multiply, reduce, non-linearity) into a single unified execution unit that can perform the entire softmax or normalization operation in one go. This consolidation eliminates the need for multiple sequential instructions, reducing computational complexity and improving hardware utilization while maintaining complete and accurate operation execution
3Reliability
If existing accelerators use multiple chained instructions for normalization operations, then the operations can be performed, but hardware utilization is low and throughput decreases
Solution Approach 1:
The patent implements a continuous datapath that allows normalization operations to proceed without interruption or memory access between instruction stages. The unified execution unit maintains continuous data flow through the pipeline, eliminating idle cycles and maximizing hardware utilization, thereby significantly improving throughput while ensuring operation correctness
Data Source
AI summary
The present disclosure relates to devices for using a configurable stacked architecture for a fixed function datapath with an accelerator for accelerating an operation or a layer of a deep neural network (DNN). The stacked architecture may have a fixed function datapath that includes one or more configurable micro-execution units that execute a series of vector, scalar, reduction, broadcasting, and normalization operations for a DNN layer operation. The fixed function datapath may be customizable based on the DNN or the operation.