Variable Output Data Format Hardware for DNN Memory Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in efficient implementation due to limited power resources and silicon area, requiring hardware configurations that minimize processing power and silicon area while being flexible to support various DNN configurations.
Innovation Solution
The hardware implementation of DNNs is configured to perform multiple passes, where each pass processes input data for a particular layer and converts the output data into a desired format based on the layer, allowing for variable output data formats to reduce memory bandwidth and improve efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fixed output data format is used for all hardware passes, then hardware design is simple, but memory bandwidth and power consumption are high
Solution Approach 1:
The patent implements dynamic output data format selection where the hardware implementation can adaptively choose different data formats (e.g., 8-bit integer, 16-bit integer, floating-point) based on the specific layer requirements. This allows the system to optimize memory bandwidth and power consumption by using lower precision formats where sufficient, rather than uniformly using high precision throughout
Solution Approach 2:
The patent changes the data format parameter dynamically based on layer characteristics. Different layers may require different precision levels, and the system adjusts the output data format parameter accordingly, converting between formats as needed to balance accuracy requirements with resource efficiency
2Device complexity
If fixed output data format is used for all hardware passes, then hardware design is simple, but memory bandwidth is high
Solution Approach 1:
The system dynamically adapts the output data format based on the specific layer being processed, allowing memory bandwidth to be optimized by using lower precision formats when appropriate rather than maintaining uniform high bandwidth requirements throughout the entire network
Solution Approach 2:
Different layers of the neural network are assigned different output data formats based on their specific requirements. The hardware implementation applies local optimization by matching the data format to the local layer characteristics, rather than applying a uniform format across all layers
3Loss of energy
If variable output data format is implemented, then memory bandwidth and power consumption are reduced, but hardware complexity increases
Solution Approach 1:
The patent introduces format conversion modules as intermediary components that handle the complexity of format transformations. These dedicated conversion units allow the main processing architecture to remain relatively simple while adding format flexibility through intermediate conversion stages
Solution Approach 2:
The hardware implementation receives format information in advance and pre-configures the output format before processing begins. This preliminary setup allows the processing modules to operate with optimized formats without requiring complex real-time format adjustment logic during the actual computation
4Area of stationary object
If variable output data format is implemented, then hardware size is reduced, but hardware complexity increases
Solution Approach 1:
The hardware implementation is designed with universal processing modules that can handle multiple data formats through a single architecture. By making the processing units multi-functional rather than dedicated to specific formats, the overall hardware size is reduced while format flexibility is maintained through shared resources
Data Source
AI summary
Hardware implementations of Deep Neural Networks (DNNs) and related methods with a variable output data format. Specifically, in the hardware implementations and methods described herein the hardware implementation is configured to perform one or more hardware passes to implement a DNN wherein during each hardware pass the hardware implementation receives input data for a particular layer, processes that input data in accordance with the particular layer (and optionally one or more subsequent layers), and outputs the processed data in a desired format based on the layer, or layers, that are processed in the particular hardware pass. In particular, when a hardware implementation receives input data to be processed, the hardware implementation also receives information indicating the desired format for the output data of the hardware pass and the hardware implementation is configured to, prior to outputting the processed data convert the output data to the desired format.