DNN Channel Extension Using Space-to-Depth for MAC Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural network operations face inefficiencies due to variations in hardware performance, leading to suboptimal utilization of hardware resources, particularly in operations involving small channel counts.
Innovation Solution
Implementing a method that includes a space-to-depth transformation operation to extend channels in the output of a first layer, followed by a channel-extended neural network operation between layers, and concluding with a depth-to-space transformation operation to reduce channels in the output of a second layer, thereby optimizing hardware utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If channel extension is performed using space-to-depth transformation, then hardware utilization efficiency is improved, but computational overhead increases
Solution Approach 1:
The space-to-depth transformation is performed in advance before the main neural network computation to extend channels, preparing the data in a format that enables better hardware utilization. This preliminary channel extension allows the accelerator to process data more efficiently during the main computation phase.
Solution Approach 2:
The transformation changes the data dimensionality by converting spatial dimensions into channel dimensions. This dimensional transformation enables the hardware accelerator to better utilize its computational resources by matching the data structure with the hardware's processing capabilities.
2Speed
If channel extension is applied to improve MAC unit utilization, then operation speed increases, but memory usage increases
Solution Approach 1:
The channel extension is dynamically applied only to specific layers where it provides the most benefit, rather than uniformly across all layers. This dynamic approach allows the system to optimize for speed in critical paths while managing memory usage by selectively applying the transformation.
Solution Approach 2:
The depth-to-space transformation is applied after computation to recover the original spatial structure from the channel-extended data. This allows the system to temporarily use extended channels during computation for improved speed, then recover the original format for subsequent processing or output.
3Adaptability or versatility
If depth-to-space transformation is performed to reduce channels, then data format compatibility is improved, but additional processing time is required
Solution Approach 1:
The depth-to-space transformation is merged with the final output processing stage, combining the format conversion with existing operations. This integration reduces the overall processing time by eliminating separate transformation steps and leveraging existing computational pipelines.
Data Source
AI summary
A method by which an electronic device performs a DNN operation includes performing a first modified operation including extending a channel of an output of a first layer with respect to a feature map input into the first layer, the first layer being one of a plurality of layers of a DNN, wherein the first modified operation includes a space-to-depth transformation operation, performing a neural network operation corresponding to layers between the first layer and a second layer as a channel-extended neural network operation, wherein the second layer is a layer of the plurality of layers of the DNN after the first layer, performing a second modified operation including reducing a channel of an output of the second layer with respect to a channel-extended feature map input into the second layer, wherein the second modified operation includes a depth-to-space transformation operation, and outputting a result of the DNN operation.