Neural Network Hardware Depth-Wise Block Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in resource-constrained environments where power consumption and silicon area are limited, requiring efficient hardware implementations that can support various configurations while minimizing memory-access overheads and energy costs.
Innovation Solution
A hardware implementation of a neural network processes input data in blocks depth-wise through multiple layers, evaluating all layers for a given block before moving on, using on-chip memory to reduce memory-access overheads and power consumption, and employing parallel processing elements to generate output data elements efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If DNNs are implemented in hardware with limited resources, then power consumption and silicon area are reduced, but processing capabilities and flexibility are limited
Solution Approach 1:
The patent divides the neural network computation into blocks of input data processed through layers in a depth-wise manner. Each block is evaluated through all layers before moving to the next block, segmenting the computation to optimize memory access patterns and reduce power consumption while maintaining processing capabilities
Solution Approach 2:
The patent processes multiple output data elements in parallel within each block evaluation, adding a dimension of parallelism at the output element level. This allows the hardware to maintain high processing capabilities while using efficient memory access patterns that reduce power consumption
2Area of stationary object
If DNNs are implemented in hardware with limited resources, then silicon area is reduced, but processing capabilities are limited
Solution Approach 1:
The patent segments the neural network computation into manageable blocks that can be processed through layers sequentially. This segmentation allows the hardware to use smaller, more area-efficient memory structures while maintaining the ability to process complex networks through systematic block evaluation
Solution Approach 2:
The patent employs dynamic memory management where memory is allocated and used efficiently based on the current block being processed. This dynamic approach allows the same hardware resources to handle different network configurations and sizes, maintaining processing capabilities with reduced silicon area
3Loss of energy
If memory access overhead is reduced by processing blocks depth-wise, then energy cost is reduced, but processing time per block increases
Solution Approach 1:
The patent loads all necessary weight data and input data for a complete block into on-chip memory before processing begins. This preliminary action eliminates repeated off-chip memory accesses during the depth-wise evaluation of all layers for that block, significantly reducing energy cost while the block processes through all layers
Data Source
AI summary
A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.