Convolutional Block Reuse Across Clock Cycles to Cut Circuit Area
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network implementations face challenges with high compute requirements and inefficient utilization of downstream processing layers due to large area demands and infrequent updates in fixed-architecture accelerators, leading to reduced throughput and circuit downtime.
Innovation Solution
Implement pseudo-multiplication circuits in convolutional blocks that defer two's complement arithmetic and use bit-shifting for weight encoding, along with configurable convolutional blocks that rotate weights across clock cycles to reduce area and re-use circuitry, optimizing hardware design and software co-design.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large parallel structures are used to implement convolutional blocks for high accuracy computation, then computational accuracy is improved, but hardware area requirements increase significantly
Solution Approach 1:
The patent segments the convolutional block computation into multiple clock cycles, processing different channels at different times. Instead of computing all channels in parallel within a single clock cycle, the system divides the computation across time, allowing reuse of the same hardware resources for different channels in subsequent cycles.
Solution Approach 2:
The patent implements periodic computation where the same convolutional block is reused across multiple clock cycles with different input data. The system computes for one channel, then reuses the same hardware for another channel in the next cycle, creating a periodic pattern of resource utilization that reduces overall hardware area requirements.
2Area of stationary object
If fixed-function hardware accelerators with hard-coded layer architectures are used, then hardware area is reduced, but downstream layers become inefficiently utilized due to infrequent updates after dimensional reduction
Solution Approach 1:
The patent introduces dynamic configurability to fixed-function hardware accelerators, allowing the same hardware block to be reconfigured for different layers and channels. This dynamic adaptation enables downstream layers to remain active and efficiently process data even after dimensional reduction, as the hardware can be dynamically allocated to different computational tasks across clock cycles.
Solution Approach 2:
The patent makes hardware blocks universal by enabling them to serve multiple functions across different layers and channels. A single convolutional block can be reused for different output channels and even different layers, transforming specialized fixed-function hardware into multi-functional resources that improve overall system throughput without increasing area.
3Reliability
If separate fixed hardware is provisioned for upstream and downstream processing layers, then layer-specific optimization is achieved, but circuit downtime increases due to starvation of downstream layers
Solution Approach 1:
The patent implements a resource sharing strategy where downstream hardware resources are temporarily held in a ready state and quickly recovered for reuse when upstream layers complete their computation. Instead of remaining idle after processing one channel, the downstream hardware is rapidly reconfigured and reused for the next channel, minimizing downtime and maximizing resource utilization.
Solution Approach 2:
The patent ensures continuous useful action by maintaining a pipeline where downstream layers are constantly engaged in processing. Through careful scheduling and the ability to reuse hardware across channels and layers, the system eliminates idle periods and maintains continuous computational flow, preventing starvation and maximizing throughput.
Data Source
AI summary
Hardware accelerator designs for neural networks are improved with various approaches to reduce circuit area, improve power consumption, and reduce starvation. Convolutional layers of a neural network may multiply a set of weights with a set of inputs. One example defers two's complement arithmetic from the parallelized multiplication circuits and completes the two's complement arithmetic when the results are accumulated. In another example, a multiplication circuit initially multiplies an input by an initial value of the maximum (or minimum) multiplication range before applying the magnitude of a multiplication encoded relative to the multiplication range. In another example, after dimensional reduction earlier in the network hardware, circuitry for a convolutional layer uses a reduced number of convolutional block circuits that are reused across a plurality of clock cycles to apply different subsets of weight channels.