Shared Weight Buffers for Sparse CNN Convolution Engines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware implementations of Deep Neural Networks (DNNs), particularly Convolutional Neural Networks (CNNs), face challenges in efficiently performing convolution operations across a wide range of applications with varying architectures, especially when power resources and silicon area are limited, and the sparsity of weights and input data is not fully exploited.
Innovation Solution
A hardware implementation of CNNs utilizing neuron engines that selectively perform convolution operations based on sparsity maps, allowing for flexible and scalable processing by skipping zero-value multiplications and independently allocating neurons to engines, thereby optimizing performance and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional hardware implementations of CNNs are used to perform convolution operations, then processing capability is maintained, but power consumption is high and silicon area is wasted due to inability to exploit weight and input data sparsity
Solution Approach 1:
The patent extracts and removes zero-value weights and input data elements from the processing pipeline. By identifying sparsity patterns and eliminating zero-value computations, the system avoids unnecessary processing operations, thereby reducing power consumption without compromising the accuracy of convolution results.
Solution Approach 2:
The patent implements dynamic sparsity exploitation where the hardware adapts its processing behavior based on the actual sparsity patterns present in the input data and weights. The system dynamically adjusts which computations are performed based on real-time detection of zero values, optimizing the balance between power consumption and processing capability for each specific input.
2Loss of energy
If hardware is designed to exploit sparsity in CNNs, then power consumption is reduced, but device complexity increases due to additional sparsity detection and selective processing mechanisms
Solution Approach 1:
The patent merges sparsity detection functionality with the existing convolution processing pipeline. Rather than adding completely separate detection and processing systems, the sparsity exploitation mechanisms are integrated into the neural network hardware architecture, allowing zero-value detection and selective computation to occur within the same processing units that perform convolution operations.
Solution Approach 2:
The hardware system performs self-service by automatically detecting sparsity patterns and adjusting its own processing behavior without requiring external control. The neural network hardware inherently identifies zero values and selectively bypasses unnecessary computations through built-in mechanisms, eliminating the need for complex external sparsity management systems.
3Adaptability or versatility
If fixed architecture hardware is used for CNNs, then manufacturing precision is maintained, but adaptability is poor when DNN definitions vary over time due to additional training
Solution Approach 1:
The patent implements universal hardware structures that can handle multiple DNN architectures and configurations. The neural network hardware is designed with configurable parameters and flexible data paths that allow it to adapt to different network definitions, layer types, and sparsity patterns while maintaining consistent processing accuracy through standardized computational kernels.
Solution Approach 2:
The hardware incorporates dynamic reconfigurability that allows it to adapt its behavior based on the specific DNN architecture being executed. The system can dynamically adjust processing parameters, sparsity exploitation strategies, and resource allocation based on the trained model's characteristics, enabling the same hardware to accurately implement multiple different neural network definitions.
Data Source
Figure 1~7
Figure 2
Figure 3
AI summary
Hardware for implementing a Deep Neural Network (DNN) having a convolution layer, the hardware comprising a plurality of convolution engines each configured to perform convolution operations by applying filters to data windows, each filter comprising a set of weights for combination with respective data values of a data window; and one or more weight buffers accessible to each of the plurality of convolution engines over an interconnect, each weight buffer being configured to provide weights of one or more filters to any of the plurality of convolution engines; wherein each of the convolution engines comprises control logic configured to request weights of a filter from the weight buffers using an identifier of that filter.