Single-Plane Convolution Filters for Sparse DNN Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Deep Neural Networks (DNNs) face challenges in efficiently performing convolution operations across a wide range of architectures due to varying filter and window sizes, leading to inefficient power consumption and resource utilization, particularly in applications with limited power resources and silicon area.
Innovation Solution
A hardware implementation of DNNs with convolution layers that utilizes sparsity data to identify zero weights and data values, allowing convolution engines to skip unnecessary multiplications and employs a flexible allocation of convolution engines to perform operations concurrently, using a control logic to manage weight buffer modules and sparsity maps for efficient data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional DNN hardware performs all multiplication operations for convolution layers, then complete processing is achieved, but power consumption increases and processing efficiency decreases due to unnecessary zero-value multiplications
Solution Approach 1:
The patent extracts and removes unnecessary multiplication operations by identifying zero-value weights and data values through sparsity maps before processing. The control logic uses these sparsity maps to skip multiplications where either operand is zero, thereby reducing power consumption while maintaining complete processing of non-zero values.
Solution Approach 2:
The patent implements skipping of redundant computational steps by using sparsity maps to identify and bypass multiplication operations involving zero values. The control logic rapidly skips over zero-value operations and proceeds directly to non-zero multiplications, improving processing efficiency and reducing energy waste on futile computations.
2Adaptability or versatility
If hardware is designed to support multiple DNN architectures with varying filter and window sizes, then versatility is improved, but device complexity increases
Solution Approach 1:
The patent implements a universal convolution engine design that can handle multiple DNN architectures through configurable parameters for filter sizes, window sizes, and sparsity map formats. The same hardware infrastructure supports different convolution configurations by loading appropriate configuration data, eliminating the need for multiple specialized hardware designs.
Solution Approach 2:
The patent employs dynamic configuration capabilities where the convolution engine can adapt its operation parameters at runtime based on the specific DNN architecture being executed. Configuration data including filter sizes, window sizes, and sparsity map formats can be changed without hardware reconfiguration, allowing flexible support for multiple architectures with moderate hardware complexity.
3Productivity
If sparsity maps and weight buffer modules are added to manage data processing, then processing efficiency is improved, but logic element usage increases
Solution Approach 1:
The patent generates sparsity maps in advance during the data preparation phase, before the actual convolution operations begin. These pre-computed sparsity maps are stored in buffer modules and used by the control logic to guide the multiplication operations, enabling efficient skipping of zero-value operations without adding complex real-time processing logic.
Solution Approach 2:
The patent introduces sparsity maps as intermediary data structures that mediate between the input data and the multiplication logic. These maps serve as a compact representation of zero-value positions, allowing the control logic to efficiently determine which multiplications to perform without requiring complex analysis of the actual data values during processing.
Data Source
Figure 1~7
Figure 2
Figure 3
AI summary
Hardware for implementing a Deep Neural Network (DNN) having a convolution layer, the hardware comprising an input buffer configured to provide data windows to a plurality of convolution engines, each data window comprising a single input plane; and each of the plurality of convolution engines being operable to perform a convolution operation by applying a filter to a data window, each filter comprising a set of weights for combination with respective data values of a data window, and each of the plurality of convolution engines comprising: multiplication logic operable to combine a weight of the filter with a respective data value of the data window provided by the input buffer; and accumulation logic configured to accumulate the results of a plurality of combinations performed by the multiplication logic so as to form an output for a respective convolution operation.