Convolution Accelerator With Embedded Kernel Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face a bottleneck in kernel data bandwidth during fully connected and recurrent neural network operations, leading to underutilization of hardware resources and high kernel data bandwidth requirements, which are costly and power-intensive when kernel data is stored off-chip.
Innovation Solution
A convolution accelerator is designed to dynamically perform embedded vector decompression of kernel data using a line buffer memory as decompression table storage, allowing for increased kernel data bandwidth with minimal additional hardware overhead, especially when kernel data is stored off-chip, by employing data markers to identify kernel decompression tables and compressed kernel values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If kernel data is stored off-chip, then cost is reduced, but kernel data bandwidth becomes a bottleneck and performance decreases
Solution Approach 1:
The patent pre-loads kernel decompression tables into the line buffer memory before processing kernel data. This preliminary action enables rapid decompression of compressed kernel data during inference, eliminating the bandwidth bottleneck when kernel data is stored off-chip while maintaining cost efficiency.
Solution Approach 2:
The patent introduces compressed kernel data as an intermediary format between off-chip storage and the MAC cluster. By compressing kernel data and using decompression tables in the line buffer, the system reduces the bandwidth requirements for kernel data transmission while maintaining computational accuracy.
2Productivity
If hardware resources are allocated for high kernel data bandwidth, then performance improves, but hardware cost and power consumption increase
Solution Approach 1:
The patent makes the line buffer memory multi-functional by using it for both feature line storage during convolution operations and kernel decompression table storage during fully connected operations. This eliminates the need for dedicated high-bandwidth hardware resources while maintaining performance.
Solution Approach 2:
The patent changes the data format parameter by using compressed kernel data instead of full-precision kernel data. This parameter change reduces the bandwidth requirements by several times while maintaining the same computational functionality, avoiding the need for expensive high-bandwidth hardware.
3Productivity
If the line buffer memory is used for feature line storage, then convolution operations are efficient, but kernel decompression table storage is not available
Solution Approach 1:
The patent dynamically reconfigures the line buffer memory between different functions based on the operation type. During convolution operations, the line buffer stores feature lines; during fully connected operations, it stores kernel decompression tables. This dynamic switching enables both convolution efficiency and kernel decompression functionality without additional hardware.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Techniques and systems are provided for implementing a convolutional neural network. One or more convolution accelerators (600) are provided that each include a feature line buffer memory, a kernel buffer memory, and a plurality of multiply-accumulate, MAC circuits arranged to multiply and accumulate data. In a first operational mode the convolutional accelerator (600) stores feature data in the feature line buffer memory and stores kernel data in the kernel data buffer memory. In a second mode of operation, the convolutional accelerator (600) stores kernel decompression tables in the feature line buffer memory.