Convolution Accelerator With Embedded Kernel Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face a bottleneck in kernel data bandwidth during fully connected and recurrent neural network operations, leading to low utilization of available MAC processing resources due to limited kernel data reuse and high kernel data bandwidth requirements, which existing solutions fail to efficiently address.
Innovation Solution
A convolution accelerator system that dynamically performs embedded vector decompression of kernel data using a line buffer memory as decompression table storage, allowing for increased kernel data bandwidth by identifying and processing kernel decompression tables and compressed data within the kernel data stream, thereby optimizing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If kernel data is stored off-chip to reduce on-chip memory usage, then device complexity is reduced, but kernel data bandwidth becomes a bottleneck limiting processing speed
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing decompression tables in off-chip memory before processing. The CPU retrieves compressed kernel data and decompression tables, performs decompression in advance, and stores results in on-chip buffer memory, thereby preparing data before the bottlenecked bandwidth path is needed for MAC operations
Solution Approach 2:
The patent introduces an intermediary decompression buffer memory structure that sits between off-chip storage and the MAC cluster. This buffer acts as a mediator that holds pre-decompressed kernel data, allowing the MAC units to access fully decompressed data without being constrained by the bandwidth limitations of the off-chip memory interface
2Speed
If kernel data is fully decompressed and stored in on-chip buffer memory to increase kernel data reuse, then processing speed is improved, but on-chip memory usage increases
Solution Approach 1:
The patent applies local quality by making the decompression capability local and selective rather than universal. The buffer memory is configured to hold only the specific decompressed kernel data currently needed for processing, while the decompression engine processes data on-demand. This allows high-speed access to active data without requiring the buffer to store all possible kernel data permanently
Solution Approach 2:
The patent implements dynamics by making the buffer memory contents dynamic rather than static. The buffer adapts its contents based on current processing needs, holding decompressed data for active kernels while allowing other regions to be overwritten or used for different purposes. The decompression engine dynamically loads and decompresses data as needed, creating a flexible memory management system
3Speed
If a separate decompression hardware module is added to increase kernel data bandwidth, then processing speed is improved, but device complexity increases
Solution Approach 1:
The patent applies universality by designing a decompression buffer and engine that serve multiple functions: they handle kernel data decompression for convolutional layers, support recurrent neural network operations, and can be configured for different data formats and compression schemes. This multi-functional design eliminates the need for separate dedicated hardware for each operation type
Solution Approach 2:
The patent merges the decompression functionality with the existing buffer memory structure and MAC cluster architecture. The decompression engine integrates with the data flow path between memory and MAC units, and the buffer serves both as decompression output storage and as kernel data cache, combining multiple functions into unified hardware structures rather than adding separate isolated components
Data Source
AI summary
Techniques and systems are provided for implementing a convolutional neural network. One or more convolution accelerators are provided that each include a feature line buffer memory, a kernel buffer memory, and a plurality of multiply-accumulate (MAC) circuits arranged to multiply and accumulate data. In a first operational mode the convolutional accelerator stores feature data in the feature line buffer memory and stores kernel data in the kernel data buffer memory. In a second mode of operation, the convolutional accelerator stores kernel decompression tables in the feature line buffer memory.