Convolution Accelerator with Embedded Kernel Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face a bottleneck in kernel data bandwidth during fully connected and recurrent neural network operations, leading to underutilization of hardware resources and high kernel data bandwidth requirements, which are costly and power-intensive when kernel data is stored off-chip.
Innovation Solution
A convolution accelerator that dynamically performs embedded vector decompression of kernel data using a line buffer memory as decompression table storage, allowing for increased kernel data bandwidth with minimal additional hardware overhead, especially when kernel data is stored off-chip, by employing data markers to identify kernel decompression tables and compressed kernel values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If kernel data is stored off-chip, then cost is reduced, but kernel data bandwidth requirement increases and access speed decreases
Solution Approach 1:
The patent applies preliminary action by pre-decompressing kernel data using decompression tables stored in the line buffer memory before the data is needed for MAC operations. This allows compressed kernel data to be decompressed in advance and stored in the kernel buffer memory, ensuring that full-speed kernel data is available when needed without requiring high bandwidth from off-chip storage.
Solution Approach 2:
The patent introduces an intermediary decompression mechanism using the line buffer memory to store decompression tables and a decompression process as a mediator between off-chip kernel data storage and the MAC cluster. This intermediary layer enables efficient data transformation and buffering, allowing compressed data to be decompressed and delivered at the required speed without directly bottlenecking the off-chip interface.
2Quantity of substance
If hardware resources are allocated for high kernel data bandwidth, then kernel data bandwidth increases, but hardware resource utilization decreases
Solution Approach 1:
The patent applies universality by making the line buffer memory multi-functional: it serves as feature line buffer memory during convolutional operations and as decompression table storage during fully connected operations. This eliminates the need for dedicated decompression table memory, improving hardware resource utilization while maintaining the ability to deliver high kernel data bandwidth when needed.
Solution Approach 2:
The patent implements dynamics by dynamically switching the operational mode of the convolution accelerator between convolutional mode and fully connected mode based on the computational task. This dynamic reconfiguration allows the same hardware resources to be optimized for different operation types, improving overall resource utilization while providing high kernel data bandwidth capability when required.
3Adaptability or versatility
If decompression tables are stored in dedicated memory, then decompression capability is improved, but hardware complexity and cost increase
Solution Approach 1:
The patent applies universality by using the existing line buffer memory for dual purposes: storing feature data during convolutional operations and storing decompression tables during fully connected operations. This eliminates the need for separate dedicated decompression table memory, reducing hardware complexity and cost while maintaining full decompression capability when needed.
Data Source
AI summary
Techniques and systems are provided for implementing a convolutional neural network. One or more convolution accelerators are provided that each include a feature line buffer memory, a kernel buffer memory, and a plurality of multiply-accumulate (MAC) circuits arranged to multiply and accumulate data. In a first operational mode the convolutional accelerator stores feature data in the feature line buffer memory and stores kernel data in the kernel data buffer memory. In a second mode of operation, the convolutional accelerator stores kernel decompression tables in the feature line buffer memory.