Convolution Engine With Distributed Caches and Delay Chain
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional solutions for convolution operations on FPGA hardware accelerators face resource inefficiencies, with adder tree structures consuming excessive resources as dimensions increase, and systolic array implementations leading to unbalanced resource usage due to excessive register consumption.
Innovation Solution
Implementing a convolution operation using a systolic array structure with distributed caches and a register delay chain for read control signals, where products are stored in cache units and read control signals are delayed per stage, reducing the need for intermediate data transfer and balancing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If adder tree structure is used for convolution operation, then implementation is simple and straightforward, but resource consumption increases exponentially with increased computing dimension
Solution Approach 1:
The patent segments the convolution computation into multiple dimensions (spatial dimensions and channel dimension) and processes them separately using different hardware structures. The spatial convolution uses a systolic array while channel convolution uses a distributed cache structure, dividing the overall resource consumption into manageable segments that don't multiply exponentially.
Solution Approach 2:
The patent transforms the channel convolution operation from a spatial arrangement to a temporal arrangement by using a distributed cache structure. Instead of allocating separate hardware resources for each channel in parallel (spatial dimension), the system processes channels sequentially using time-multiplexed cache units, effectively moving the problem from spatial to temporal domain.
2Productivity
If systolic array structure is used for convolution operation, then resource utilization is improved, but register resources are excessively consumed leading to unbalanced resource usage
Solution Approach 1:
The patent introduces distributed caches as intermediary structures between the systolic array and the output. These caches temporarily store intermediate computation results, allowing the systolic array to operate efficiently without requiring excessive register resources for data retention and transfer. The caches act as a buffer that decouples the high-speed computation from the resource-constrained register architecture.
Solution Approach 2:
The patent changes the architectural parameters of the convolution engine by separating spatial and channel processing into different hardware structures. By using a distributed cache structure for channel convolution instead of extending the systolic array, the system changes the resource allocation parameters to achieve better balance between different resource types (logic resources vs. register resources).
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed are an apparatus and a method for convolution operation. The apparatus comprises: a plurality of multipliers configured to receive matrix elements of two matrices, and multiply respective matrix elements to generate a plurality of product elements; a plurality of distributed caches each comprising a plurality of cache units and configured to respectively store the plurality of product elements in respective cache units; a register delay chain comprising a plurality of registers coupled in series and configured to receive a read control signal, delay the read control signal per stage to output a plurality of per-stage delayed read control signals, and respectively provide, to the plurality of distributed caches, the read control signal and the plurality of per-stage delayed read control signals; and an accumulator circuit configured to receive product elements in cache units corresponding to the read control signal and the plurality of per-stage delayed read control signals, and to generate an accumulation data based on the received product elements. This solution can be used in the field of artificial intelligence, especially in the field of automated driving (including autonomous parking).