Multi-Dimensional Convolution Processing Circuit With Asynchronous Buffer Fetching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer vision processing designs face inefficiencies in memory bandwidth and hardware usage due to redundant data fetches and limited flexibility in window sizes during convolution operations, particularly in multi-dimensional calculations.
Innovation Solution
A system comprising a memory, buffers, and a processing circuit that efficiently calculates convolution operations in multiple dimensions by fetching kernel values slower than input tiles, allowing for parallel multiplication and output tile generation, minimizing memory bandwidth consumption and supporting flexible window configurations and kernel sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If row buffers are used to avoid re-fetching data, then memory bandwidth is reduced, but hardware complexity increases and flexibility of window sizes is limited
Solution Approach 1:
The patent segments the convolution operation into independent processing elements that handle different spatial locations. Each processing element computes convolutions for its assigned region independently, eliminating the need for large row buffers while maintaining data efficiency. This segmentation allows the system to process multi-dimensional data without requiring extensive hardware buffering.
Solution Approach 2:
The patent extends the convolution operation to multiple dimensions (beyond traditional 2D), allowing data to be processed in additional spatial or feature dimensions. This dimensional extension enables more efficient memory utilization by distributing data access patterns across multiple dimensions, reducing the need for large buffers in any single dimension while maintaining flexibility for various window sizes.
2Adaptability or versatility
If conventional separate fetches are used for each output element, then flexibility is maintained, but memory bandwidth is wasted due to redundant fetches
Solution Approach 1:
The patent merges the fetching and processing operations by having processing elements that can operate on shared data segments. Multiple output elements share common input data segments, and the system fetches these segments once while having multiple processing elements work on them in parallel. This merging eliminates redundant fetches while maintaining the flexibility to handle different convolution configurations through the parallel processing architecture.
Solution Approach 2:
The processing elements are designed to be universal and multi-functional, capable of handling various convolution configurations, window sizes, and data dimensions. Each processing element can operate on different data segments and produce different output elements, providing flexibility without requiring separate dedicated hardware for each operation, thus avoiding redundant data fetches.
Data Source
AI summary
An apparatus includes a memory, a first buffer, a second buffer, and a processing circuit. The memory may be configured to store data. The first buffer may be configured to store a plurality of kernel values fetched from the memory and present a first signal communicating the kernel values as stored. The second buffer may be configured to store a plurality of input tiles fetched from the memory and present a second signal communicating the input tiles as stored. The processing circuit may be configured to (i) receive the first signal and the second signal, (ii) calculate a plurality of intermediate values in parallel by multiplying the input tiles with a corresponding one of the kernel values, and (iii) calculate an output tile comprising a plurality of output values based on the intermediate values. The kernel values are generally fetched from the memory to the first buffer slower than the input tiles are fetched from the memory to the second buffer.