Systolic Array Data Reuse for Memory Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing the Sum of Absolute Differences (SAD) algorithm in high-performance platforms is challenging due to complexities in computing and bandwidth requirements, especially when dealing with large datasets and mixed sparse and dense matrices, which are common in applications like object recognition and video compression.
Innovation Solution
The implementation of a systolic array structure that efficiently processes and reuses data across multiple processing elements, reducing memory bandwidth and accelerating computation by using data reuse schemes in both x and y dimensions, suitable for programmable logic devices like FPGAs and ASICs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the SAD algorithm is implemented using multiple processing elements to handle large datasets, then the computing throughput is improved, but the device complexity and cost increase
Solution Approach 1:
The patent divides the large matrix datasets into smaller blocks that can be processed by individual processing elements in a systolic array. Each processing element handles a specific block, allowing parallel processing without requiring overly complex individual elements. The segmentation enables high throughput by distributing work across many simple, identical processing units.
Solution Approach 2:
The patent introduces a two-dimensional systolic array structure where data flows through processing elements in both horizontal and vertical dimensions. This 2D arrangement allows for efficient data reuse and parallel computation, achieving high throughput without increasing the complexity of individual processing elements beyond what is necessary for basic SAD operations.
2Measurement precision
If the SAD algorithm processes large sliding windows, then the measurement precision is improved, but the bandwidth requirement increases
Solution Approach 1:
The patent loads data blocks into the systolic array processing elements before the actual SAD computation begins. This preliminary data loading allows the processing elements to have immediate access to all necessary data for large window processing, reducing the need for repeated memory access during computation and thereby reducing overall bandwidth requirements while maintaining precision.
Solution Approach 2:
The systolic array structure enables continuous data flow and computation without interruption. Data is streamed through the processing elements in a continuous manner, allowing large sliding window operations to be performed efficiently with sustained bandwidth utilization rather than repeated peak bandwidth demands, thus reducing total bandwidth requirements.
3Ease of operation
If a unified processing scheme is used for both sparse and dense matrices, then the ease of operation is improved, but the productivity decreases
Solution Approach 1:
The patent designs the systolic array processing elements to be universal units that can handle both sparse and dense matrix operations using the same basic SAD computation logic. The processing elements are configured to accept different data patterns (sparse or dense) without requiring different hardware architectures, thus maintaining ease of operation while achieving high productivity through optimized data flow patterns for different matrix types.
Data Source
AI summary
A systolic array implemented in circuitry of an integrated circuit, includes a processing element array having processing elements arranged in a vertical direction and a horizontal direction, first loaders communicatively coupled to the processing element array to load samples Am,n from at least one external memory to the processing element array, and second loaders communicatively coupled to the processing element array to load samples Bk,l from the at least one external memory to the processing element array. Each row of the samples Am,n is loaded one row at a time to a single processing element along the horizontal direction, and each row of the samples Bk,l is loaded one row at a time to a single processing element along the vertical direction, wherein pairing between the samples Am,n and Bk,l in the horizontal direction and the vertical direction enables data reuse to reduce bandwidth usage of the external memory.