Dynamic Banding Convolution for Multi-Core Memory Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-core processing systems face bottlenecks in convolution operations due to shared memory limitations, particularly in electronic design automation (EDA) tasks like aerial image simulation, where large data sets and memory accessibility hinder performance improvements.
Innovation Solution
A dynamic banding and stride-based algorithm is employed, where the layout is divided into sectors (dynamic bands) with only relevant data loaded into shared memory, and basis data is iteratively loaded in parts (strides) based on available memory, allowing concurrent processing by multiple threads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large data sets are loaded into shared memory for multi-core processing, then computation speed is improved, but memory cost and accessibility become bottlenecks
Solution Approach 1:
The patent divides the large data set into multiple smaller tiles that can be independently loaded into shared memory. Each thread processes a specific tile, allowing parallel computation without requiring the entire data set to reside in shared memory simultaneously. This segmentation enables efficient use of limited memory resources while maintaining high computation speed through concurrent processing of multiple tiles.
2Ease of manufacture
If cache memory size is reduced to lower cost, then memory accessibility becomes a bottleneck, but if cache memory size is increased, then cost increases
Solution Approach 1:
The patent pre-loads relevant data tiles into shared memory before computation begins. By preparing the data in advance and organizing it into manageable tiles, the system ensures that all necessary data is available in fast memory before the computational kernels are executed, eliminating memory access bottlenecks during the actual computation phase.
3Productivity
If layout data and basis data are stored in shared memory for concurrent access, then computation efficiency improves, but memory requirements exceed available capacity
Solution Approach 1:
The patent segments both the layout data and basis data into smaller manageable tiles that fit within the available shared memory capacity. By dividing the large data sets into chunks and processing them in parallel across multiple threads, the system achieves high computation efficiency without exceeding memory limits.
Solution Approach 2:
The patent introduces a new dimension of parallelism by processing multiple tiles simultaneously across different thread blocks. This dimensional approach to data organization and processing allows the system to handle large data sets that exceed shared memory capacity by distributing different tiles to different threads for concurrent processing.
Data Source
AI summary
A convolution of the kernel over a layout in a multi-core processor system includes identifying a sector, called a dynamic band, of the layout including a plurality of evaluation points. Layout data specifying the sector of the layout is loaded in shared memory, which is shared by a plurality of processor cores. A convolution operation of the kernel and the evaluation points in the sector is executed. The convolution operation includes iteratively loading parts of the basis data set, called a stride, into space available in shared memory given the size of the layout data specifying the sector. A plurality of threads is executed concurrently using the layout data for the sector and the currently loaded part of the basis data set. The iteration for the loading basis data set proceeds through the entire data set until the convolution operation is completed.