Memory Shuffle Engine Parallel Computing Data Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional parallel computing systems face inefficiencies in data access and power consumption due to repeated memory access for the same data points, particularly in operations like data filtering and matrix multiplication, where each thread processing unit individually accesses its operands, leading to suboptimal performance and energy usage.
Innovation Solution
A memory shuffle engine (MSE) is introduced to efficiently manage data access by loading data elements into an array of registers and using input selectors to transfer these elements to thread processing units, reducing redundant memory access and enabling efficient data reuse, along with a mechanism for configurable workgroup assignments to optimize parallel computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If each thread processing unit individually accesses its operands from memory, then each processing unit can operate independently, but memory access time and power consumption increase due to repeated access to the same data points
Solution Approach 1:
The patent applies preliminary action by loading data elements from memory into a shared register file before parallel processing begins. The memory interface unit prefetches and stores operands in the register file, so that when multiple thread processing units need the same data, it is already available locally without requiring repeated memory access. This resolves the contradiction by preparing data in advance, reducing both memory access time and maintaining parallel throughput.
Solution Approach 2:
The patent introduces a shared register file as an intermediary between memory and thread processing units. Instead of each processing unit directly accessing memory, they access the shared register file which acts as a buffer. This intermediary structure allows multiple processing units to share common data efficiently, reducing redundant memory accesses while preserving independent processing capability.
2Productivity
If each thread processing unit individually accesses its operands from memory, then processing independence is maintained, but power consumption increases due to redundant memory access operations
Solution Approach 1:
The memory interface unit performs preliminary data loading into the shared register file before processing operations begin. By prefetching and storing operands in advance, the system eliminates the need for repeated memory access during parallel processing, thereby reducing power consumption while maintaining high processing throughput.
Solution Approach 2:
The shared register file serves as an intermediary that reduces power consumption by localizing data access. Instead of multiple processing units independently accessing high-power memory interfaces, they access the lower-power shared register file, which caches commonly accessed data elements for efficient reuse.
3Ease of manufacture
If conventional parallel computing platforms are used with fixed hardware architecture, then implementation is straightforward, but flexibility in workgroup assignment and data access patterns is limited
Solution Approach 1:
The patent implements dynamics by making the number of workgroups per batch configurable rather than fixed. The system allows dynamic adjustment of workgroup batch size according to different computational needs and data access patterns. This dynamic configuration capability provides flexibility in workgroup assignment while maintaining a relatively simple hardware architecture that can adapt to different scenarios.
Data Source
AI summary
A computing device operative to perform parallel computations. The computing device includes a controller unit to assign workgroups to a set of batches. Each batch includes a program counter shared by M workgroups assigned to the batch, where M is a positive integer determined according to a configurable batch setting. Each batch further includes a set of thread processing units operative to execute, in parallel, a subset of work items in each of the M workgroups. Each batch further includes a spilling memory to store intermediate data of the M workgroups when one or more workgroups in the M workgroups encounters a synchronization barrier.