Read-Write Page Replication for Multi-Compute Unit Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multiple compute unit environments, synchronization between compute units is challenging due to high latency and low bandwidth of interconnects, leading to bottlenecks and resource costs in maintaining memory coherence across different memory units.
Innovation Solution
The method involves creating duplicate memory pages for each compute unit, intercepting memory instructions, and collapsing them into a shared memory page when coherence is required, ensuring that memory operations are performed on a single instance of the page to maintain data consistency across units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a shared memory is used for synchronization between compute units, then data sharing is achieved, but latency increases and bandwidth decreases due to interconnect bottlenecks
Solution Approach 1:
The shared memory is segmented into multiple duplicate memory pages, with each compute unit having its own local copy. This segmentation allows each compute unit to access its local copy without contention, eliminating the bottleneck while maintaining synchronization through selective merging when coherence is required.
Solution Approach 2:
A memory management unit acts as an intermediary that monitors memory access patterns and determines when duplicate memory pages should be merged into a shared memory page. This intermediary coordinate the synchronization without requiring constant communication between compute units, reducing latency while maintaining data consistency.
2Speed
If duplicate memory pages are created for each compute unit, then read/write speed increases due to local memory access, but device complexity increases due to memory management overhead
Solution Approach 1:
Each compute unit independently manages its own duplicate memory pages through local memory management units that autonomously handle creation, maintenance, and merging of memory pages based on coherence requirements. This self-service approach distributes the management complexity across multiple independent units rather than requiring a centralized complex memory management system.
Solution Approach 2:
Instead of managing complex shared memory protocols, the system creates simple duplicate copies of memory pages for each compute unit. These copies are identical to the original and can be independently accessed, simplifying the memory management while maintaining performance. The copying mechanism is straightforward and does not require complex coordination.
3Speed
If multiple different memory units are employed for each compute unit, then access speed improves, but resource costs increase for synchronizing memory units
Solution Approach 1:
The system employs periodic monitoring of memory access patterns by memory management units that detect when coherence is required. Instead of continuous synchronization, the system periodically merges duplicate memory pages into shared memory pages only when necessary, reducing the energy costs of synchronization while maintaining data consistency. This periodic action replaces continuous expensive synchronization operations.
Solution Approach 2:
When duplicate memory pages are merged into a shared memory page, the individual duplicate copies are discarded and their resources recovered. This allows the system to maintain high-speed local access when duplicates are needed while efficiently releasing resources when synchronization is achieved, optimizing the balance between performance and resource utilization.
Data Source
AI summary
In general, an application executes on a compute unit, such as a central processing unit (CPU) or graphics processing unit (GPU), to perform some function(s). In some circumstances, improved performance of an application, such as a graphics application, may be provided by executing the application across multiple compute units. However, when using multiple compute units in this manner, synchronization must be provided between the compute units. Synchronization, including the sharing of the data, is typically accomplished through memory. While a shared memory may cause bottlenecks, employing local memory for each compute unit may itself require synchronization (coherence) which can be costly in terms of resources, delay, etc. The present disclosure provides read-write page replication for multiple compute units that avoids the traditional challenges associated with coherence.