Non-uniform Compute Device Data Coherence via Near-Memory Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing systems with multiple processor hosts and shared data resources face inefficiencies due to delayed processing caused by poor data locality and idle host processors, as they do not fully utilize processing resources and lack speculative execution capabilities.
Innovation Solution
A Non-uniform Compute Device (NUCD) architecture that integrates near-memory processing units, allowing instructions to be routed selectively to host or near-memory processing units based on availability and data location, with mechanisms for speculative execution and data coherence maintenance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is retrieved from shared data resources or lower level cache, then data availability is improved, but processing is delayed
Solution Approach 1:
The patent implements preliminary action by pre-fetching data from shared data resources or lower level caches to higher level caches before the processing units actually need it. This allows data to be staged in advance, so when processing is initiated, the data is already available in fast memory, eliminating processing delays while ensuring data availability.
Solution Approach 2:
The patent introduces higher level caches as intermediary structures between shared data resources/lower level caches and processing units. These intermediary caches buffer data transfers, allowing processing units to access data quickly without directly waiting for slower shared resources, thus reducing processing delay while maintaining data availability.
2Loss of energy
If PIM instructions are executed in near-memory processing units, then data movement is reduced, but host processor resources are not fully utilized
Solution Approach 1:
The patent implements dynamic resource allocation where the system can flexibly route instructions to either near-memory processing units or host processors based on real-time conditions. When host processors are idle, PIM instructions are executed in near-memory units to reduce data movement energy. When host processors become available, they can take over processing tasks, ensuring full utilization of host resources while maintaining energy efficiency.
Solution Approach 2:
The patent creates a universal processing architecture where both host processors and near-memory processing units can execute processing instructions. The system is designed to accommodate multiple processing paths, allowing the same instruction stream to be handled by different processing units depending on their availability and suitability, thus maximizing both energy efficiency and resource utilization.
3Ease of operation
If PIM instructions are executed atomically without speculation, then execution simplicity is improved, but processing performance is limited
Solution Approach 1:
The patent applies preliminary action to speculative execution by performing speculative PIM operations in advance and validating results before committing them. The system can speculatively execute instructions out-of-order or predictively, then verify correctness before finalizing, thus improving processing performance while maintaining execution simplicity through controlled speculation rather than complex atomic operations.
Data Source
AI summary
A data processing apparatus includes one or more host processors with first processing units, one or more caches with second processing unit, a non-cache memory having a third processing unit and a reorder buffer operable to maintain data order during execution of a program of instructions. An instruction scheduler routes instructions to the processing units. Data coherence is maintained by control logic that blocks access to data locations in use by a selected processing unit other than the selected processing unit until data associated with the data locations are released from the reorder buffer. Data stored in the cache is written to the memory if it is already in a modified state, otherwise the state is set to the modified state. A memory controller may be used to restrict access to memory locations to be operated on.