Non-blocking Load Unit Asynchronous Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing latency in non-volatile memory access leads to significant cache misses and degraded system performance, particularly when using non-volatile memory as main memory, as it results in longer clock cycles to retrieve data compared to traditional DRAM systems.
Innovation Solution
A computing system with a processing unit that supports non-blocking load instructions, which allow for asynchronous data access by verifying address translations and sending information to a non-blocking load unit for cache coherent read access, enabling continued processing without stalls, even during data retrieval from non-volatile memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If non-volatile memory is used as main memory, then cost and power consumption are improved, but access latency increases significantly
Solution Approach 1:
The patent implements a non-blocking load unit that pre-issues memory access requests to non-volatile memory before the processor needs the data. By preparing data transfers in advance and overlapping them with other processing operations, the system mitigates the inherent latency of non-volatile memory while maintaining low power consumption benefits.
Solution Approach 2:
The patent enables continuous processor operation by implementing non-blocking load instructions that allow the processor to execute subsequent instructions without waiting for memory data to return. The non-blocking load unit continues to fetch data in the background, ensuring useful work is performed continuously despite the slower memory access characteristics of non-volatile memory.
2Quantity of substance
If non-volatile memory density is increased, then storage capacity and cost-effectiveness are improved, but read and write latencies increase
Solution Approach 1:
The non-blocking load unit anticipates future data needs and initiates memory access requests in advance. By pre-fetching data from high-density non-volatile memory before it is actually needed by the processor, the system compensates for increased access latencies inherent in higher-density memory configurations.
Solution Approach 2:
The patent introduces a non-blocking load unit as an intermediary between the processor and non-volatile memory. This intermediary layer manages the latency issue by buffering and coordinating memory access requests, allowing the processor to continue operating at full speed while data is retrieved from high-density non-volatile memory in the background.
3Reliability
If traditional blocking load instructions are used, then data accuracy is ensured, but processor productivity decreases due to stalls
Solution Approach 1:
The patent segments the data transfer operation from the processor execution flow by implementing a separate non-blocking load unit. This segmentation allows the processor to execute instructions independently without being blocked by memory operations, while the non-blocking load unit handles data transfer separately, ensuring both productivity and data accuracy.
Solution Approach 2:
The non-blocking load instruction enables continuous processor execution by allowing the processor to proceed with subsequent instructions immediately after issuing the load command. The data transfer continues in the background without causing processor stalls, thereby maintaining both productivity and data accuracy through the completion of the load operation.
Data Source
AI summary
A system and method for reducing latencies of main memory data accesses are described. A non-blocking load (NBLD) instruction identifies an address of requested data and a subroutine. The subroutine includes instructions dependent on the requested data. A processing unit verifies that address translations are available for both the address and the subroutine. The processing unit continues processing instructions with no stalls caused by younger-in-program-order instructions waiting for the requested data. The non-blocking load unit performs a cache coherent data read request on behalf of the NBLD instruction and requests that the processing unit perform an asynchronous jump to the subroutine upon return of the requested data from lower-level memory.