Persistent Predictive Prefetching Memory Device
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current predictive prefetching in computer systems is limited by the lack of persistence and independence from operating system (OS) memory allocation, leading to inefficiencies in caching and increased latency, as it fails to leverage past learnings across power cycles and program executions.
Innovation Solution
A memory device with persistent predictive prefetching using a prediction engine that employs reinforcement learning and non-volatile memory to store memory access patterns, allowing it to maintain predictions and improve caching efficacy across reboots and OS changes, by mapping page numbers to indices and using volatile memory for caching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If predictive prefetching is implemented using volatile memory only, then the CPU can quickly access cached data, but the cache information is lost after power cycles and the prefetching cannot leverage past learnings
Solution Approach 1:
The memory system is divided into two distinct segments: volatile memory (SRAM/DRAM) for fast caching during execution and non-volatile memory (NVM) for persistent storage of predictions and access patterns. This segmentation allows each memory type to fulfill its specialized function - speed for volatile, persistence for non-volatile - resolving the contradiction between fast access and information retention across power cycles.
Solution Approach 2:
A prediction engine acts as an intermediary component that operates between the CPU and memory subsystem. It receives access patterns from volatile memory, processes them through machine learning models, stores predictions in non-volatile memory, and feeds back prefetching instructions. This intermediary enables the system to leverage past learnings while maintaining fast access paths during execution.
2Quantity of substance
If predictive prefetching is implemented on the CPU die, then prefetching can occur quickly, but the amount of memory available for storing predictions is very limited
Solution Approach 1:
The prediction storage capacity is extended from the two-dimensional constraint of CPU die area to a three-dimensional architecture that incorporates non-volatile memory layers. By moving prediction storage to a separate memory dimension (NVM), the system achieves both large capacity for predictions and maintains fast prefetching execution through the volatile memory layer, effectively resolving the space-speed tradeoff.
3Reliability
If caching is performed in volatile memory, then data can be accessed quickly during program execution, but the caching information is not preserved when the OS changes page placement between runs
Solution Approach 1:
The prediction engine creates a persistent copy of access patterns and predictions in non-volatile memory that survives power cycles and OS reconfigurations. This copy is then loaded back into the volatile memory prediction table when needed, allowing the system to maintain caching efficiency across different execution contexts without requiring the OS to preserve its own page placement information.
Data Source
AI summary
A memory device with non-volatile memory and persistent predictive prefetching provides highspeed storage to a computer system. The memory device uses a non-volatile memory to store data and a volatile memory to cache the data from the non-volatile memory. The computer system sends access requests to obtain data in the non-volatile memory. A prediction engine in the memory device receives the access requests. The prediction engine compute access histories based on the access requests and stores them in an access history table. The prediction engine computes prediction of non-volatile memory addresses that will be accessed in the future based on the stored access history table. The prediction engine causes to store the data from the predicted addresses of the non-volatile memory in the volatile memory. The memory device stores the prediction in the non-volatile memory so the past predictions can be used after restarting the computer system.