Object Cache Prefetching for Distributed Memory Subsystems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer memory architectures face inefficiencies in prefetching due to inaccurate predictions and large communication overheads in distributed systems, leading to suboptimal cache utilization and increased latency.
Innovation Solution
A compute node architecture with a memory subsystem that uses object-based management, assigning unique identifiers to memory objects, and a runtime management unit to prefetch data based on metadata and object types, enabling accurate and efficient data retrieval from a distributed store.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If predictive prefetching is used with cache-based memory hierarchies, then memory access efficiency is improved, but prediction accuracy is limited by reliance on memory address information
Solution Approach 1:
The patent introduces an object cache as an intermediary layer between the compute core and the distributed store. This object cache stores complete memory objects (rather than individual fragments) and uses object identifiers as keys, enabling the system to prefetch entire objects based on object-level predictions rather than fragment-level address patterns, thus improving prediction accuracy
Solution Approach 2:
The patent segments the memory system into distinct layers: the object cache at the compute node storing complete objects, and the distributed store storing object fragments. This segmentation allows the object cache to handle prediction and prefetching at the object level while the distributed store maintains fragments, resolving the contradiction between efficiency and prediction accuracy
2Speed
If memory address information is cached locally at compute node level, then memory access speed is improved, but large memory capacities and complex cache synchronization protocols are required
Solution Approach 1:
The object cache acts as an intermediary that stores complete memory objects locally at the compute node. By caching entire objects rather than individual address information, the system achieves fast local access without requiring complex synchronization protocols, as each compute node's object cache is independently managed
Solution Approach 2:
The system copies complete memory objects from the distributed store to the local object cache when they are accessed or predicted to be accessed. This copying approach eliminates the need for complex address translation and synchronization mechanisms, as the object cache contains self-contained copies of entire objects
3Quantity of substance
If memory objects are fragmented and stored in distributed store, then storage capacity is improved, but memory access latency increases
Solution Approach 1:
The system performs preliminary action by prefetching complete memory objects from the distributed store to the local object cache before they are actually needed by the compute core. This advance loading of entire objects eliminates the latency of accessing fragmented data from the distributed store during actual computation
Solution Approach 2:
The object cache serves as an intermediary buffer between the fragmented distributed store and the compute core. It assembles complete memory objects from stored fragments and provides them to the compute core in full, thereby hiding the latency of fragmented storage access while maintaining high storage capacity
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A compute node (500) for a multi-node computer and related prefetching method are disclosed. The compute node comprises a compute core (511) for executing tasks on memory objects, a runtime management unit (516), a memory subsystem (503; 700) with local storage controller (710), and an object management unit (502). The object management unit is configured to calculate the physical address of each memory object fragment accessible on a distributed store from a combination of an object identifier (OID) and fragment offset. The runtime management unit is configured to notify the memory subsystem of the memory object identifiers (OID) related to scheduled tasks. The local storage controller is configured to detect the object type included in the metadata portion of each notified memory object and command prefetching of a type-dependent number of fragments of the notified memory object, or another memory object referred to in the metadata portion, from a deeper level of the memory subsystem or the distributed store.