Tiered Storage Data Allocation for Persistent Memory Cost
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently storing and retrieving data across different memory types, particularly in distributed networks, where persistent memory is expensive and not scalable like traditional storage devices.
Innovation Solution
The implementation of a system that dynamically allocates data across tiered storage devices, including persistent memory, DRAM, and HDDs, based on a cost metric that considers access frequency, access cost, latency, and re-computation costs, to optimize data retrieval and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If persistent memory is used to store frequently accessed data, then data retrieval speed is improved, but storage cost increases
Solution Approach 1:
The patent segments data into different types based on access patterns (frequently accessed vs. rarely accessed) and stores them in different memory tiers. Frequently accessed data is stored in persistent memory for fast retrieval, while rarely accessed data is stored in cheaper storage devices, resolving the contradiction between retrieval speed and storage cost.
Solution Approach 2:
Different data structures have different storage requirements based on their local characteristics. The patent applies local quality by storing only the necessary fields of composite data structures in persistent memory rather than the entire structure, optimizing the balance between access speed and storage cost for each specific data field.
2Loss of time
If composite data structures are stored entirely in persistent memory, then data access latency is reduced, but memory utilization efficiency decreases
Solution Approach 1:
The patent extracts only the essential fields from composite data structures and stores them in persistent memory, while leaving non-essential fields in cheaper storage. This extraction approach reduces data access latency for critical fields while improving overall memory utilization efficiency by not consuming expensive persistent memory for less important data.
Solution Approach 2:
Instead of storing entire composite data structures in persistent memory (excessive action), the patent applies partial action by storing only the necessary fields. This partial storage approach maintains low latency for critical data access while significantly improving memory utilization efficiency.
3Adaptability or versatility
If data is stored in distributed network nodes, then system scalability is improved, but network data transfer overhead increases
Solution Approach 1:
The patent implements preliminary action by pre-computing and storing aggregate data or frequently accessed data fields at distributed nodes before they are needed. This reduces the need for real-time network data transfers when queries are executed, maintaining system scalability while minimizing network transfer overhead.
Data Source
AI summary
Aspects of the subject technology relate to ways to determine the optimal storage of data structures in a hierarchy of memory types. In some aspects, a process of the technology can include steps for identifying a retrieval cost associated with retrieving a field in an object from data storage, comparing the retrieval cost for the field to a cost threshold for storing data in persistent memory, and selectively storing the field in either a persistent memory device or a non-persistent memory device based on a comparison of the retrieval cost for the field to the cost threshold. Systems and machine-readable media are also provided.