Storage Controller Automatic Data Phasing via Lifespan Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In data analysis applications, existing storage systems lack efficient mechanisms to automatically manage data lifespan, leading to unnecessary garbage collection and data movement, as the host typically relies on overwriting logical data to delete previous versions, while the storage system performs garbage collection without knowing the data content, potentially moving data before erasure commands are received.
Innovation Solution
A storage system with a controller that receives data and an indication of its expected lifespan, invalidates data in the logical-to-physical address table after the lifespan elapses, thereby avoiding garbage collection and allowing for optimized storage decisions, such as choosing data protection methods and memory regions based on data endurance and lifetime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the storage system performs garbage collection on all data without knowing data lifespan, then garbage collection operations are executed uniformly, but unnecessary data movement occurs and storage efficiency deteriorates
Solution Approach 1:
The host performs preliminary action by informing the storage system of data lifespan before the data is actually deleted. This allows the storage system to proactively invalidate L2P table entries and skip garbage collection for short-lived data, preventing unnecessary data movement before it occurs.
Solution Approach 2:
The host provides feedback to the storage system about data lifespan characteristics. This feedback mechanism enables the storage system to adjust its garbage collection behavior dynamically, using the lifespan information to make intelligent decisions about which data to collect and which to ignore.
2Ease of operation
If the host tracks data movement and logical addresses to manage data deletion, then data can be deleted on demand, but host complexity and management overhead increase
Solution Approach 1:
The storage system performs self-service by automatically managing data lifecycle based on lifespan information provided by the host. The host simply provides lifespan metadata, and the storage system autonomously handles invalidation and garbage collection decisions, relieving the host of complex tracking and management responsibilities.
Solution Approach 2:
Data lifespan information acts as an intermediary mechanism between the host and storage system. Instead of complex direct communication about data movement and deletion, the host provides lifespan metadata that mediates the interaction, allowing the storage system to autonomously make management decisions.
3Reliability
If the host sends erase commands to logically erase data, then data deletion is achieved, but the storage system may have already moved the data through garbage collection
Solution Approach 1:
The storage system performs preliminary action by invalidating L2P table entries before garbage collection occurs. By proactively marking short-lived data as invalid based on lifespan information, the system prevents data movement before the host's erase command is even sent, ensuring data is truly deleted and not relocated.
Solution Approach 2:
The storage system applies preliminary anti-action by preemptively preventing garbage collection on short-lived data through L2P invalidation. This counteracts the potential harmful effect of moving data that is destined for deletion, eliminating the problem before it can occur.
Data Source
AI summary
A storage system and method for automatic data phasing are disclosed. In one embodiment, a storage system is configured to receive, from a host, data to be written in the memory and an indication of an expected lifespan of the data; and determine whether to perform a garbage collection operation on the data based on the expected lifespan of the data. Other embodiments are provided.