Persistent Address Space Management via Log-Based Metadata Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems fail to properly manage data on cache and solid-state storage devices after improper shutdowns, leading to invalid data consideration and loss of volatile metadata, which affects data integrity and system reliability.
Innovation Solution
The implementation of a system that uses a non-volatile storage device with a storage controller and write/read data pipelines, along with indexes and metadata management, to maintain data validity and integrity by storing data in a sequential log-based format and employing error correction codes, ensuring data can be reconstructed even after an invalid shutdown.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If volatile metadata is used to manage cache data, then data access speed is improved, but data integrity is lost after improper shutdown
Solution Approach 1:
The system performs preliminary actions by writing data sequentially to non-volatile storage media before cache operations complete, ensuring a persistent record exists. The sequential log format pre-establishes a recoverable state that can be used after improper shutdowns, preventing data integrity loss while maintaining cache speed benefits.
Solution Approach 2:
The patent introduces an intermediary mechanism (sequential log with metadata on non-volatile storage) that bridges the volatile cache and persistent storage. This intermediary maintains the mapping between logical and physical addresses across power cycles, allowing the system to recover data integrity without sacrificing cache access speed.
2Productivity
If TRIM messages are used to identify removable data, then storage efficiency is improved, but data validity is lost due to volatile metadata
Solution Approach 1:
The system pre-records data validity information and TRIM message states in the sequential log on non-volatile storage before operations complete. This preliminary recording ensures that when the system restarts after improper shutdown, it can recover the validity state and continue TRIM operations without losing information about which data should be removed.
Solution Approach 2:
The patent creates a copy of the metadata and validity information in the sequential log on non-volatile storage media. This copy serves as a backup that preserves TRIM message states and data validity information even when the volatile cache is lost, allowing the system to maintain storage efficiency by recovering which data should be removed.
3Reliability
If sequential log-based format is used for data storage, then data integrity is improved, but storage access complexity increases
Solution Approach 1:
The patent segments the storage system into distinct components: a sequential log area for persistent metadata and data, and a cache area for fast access. This segmentation allows the complex sequential log format to be isolated from the fast cache operations, maintaining data integrity while managing access complexity through clear separation of concerns.
Solution Approach 2:
The sequential log acts as an intermediary layer between the host system and the cache/storage media. It provides a standardized format for recording operations that simplifies the interface while handling the complexity of recovery and integrity checks internally, reducing the apparent complexity for external access.
Data Source
AI summary
Data is stored on a non-volatile storage media in a sequential, log-based format. The formatted data defines an ordered sequence of storage operations performed on the non-volatile storage media. A storage layer maintains volatile metadata, which may include a forward index associating logical identifiers with respective physical storage units on the non-volatile storage media. The volatile metadata may be reconstructed from the ordered sequence of storage operations. Persistent notes may be used to maintain consistency between the volatile metadata and the contents of the non-volatile storage media. Persistent notes may identify data that does not need to be retained on the non-volatile storage media and/or is no longer valid.