Table of Contents Entries for Persistent Storage Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for reading and writing data to persistent storage introduce significant latency due to processing by multiple system kernel layers and hardware entities, affecting overall system performance.
Innovation Solution
A method that involves receiving requests to write data to persistent storage, determining physical addresses, generating table of contents entries, and populating an in-memory data structure with mappings between logical and physical addresses, allowing direct access to user data using the in-memory data structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional multi-layer kernel processing is used for reading and writing data to persistent storage, then data can be accessed through standard system interfaces, but significant latency is introduced reducing overall system performance
Solution Approach 1:
The patent segments the address translation process by separating the TOC (table of contents) from the data blocks. The TOC contains metadata mappings that enable direct address translation without requiring traversal through multiple kernel layers, thus reducing latency while maintaining data accessibility through standard interfaces.
Solution Approach 2:
The patent introduces an intermediary in-memory data structure that caches the TOC metadata. This intermediary enables fast address translation by providing a direct mapping between logical and physical addresses, eliminating the need for repeated traversals through the multi-layer kernel processing stack.
2Ease of operation
If multiple layers in the system kernel process read and write requests, then standard system interfaces can be used, but the processing complexity and time increase
Solution Approach 1:
The patent performs preliminary action by pre-computing and caching the address mappings in the TOC data structure before they are needed for actual data access. This allows the system to use standard interfaces without repeatedly traversing multiple kernel layers, as the address translation is already prepared and cached in memory.
3Stability of the object's composition
If address mappings are stored in intermediate metadata hierarchies, then data can be organized and accessed systematically, but additional processing steps are required increasing latency
Solution Approach 1:
The patent extracts the essential address mapping information from the complex multi-layer kernel processing hierarchy and places it directly into an in-memory data structure. This extraction eliminates the need to traverse intermediate metadata hierarchies during data access, reducing processing steps while maintaining systematic data organization through the TOC structure.
Data Source
Figure 1A~1B
Figure 1C~1D
Figure 1E
AI summary
A method for storing data, including receiving a request to write a first datum defined using a first object ID and a first offset ID to persistent storage. The method further including determining a first physical address in the persistent storage, where the first physical address comprises a first block ID and first sub block ID. The method further includes writing the first datum to the first physical address, generating a first table of contents entry (TE) comprising the first object ID, the first offset ID, and the first sub block ID, and writing the first TE to a second physical address in the persistent storage, where the second physical address comprises the first block ID and a second sub block ID corresponding to the second sub block ID, and where the second sub block is located within a first block corresponding to the first block ID. storage system with self describing data