Virtual Block Mapping for Direct Data Flush in Non-Volatile Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage systems face challenges in efficiently flushing user data directly ingested into non-volatile data storage while maintaining accessibility through logical addresses, especially when data sizes meet a minimum threshold, and handling errors in metadata.
Innovation Solution
The technology allocates virtual layer blocks for physical blocks of non-volatile data storage to store virtual block pointers, which are then mapped to logical address mapping trees, enabling direct ingestion and error handling without copying or moving data, and marking blocks with a 'known bad' status during errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If user data is directly stored into non-volatile data storage bypassing cache, then storage efficiency and speed are improved, but data accessibility through logical addresses becomes difficult to maintain
Solution Approach 1:
The patent introduces a virtual layer block as an intermediary structure between the physical storage block and the logical address mapping tree. This virtual layer block contains virtual block pointers that map logical addresses to physical storage locations, enabling the system to maintain both direct storage efficiency and logical address accessibility. The virtual layer acts as a mediator that resolves the contradiction by providing the necessary mapping information without requiring data copying.
Solution Approach 2:
The patent adds a virtual dimension to the storage architecture by introducing virtual block pointers alongside physical block pointers. This creates a multi-dimensional addressing system where data can be accessed both through direct physical storage and through logical address mapping, resolving the contradiction between storage efficiency and accessibility by operating in multiple address spaces simultaneously.
2Ease of operation
If virtual layer blocks are allocated for directly stored data, then data accessibility through logical addresses is maintained, but system complexity increases
Solution Approach 1:
The patent merges the virtual layer block allocation with the existing flush request processing mechanism. When a flush request is received for directly stored data, the system simultaneously performs the flush operation and allocates the virtual layer block with appropriate pointers in a single coordinated operation. This merging approach maintains data accessibility while minimizing the increase in system complexity by combining multiple functions into one process flow.
Solution Approach 2:
The system performs preliminary allocation of virtual layer blocks when flush requests are received for directly stored data, before any read operations need to access the data through logical addresses. This preliminary action ensures that the mapping structure is ready in advance, reducing the complexity of on-demand allocation and enabling faster data access when needed.
3Reliability
If data is flushed from cache to non-volatile storage, then data persistence is ensured, but data copying and moving operations increase
Solution Approach 1:
The patent extracts the data copying operation from the flush process for directly stored data. Instead of copying data from cache to non-volatile storage, the system directly stores data in non-volatile storage and then extracts only the necessary mapping information (virtual block pointers) to create the virtual layer block. This extraction eliminates redundant data copying while maintaining persistence, as the data is already in its final destination.
Solution Approach 2:
The patent selectively applies copying only where necessary - copying virtual block pointers and mapping information rather than copying the actual data. This selective copying approach maintains data persistence through proper mapping while minimizing data copying overhead, as the actual data remains in place and only the metadata is copied to establish the virtual layer mapping.
Data Source
AI summary
For a flush request for a first block of user data directly stored into a physical block of non-volatile data storage, allocating a virtual layer block, and storing, in the virtual layer block, a first set of virtual block pointers indicating pages of the first block within the physical block. Pointers to the virtual block pointers in the first set are then stored into a leaf layer of a logical address mapping tree. The virtual layer block is identified for a subsequent flush request issued for a second block of user data directly stored into the physical block, and a second set of virtual block pointers indicating pages of the second block within the physical block is stored in the virtual layer block. Pointers to the virtual block pointers in the second set are stored into a leaf layer node of another logical address mapping tree.