Page-Level Storage Access Without Data Index Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies due to large data indices occupying significant storage space and requiring extensive computing power for data access, especially in storage-compute decoupled architectures.
Innovation Solution
Implement a storage system with a computing node and storage node architecture where page-level storage is provided by the storage node, using logical chunk groups as the minimum allocation unit, allowing direct access to storage locations via page identifiers that include logical chunk group identifiers and offsets, eliminating the need for data indices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored at block, file, or object granularity with traditional indexing, then data can be accessed through established methods, but the data index occupies large storage space and reduces access efficiency
Solution Approach 1:
The patent segments the storage space into logical chunk groups, where each group contains multiple contiguous chunks. This segmentation allows the system to manage storage at a coarser granularity while maintaining fine-grained access capabilities, reducing the overhead of traditional fine-grained indexing.
Solution Approach 2:
The patent introduces a new dimensional organization by mapping pages to logical chunk groups and chunks rather than using traditional flat indexing. The page identifier encodes multiple dimensions including logical chunk group identifier and chunk identifier, enabling direct calculation of storage locations without linear index traversal.
2Ease of operation
If traditional data indexing is used to manage page-level storage, then data can be located through index queries, but the indexing process consumes extensive computing power and time
Solution Approach 1:
The patent performs preliminary organization of storage space into logical chunk groups and assigns page identifiers with embedded location information in advance. This preliminary structuring eliminates the need for runtime index queries, as the storage location can be directly calculated from the page identifier without additional computing operations.
Solution Approach 2:
The page identifier itself contains the necessary location information (logical chunk group identifier and chunk identifier), making the identifier self-descriptive. The system does not need external indexes to locate data because the identifier carries its own addressing information, enabling self-service location without additional computational overhead.
Data Source
AI summary
An storage system in a storage-compute decoupled architecture includes a computing node and a storage node. A hard disk in the storage node provides page-level storage space for the computing node, and a minimum allocation unit in the page-level storage space is defined as a logical chunk group. The computing node sends an access instruction to the storage node, where the access instruction carries a page identifier of a target page. The storage node parses out a logical chunk group identifier and an offset from the page identifier of the target page, where the offset indicates a location of the target page in a logical chunk group identified by the logical chunk group identifier; and accesses the target page based on the storage location to which the logical chunk group identifier and the offset point.