Distributed Storage File Management via Global Virtual Address Space Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional storage systems using next-generation nonvolatile memory face inefficiencies in file management due to large metadata storage requirements and complex storage location information processing, leading to low file management efficiency.
Innovation Solution
A distributed storage system with a management node that allocates virtual space from a global virtual address space, recording metadata to point to local storage nodes, allowing for efficient file creation, expansion, and deletion without processing complex storage location information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional storage systems use metadata to record storage location information for each data block, then file management can be achieved, but metadata occupies large storage space and increases management burden
Solution Approach 1:
The patent segments the global virtual address space into multiple local virtual address spaces, each corresponding to a specific storage node. This segmentation allows the system to manage storage locations more efficiently by using smaller, localized address spaces instead of a single large global metadata structure, thereby reducing the overall metadata storage requirements while maintaining file management capabilities.
Solution Approach 2:
The patent introduces a new dimension of virtual address space management by creating a mapping between global virtual address space and local virtual address spaces. This dimensional transformation enables the system to resolve storage locations through address space mapping rather than traditional metadata recording, reducing metadata overhead while preserving file management functionality.
2Loss of information
If metadata records detailed storage location information for each data block, then data can be located, but efficiency of accessing storage node by host is low
Solution Approach 1:
The patent performs preliminary action by pre-establishing the mapping relationship between global virtual address space and local virtual address spaces before actual file access operations. This pre-computation of address mappings enables hosts to efficiently locate storage nodes without requiring complex metadata processing at access time, thereby improving file management efficiency while preserving complete storage location information.
Solution Approach 2:
The patent introduces local virtual address space as an intermediary between the host and the physical storage location. This intermediary layer simplifies the access process by providing a direct mapping from global addresses to local storage nodes, eliminating the need for hosts to process complex metadata and thereby improving access efficiency while maintaining information completeness.
3Adaptability or versatility
If a global virtual address space is used for distributed storage system, then space allocation can be unified, but address bit quantity is larger compared to local virtual address space
Solution Approach 1:
The patent segments the large global virtual address space into multiple smaller local virtual address spaces, each managed by a specific storage node. This segmentation reduces the address bit quantity required at each node while maintaining unified space allocation capabilities through the global-to-local mapping mechanism, thereby resolving the contradiction between adaptability and complexity.
Solution Approach 2:
The patent transforms the single-dimension global address space problem into a multi-dimensional solution by introducing the global-to-local address space mapping dimension. This dimensional change allows the system to achieve unified space allocation across the distributed system while each node only needs to handle smaller local address spaces, reducing the effective address bit quantity at each node.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The present invention discloses a file management method, a distributed storage system, and a management node. The method includes: receiving, by a management node, a file creation request sent by a host, where the file creation request is used to create a file in a distributed storage system; allocating, by the management node to the file, first virtual space from global virtual address space of the distributed storage system according to the file creation request, where local virtual address space of each storage node in the distributed storage system is corresponding to a part of the global virtual address space; recording, by the management node, metadata of the file, where the metadata of the file includes information about the first virtual space allocated to the file, and the information about the first virtual space is used to point to local virtual address space of a storage node that is used to store the file; and sending, by the management node, a file creation success response to the host, where the file creation success response carries the information about the first virtual space. In embodiments of the present invention, file management efficiency can be improved.