Dynamic mapping strategy for improving space utilization rate of ZNS solid state disk

Abstract

The invention discloses a dynamic mapping strategy for improving the space utilization rate of a ZNS solid state disk. The dynamic mapping strategy comprises the following steps that 1, Strip strips are distributed according to needs; the method comprises the following steps: dynamically allocating different numbers of Strip strips for a logic Zone according to different sizes of SST / WAL files during load operation; 2, only one file is stored in one Zone; a new null logic Zone is allocated to each SST / WAL file, and it is guaranteed that only one SST / WAL file is stored in one Zone; and 3, providing an infinite logic Zone number. The equipment provides an infinite logic Zone number for an upper-layer application so as to fully utilize resources in the equipment; and 4, resetting the Zone if the file fails. And when one SST / WAL file is invalid, the upper application directly sends a reset command to the Zone, and the Zone space is quickly recovered. According to the method, the problems of space amplification and data migration of LSM tree storage on the ZNS SSD can be well solved, the space utilization rate can be remarkably increased, the data migration overhead is eliminated, and then the overall I / O performance is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer storage technology, and in particular to a storage space utilization optimization method in a Log-Structured Merge-tree (LSM-tree) scenario for a Zoned Namespace (ZNS) solid-state hard disk. Background Art

[0002] ZNS is an emerging SSD interface specification that circumvents garbage collection inside the device, simplifies firmware complexity, and reduces hardware costs by dividing the logical address space into multiple zones and requiring sequential writes. Thanks to partition management, data is organized and erased in units of zones, so ZNS SSDs are very suitable for LSM-tree type data storage systems.

[0003] However, the existing LSM-tree storage system has serious space amplification and data migration problems on ZNS solid-state drives. This is because the basic failure unit in the LSM-tree is the Sorted String Table (SST) file or the Write-Ahead Log (WAL) file, while the erase unit in the ZNS solid-state drive is the Zone. If multiple SST / WAL files with inconsistent life cycles are placed in the same Zone, it will cause serious space amplification in the Zone and require a large number of data migration operations. Existing studies have attempted to ensure that a Zone only stores one file by limiting the size of the SST / WAL file at the application layer, but the actual effect is not ideal.

[0004] In addition, LSM-tree faces other challenges on ZNS SSDs. For example, since the size of SST / WAL files is not fixed, it is difficult to perfectly match them with fixed-size zones, resulting in a large amount of fragmented space inside the zones. At the same time, due to the lack of accurate prediction of the zone lifespan, SST / WAL files of different life cycles may still be mixed in the same zone, delaying the reset time of the zone and reducing space utilization efficiency. These problems seriously restrict the energy efficiency of ZNS SSDs in LSM-tree storage scenarios. Summary of the invention

[0005] In response to the above technical problems, the present invention proposes a dynamic storage resource mapping strategy that combines software and hardware, which can achieve high space utilization and zero-copy LSM-tree data management on the ZNS solid-state drive without changing the ZNS specification. The basic idea is to store an SST / WAL file in a separate Zone. When the SST / WAL file fails, the application layer directly sends a Zone reset command to the ZNS solid-state drive through the ZNS interface, completely avoiding space expansion and data migration.

[0006] The technology adopted by the present invention comprises the following steps:

[0007] Step 1: Only one SST / WAL file is placed in each zone. Since each SST / WAL file occupies a zone exclusively, the zone can be reset immediately when the file fails to reclaim the space, thus avoiding space waste and data migration overhead to the greatest extent.

[0008] Step 2: Allocate storage resources in the zone as needed. For SST / WAL files of different sizes, Hi-ZNS allocates different numbers of flash blocks to the zone at runtime. It should be noted that Hi-ZNS still follows the ZNS specification to ensure that the logical zone size is fixed, but only adjusts the actual number of flash resources mapped to the zone. Through this flexible resource configuration method, Hi-ZNS can maximize the matching of SST / WAL file characteristics while complying with the ZNS interface specification and improving space utilization.

[0009] Step 3: Provide unlimited logical zone space. Since the logical zone size is not equal to the actual allocated flash resources, Hi-ZNS needs to continuously track the available resources of the ZNS SSD. As long as there are remaining resources in the device, Hi-ZNS can continuously allocate logical zones (by incrementing the zone number) to make full use of the flash space. This design breaks through the limitation of the number of physical zones and provides greater flexibility. At the same time, Hi-ZNS maintains the mapping relationship between the logical zone number and the actual physical resources to ensure the correct location and access of data.

[0010] The beneficial effect of the present invention is that by adopting mechanisms such as allocating a single logical zone by file, dynamic resource mapping, and providing unlimited logical space to the device upward, the space utilization rate of the ZNS solid-state hard disk can be significantly improved, and data migration at the file system level is also completely eliminated, thereby further improving the I / O performance of the LSM tree storage on the ZNS solid-state hard disk and improving the overall performance of the storage system. The design idea of ​​the present invention is simple and elegant. Without changing the ZNS interface specification, it taps the potential of the ZNS solid-state hard disk, while ensuring good versatility and compatibility, and can be easily integrated into the existing ZNS ecosystem to provide efficient support for upper-level applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of the system architecture of the present invention;

[0012] Figure 2 It is a flow chart of the dynamic mapping strategy of the present invention;

[0013] Figure 3 This is a schematic diagram of flash resource management of the present invention; DETAILED DESCRIPTION

[0014] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0015] Figure 1 The overall architecture diagram and implementation principle diagram of the present invention. The present invention spans the file system layer and the ZNS solid-state device layer. Inside the ZNS solid-state hard drive, a new resource management unit, Strip, is introduced. A Strip consists of a block of each flash chip. Strip Allocator is responsible for managing the allocation and recycling of all Strips. The Zone Mapping Table records the mapping relationship between logical Zone numbers and Strips. At the file system level, the Device Capacity Maintainer is responsible for tracking the actual available capacity of the entire ZNS solid-state hard drive. The Zone Management module is responsible for the management of logical zones and provides unlimited logical Zone numbers to make full use of flash resources. The upper-level LSM-tree storage engine (such as RocksDB) performs read and write operations based on the file system interface, but is unaware of the internal implementation details of the present invention.

[0016] Figure 2 This is a schematic diagram of flash resource management of the present invention. Each logical zone in the device is dynamically allocated Strip storage resources as needed, and at the same time the logical zone maintains a fixed logical size to comply with the ZNS interface specification.

[0017] Figure 3The dynamic mapping strategy flow chart of the present invention. Starting from step 101, the specific steps are as follows:

[0018] In step 102: an independent logical zone is allocated to each file. When the application layer writes an SST / WAL file, a new empty logical zone is allocated to it at the file system layer.

[0019] Step 103: Initiate a write request to the target address in the Zone.

[0020] Step 104: Determine whether the target address of the write request has actual storage resource mapping. If yes, directly proceed to step 106. If not, first execute 105 to allocate resources.

[0021] Step 105: Obtain sufficient Strip storage resources from the Strip resource pool at one time and allocate them to the target logical Zone to ensure that it can meet the current write demand.

[0022] Step 106: Execute the actual write operation to write the requested content into the flash medium.

[0023] Step 107: Determine whether the file has been written. If it needs to be written, return to 103 to continue writing. If it has been written, go to 108.

[0024] Step 108: Change the status of the target logical zone just used to full to prevent other files from being written into the zone.

Claims

1.S1: A dynamic mapping strategy to improve the space utilization of ZNS solid state drives, characterized by , including the following steps: Step 1: Allocate resources on demand to build logical zones. When writing SST / WAL files, dynamically allocate different numbers of Strip resources to logical zones based on the file size in real time. Each Strip contains a physical block across all flash chips. Step 2: Provide unlimited logical zone numbers. Consider the logical zone numbers as unlimited. As long as there are strip resources available, continue to increase the logical zone numbers to fully utilize the storage resources in the device. Step 3: Each logical zone stores only one SST / WAL file. In the present invention, the traditional strategy of allocating zones according to life cycle prompts is abandoned, and a new empty logical zone is allocated for each SST / WAL file. Step 4: Reclaim the Zone space immediately when the SST / WAL file fails. When the SST / WAL file fails, the upper-layer application directly sends a Zone reset command to the ZNS SSD device, and the Zone space can be reclaimed without any data migration. S2: The mechanism according to claim S1 is characterized in that: in step 1, the ZNS specification is followed to ensure that the logical zone size is fixed, and only the number of physical blocks allocated to the zone is adjusted. At the device level, the Strip Allocator is used to record the valid / invalid status of all strips, and the Zone mapping table is used to record the mapping relationship between the logical zone number and the strip. S3: According to the mechanism described in claim S1, it is characterized in that: in step 2, since the size of the logical zone is not equal to the actual physical space mapped, the file system level tracks the actual available capacity of the ZNS SSD device through the device capacity maintainer. S4: The mechanism according to claim S1 is characterized in that: in step 3, the present invention ensures that each logical zone at the file system level contains only one SST / WAL file. For other types of files with smaller volumes, they are placed in the same zone. S5: The mechanism according to claim S1, characterized in that : When building the logical zone in step 1, it is preferred to use fully parallel strips, that is, the strip width is equal to the total number of flash chips in the ZNS SSD device, allowing each strip to utilize the full parallelism of the device. Although fully parallel strips will produce larger fragments, they can still significantly improve space utilization without reducing I / O performance.

Images