Partition namespace solid-state disk system architecture based on dynamic region

Abstract

The invention discloses a partition namespace solid-state disk system architecture based on a dynamic area, and belongs to the technical field of ZNS storage, the sizes of small zone and large zone storage and hierarchical boundaries of data placed in the small zone and the large zone are preset, compression is carried out in a solid-state disk to generate SSTtables, the overlapped SSTtables in the compression are placed in the same zone, and the SSTtables are stored in the same zone. And data of different levels are put into zones of different sizes. By adopting the partition namespace solid-state disk system architecture based on the dynamic region, effective data migration can be remarkably reduced, free space of zone is fully utilized, the service life of a ZNS SSD is prolonged, the write amplification problem of a ZNS platform is effectively relieved, and the overall performance of the system is improved.

Description

Technical Field

[0001] The present invention relates to the field of ZNS storage technology, and in particular to a partition namespace solid state disk system architecture based on dynamic areas. Background Art

[0002] As a new type of SSD with a new interface, ZNS SSD transfers the data placement and garbage collection of traditional SSDs to the host, allowing the host to directly control the device and improve the stability of the device. In the ZNS scenario, different applications store business data according to the zone configuration information. Since the host manages data placement, garbage collection can be minimized to the greatest extent, thereby reducing the write amplification inside the SSD and increasing the life of the SSD. In addition, since garbage collection is controlled by the host, the OP space originally used for GC optimization is no longer needed, and the GB-level DRAM overhead required for the original page-level mapping table is also reduced to the KB level of ZNS coarse granularity, which greatly reduces the storage cost. In order to ensure a high degree of parallelism, a zone contains multiple flash blocks. In this way, ZNS performs reset operations at the zone granularity. When garbage collection is triggered, after the host selects the zone to be recycled, even if the data in the zone is only partially invalid, the entire zone will be erased. Therefore, it is necessary to migrate the valid data in it to other writable zones, and then reset the original zone to an "empty" writable state.

[0003] In addition, LSMT (Log Structured Merge Tree) is a structure that converts random writes of write-intensive loads into sequential writes. It is widely used in key-value storage systems. LSMT stores data in a tree hierarchy with SSTable (Sorted String Table) as the basic unit, and each layer contains multiple SSTables. Data is first stored in the MemTable (Memory Table) in the memory. As data is continuously written, when the size of the MemTable reaches a certain level, LSMT will trigger a compaction operation to merge the upper SSTable into the next layer. This process reduces data fragmentation and improves query performance.

[0004] In a key-value storage system using LSMT, data is written to SSTable in batches, and during the merge sort process of the compaction operation, the data in the upper layer is selected together with the key-value pairs that overlap with the lower layer and are written to the lower layer. The size of each layer of LSMT increases layer by layer, and the layers closer to the memory are smaller, and the layers farther away from the memory are larger. When the merge sort is executed, the data needs to undergo multiple reads and rewrites, which leads to the so-called write amplification phenomenon. In the case of using LSMT in ZNS SSD, storage management is based on larger zones. During garbage collection, data in the entire zone may need to be migrated to obtain continuous free space. This process also generates additional write activity, exacerbating the problem of write amplification.

[0005] In addition, the current mainstream ZNS platform uses the ZenFS file system, a backend plug-in of RocksDB, to control the device. However, ZenFS assumes that the lifespan of SSTables at the same level is similar, so try to place SSTables at the same level in the same zone to ensure that the amount of valid data that needs to be migrated when GC is triggered is very small, reducing migration overhead. However, through experimental analysis, we found that the lifespan of SSTables at the same level is not necessarily similar. This prediction of lifespan is only limited to lower levels and is very inaccurate for higher levels. This leads to a significant increase in migration overhead. The simultaneous effect of the two will seriously affect the write performance of key-value storage applications. Summary of the invention

[0006] The purpose of the present invention is to provide a partition namespace solid-state disk system architecture based on dynamic zones, which can significantly reduce effective data migration, make full use of the free space of the zone, extend the life of the ZNS SSD, effectively alleviate the write amplification problem of the ZNS platform, and improve the overall performance of the system.

[0007] To achieve the above objectives, the present invention provides a partitioned namespace solid-state disk system architecture based on dynamic zones, which pre-determines the storage size of small zones and large zones and the hierarchical boundaries of data placement in small zones and large zones, performs compaction merging and compression in the solid-state disk and then generates SSTables, places overlapping SSTables in the compaction into the same zone, and places data of different levels into zones of different sizes.

[0008] Preferably, the specific steps are: first, all low-level related data are placed in a small zone, and all high-level related data are placed in a large zone across flash memory chips, while retaining the advantages of small zones and large zones; When compaction between Li and Li+1 is triggered, some new SSTables are generated in Li+1, and all zones in the Li+2 layer that overlap with the newly generated SSTable key value range are sorted according to the remaining space; Select the zone with the smallest remaining space from the sorted zones and write it into the newly generated SSTable. If there is insufficient space or there are no overlapping zones, select a new zone to write into. When the compaction between Li+1 and Li+2 is triggered next time, the victim SSTable of Li+1 and the overlapping SSTable in Li+2 are located in the same zone and become invalid data.

[0009] Therefore, the present invention adopts the above-mentioned partitioned namespace solid-state disk system architecture based on dynamic areas. By placing data of different levels into zones of different sizes, the effective data migration of ZNS devices can be reduced, and placing SSTables of different levels but overlapping into the same zone can reduce the problem of effective data dispersion caused by compaction merge sorting operations and thus increase migration overhead. It can significantly reduce effective data migration, make full use of the free space of the zone, extend the life of the ZNS SSD, reduce the number of write operations, effectively alleviate the write amplification problem of the ZNS platform, and improve the overall performance of the system.

[0010] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is the data placement diagram of LIZA; Figure 2 It is a structural schematic diagram of a dynamic area of ​​an embodiment of a partition namespace solid state disk system architecture based on a dynamic area of ​​the present invention; Figure 3 It is a solid state disk system architecture diagram of a dynamic area according to an embodiment of a partition namespace solid state disk system architecture based on a dynamic area of ​​the present invention. DETAILED DESCRIPTION

[0012] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0013] Embodiment 1

[0014] The file system ZenFS used by the existing ZNS SSD adopts LIZA (Liftime Zone Allocation) for the data placement of LSMT's SSTable. It is based on a prediction that SSTables at the same level have the same lifespan, so placing them in the same zone can make their lifespans approximately equal. In this way, the amount of valid data during GC is not that large, and most of it is invalid data with similar lifespans, which greatly reduces the data migration overhead. However, this prediction is not accurate. Through experimental analysis, it is found that the level-based lifespan prediction is only effective at the low level, and is not accurate at the high level.

[0015] like Figure 1 As shown in the figure, after the compaction, there are still many valid SSTables in zone1. If garbage collection is triggered at this time, the valid data in this zone will be migrated to zone3. If compaction is continued, the data in level2 will be written repeatedly, causing serious write amplification.

[0016] In addition, for low-level levels such as level 0 and level 1, the amount of data is small. If all global levels use zones of the same size, the low-level zones may not be fully used, which will not only cause space enlargement, but also cause a lot of remaining valid data, which will also bring considerable migration overhead.

[0017] These problems are mainly determined by the structure and compression characteristics of LSMT. Only the lifespans of SSTables participating in the same compaction are similar. Using larger zones at lower levels will bring a lot of data migration overhead.

[0018] The present invention provides a partition namespace solid-state disk system architecture based on dynamic regions, presetting the storage size of small zones and large zones and the hierarchical boundaries of data placement in small zones and large zones, performing compaction merging and compression in the solid-state disk and then generating SSTables, placing overlapping SSTables in the compaction into the same zone, and retaining the advantages of large zones and small zones at the same time (the feature of large zones crossing flash memory chips can make full use of device parallelism, and small zones can avoid large data migration overhead), and placing data of different levels into zones of different sizes. The specific process is as follows: First, the size of small zones is uniformly set to 96MB, and that of large zones is uniformly set to 1024MB to accommodate the data placement of SSTables at different levels. The level boundary for placing small zones and large zones is set to 2 (that is, which level is used as the boundary for placing small zones and large zones. For example, if level k is used as the boundary, then SSTables from level 0 to level k are placed in small zones, and SSTables in levels greater than level k are placed in large zones). The zone size setting and level boundary setting are derived from experimental data analysis ( Figure 2 The case of medium and large zones and small zones is only used to illustrate parallelism, not to say that a small zone really has only 2 flash blocks). The specific process is as follows: The data is first accumulated in the memory to form a MemTable. When the MemTable in the memory reaches the capacity threshold, the flush operation is triggered to reorganize the data from the memory into the SSTable file format and dump it to level 0. According to the pre-set settings, level 0 should be placed in a small zone, so an empty zone is allocated from the free zone queue and the SSTable data generated by the flush is placed in it.

[0019] As subsequent data is continuously written, more and more data is flushed from memory to level 0. When the number of SSTables in level 0 reaches the limit of level 0, the compaction operation is triggered, and the SSTables in level 1 that have overlapping keys with level 0 participate in the merge sort, and a new SSTable is generated and output to level 1, but it is not placed in the same zone as the original SSTable in level 1. Instead, the newly generated SSTable is compared with the SSTable in level 2. The zones where all level 2 SSTables that have overlapping keys with the new SSTable are located form a priority queue, and are sorted in descending order according to the number of overlapping SSTables in each zone, and the zones with the largest number of overlapping SSTables are allocated in turn to store these new SSTables.

[0020] For example, suppose L0-L1 generates new SST1 and SST2 and outputs them to L1, then the files overlapping with these newly generated SSTs are checked one by one from L2. The result is: In L2, SST3 (located in zone 1) overlaps with SST1, and SST4 and SST5 (located in zone 2) overlap with SST1; In L2, SST6 and SST7 (located in zone 3) overlap with SST2, and 8 (located in zone 4) overlaps with SST2; Then, for the overlap with SST1, zone1 and zone2 are arranged in descending order according to the number of overlapping SSTs: zone2, zone1 (zone2 has two overlapping SST4 and SST5, and zone1 has only SST3), so zone2 is preferred to store the newly generated SST1, and if zone2 does not have enough space, it will be postponed to zone1. For the overlap with SST2, zone3 is given priority, and if there is not enough space, zone4 will be considered.

[0021] The same is true for the data placement of the newly generated SSTables of L1-L2 compaction. In this way, the next time level 1 triggers L1-L2 compaction, most of the overlapping SSTables are located in the same zone and the data are invalid at the same time, reducing the effective data migration overhead during background GC.

[0022] It should be noted that the SST output by the L1-L2 compaction is logically output to L2, but is actually placed in other zones. Since the boundary level between large zones and small zones is set to 2, the output data of flush, L0-L1, and L1-L2 compaction are all placed in small zones. Subsequent L2-L3, L3-L4, ... compactions are placed in large zones for parallelism considerations.

[0023] According to experimental data analysis, the traditional LIZA algorithm is accurate in predicting the lifespan of low-level data, but inaccurate for high-level data. This leads to a problem that different lifespans lead to a significant increase in data migration overhead. Therefore, in this embodiment, different levels are placed in different zones to reduce the lifespan gap of individual data in the zone.

[0024] The capacity of the lower level is limited, and its data volume is far less than that of the higher level. If large zones are used to store SSTable data with similar lifespans from different levels, and large-capacity zones store a small amount of data, the free zones will be allocated quickly, and GC will be triggered when the number of free zones is insufficient. But in fact, the space in each zone is not completely used up, and only a small amount of data is stored in it. The write pointer of the zone is far from reaching the zone capacity, which seriously wastes space. In addition, the rapid lack of space leads to frequent GC, which seriously reduces the throughput.

[0025] Therefore, the size of the zone is separated. When it exceeds level 2, the data placement process is the same as the above steps, except that the zone is a large zone instead of a small zone. This can reduce the negative impact of frequent GC. Large zones across multiple flash chips can also make full use of the parallelism of the channel and significantly improve performance.

[0026] The specific operation process in this embodiment is as follows: (1) First, place low-level related data in small zones, and high-level related data in large zones across flash memory chips, while retaining the advantages of small and large zones. Figure 2 As shown in the figure, data related to level0, level1 and level2 are placed in the small zone in the figure, and other levels are placed in the large zone across the flash memory chips in the figure.

[0027] (2) When the compaction between Li and Li+1 is triggered, some new SSTables are generated in Li+1. Then, all zones in the Li+2 layer that overlap with the key value ranges of these newly generated SSTables are sorted according to their remaining space.

[0028] It should be noted that the data of level 0, level 1 and level 2 are placed in small zones.

[0029] (3) Select the zone with the smallest remaining space from the sorted zones and write these new SSTables. If there is insufficient space or there are no overlapping zones, select a new zone to write.

[0030] (4) When the compaction between Li+1 and Li+2 is triggered next time, the victim SSTable of Li+1 and the overlapping SSTable in Li+2 are located in the same zone, so they will be invalidated almost at the same time and become invalid data, significantly reducing the amount of valid data that needs to be migrated. In addition, for low-level levels, since the assigned zone is a small zone, the amount of valid data to be migrated can be further reduced.

[0031] like Figure 3 As shown in the figure, after the compaction operation, invalid data is concentrated in zone0. During garbage collection, only a small amount of valid data in zone0 needs to be migrated, and then the entire zone0 is directly reset. This significantly reduces the data migration overhead of GC and reduces the double write amplification effect caused by compaction and garbage collection.

[0032] Therefore, the present invention adopts the above-mentioned partitioned namespace solid-state disk system architecture based on dynamic areas, which can significantly reduce effective data migration, make full use of the free space of the zone, extend the life of the ZNS SSD, effectively alleviate the write amplification problem of the ZNS platform, and improve the overall performance of the system.

[0033] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A partition namespace solid state disk system architecture based on dynamic regions, characterized in that: The storage size of small and large zones and the hierarchical boundaries of data placement are pre-set. Compaction is performed on the SSD to merge and compress the data and then generate SSTables. The overlapping SSTables in the compaction are placed in the same zone, and data of different levels are placed in zones of different sizes.

2. The dynamic zone-based partition namespace solid state disk system architecture according to claim 1, characterized in that: The specific steps are as follows: first, place all low-level related data into small zones, and place all high-level related data into large zones across flash memory chips, while retaining the advantages of small and large zones; When compaction between Li and Li+1 is triggered, some new SSTables are generated in Li+1, and all zones in the Li+2 layer that overlap with the newly generated SSTable key value range are sorted according to the remaining space; Select the zone with the smallest remaining space from the sorted zones and write it into the newly generated SSTable. If there is insufficient space or there are no overlapping zones, select a new zone to write into. When the compaction between Li+1 and Li+2 is triggered next time, the victim SSTable of Li+1 and the overlapping SSTable in Li+2 are located in the same zone and become invalid data.

Images