ZNS SSD operation partition resetting and redistribution method based on LSM tree popularity perception
By using a heat-aware method based on LSM trees, data is divided into hot, warm, and cold data, and different partition runtime reset thresholds are set for different hot partitions. Combined with the heat-aware runtime partition reallocation strategy, the problem that a single threshold write pointer in ZNS SSD cannot balance partition space utilization and data migration volume is solved, achieving more efficient partition management and performance improvement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-04-03
AI Technical Summary
In existing ZNS SSDs, during runtime partition reset, a single threshold write pointer cannot balance the space utilization and data migration volume of partitions with different activity levels, resulting in increased zone wear and reduced space utilization.
A heat perception method based on LSM tree is adopted to divide the data into hot, warm and cold data, and different partition runtime reset thresholds are set for different heat partitions. Combined with the heat perception runtime partition redistribution strategy, the timing of partition reset and redistribution is dynamically adjusted to optimize partition space utilization.
By taking into account the differences in the lifespan of data with varying degrees of popularity, partition reset and reallocation are optimized, reducing the number of partition erases, improving space utilization, preventing cold data from occupying unused space for extended periods, and enhancing the overall performance of ZNS SSDs.
Smart Images
Figure CN121785533A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer storage technology, and in particular relates to a method for resetting and reallocating ZNS SSD runtime partitions based on LSM tree heat awareness. Background Technology
[0002] The current surge in demand for high-capacity storage poses a significant challenge to traditional block interface SSDs, primarily in terms of internal DRAM, operational processing (OP), and garbage collection (GC). To address these challenges, the emerging ZNS SSD, evolved from open-channel SSDs, has emerged, offering a highly efficient solution to the technical difficulties of traditional SSDs. By adopting ZNS SSDs, not only are storage system costs significantly reduced, but data writing and retrieval efficiency are also greatly improved, ensuring stable performance under high load conditions and providing robust support for data centers and enterprise applications.
[0003] ZNS SSD divides the storage space into multiple Zones (partitions). Each Zone can be managed independently and has a fixed-size logical address space (e.g., 512MB). Data can be read randomly, but it must be written sequentially. Before rewriting data to a Zone, it must be erased, i.e., the Zone must be reset.
[0004] ZenFS is a file system plugin that allows RocksDB to place files in zones on raw partition block devices. When ZenFS performs a zone reset, it checks all I / O zones, identifying zones where data is invalid up to the write pointer (WP). If such a zone exists, ZenFS performs a zone reset operation on it. This reset method, occurring in zones that haven't undergone garbage collection, is called runtime zone reset. While runtime zone reset can promptly release zone resources, reducing data migration, it can lead to excessive zone reset calls, increasing zone wear and tear and severely reducing zone space utilization.
[0005] To improve space utilization during zone resets, Sungjin Byeon et al. proposed the FAR algorithm (AFree-Space Adaptive Runtime Zone-Reset Algorithm for Enhanced ZNSEfficiency). The FAR algorithm provides a globally dynamically adjusted threshold write pointer (Twp) for the ZNS SSD based on the remaining available space, thereby improving zone space utilization and reducing the number of zone erases. However, because FAR ignores the different data expiration times at different activity levels, a single threshold write pointer (Twp) cannot meet the reset requirements of zones with varying activity levels. In other words, a single threshold write pointer (Twp) cannot balance partition space utilization and data migration during runtime partition resets. Summary of the Invention
[0006] To address the problems existing in the background technology, this invention provides a method for runtime partition reset and reallocation of ZNS SSDs based on LSM tree heat awareness, including:
[0007] S1: For LSM tree-based key-value databases, a hierarchical data popularity classification method is proposed, which divides data popularity into hot data, warm data and cold data, writes data of different popularity into different partitions of ZNSSSD, and assigns popularity labels to partitions;
[0008] S2: Based on the heat label of the partition, a heat-aware runtime partition reset method is proposed. Different partition runtime reset thresholds are set for different heat partitions to allow partitions storing cold data to be reset as soon as possible, so as to avoid cold data occupying unused space for a long time. At the same time, partitions storing hot data are allowed to write as much data as possible to improve the space utilization of partitions storing hot data.
[0009] S3: For invalid data partitions with low space utilization for storing low-hot data, a hotness-aware runtime partition reallocation method is proposed to make full use of the unused space of low-hot partitions and improve the space utilization of ZNS SSD.
[0010] The present invention has at least the following beneficial effects
[0011] This paper proposes a method to optimize the runtime partition reset and allocation strategy of ZNS SSD by comprehensively considering the differences in the popularity of different data types, thereby significantly improving the overall performance of ZNS SSD. By leveraging the varying lifespans of data with different popularity levels, the timing of runtime resets and partition allocation methods for partitions storing data of varying popularity are optimized, thus improving partition space utilization and the number of partition erases. The runtime partition reset and reallocation method of this invention addresses the issue that a single threshold write pointer cannot meet the reset requirements of partitions with different popularity levels; that is, a single threshold write pointer cannot balance partition space utilization and data migration during runtime partition reset, achieving the goal of reducing the number of partition erases without increasing the amount of data migration. In summary, the runtime partition reset and reallocation method based on data popularity proposed in this invention optimizes the runtime partition reset of ZNS SSD by considering the differences in the lifespans of data with different popularity levels. Attached Figure Description
[0012] Figure 1 This is a schematic diagram of the process of the present invention;
[0013] Figure 2 This is a diagram of the overall architecture of the present invention;
[0014] Figure 3 This is a flowchart of the LSM tree-based heatsense-based ZNS SSD runtime partition reset process of the present invention;
[0015] Figure 4 This is an example diagram of the LSM tree-based heatsense-based ZNS SSD runtime partition resetting according to the present invention.
[0016] Figure 5 This is an overall flowchart of the LSM tree-based heatsense-based ZNS SSD runtime partition reallocation based on the present invention. Detailed Implementation
[0017] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0018] like Figure 1 This invention provides a method for resetting and reallocating ZNS SSD runtime partitions based on LSM tree heat awareness, comprising:
[0019] S1: For LSM-based key-value databases, a hierarchical data popularity classification method is proposed, which divides data popularity into hot data, warm data, and cold data, and writes data of different popularity into ZNS.
[0020] Different partitions of the SSD, and assign hotness tags to the partitions;
[0021] S2: Based on the heat label of the partition, a heat-aware runtime partition reset method is proposed. Different partition runtime reset thresholds are set for different heat partitions to allow partitions storing cold data to be reset as soon as possible, so as to avoid cold data occupying unused space for a long time. At the same time, partitions storing hot data are allowed to write as much data as possible to improve the space utilization of partitions storing hot data.
[0022] S3: For invalid data partitions with low space utilization that store low-popularity data, a runtime partition reallocation method based on popularity is proposed to fully utilize the unused space of low-popularity partitions and improve efficiency.
[0023] ZNS SSD space utilization;
[0024] To better categorize data with different levels of popularity, this invention proposes a data popularity classification and data placement strategy that combines LSM tree management, specifically including the following steps:
[0025] S11: According to the layer-by-layer compression rules of LSM tree-based key-value databases, since the data in layer 0 is unsorted, it usually overlaps. During compression, all data in layer 0 will be selected. Since all data in layer 1 usually overlaps with the data in layer 0, the data in layer 0 and layer 1 usually participate in compression together. Their compression frequency is relatively high and consistent. The higher the level of data, the lower the compression frequency and the longer the data life. Therefore, the data in layer 0 and layer 1 are divided into hot data, the data in layer 2 are divided into warm data, and the data in layer 3 and above are divided into cold data.
[0026] S12: When data is written to a partition, determine whether there is a heat label that matches the heat of the data in the partition where the data has been written;
[0027] S13: If there are consistent popularity labels, write the data to the partition corresponding to the label, so that data with the same popularity are written to the same partition, thereby making the data invalid at the same time and reducing data write amplification.
[0028] S14: Otherwise, find an empty partition to write to, and use the data's popularity as the partition's popularity label, i.e., hot data, warm data, cold data.
[0029] After the heat classification and partition allocation strategy are completed, the timing of different partition resets can be dynamically determined based on heat. In step S2, the specific details of the ZNS SSD runtime partition reset based on LSM tree heat awareness are as follows:
[0030] S21: Proposes a novel LSM tree-based heat-aware ZNS SSD runtime reset threshold write pointer (T) method. wp The dynamic adjustment method dynamically adjusts the partition runtime reset threshold write pointer based on characteristics such as partition popularity, ZNS SSD remaining available space percentage, and partition capacity.
[0031] S22: Determine if the write pointer (WP) position of the partition has reached the threshold write pointer (T) position. wp If the condition is not met, return; otherwise, proceed to step S23. The specific judgment process is as follows:
[0032] For hot data partitions: Determine if the partition write pointer WP is greater than or equal to the hot data threshold write pointer Twp. hot If the target is not reached, return; otherwise, proceed to step S24.
[0033] For warm data partitions: when the partition write pointer WP is greater than or equal to the warm data threshold write pointer Twp warm If the target is not reached, return; otherwise, proceed to step S24.
[0034] For cold data partitions: when the partition write pointer WP is greater than or equal to the cold data threshold write pointer Twp cold If the target is not reached, return; otherwise, proceed to step S24.
[0035] S23: Further determine the validity of the partition data. If the amount of valid data in the partition is 0, then reset the partition to release the space occupied by the partition, reset the partition write pointer (WP) to the beginning position and save the partition hot label as empty (none); otherwise, return.
[0036] Based on the defined data heat, allocate a threshold write pointer (T) for runtime partition reset based on the data heat. wp The adjustment algorithm is as follows. In S2, the specific algorithm is executed as follows:
[0037]
[0038] Where i represents the current partition's popularity (hot, warm, or cold); n represents the number of popular partitions; freespace represents the percentage of remaining available space on the ZNS SSD; and zone_capacity represents the partition's capacity. The threshold write pointer is dynamically adjusted at runtime using a formula that resets the threshold write pointer to set different threshold write pointers (T) for partitions with different popularity levels. wp The higher the popularity of a partition, the larger its i value, and thus its threshold write pointer (T) wp The larger the i value, the more data can be written to this type of partition, thus improving the partition space utilization. The smaller the i value, the less popular the partition, thus ensuring that the partition reset is performed as early as possible to release space and prevent cold data from occupying unused space for a long time.
[0039] When there is sufficient remaining available space, for partitions with no valid data and low activity, the partition space utilization can be further improved. This includes the following steps:
[0040] S31: When a write request arrives, determine whether the remaining available space of the ZNS SSD is sufficient. Set the judgment threshold to 20%. If the remaining space is greater than 20%, execute step S32 to perform the reallocation method for invalid data partitions. Otherwise, execute step S1 to perform the original data writing method.
[0041] S32: Find the write pointer T where all stored data is invalid and the write pointer WP is less than the threshold corresponding to the write data popularity label. wp The set of partitions (Z) invalid data );
[0042] S33: In Z invalid data Within the partition set, further search for partition sets (RZ) where the partition popularity is less than the write data popularity. invalid data The specific criteria for selection are as follows:
[0043] If the written data is hot data, the matching partition's heat label is warm or...
[0044] (cold) temperature;
[0045] If the data written is warm data, the matching partition's heat label is cold.
[0046] S34: Determine set RZ invalid data If the value is empty, execute S1 to write the existing data; otherwise, execute RZ. invalid data In the set of partitions, find the partition with the smallest remaining space and reallocate it.
[0047] S35: Modify the hotness label of the reassigned partition to the hotness label of the data, and write the data to the partition;
[0048] The above-mentioned redistribution mechanism allows low-frequency partitions to be reused by high-frequency data before they reach the original reset threshold, avoiding space waste caused by premature partition reset, while reducing the number of partition erases and improving overall space utilization.
[0049] like Figure 2 This invention is mainly divided into two modules: a ZNS SSD runtime partition reset module based on LSM tree heat perception and a partition reallocation module.
[0050] The LSM tree-based hotspot-aware ZNS SSD runtime partition reset module dynamically sets different threshold write pointers for partitions based on their hotspot labels, the percentage of remaining available space in the ZNS SSD, and the partition size, thereby determining when to reset the partition. The LSM tree-based LNS SSD runtime partition reallocation module uses partition hotspot labels, threshold write pointers, and the percentage of remaining available space to determine whether to reallocate the partition.
[0051] To implement runtime partition reset of ZNS SSDs based on LSM tree heat awareness, such as Figure 3 The following describes the specific steps for runtime partition reset of ZNS SSD based on LSM tree heat awareness:
[0052] In step 101, the runtime partition reset process begins;
[0053] In step 102, all I / O partitions are traversed sequentially;
[0054] In step 103, determine whether all IO partitions have been traversed. If yes, proceed to step 109; otherwise, proceed to step 104.
[0055] In step 104, determine whether the traversed partition is in a busy state. If yes, skip the partition and proceed to step 101; otherwise, proceed to step 105.
[0056] In step 105, determine whether the traversed partition contains data and whether the data is invalid. If yes, proceed to step 106; otherwise, skip the partition and proceed to step 101.
[0057] In step 106, calculate the threshold write pointer corresponding to the heat of the corresponding partition;
[0058] In step 107, the partition write pointer is compared with the threshold write pointer size calculated in step 106. If the write pointer is less than the threshold write pointer, then skip the partition and proceed to step 101; otherwise, proceed to step 108.
[0059] In step 108, reset the traversed partitions, and then return to step 103 after completion;
[0060] In step 109, the process ends.
[0061] An example diagram of runtime partition reset for ZNS SSD based on LSM tree heat awareness is shown below. Figure 4 As shown in the diagram, the data in Zones 0 and 1 is completely invalid. However, because the write pointer WP of Zone 0 is less than its threshold write pointer Twp_hot, a runtime partition reset cannot be performed. Meanwhile, the write pointer WP of Zone 1 is greater than its threshold write pointer Twp_warm, so Zone 1 meets the conditions for a runtime partition reset, and a reset is performed on Zone 1. The write pointer WP of Zone 2 exceeds its threshold write pointer Twp_cold, but because its internal data is not invalid, the conditions for a runtime partition reset are not met.
[0062] To enable the reallocation of partitions with lower activity levels, this invention proposes a runtime partition reallocation strategy for ZNS SSDs based on LSM tree activity awareness. The execution process is as follows: Figure 5 As shown, it specifically includes:
[0063] Step 201: Begin executing the partition reallocation process;
[0064] Step 202: Determine if there is sufficient remaining available space. If so, proceed to step 203; otherwise, proceed to step 211.
[0065] Step 203: Initialize the partition variables that need to be returned;
[0066] Step 204: Traverse each I / O partition;
[0067] Step 205: Determine whether all IO partitions have been traversed. If so, proceed to step 211; otherwise, execute step 206.
[0068] Step 206: Determine if the currently traversed partition is in a busy state. If yes, proceed to step 204 to continue traversing the next IO partition; otherwise, execute step 207.
[0069] Step 207: Determine whether the partition is not empty, not full, and all data inside the partition is invalid. If yes, proceed to step 208; otherwise, go to step 204.
[0070] Step 208: Calculate the value of the threshold write pointer Twp_i corresponding to the SSTable's popularity based on the popularity i of the written SSTable, and then execute step 209.
[0071] Step 209: Determine whether the write pointer WP of the partition is greater than or equal to the value of the threshold write pointer Twp_i corresponding to the SSTable's heat level. If it is, it means that the partition meets the reset condition under the heat level corresponding to this SSTable, so skip the partition and go to step 204; otherwise, execute step 210.
[0072] Step 210: Add the partition to the out_zone_set set, then proceed to step 204;
[0073] Step 211: Determine whether the reallocated partition set out_zone_set is empty. If it is, proceed to step 213; otherwise, execute step 212.
[0074] Step 212: Select the partition with the smallest remaining space from the out_zone_set as the out_zone;
[0075] Step 213: Process ends.
[0076] In summary, this invention, by comprehensively considering the differences in data lifespan at different levels of the LSM tree, divides data into different levels of "hotness" and optimizes the runtime partition reset strategy of ZNS SSD accordingly, significantly improving the overall performance of ZNS SSD. By utilizing partition hotness, remaining available space, and partition capacity, threshold write pointers are dynamically set for different partitions to balance partition space utilization and data migration overhead. Furthermore, to further improve partition space utilization and reduce partition erase frequency, a runtime partition reallocation scheme is proposed. When remaining available space is sufficient, partitions with low hotness and no valid data are reallocated to areas with higher hotness for data writing, successfully reducing partition erase frequency and improving partition space utilization without increasing data migration. In conclusion, the runtime partition reset and reallocation method based on data hotness proposed in this invention optimizes the runtime partition reset of ZNS SSD.
[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for runtime partition reset and reallocation of ZNS SSDs based on LSM tree heat awareness, characterized in that, Includes the following steps: S1: For LSM-based key-value databases, a hierarchical data popularity classification method is proposed, which divides data popularity into hot data, warm data, and cold data, and writes data of different popularity into ZNS. Different partitions of the SSD, and assign hotness tags to the partitions; S2: Based on the heat label of the partition, a heat-aware runtime partition reset method is proposed. Different partition runtime reset thresholds are set for different heat partitions to allow partitions storing cold data to be reset as soon as possible, so as to avoid cold data occupying unused space for a long time. At the same time, partitions storing hot data are allowed to write as much data as possible to improve the space utilization of partitions storing hot data. S3: For invalid data partitions with low space utilization for storing low-hot data, a hotness-aware runtime partition reallocation method is proposed to make full use of the unused space of low-hot partitions and improve the space utilization of ZNS SSD.
2. The method for resetting and reallocating ZNS SSD runtime partitions based on LSM tree heat awareness as described in claim 1, wherein the heat label feature of step S1 is that when the partition is empty, the heat of the data first written to the partition is used as the heat label of the partition to identify the core data heat attribute of the partition.
3. The method for runtime partition reset and reallocation of ZNS SSD based on LSM tree heat awareness according to claim 1, characterized in that, Step S1 specifically includes: S11: According to the layer-by-layer compression rules of LSM tree-based key-value databases, since the data in layer 0 is unsorted, it usually overlaps. During compression, all data in layer 0 will be selected. Since all data in layer 1 usually overlaps with the data in layer 0, the data in layer 0 and layer 1 usually participate in compression together. Their compression frequency is relatively high and consistent. The higher the level of data, the lower the compression frequency and the longer the data life. Therefore, the data in layer 0 and layer 1 are divided into hot data, the data in layer 2 are divided into warm data, and the data in layer 3 and above are divided into cold data. S12: When data is written to a partition, determine whether there is a heat label in the partition where the data has been written that matches the data heat. S13: If there are consistent popularity labels, write the data to the partition corresponding to the label, so that data with the same popularity are written to the same partition, thereby making the data invalid at the same time and reducing data write amplification. S14: Otherwise, find an empty partition to write to, and use the data's popularity as the partition's popularity label, i.e., hot data, warm data, cold data.
4. The method for runtime partition reset and reallocation of ZNS SSD based on LSM tree heat awareness according to claim 1, characterized in that, The heat-aware runtime partition reset method proposed in step S2 includes the following steps: S21: Proposes a novel runtime reset threshold write pointer (T) based on LSM tree heat awareness. wp The dynamic adjustment method dynamically adjusts the partition runtime reset threshold write pointer based on characteristics such as partition popularity, ZNS SSD remaining available space percentage, and partition capacity. S22: Determine if the write pointer (WP) position of the partition has reached the threshold write pointer (T) position. wp If the condition is not met, return; otherwise, proceed to step S23. S23: Further determine the validity of the partition data. If the amount of valid data in the partition is 0, then reset the partition to release the space occupied by the partition, reset the partition write pointer (WP) to the beginning position and save the partition hot label as empty (none), otherwise return.
5. The method for runtime partition reset and reallocation of ZNS SSD based on LSM tree heat awareness according to claim 1, characterized in that, Step S2 proposes a new runtime reset threshold write pointer (T) wp The dynamic adjustment method, based on characteristics such as partition popularity, partition capacity, and the percentage of remaining available space on the ZNS SSD, constructs a runtime reset threshold write pointer (T) mechanism. wp The formula for dynamic adjustment is as follows: Where i represents the current partition's popularity (hot, warm, or cold); n represents the number of popular partitions; freespace represents the percentage of ZNSSSD's remaining available space; and zone_capacity represents the partition's capacity. The threshold write pointer is dynamically adjusted at runtime using a formula that sets different threshold write pointers (T) for partitions with different popularity levels. wp The higher the popularity of a partition, the larger its i value, and thus its threshold write pointer (T) wp The larger the i value, the more data can be written to this type of partition, thus improving the partition space utilization. The smaller the i value, the less popular the partition, thus ensuring that the partition reset is performed as early as possible to release space and prevent cold data from occupying unused space for a long time.
6. The method for runtime partition reset and partition reallocation of ZNS SSD based on LSM tree heat awareness according to claim 1, characterized in that, Step S3 includes: S31: When a write request arrives, determine whether the remaining available space of the ZNS SSD is sufficient. Set the judgment threshold to 20% (ZenFS GC threshold). If the remaining space is greater than 20%, execute step S32 to perform the reallocation method for invalid data partitions. Otherwise, execute step S1 to perform the original data writing method. S32: Find the write pointer T where all stored data is invalid and the write pointer WP is less than the threshold corresponding to the write data popularity label. wp The set of partitions (Z) invalid data ); S33: In Z invalid data Within the partition set, further search for partition sets (RZ) where the partition popularity is less than the write data popularity. invalid data ); S34: Determine set RZ invalid data If the value is empty, execute S1 to write the existing data; otherwise, execute RZ. invalid data In the set of partitions, find the partition with the smallest remaining space and reallocate it. S35: Modify the hotness label of the reassigned partition to the hotness label of the data, and write the data to the partition.