A garbage collection method for ZNS-SSD storage system

Abstract

The present invention belongs to the technical field of computer storage, and in particular relates to a garbage collection method for a ZNS-SSD storage system, comprising: adding a Zone_MD command dedicated to effective data migration between partitions during garbage collection on a host side in a ZNS-SSD; when the host needs to migrate effective data, sending a command to a device side, and the device side executes effective data migration internally after receiving the command; the present invention reduces the transmission overhead and data copy overhead of effective data migration during garbage collection respectively by adding a Zone_MD command and a block-level remapping mechanism, thereby significantly reducing the execution time of garbage collection in a ZNS-SSD storage system and effectively improving the I / O performance and life of a ZNS-SSD device.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer storage, and in particular relates to a garbage collection method for a ZNS-SSD storage system. Background Art

[0002] With the continuous upgrading of storage media and the increasing demand of users for storage system performance, the cost of maintaining traditional block interfaces for flash-based solid-state drives (SSDs) (such as garbage collection overhead on the device side, DRAM overhead required for the mapping table of storage address conversion, etc.) is also increasing. These overheads hinder the further improvement of the performance of current SSD storage systems. Therefore, the ZNS (Zoned Namespace) interface is proposed as an emerging NVMe storage interface protocol to avoid the operating costs of traditional block interfaces. The ZNS interface divides the logical address space into multiple fixed-size partitions (Zones), and the partitions can only be written sequentially and rewritten after reset. Among them, sequential writing and rewriting after reset make data writing more in line with the characteristics of flash storage media, thereby avoiding unnecessary conversion overhead. Partitions enable the host to reduce the write amplification caused by garbage collection through reasonable partition data placement. SSD devices that use the ZNS interface to interact with the host are called ZNS-SSDs.

[0003] Based on the current write mode, ZNS-SSD eliminates garbage collection on the device side to improve I / O performance. However, in order to improve the space utilization of ZNS-SSD, applications using ZNS-SSD need to perform partition recycling operations. When the selected partition still contains valid data, the valid data must be migrated to another partition before recycling the partition (host-side garbage collection), and migrating valid data will inevitably lead to write amplification and reduced I / O performance. In the prior art, in order to reduce the negative impact of host-side garbage collection, most technologies focus on proposing more reasonable data placement solutions and better garbage collection algorithms. However, these technologies all ignore the inefficient and high-latency effective data migration methods during host-side garbage collection in ZNS-SSD storage systems. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a garbage collection method for a ZNS-SSD storage system, comprising:

[0005] S1: When a partition needs to be reclaimed, the host uses the Greedy algorithm or the Cost-benefit algorithm to determine the victim partition to be reclaimed and the destination partition for valid data migration;

[0006] S2: The host determines all valid data of the victim partition according to the metadata, and divides the valid data of the victim partition into multiple data blocks according to whether all valid data of the victim partition are continuous addresses;

[0007] S3: The host determines the destination address of data migration in the current partition according to the write pointer in the destination partition;

[0008] S4: using the first address of the data block as the source address of the data migration, the write pointer in the target partition as the destination address of the data migration, and the size of the data block as the size of the data migration as the data migration information;

[0009] S5: The host sends the data migration information to the device through the newly added Zone_MD command;

[0010] S6: After receiving the Zone_MD command, the device side parses the address information carried in the Zone_MD command to determine the victim partition and the destination partition of the data migration, and determines the source physical space and the destination physical space corresponding to the data being migrated according to the partition mapping relationship;

[0011] S7: Migrate all valid data blocks in the victim partition according to the source physical space state and the destination physical space state of the migrated data;

[0012] S8: After all valid data is migrated, the host sends a reset command for the victim partition. After receiving the reset command for the partition, the device erases all mapped physical blocks according to its partition mapping relationship.

[0013] Beneficial effects of the present invention:

[0014] The present invention improves the efficiency of garbage collection by providing a new ZNS command Zone_MD to avoid unnecessary transmission overhead during effective data migration; avoids most data copies during effective data migration through a remapping mechanism, thereby minimizing the overhead such as I / O performance degradation and life reduction caused by data copying; the present invention can significantly reduce the execution time of garbage collection in the ZNS-SSD storage system, while effectively improving the I / O performance and life of the ZNS-SSD. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the garbage collection method of the present invention;

[0016] Figure 2 It is a structural diagram of the ZNS-SSD storage system of the present invention;

[0017] Figure 3 A comparison diagram of the data migration method of the present invention and the traditional data migration method;

[0018] Figure 4is a schematic diagram of the remapping mechanism of the present invention;

[0019] Figure 5 It is an overall flow chart of the garbage collection method of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] A garbage collection method for ZNS-SSD storage system, such as Figure 1 As shown, including:

[0022] When a partition needs to be reclaimed, the host side (application or file system) can use any optimal garbage collection algorithm to determine the target partition (victim partition) for this recycling and the destination partition (destination partition) for effective data migration.

[0023] In this embodiment, the Greedy algorithm or the Cost-benefit algorithm is used to determine the victim partition to be recycled and the destination partition for valid data migration. For example, the Greedy algorithm can be used to greedily select the partition with the least valid data among all used partitions, or the Cost-benefit algorithm can be used. The selection of the destination partition is usually to select an idle partition and a partition with sufficient free space that is close to the life cycle of the valid data in the victim partition.

[0024] Furthermore, the host determines all valid data of the victim partition based on the metadata, and divides them into multiple data blocks based on whether they are continuous addresses. The valid data in each data block is continuous, that is, the data block can be described using the first address of each data block and the size of the data it contains.

[0025] Furthermore, the host determines the current write logic address of the current partition (the destination address of data migration) according to the write pointer in the destination partition.

[0026] Furthermore, data migration is performed in units of data blocks, that is, the first address of the data block is the source address of the data migration, the write pointer in the destination partition is the destination address of the data migration, and the size of the data block is the size of the data migration.

[0027] Furthermore, the host sends the information of the data migration (source address, destination address and size) to the device through the newly added ZNS command Zone_MD (Src, Dst, Size).

[0028] Furthermore, after receiving the Zone_MD command, the device parses the address information carried therein to determine the victim partition and the destination partition of the data migration, and further determines the source physical space and the destination physical space corresponding to the migrated data according to the partition mapping relationship.

[0029] Furthermore, there are four situations for the source physical space status and the destination physical space status of the migrated data:

[0030] ①The source physical space contains only valid data;

[0031] ② The source physical space contains both valid data and invalid data;

[0032] ③ All spaces in the destination physical space have been written with data;

[0033] ④There is still some space in the destination physical space where data has not been written.

[0034] Furthermore, based on these four situations, the present invention adopts four different data migration mechanisms, namely:

[0035] S1. When the source physical space is in state ① and the destination physical space is in state ③. At this time, the physical blocks in the source physical space are directly and sequentially remapped to the destination physical space in the destination partition, and the mapping relationship is saved in the mapping table in the destination partition. In addition, the write pointer of the destination partition is updated to the new address after adding the size of the source physical space.

[0036] S2. When the source physical space is in state ① and the destination physical space is in state ④. Similar to S1, the physical blocks in the source physical space are directly and sequentially remapped to the destination partition, and the mapping table and write pointer of the destination partition are updated. Furthermore, in order to utilize the part of the destination physical space that has not yet been written to the data, the data that continues to be written in the destination partition after remapping will be preferentially written to this part of the space until it is full. However, at this time, the physical address and the logical address are not one-to-one corresponding. Therefore, the present invention adds a remapping table for each partition, which is used to record the offset of the physical address and the logical address in the partition, and when recording, the addresses with the same offset and continuous are recorded as one member to save the space overhead of the table.

[0037] S3. When the source physical space is in state ② and the destination physical space is in state ③. The mapping of the source physical space and the updating of the mapping table and write pointer in the destination partition are similar to S1. However, since the invalid data and valid data in the source physical space share physical blocks and the physical blocks are used as the basic units for mapping, the invalid data in the source physical space will also be occupied by the destination partition after mapping, and the space for invalid data can only be reclaimed when the destination partition is reset. This is equivalent to remapping at this time to avoid the situation where data migration will cause the destination partition to occupy more space for this part of invalid data for a period of time. Therefore, the present invention strikes a balance between space utilization and performance: only the source physical space where the proportion of valid data is higher than 0.5 is remapped, and data is copied for the source physical space where the proportion is lower than 0.5.

[0038] S4. When the source physical space is in state ② and the destination physical space is in state ④. At this time, the remapping method combines S2 and S3. First, determine whether to remap based on the remapping conditions. If remapping is performed, update the mapping table and write pointer, and the data written after remapping in the destination partition is written to the space where no data was written before. Finally, record all the offsets of the physical address and the logical address that are not aligned in the remapping table.

[0039] The remapping condition includes: whether the ratio of the effective data size to the total size in the source physical space exceeds a set threshold.

[0040] The threshold includes: the ratio of the effective data size to the total size in the source physical space, a fixed value, which is usually 0.5, and can be determined according to your own performance requirements: the smaller the ratio, the better the performance but the more space is occupied; the larger the ratio, the worse the performance but the less space is occupied.

[0041] Furthermore, all valid data blocks in the victim partition are migrated based on the above method.

[0042] Further, after all valid data is migrated, the host issues a reset command for the victim partition to reclaim the partition for rewriting. After receiving the reset command for the partition, the device erases all mapped physical blocks according to its partition mapping relationship.

[0043] The partition mapping relationship includes: mapping the physical blocks to the corresponding partitions to obtain the number of physical blocks contained in the partitions.

[0044] Figure 2This is a schematic diagram of the structure of the ZNS-SSD storage system based on the flash memory medium for the present invention. First, the ZNS interface divides the entire logical address space into multiple fixed-size partitions for partition data placement to reduce the write amplification caused by garbage collection; secondly, the ZNS interface stipulates that each partition can only be written sequentially and rewritten after reset so that data writing is more in line with the hardware characteristics of the flash memory medium; further, the ZNS-SSD eliminates garbage collection on the device side; ideally, the ZNS-SSD can completely avoid effective data migration during garbage collection through perfect partition data placement, that is, the data in the partition always fails at almost the same time. However, in actual situations, such a partition data placement strategy is difficult to implement. Therefore, in order to improve the utilization rate of the ZNS-SSD device, garbage collection on the host side is still needed; like garbage collection on the device side, garbage collection on the host side will also affect the effective throughput, I / O performance and life of the device. Therefore, based on this, the present invention proposes a high-performance host-side garbage collection method to reduce the negative impact caused by garbage collection.

[0045] Embodiment 1:

[0046] The ZNS interface transfers data management, including garbage collection, from the device side to the host side, but does not provide an interface for migrating data between related partitions. Therefore, when performing garbage collection on the host side, the traditional way of migrating valid data is to first read the valid data from the device side to the host's cache, and then write it back from the host's cache to the device side. Obviously, in this process, the valid data undergoes two end-to-end transmissions (device to host, host to device), which will cause corresponding transmission delays. And this is actually unnecessary because the valid data is ultimately on the device side.

[0047] like Figure 3 As shown, the present invention proposes a more efficient data migration method. First, the current ZNS specification is expanded, and a new ZNS command is added on its basis: Zone_MD. The specific format of the Zone_MD command is Zone_MD (Src, Dst, Size), where Src is the starting logical address (source address) of the valid data, Dst is the write pointer (destination address) of the target partition for this data migration, and Size is the length of this valid data. The Zone_MD command is dedicated to data migration between partitions. Secondly, during garbage collection, the command is called to notify the device side to migrate valid data. Finally, after receiving the command, the device side performs data migration internally. Therefore, valid data is only transmitted within the device and does not need to be transmitted to the host side, thereby avoiding unnecessary end-to-end transmission during valid data migration and improving the efficiency of valid data migration.

[0048] Embodiment 2:

[0049] When the device receives the Zone_MD command from the host, it will first parse the source address and destination address carried in the command to determine the victim partition and destination partition of this data migration. Figure 4 As shown, further, the source physical space and the destination physical space corresponding to the data to be migrated are determined according to the partition mapping relationship. Figure 4 (a) shows two possible states of the source physical space and the destination physical space of the migrated data: ① The source physical space contains only valid data; ② The source physical space contains both valid data and invalid data; ③ All the space in the destination physical space has been written with data; ④ Some space in the destination physical space has not been written with data. The combination of two of them will form four basic situations during data migration. Based on these four situations, the present invention adopts four different data migration mechanisms, namely:

[0050] S1. Figure 4 (b) shows the data migration method when the source physical space is in state ① and the destination physical space is in state ③. In this case, the source physical space is directly and sequentially remapped to the destination physical space in the destination partition, and the mapping relationship is saved in the mapping table in the destination partition. In addition, the write pointer of the destination partition is updated to the new address with the size of the source physical space added.

[0051] S2. Figure 4 (c) shows the data migration method when the source physical space is in state ① and the destination physical space is in state ④. Similar to S1, the physical blocks in the source physical space are directly and sequentially remapped to the destination partition, and the mapping table and write pointer of the destination partition are updated. However, at this time, the destination partition still contains some unused space. In order to utilize this part of the space, the data that continues to be written in the destination partition after remapping (referred to as continued writing) will be written to this part of the space first until it is full. As shown in the figure, at this time, its physical address and logical address are not one-to-one corresponding. Therefore, the present invention adds a remapping table for each partition to record the offset of the physical address and logical address in the partition. And when recording, the addresses with the same offset and continuous are recorded as a member (referred to as range) to save the space overhead of the remapping table.

[0052] S3. Figure 4(d) shows the data migration method when the source physical space is in state ② and the destination physical space is in state ③. Similarly, the physical blocks in the source physical space are directly and sequentially remapped to the destination partition, and the mapping table and write pointer of the destination partition are updated. However, since the invalid data and valid data in the source physical space share the physical blocks and the physical blocks are used as the basic unit for mapping, the invalid data in the source physical space will also be occupied by the destination partition after mapping, and the space for the invalid data can only be reclaimed when the destination partition is reset. It is equivalent to remapping at this time to avoid the situation that data migration will cause the destination partition to occupy more space for this part of invalid data for a period of time. Therefore, the present invention strikes a balance between space utilization and performance: only the source physical space with a valid data ratio higher than 0.5 is remapped, and the source physical space with a ratio lower than 0.5 is directly copied.

[0053] S4. When the source physical space is in state ② and the destination physical space is in state ④, the data migration method combines S2 and S3. First, determine whether to perform remapping based on the remapping conditions. If remapping is performed, the mapping table and write pointer are updated after remapping, and the data written after remapping in the destination partition is written to the space where no data was written before. Finally, all the misaligned offsets of the physical address and the logical address are recorded in the remapping table. If remapping is not performed, the data is copied directly.

[0054] Embodiment 3:

[0055] Based on Example 1 and Example 2, the process of garbage collection performed by the ZNS-SSD storage system is as follows: Figure 5 As shown. First, the host can select any excellent garbage collection algorithm to determine the triggering conditions for garbage collection and select the corresponding victim partition. Afterwards, if there is no valid data in the victim partition, it can be reset directly. If the victim partition still contains valid data, it is necessary to first select the destination partition according to the corresponding algorithm, so as to migrate the valid data to the partition with a life cycle close to it as much as possible. Then, the valid data is divided into multiple data blocks according to whether its address is continuous, and data migration is performed in units of data blocks, and only one data block is migrated each time. Furthermore, the host notifies the device side to perform data migration by sending a Zone_MD command. After receiving the command, the device side determines whether to perform remapping or a real data copy based on the proportion of valid data in the source physical space. Finally, after all physical blocks have been migrated, the victim partition can be reset to reclaim the corresponding physical space for rewriting.

[0056] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A garbage collection method for a ZNS-SSD storage system, characterized in that: include: S1: When a partition needs to be reclaimed, the host uses the Greedy algorithm or the Cost-benefit algorithm to determine the victim partition to be reclaimed and the destination partition for valid data migration; S2: The host determines all valid data of the victim partition according to the metadata, and divides the valid data of the victim partition into multiple data blocks according to whether all valid data of the victim partition are continuous addresses; S3: The host determines the destination address of data migration in the current partition according to the write pointer in the destination partition; S4: using the first address of the data block as the source address of the data migration, the write pointer in the target partition as the destination address of the data migration, and the size of the data block as the size of the data migration as the data migration information; S5: The host sends the data migration information to the device through the newly added Zone_MD command; S6: After receiving the Zone_MD command, the device side parses the address information carried in the Zone_MD command to determine the victim partition and the destination partition of the data migration, and determines the source physical space and the destination physical space corresponding to the data being migrated according to the partition mapping relationship; S7: Migrate all valid data blocks in the victim partition according to the source physical space state and the destination physical space state of the migrated data; S8: After all valid data is migrated, the host sends a reset command for the victim partition. After receiving the reset command for the partition, the device erases all mapped physical blocks according to its partition mapping relationship.

2. A garbage collection method for a ZNS-SSD storage system according to claim 1, characterized in that: The partition mapping relationship includes: mapping the physical blocks to the corresponding partitions to obtain the number of physical blocks contained in the partitions.

3. A garbage collection method for a ZNS-SSD storage system according to claim 1, characterized in that: The source and destination physical space status of the migrated data, including: The source physical space contains only valid data; The source physical space contains both valid data and invalid data; All the space in the destination physical space has been written with data; There is still free space in the destination physical space where no data has been written.

4. A garbage collection method for a ZNS-SSD storage system according to claim 1, characterized in that: Migrate all valid data blocks in the victim partition according to the source physical space status and the destination physical space status of the migrated data, including: When the source physical space contains only valid data and all the space in the destination physical space has been written with data, the physical blocks in the source physical space are directly remapped to the destination physical space in the destination partition in order, and the mapping relationship is saved in the mapping table in the destination partition, and the write pointer of the destination partition is updated to the new address with the size of the source physical space added; When the source physical space contains only valid data and there is still free space in the destination physical space where no data has been written, the physical blocks in the source physical space are directly and sequentially remapped to the destination partition, and the mapping table and write pointer of the destination partition are updated. After the remapping, the data that is subsequently written to the destination partition will be preferentially written to the free space where no data has been written until it is full. A new remapping table is created for each destination partition to record the offsets of the physical address and the logical address in the partition, and the addresses with the same offset and continuous are recorded as one member during the recording. When the source physical space contains both valid data and invalid data, and all the space in the destination physical space has been written with data, the valid data in the source physical space whose valid data ratio is higher than the set threshold is remapped to the destination partition, and the valid data in the source physical space whose valid data ratio is lower than the set threshold is copied to the destination partition, and the mapping table and write pointer of the destination partition are updated; When the source physical space contains both valid data and invalid data, and there is still free space in the destination physical space where no data has been written, whether to perform remapping is determined based on the remapping conditions. If remapping is performed, the mapping table and write pointer are updated, and the data written after remapping in the destination partition is written preferentially to the free space where no data has been written, and all misaligned offsets of physical addresses and logical addresses are recorded in the remapping table.

5. A garbage collection method for a ZNS-SSD storage system according to claim 4, characterized in that: The remapping condition includes: whether the ratio of the effective data size to the total size in the source physical space exceeds a set threshold.

6. A garbage collection method for a ZNS-SSD storage system according to claim 4 or 5, characterized in that: The threshold includes: the ratio of the effective data size to the total size in the source physical space.

Images