A data placement method and system suitable for ZNSSSD write amplification optimization

Abstract

The application discloses a data placement method suitable for ZNS SSD write amplification optimization, which obtains a block heat value through a two-stage mode according to a write frequency and a block lifetime, then classifies hot and cold data blocks according to the block heat value, and writes data with the same block heat value into the same partition. In order to adapt to frequent changes of data heat distribution, the method timely updates the heat value of the data block to adjust the category thereof. The basic idea of the application is that the block heat value is obtained through a calculation plus strategy mode according to the write frequency and the block lifetime of the data block, cold and hot data is dynamically identified with low cost, and then the data is classified and placed according to the block heat value, so that the data migration in the garbage collection process is reduced, and the overall write amplification is reduced. The application can solve the technical problem that the existing default placement method does not distinguish between cold and hot data, frequently triggers a garbage collection operation, causes serious write amplification, and further affects the usability and reliability of the system.

Description

Technical Field

[0001] This invention belongs to the field of computer storage technology, and more specifically, relates to a data placement method and system suitable for write amplification optimization of ZNS SSD. Background Technology

[0002] In recent years, with the explosive growth of internet data, more and more users and enterprises are choosing to store their data in the cloud. Among them, cloud block storage systems manage user data at the "block" level. To improve the availability and reliability of the system, when writing user data to a Zoned Namespace Solid State Disk (ZNSSSD), this data needs to be categorized into hot and cold data and placed in partitions to reduce data migration during garbage collection.

[0003] Existing data placement methods mainly include default placement, placement methods based on machine learning classification models, and placement methods based on file semantics. Default placement is very simple, performing no processing on data blocks and writing them sequentially to the hard drive. Placement methods based on machine learning classification models generally use clustering algorithms to classify data blocks and then write data of the same category into the same region. For example, the MLDC method uses the K-means algorithm to cluster data blocks based on four features: logical address, request size, historical write frequency, and age. Then, it uses the CART algorithm to generate a decision tree as a classifier to classify the data blocks. Placement methods based on file semantics classify files by using the file system's attribute information or application context information. For example, the PCStream method classifies data based on the specific execution path followed by the application when writing data.

[0004] However, all of the above data placement methods have some undeniable drawbacks: First, the default placement method does not distinguish between hot and cold data, which frequently triggers garbage collection operations, leading to severe write amplification problems and affecting the availability and reliability of the system; Second, the placement method based on machine learning classification models requires a large amount of system resources to store historical access information, execute classification algorithms, and train classifiers, resulting in excessive overhead; in addition, model training also needs to be carried out offline, which makes it impossible to guarantee the effectiveness of data classification; Third, the placement method based on file semantics requires the upper-level file system to provide the attribute information of the file itself, but block storage systems that manage data at the block level cannot obtain this information. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a data placement method and system suitable for ZNS SSD write amplification optimization. Its purpose is to solve the technical problems of existing default placement methods, which, due to their failure to distinguish between hot and cold data, frequently trigger garbage collection operations, leading to severe write amplification and thus affecting system availability and reliability; existing placement methods based on machine learning classification models, which require significant system resources to store historical access information, execute classification algorithms, and train classifiers, resulting in excessive overhead; the need for offline model training, which compromises the effectiveness of data classification; and existing placement methods based on file semantics, which require upper-layer file systems to provide file attribute information, but which block storage systems managing data at the block level cannot access.

[0006] To achieve the above objectives, according to one aspect of the present invention, a data placement method suitable for write amplification optimization of ZNS SSDs is provided, comprising the following steps:

[0007] (1) Obtain input / output (IO) data in bytes from the test dataset and convert the IO data into N data blocks in blocks;

[0008] (2) Set the counter cnt_uw = 1;

[0009] (3) Determine whether the metadata information corresponding to the cnt_uw-th data block among the N data blocks obtained in step (1) exists in memory. If it does, proceed to step (5); otherwise, proceed to step (4). The metadata information includes the LBA interval corresponding to the cnt_uw-th data block among the N data blocks obtained in step (1) and the number of user writes (freq). uw Garbage collection write count (freq) gw And the value of the global counter gt when the data in the LBA interval corresponding to the data block was last modified. pre .

[0010] (4) Perform a new user write operation on the cnt_uw-th data block among the N data blocks obtained in step (1), add the metadata information of the data block to memory, and set the user write count freq in the metadata information of the data block. uw and garbage collection write count (freq) gw Initialize to 0, and simultaneously set the block lifetime value of this data block to 2. 64 -1;

[0011] (5) Perform a user write operation on the cnt_uw-th data block among the N data blocks obtained in step (1) to obtain the block lifetime value of the data block;

[0012] (6) Retrieve the amount of invalid data from each partition in memory. inv and total data volume zone And based on the amount of invalid data dat inv and total data volume zone Get the percentage of invalid data in this partition (gp), and the percentage of invalid data in all M partitions in memory (gp). all ;

[0013] (7) Determine the percentage of invalid data (gp) of all M partitions obtained in step (6). all If the value is greater than the global garbage collection threshold, proceed to step (8) if yes, otherwise proceed to step (14);

[0014] (8) Use the garbage collection algorithm (Cost-benefit-enhanced, or CBE) to select the partition to be garbage collected from all M partitions in memory;

[0015] (9) Set the counter cnt_gw = 1;

[0016] (10) Determine whether the cnt_gw data block in the partition to be garbage collected in step (8) is valid. If it is, proceed to step (11); otherwise, proceed to step (12).

[0017] (11) Perform a garbage collection write operation on the cnt_gw data block in the partition selected in step (8) to modify some of its metadata information, and then move the data block with modified metadata information to other partitions in memory that are not full.

[0018] (12) Determine whether cnt_gw is equal to the total number of data blocks in the partition selected in step (8) for garbage collection. If yes, proceed to step (13); otherwise, set cnt_gw to cnt_gw+1 and return to step (10).

[0019] (13) Reset the partition selected in step (8) to be garbage collected, delete invalid data in the partition, and return to step (7);

[0020] (14) For the cnt_uw-th data block among the N data blocks obtained in step (1), the garbage collection write count freq of this data block is used as the basis for the calculation. gw and the number of times the user writes (freq) uwThe system retrieves the final heat value of a data block from the partition lifetime queue and the block lifetime value, and writes the data block into the partition corresponding to its final heat value.

[0021] (15) Determine whether cnt_uw is equal to the number of data blocks N after splitting in step (1). If yes, the process ends; otherwise, set cnt_uw to cnt_uw+1 and return to step (3).

[0022] Preferably, step (5) includes the following sub-steps:

[0023] (5-1) Obtain the current global counter value gt in memory and the global counter value gt of the LBA interval corresponding to the N data blocks obtained in step (1) when it was last modified. pre And based on the values ​​gt and gt pre Obtain the Block Lifetime (BILT) value for this data block; the formula used in this step is as follows:

[0024] BILT = gt-gt pre

[0025] (5-2) Extract the user write count freq from the metadata information of the cnt_uw-th data block among the N data blocks obtained in step (1). uw Set to freq uw +1, and simultaneously increment the global counter value gt for the data in the LBA interval corresponding to the data block at the time of the last modification. pre Set the value of the current global counter to gt, and set the value of the current global counter to gt+1.

[0026] Preferably, step (6) uses the following formula:

[0027]

[0028]

[0029] Preferably, step (8) includes the following steps:

[0030] (8-1) Set the counter cnt_zone = 1;

[0031] (8-2) Determine whether the cnt_zone-th partition among all M partitions in memory has been filled. If so, proceed to step (8-3); otherwise, proceed to step (8-4).

[0032] (8-3) Retrieve the global counter value gt from memory when the cnt_zone-th partition among all M partitions is full. fulledCombined with the invalid data percentage gp of the partition obtained in step (6) and the current global counter value gt, the partition lifetime value t of the partition is obtained. zone And the profit value cb, then proceed to step (8-4);

[0033] (8-4) Determine if cnt_zone is equal to M. If yes, proceed to step (8-5). Otherwise, set cnt_zone to cnt_zone+1 and return to step (8-2).

[0034] (8-5) Sort the profit values ​​of all M partitions in memory obtained in step (8-3) in descending order, and take the partition with the largest profit value as the partition to be garbage collected.

[0035] (8-6) The partition lifetime t of the partition with the highest profit value obtained in step (8-5) is calculated. zone Add it to the partition lifetime queue of the corresponding type in memory;

[0036] Preferably, step (8-3) uses the following formula:

[0037] t zone =gt-gt fulled

[0038]

[0039] Preferably, the partition lifetime queue is a first-in-first-out (FIFO) queue, and the number of partition lifetime queues is equal to the number of partition types in memory, L, corresponding to the maximum number of partitions that the ZNS SSD can open simultaneously. Preferably, step (11) includes the following steps:

[0040] (11-1) Obtain the current global counter value gt in memory and the global counter value gt of the data in the LBA interval corresponding to the cnt_gw-th data block in the partition selected in step (8) for garbage collection when it was last modified. pre And obtain the block lifetime value of the data block;

[0041] (11-2) Determine whether the block lifetime value obtained in step (11-1) exceeds the refresh threshold R1. If it does, proceed to step (11-3); otherwise, proceed to step (11-6).

[0042] (11-3) Take the user write count freq from the metadata information of the cnt_gw-th data block in the partition selected in step (8) for garbage collection. ue Setting it to 0 sets the garbage collection write count (freq). gw Set to 1;

[0043] (11-4) Determine whether the block lifetime value obtained in step (11-1) exceeds the extreme cold data threshold R2. If it does, proceed to step (11-5); otherwise, proceed to step (11-7).

[0044] (11-5) Calculate the number of garbage collection writes (freq) of the cnt_gw-th data block in the partition selected in step (8) for garbage collection. gw Set it to L–C, then proceed to step (11-7);

[0045] (11-6) Extract the garbage collection write count freq from the metadata information of the cnt_gw-th data block in the partition selected in step (8) for garbage collection. gw Set to freq gw +1, then proceed to step (11-7).

[0046] (11-7) Retrieve from memory the garbage collection write count freq of the cnt_gw-th data block in the partition selected for garbage collection in step (8). gw and the number of times the user writes (freq) uw And based on the number of times garbage collection is written (freq) gw and the number of times the user writes (freq) uw Get the initial heat value bh of this data block. pre ;

[0047] (11-8) Retrieve the partition lifetime queue obtained in step (8-6) from memory, and use the initial heat value bh obtained in step (11-7) in the partition lifetime queue. pre The lifetime values ​​t of all i partitions in the queue to which the corresponding type of partition belongs. zone The decision threshold l is obtained;

[0048] (11-9) Using the initial heat value bh obtained in step (11-7) pre The final heat value of the cnt_gw data block in the partition to be garbage collected in step (8) is obtained by taking the block lifetime value BILT obtained in step (11-1) and the judgment threshold l obtained in step (11-8).

[0049] (11-10) Write the cnt_gw data block in the partition selected in step (8) to be garbage collected into the partition corresponding to the final heat value obtained in step (11-9).

[0050] Preferably, the formula for calculating the initial heat value is as follows:

[0051] bh pre =freq gw-freq uw +C

[0052] The formula for calculating the threshold is as follows:

[0053]

[0054] The formula for calculating the final heat value is as follows:

[0055]

[0056] According to another aspect of the present invention, a data placement system suitable for write amplification optimization of ZNS SSDs is provided, comprising:

[0057] The first module is used to obtain input / output (IO) data in bytes from the test dataset and convert the IO data into N data blocks in blocks.

[0058] The second module is used to set the counter cnt_uw = 1;

[0059] The third module determines whether the metadata information corresponding to the cnt_uw-th data block among the N data blocks obtained from the first module exists in memory. If it does, the process proceeds to the fifth module; otherwise, it proceeds to the fourth module. The metadata information includes the LBA interval corresponding to the cnt_uw-th data block among the N data blocks obtained from the first module, and the number of user writes (freq). uw Garbage collection write count (freq) gw And the value of the global counter gt when the data in the LBA interval corresponding to the data block was last modified. pre .

[0060] The fourth module performs a new user write operation on the cnt_uw-th data block out of the N data blocks obtained by the first module. It adds metadata information for this data block to memory and sets the user write count (freq) in the metadata information of this data block. uw and garbage collection write count (freq) gw Initialize to 0, and simultaneously set the block lifetime value of this data block to 2. 64 -1;

[0061] The fifth module is used to perform a user write operation on the cnt_uw-th data block among the N data blocks obtained by the first module, so as to obtain the block lifetime value of the data block;

[0062] The sixth module is used to retrieve the amount of invalid data from each partition in memory. inv and total data volume zone And based on the amount of invalid data invand total data volume zone Get the percentage of invalid data in this partition (gp), and the percentage of invalid data in all M partitions in memory (gp). all ;

[0063] The seventh module is used to determine the percentage of invalid data (gp) across all M partitions obtained from the sixth module. all If the value is greater than the global garbage collection threshold, proceed to module 8; otherwise, proceed to module 14.

[0064] The eighth module is used to select the partition to be garbage collected from all M partitions in memory using the Cost-benefit-enhanced (CBE) garbage collection algorithm.

[0065] The ninth module is used to set the counter cnt_gw = 1;

[0066] The tenth module is used to determine whether the cnt_gw-th data block in the partition selected by the eighth module for garbage collection is valid. If it is valid, proceed to the eleventh module; otherwise, proceed to the twelfth module.

[0067] The eleventh module is used to perform a garbage collection write operation on the cnt_gw-th data block in the partition selected by the eighth module for garbage collection, so as to modify some of its metadata information, and then migrate the data block with modified metadata information to other unfilled partitions in memory.

[0068] The twelfth module is used to determine whether cnt_gw is equal to the total number of data blocks in the partition selected by the eighth module for garbage collection. If so, proceed to the thirteenth module; otherwise, set cnt_gw to cnt_gw+1 and return to the tenth module.

[0069] The thirteenth module is used to reset the partition selected by the eighth module for garbage collection, delete invalid data in the partition, and return to the seventh module.

[0070] The fourteenth module, for the cnt_uw-th data block out of the N data blocks obtained in the first module, calculates the garbage collection write count freq for that data block. gw and the number of times the user writes (freq) uw The system retrieves the final heat value of a data block from the partition lifetime queue and the block lifetime value, and writes the data block into the partition corresponding to its final heat value.

[0071] The fifteenth module is used to determine whether cnt_uw is equal to the number of data blocks N after the first module. If so, the process ends; otherwise, cnt_uw is set to cnt_uw+1 and the process returns to the third module.

[0072] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0073] (1) The present invention can solve the serious write amplification problem caused by the default placement method: Since the present invention adopts steps (11) and (14), data is classified according to the data block heat value, which can effectively distinguish between hot and cold data. After partitioning and placing them, the amount of data migration in the garbage collection process can be significantly reduced, and the write amplification factor can be reduced.

[0074] (2) The present invention can solve the problem that the effectiveness of data classification cannot be guaranteed in the existing methods: because the present invention uses steps (5) and (11) to modify the metadata information of the data block during the writing process, the hotness value of the data block is dynamically adjusted online, thus ensuring the effectiveness of hot and cold data classification.

[0075] (3) The present invention can solve the problem of excessive overhead in the existing methods: Since the present invention adopts steps (11-7) to (11-9), it only uses the number of writes and the block lifetime value to obtain the data block heat value by calculating the strategy. It does not require a lot of additional historical information and classification model training. Compared with machine learning classification models, it greatly reduces storage and computing overhead while ensuring classification accuracy.

[0076] (4) The present invention can avoid the negative benefit phenomenon in the garbage collection process: Since the present invention uses the CBE algorithm in step (8) to select the partition to be collected, it skips the partitions with invalid data ratio less than the global garbage collection threshold during the selection process, which can achieve better garbage collection effect compared with the traditional cost-benefit algorithm.

[0077] (5) In simulation experiments, the write amplification factor of the data placement algorithm designed in this invention was reduced by an average of 34.8% compared with the default placement method and by an average of 19.0% compared with the latest placement method. Attached Figure Description

[0078] Figure 1 This is a flowchart of the data placement method of the present invention applicable to ZNS SSD write amplification optimization. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0080] This invention employs a data placement method suitable for write amplification optimization of ZNS SSDs. Based on the number of writes and block lifetime, it obtains block popularity values ​​in a two-stage process, then categorizes data blocks into hot and cold categories according to these values, writing data with the same popularity value to the same partition. To adapt to frequent changes in data popularity distribution, this method updates the popularity values ​​of data blocks in a timely manner to adjust their category.

[0081] The basic idea of ​​this invention is to obtain the block heat value by calculation and strategy based on the number of writes and block life of the data block, dynamically identify hot and cold data with low overhead, and then classify and place the data according to the block heat value, thereby reducing data migration during the garbage collection process and reducing the overall write amplification factor.

[0082] like Figure 1 As shown, this invention provides a data placement method suitable for write amplification optimization of ZNS SSDs, including the following steps:

[0083] (1) Obtain input / output (IO) data in bytes from the test dataset and convert the IO data into N data blocks in blocks;

[0084] Specifically, the conversion process in this step first obtains the logical block address (LBA) range containing the IO data. The start and end addresses of this LBA range must be integer multiples of the block size (in this invention, the block size is set to 4KB). The portion of the LBA range excluding the IO data to be modified is supplemented by reading existing user data. Then, the obtained LBA range is divided into N data blocks. For example, if the starting address of the current IO data is 1024 and its size is 5KB, then the required LBA range is [0, 8192]. Besides the data address [1024, 6144] in the IO data to be modified, existing user data at addresses [0, 1024] and [6144, 8192] must be read first. Then, the LBA range [0, 8192] is divided into two data blocks: [0, 4096] and [4096, 8192].

[0085] (2) Set the counter cnt_uw = 1;

[0086] (3) Determine whether the metadata information corresponding to the cnt_uw data block among the N data blocks obtained in step (1) exists in memory. If it does, proceed to step (5); otherwise, proceed to step (4).

[0087] Specifically, the metadata information includes the LBA interval corresponding to the cnt_uw-th data block among the N data blocks obtained in step (1), and the number of user writes freq. uw Garbage collection write count (freq) gw And the value of the global counter gt when the data in the LBA interval corresponding to the data block was last modified. pre .

[0088] (4) Perform a new user write operation on the cnt_uw-th data block among the N data blocks obtained in step (1), add the metadata information of the data block to memory, and set the user write count freq in the metadata information of the data block. uw and garbage collection write count (freq) gw Initialize to 0, and simultaneously set the block lifetime value of this data block to 2. 64 -1;

[0089] (5) Perform a user write operation on the cnt_uw-th data block among the N data blocks obtained in step (1) to obtain the block lifetime value of the data block;

[0090] Specifically, step (5) includes the following sub-steps:

[0091] (5-1) Obtain the current global counter value gt in memory and the global counter value gt of the LBA interval corresponding to the N data blocks obtained in step (1) when it was last modified. pre And based on the values ​​gt and gt pre Get the block lifetime value (BILT) for this data block;

[0092] Specifically, the formula used in this step is as follows:

[0093] BILT = gt-gt pre

[0094] The advantage of this step is that the block lifetime value is very simple to obtain and can well reflect the recent popularity of the data block.

[0095] (5-2) Extract the user write count freq from the metadata information of the cnt_uw-th data block among the N data blocks obtained in step (1). uw Set to freq uw +1, and simultaneously increment the global counter value gt for the data in the LBA interval corresponding to the data block at the time of the last modification. pre Set the value of the current global counter to gt, and set the value of the current global counter to gt+1.

[0096] The advantage of this step is that it allows for online modification of the metadata information of data blocks, ensuring the timeliness and accuracy of the data block popularity value.

[0097] (6) Retrieve the amount of invalid data from each partition in memory. inv and total data volume zone And based on the amount of invalid data inv and total data volume zone Get the percentage of invalid data in this partition (gp), and the percentage of invalid data in all M partitions in memory (gp). all ;

[0098] Specifically, this step uses the following formula:

[0099]

[0100]

[0101] (7) Determine the percentage of invalid data (gp) of all M partitions obtained in step (6). all If the value is greater than the global garbage collection threshold, proceed to step (8) if yes, otherwise proceed to step (14);

[0102] Specifically, the global garbage collection threshold in this invention ranges from 10% to 30%, preferably 15%.

[0103] (8) Use the garbage collection algorithm (Cost-benefit-enhanced, or CBE) to select the partition to be garbage collected from all M partitions in memory;

[0104] Specifically, step (8) includes the following steps:

[0105] (8-1) Set the counter cnt_zone = 1;

[0106] (8-2) Determine whether the cnt_zone-th partition among all M partitions in memory has been filled. If so, proceed to step (8-3); otherwise, proceed to step (8-4).

[0107] (8-3) Retrieve the global counter value gt from memory when the cnt_zone-th partition among all M partitions is full. fulled Combined with the invalid data percentage gp of the partition obtained in step (6) and the current global counter value gt, the partition lifetime value t of the partition is obtained. zone And the profit value cb, then proceed to step (8-4);

[0108] Specifically, this step uses the following formula:

[0109] t zone =gt-gt fulled

[0110]

[0111] The advantage of this sub-step is that it considers two key factors—the proportion of invalid data and partition lifetime—when calculating the benefit value, which can reduce the amount of subsequent invalid data migration and lower the write amplification factor.

[0112] (8-4) Determine if cnt_zone is equal to M. If yes, proceed to step (8-5). Otherwise, set cnt_zone to cnt_zone+1 and return to step (8-2).

[0113] (8-5) Sort the profit values ​​of all M partitions in memory obtained in step (8-3) in descending order, and take the partition with the largest profit value as the partition to be garbage collected.

[0114] (8-6) The partition lifetime t of the partition with the highest profit value obtained in step (8-5) is calculated. zone Add it to the partition lifetime queue of the corresponding type in memory;

[0115] Specifically, the partition lifetime queue is a first-in-first-out (FIFO) queue, and the number of partition lifetime queues is the number of partition types in memory, L, which corresponds to the maximum number of partitions that ZNS SSD can open at the same time. In this invention, it is set to 7, and the length K of the partition lifetime queue ranges from 1 to 1024, preferably 10.

[0116] (9) Set the counter cnt_gw = 1;

[0117] (10) Determine whether the cnt_gw data block in the partition to be garbage collected in step (8) is valid. If it is, proceed to step (11); otherwise, proceed to step (12).

[0118] (11) Perform a garbage collection write operation on the cnt_gw data block in the partition selected in step (8) to modify some of its metadata information, and then move the data block with modified metadata information to other partitions in memory that are not full.

[0119] Specifically, step (11) includes the following steps:

[0120] (11-1) Obtain the current global counter value gt in memory and the global counter value gt of the data in the LBA interval corresponding to the cnt_gw-th data block in the partition selected in step (8) for garbage collection when it was last modified. pre And obtain the block lifetime value of the data block;

[0121] Specifically, the formula for calculating the block lifetime value of the data block in this step is exactly the same as that in step (5-1) above.

[0122] (11-2) Determine whether the block lifetime value obtained in step (11-1) exceeds the refresh threshold R1. If it does, proceed to step (11-3); otherwise, proceed to step (11-6).

[0123] Specifically, the refresh threshold R1 ranges from 0 to 128*data. zone , where data zone The total amount of data within a single partition in step (6) is preferably 64 * data. zone .

[0124] (11-3) Take the user write count freq from the metadata information of the cnt_gw-th data block in the partition selected in step (8) for garbage collection. ue Setting it to 0 sets the garbage collection write count (freq). gw Set to 1;

[0125] (11-4) Determine whether the block lifetime value obtained in step (11-1) exceeds the extreme cold data threshold R2. If it does, proceed to step (11-5); otherwise, proceed to step (11-7).

[0126] Specifically, the refresh threshold R2 ranges from 128 * data. zone Up to 1024*data zone , where data zone The total amount of data within a single partition in step (6) is preferably 256 * data. zone .

[0127] (11-5) Calculate the number of garbage collection writes (freq) of the cnt_gw-th data block in the partition selected in step (8) for garbage collection. gw Set it to L–C, then proceed to step (11-7);

[0128] Specifically, L is the number of partition types in step (8-6), and C is the default heat value of the data block, which ranges from 1 to L, preferably 5.

[0129] (11-6) Extract the garbage collection write count freq from the metadata information of the cnt_gw-th data block in the partition selected in step (8) for garbage collection. gw Set to freq gw +1, then proceed to step (11-7).

[0130] The advantage of steps (11-2) to (11-6) above is that by using the threshold to select the hot data blocks that have cooled down in time, it can effectively avoid multiple unnecessary migrations of the data blocks and reduce the write amplification factor.

[0131] (11-7) Retrieve from memory the garbage collection write count freq of the cnt_gw-th data block in the partition selected for garbage collection in step (8). gw and the number of times the user writes (freq) uw And based on the number of times garbage collection is written (freq) gw and the number of times the user writes (freq) uw Get the initial heat value bh of this data block. pre ;

[0132] Specifically, the formula for calculating the initial heat value is as follows:

[0133] bh pre =freq gw -freq uw +C

[0134] (11-8) Retrieve the partition lifetime queue obtained in step (8-6) from memory, and use the initial heat value bh obtained in step (11-7) in the partition lifetime queue. pre The lifetime values ​​t of all i partitions in the queue to which the corresponding type of partition belongs. zone The decision threshold l is obtained;

[0135] Specifically, the formula for determining the threshold is as follows:

[0136]

[0137] (11-9) Using the initial heat value bh obtained in step (11-7) pre The final heat value of the cnt_gw data block in the partition to be garbage collected in step (8) is obtained by taking the block lifetime value BILT obtained in step (11-1) and the judgment threshold l obtained in step (11-8).

[0138] Specifically, the formula for calculating the final popularity value is as follows:

[0139]

[0140] The advantage of steps (11-7) to (11-9) above is that the initial hotness value is obtained by first using the number of user writes and garbage collection writes of the data block, which can achieve preliminary hot and cold data classification with low overhead. Then, the hotness value of the data block is corrected according to the block lifetime value and the judgment threshold, which can obtain a more accurate hot and cold data classification effect.

[0141] (11-10) Write the cnt_gw data block in the partition selected in step (8) to be garbage collected into the partition corresponding to the final heat value obtained in step (11-9).

[0142] (12) Determine whether cnt_gw is equal to the total number of data blocks in the partition selected in step (8) for garbage collection. If yes, proceed to step (13); otherwise, set cnt_gw to cnt_gw+1 and return to step (10).

[0143] (13) Reset the partition selected in step (8) to be garbage collected, delete invalid data in the partition, and return to step (7);

[0144] (14) For the cnt_uw-th data block among the N data blocks obtained in step (1), the garbage collection write count freq of this data block is used as the basis for the calculation. gw and the number of times the user writes (freq) uw The system retrieves the final heat value of a data block from the partition lifetime queue and the block lifetime value, and writes the data block into the partition corresponding to its final heat value.

[0145] Specifically, the process of obtaining the final heat value of the data block in this step is exactly the same as in step (11) above, and will not be repeated here.

[0146] (15) Determine whether cnt_uw is equal to the number of data blocks N after splitting in step (1). If yes, the process ends; otherwise, set cnt_uw to cnt_uw+1 and return to step (3).

[0147] Test Results

[0148] The test environment for this invention is as follows: under the CentOS 7.9 operating system, the CPU is an AMD Ryzen 55600X, the DRAM is DDR4 32GB, the hard drive is a ZNS SSD emulated by FEMU, and the method and system of this invention are implemented by C++ programming, and compiled by cmake 3.23.2 and g++ (GCC) 8.3.1. The specific parameter settings of ZNS SSD are shown in Table 1.

[0149] To demonstrate the effectiveness of the method of this invention, a 6-day client access module IO log record of a certain cluster of Tencent Cloud block storage system was used as a dataset for testing. The statistical information of this dataset is shown in Table 2. The test results obtained by this invention are compared with the current state-of-the-art methods, and the evaluation results are shown in Table 3.

[0150] According to the test results under real load in the Tencent Cloud block storage system recorded in Table 3, it can be seen that the data placement method proposed in this invention for ZNS SSD write amplification optimization is superior to existing methods in terms of average write amplification factor, average garbage collection write volume, and average garbage collection times.

[0151] Table 1 ZNS SSD Simulation Parameters

[0152]

[0153] Table 2. Statistical information of the dataset

[0154]

[0155] Table 3 Comparison of Test Results

[0156]

[0157] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A data placement method suitable for write amplification optimization of ZNS SSD, characterized in that, Includes the following steps: (1) Obtain input / output IO data in bytes from the test dataset and convert the IO data into N data blocks in blocks; (2) Set the counter cnt_uw = 1; (3) Determine whether the metadata information corresponding to the cnt_uwth data block among the N data blocks obtained in step (1) exists in memory. If it does, proceed to step (5); otherwise, proceed to step (4). The metadata information includes the LBA interval corresponding to the cnt_uwth data block among the N data blocks obtained in step (1) and the number of user writes. Number of times garbage collection is written And the value of the global counter when the data in the LBA interval corresponding to the data block was last modified. ; (4) Perform a new user write operation on the cnt_uw-th data block among the N data blocks obtained in step (1), add the metadata information of the data block in memory, and record the number of user writes in the metadata information of the data block. and the number of times garbage is written Initialize to 0, and simultaneously set the block lifetime value of this data block to 2. 64 -1; (5) Perform a user write operation on the cnt_uw-th data block among the N data blocks obtained in step (1) to obtain the block lifetime value of the data block; step (5) includes the following sub-steps: (5-1) Retrieve the current global counter value in memory The value of the global counter for the data within the LBA interval corresponding to the cnt_uw-th data block among the N data blocks obtained in step (1) when it was last modified. And according to the value and Get the block lifetime value of this data block. The formula used in this step is as follows: ； (5-2) The number of user writes in the metadata information of the cnt_uw-th data block out of the N data blocks obtained in step (1) Set as At the same time, it sets the value of the global counter for the data in the LBA range corresponding to the data block at the time of the last modification. Set to the value of the current global counter and set the current global counter value Set as ; (6) Obtain the amount of invalid data for each partition from memory. and total data volume And based on the amount of invalid data and total data volume Get the percentage of invalid data in this partition. And the percentage of invalid data across all M partitions in memory. ; (7) Determine the percentage of invalid data in all M partitions obtained in step (6). If the value is greater than the global garbage collection threshold, proceed to step (8) if yes, otherwise proceed to step (14). (8) Use the garbage collection algorithm CBE to select the partition to be garbage collected from all M partitions in memory; (9) Set the counter cnt_gw = 1; (10) Determine whether the cnt_gw-th data block in the partition to be garbage collected in step (8) is valid. If it is, proceed to step (11); otherwise, proceed to step (12). (11) Perform a garbage collection write operation on the cnt_gw data block in the partition selected in step (8) to modify some of its metadata information, and then migrate the data block with modified metadata information to other unwritten partitions in memory; (12) Determine whether cnt_gw is equal to the total number of data blocks in the partition selected in step (8) for garbage collection. If yes, proceed to step (13); otherwise, set cnt_gw to cnt_gw + 1 and return to step (10). (13) Reset the partition selected in step (8) to be garbage collected, delete invalid data in the partition, and return to step (7); (14) For the cnt_uw-th data block among the N data blocks obtained in step (1), the number of garbage collection writes for that data block is calculated. Number of times users write The system retrieves the final heat value of a data block from the partition lifetime queue and the block lifetime value, and writes the data block into the partition corresponding to its final heat value. (15) Determine whether cnt_uw is equal to the number of data blocks N after splitting in step (1). If yes, the process ends; otherwise, set cnt_uw to cnt_uw + 1 and return to step (3).

2. The data placement method for ZNS SSD write amplification optimization according to claim 1, characterized in that, Step (6) uses the following formula: ； 。 3. The data placement method for ZNS SSD write amplification optimization according to claim 2, characterized in that, Step (8) includes the following steps: (8-1) Set the counter cnt_zone = 1; (8-2) Determine whether the cnt_zone-th partition among all M partitions in memory has been filled. If so, proceed to step (8-3); otherwise, proceed to step (8-4). (8-3) Retrieve the value of the global counter when the cnt_zone-th partition out of all M partitions is full in memory. And combined with the percentage of invalid data in the partition obtained in step (6) and the current global counter value Get the partition lifetime value of this partition. and earnings Then proceed to step (8-4); (8-4) Determine if cnt_zone is equal to M. If yes, proceed to step (8-5). Otherwise, set cnt_zone to cnt_zone + 1 and return to step (8-2). (8-5) Sort the profit values ​​of all M partitions in memory obtained in step (8-3) in descending order, and select the partition with the largest profit value as the partition to be garbage collected. (8-6) The partition lifetime value of the partition with the highest profit value obtained in step (8-5) is determined. Add it to the partition lifetime queue of the corresponding type in memory.

4. The data placement method for ZNS SSD write amplification optimization according to claim 3, characterized in that, Step (8-3) uses the following formula: ； 。 5. The data placement method for ZNS SSD write amplification optimization according to claim 4, characterized in that, The partition lifetime queue is a first-in-first-out (FIFO) queue, and the number of partition lifetime queues is equal to the number of partition types in memory, L, which corresponds to the maximum number of partitions that ZNS SSD can open simultaneously.

6. The data placement method for ZNS SSD write amplification optimization according to claim 5, characterized in that, Step (11) includes the following steps: (11-1) Retrieve the current global counter value in memory The value of the global counter when the data in the LBA interval corresponding to the cnt_gw-th data block in the partition selected for garbage collection in step (8) was last modified. And obtain the block lifetime value of the data block; (11-2) Determine whether the block lifetime value obtained in step (11-1) exceeds the refresh threshold R1. If it does, proceed to step (11-3); otherwise, proceed to step (11-6). (11-3) The number of user writes in the metadata information of the cnt_gw-th data block in the partition to be garbage collected in step (8) Setting it to 0 reduces the number of write operations during garbage collection. Set to 1; (11-4) Determine whether the block lifetime value obtained in step (11-1) exceeds the extreme cold data threshold R2. If it does, proceed to step (11-5); otherwise, proceed to step (11-7). (11-5) Calculate the number of garbage collection writes for the cnt_gw-th data block in the partition selected in step (8) for garbage collection. Set it to L – C, and then proceed to step (11-7); where L is the number of partition types in step (8-6), and C is the default hotness value of the data block, with a value range of 1 to L; (11-6) The number of garbage collection writes in the metadata information of the cnt_gw-th data block in the partition to be garbage collected in step (8) is used. Set as Then proceed to step (11-7). (11-7) Obtain from memory the number of garbage collection writes of the cnt_gw-th data block in the partition selected for garbage collection in step (8). Number of times users write And based on the number of times garbage is recycled Number of times users write Get the initial heat value of this data block. ; (11-8) Retrieve the partition lifetime queue obtained in step (8-6) from memory, and use the initial heat value obtained in step (11-7) from the partition lifetime queue. All of the corresponding type partitions in the queue Partition lifespan value Obtain the decision threshold ; (11-9) Using the initial heat value obtained in step (11-7) The block lifetime value obtained in step (11-1) and the judgment threshold obtained in step (11-8) , obtain the final heat value of the cnt_gw data block in the partition to be garbage collected in step (8); (11-10) Write the cnt_gw data block in the partition selected in step (8) to be garbage collected into the partition corresponding to the final heat value obtained in step (11-9).

7. The data placement method for ZNS SSD write amplification optimization according to claim 6, characterized in that, The formula for calculating the initial heat value is as follows: ； The formula for calculating the threshold is as follows: ； The formula for calculating the final heat value is as follows: 。 8. A data placement system suitable for write amplification optimization of ZNS SSDs, implemented by the data placement method for write amplification optimization of ZNS SSDs as described in claim 1, characterized in that, The data placement system includes: The first module is used to obtain input / output IO data in bytes from the test dataset and convert the IO data into N data blocks in blocks. The second module is used to set the counter cnt_uw = 1; The third module determines whether the metadata information corresponding to the cnt_uw-th data block among the N data blocks obtained from the first module exists in memory. If it does, the process proceeds to the fifth module; otherwise, it proceeds to the fourth module. The metadata information includes the LBA range corresponding to the cnt_uw-th data block among the N data blocks obtained from the first module, and the number of user writes. Number of times garbage collection is written And the value of the global counter when the data in the LBA interval corresponding to the data block was last modified. ; The fourth module is used to perform a new user write operation on the cnt_uw-th data block out of the N data blocks obtained by the first module. It adds the metadata information of the data block to memory and sets the number of user writes in the metadata information of the data block. and the number of times garbage is written Initialize to 0, and simultaneously set the block lifetime value of this data block to 2. 64 -1; The fifth module is used to perform a user write operation on the cnt_uw-th data block among the N data blocks obtained by the first module, so as to obtain the block lifetime value of the data block; The sixth module is used to retrieve the amount of invalid data from each partition in memory. and total data volume And based on the amount of invalid data and total data volume Get the percentage of invalid data in this partition. And the percentage of invalid data across all M partitions in memory. ; The seventh module is used to determine the percentage of invalid data across all M partitions obtained from the sixth module. If the value is greater than the global garbage collection threshold, proceed to module 8; otherwise, proceed to module 14. The eighth module is used to select the partition to be garbage collected from all M partitions in memory using the CBE garbage collection algorithm; The ninth module is used to set the counter cnt_gw = 1; The tenth module is used to determine whether the cnt_gw-th data block in the partition selected by the eighth module for garbage collection is valid. If it is valid, proceed to the eleventh module; otherwise, proceed to the twelfth module. The eleventh module is used to perform a garbage collection write operation on the cnt_gw-th data block in the partition selected by the eighth module for garbage collection, so as to modify some of its metadata information, and then migrate the data block with modified metadata information to other unfilled partitions in memory. The twelfth module is used to determine whether cnt_gw is equal to the total number of data blocks in the partition selected by the eighth module for garbage collection. If so, proceed to the thirteenth module; otherwise, set cnt_gw to cnt_gw + 1 and return to the tenth module. The thirteenth module is used to reset the partition selected by the eighth module for garbage collection, delete invalid data in the partition, and return to the seventh module. The fourteenth module is used to determine the number of garbage collection writes for the cnt_uw-th data block out of the N data blocks obtained in the first module. Number of times users write The system retrieves the final heat value of a data block from the partition lifetime queue and the block lifetime value, and writes the data block into the partition corresponding to its final heat value. The fifteenth module is used to determine whether cnt_uw is equal to the number of data blocks N after the first module. If so, the process ends; otherwise, cnt_uw is set to cnt_uw + 1, and the process returns to the third module.

Images