Solid state disk wear leveling algorithm based on log structure merging tree

Abstract

The invention relates to the technical field of wear leveling, and particularly discloses a solid state disk wear leveling algorithm based on a log structure merging tree. Comprising the following steps: S1, dividing the hierarchy of LSM-Tree, and triggering a merging operation when write-in data is increased and reaches a threshold value; s2, distinguishing cold and hot data according to the data layering characteristic of the LSM-Tree, modifying a corresponding interface to transmit layering information to an FTL of an underlying SSD, and confirming a data placement area through the layering information transmitted by an upper layer; s3, storing the erasing times of each flash memory block by adopting a cold and hot block table, and presetting a threshold value; s4, judging and distinguishing cold and hot blocks for the flash memory blocks which are subjected to read-write operation and cannot be updated on site; and S5, organizing all the blocks into a double linked list according to the number of wear times, distributing different hierarchical data storage positions according to hierarchical information of the LSM-Tree transmitted by an upper layer, and performing data exchange. According to the method, resource waste caused by the fact that cold data occupies a low-erasure-frequency block for a long time is avoided, and the problem of unbalanced abrasion of the storage block is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of wear leveling, and in particular to a wear leveling algorithm for a solid state hard disk based on a log structure merge tree. Background Art

[0002] With the development of storage technology, solid-state drives have achieved remarkable development, including a significant increase in read and write speeds, reduced energy consumption, and a continuous increase in storage capacity. With its unique solid-state storage chip technology, solid-state drives have gradually replaced traditional mechanical hard drives and become an indispensable storage solution in data centers. Key-value databases make full use of the advantages of solid-state drives and widely use LSM-Tree as their storage structure to further optimize data writing performance, improve storage efficiency, and meet the needs of high-performance and highly scalable storage systems in the big data era.

[0003] However, as the main storage medium of solid-state drives (SSDs), NAND Flash has a limited number of physical blocks that can be erased and written. Usually, the number of times a NAND Flash block can be erased and written ranges from thousands to tens of thousands, which is much lower than the life of a mechanical hard drive. This means that if a block is erased and written frequently, it will soon reach its life limit, resulting in data loss or hard drive failure. In order to solve the problem of flash memory life, wear leveling is usually used as one of the solutions to increase the life of solid-state drives. The wear leveling method intelligently manages the erase and write operations of data blocks to ensure that all blocks receive a relatively uniform number of erase and write times, thereby extending the overall life of the SSD.

[0004] Wear leveling methods can be divided into two types according to the timing of wear leveling: 1. Dynamic wear leveling. Triggered when data is written. When data needs to be updated or written, the algorithm calculates the wear of each flash memory unit and writes it to the flash memory block with the least number of erases. The advantages of this method are fast processing speed and little impact on performance, but it mainly focuses on the balance between dynamic data and unused space. 2. Static wear leveling. Static wear leveling can run in the background. It monitors the wear of each block in the SSD and migrates data from high-wear blocks to low-wear blocks. Through this operation, the number of erases of storage blocks with a large number of erases increases very slowly, while the number of erases of storage blocks with a small number of erases increases rapidly, thereby achieving the purpose of balancing the wear conditions of each storage block.

[0005] However, in the first method, if there is a large amount of cold data in the data, the cold data may not be modified for a long time, and the blocks where the cold data are located cannot be recycled, making it impossible for other data to be written into these blocks with a small number of erase times, thereby weakening the effect of wear leveling. In actual applications, most of the data is cold data, and a small part of the data is hot data. In order to achieve a better wear leveling effect in the second method, a large amount of memory is required to store information such as the number of times the block is erased and the storage time of the data in the storage block. When performing wear leveling operations, this information is used to decide when to trigger the wear leveling operation. And it is easily affected by the coldness and heat of the data, so how to judge the coldness and heat of the data is also a problem that needs to be considered.

[0006] Therefore, based on the hierarchical and ordered characteristics of LSM-Tree to distinguish between hot and cold data, a wear leveling algorithm is redesigned to extend the overall life of the SSD. Summary of the invention

[0007] In view of the above problems in the prior art, the present invention provides a wear leveling algorithm for solid-state drives based on a log-structured merge tree. The algorithm utilizes the multi-layer structure of the LSM-Tree and the high write frequency and fast access characteristics of the low-level data to preferentially store hot data in areas of the SSD with less wear times, while high-level data is written into the cold data area of ​​the SSD, thereby achieving hot and cold data separation and maximizing storage efficiency.

[0008] The technical solution adopted by the present invention is as follows: A wear leveling algorithm for a solid state drive based on a log structure merge tree comprises the following steps: S1. Divide the LSM-Tree into layers, and trigger a merge operation when the amount of written data increases to a threshold; S2. Differentiate between hot and cold data based on the data stratification characteristics of the LSM-Tree, modify the corresponding interface to pass the stratification information to the FTL of the underlying SSD, and confirm the data placement area through the hierarchical information passed by the upper layer; S3, using a hot and cold block table to store the number of erase times C of each flash memory block, and presetting a threshold K to distinguish between hot and cold blocks; S4, for the flash memory blocks that cannot be updated in situ after the read and write operations, distinguish between hot and cold blocks. If the C value is greater than the threshold K, the system will determine it as a hot block, otherwise it will be determined as a cold block; S5. Organize all blocks into a bidirectional linked list according to the number of times they are worn, allocate data storage locations at different levels according to the level information of the LSM-Tree transmitted from the upper layer, and exchange data between the free block with the least number of erases and the free block with the most number of erases.

[0009] Preferably, the LSM-Tree hierarchy in step S1 includes multiple levels, Level 0 is located in the memory, Level 1 and above are located on the disk, and each level stores sorted and merged data blocks (SSTable).

[0010] Preferably, in the initial state of the solid state hard disk in step S3, the erasure count C of all blocks is 0, and the data written in the upper layer is written into the flash memory normally.

[0011] Preferably, the flash memory blocks that cannot be updated in-place after the read and write operations in step S4 are flash memory blocks that need to be erased by garbage collection.

[0012] Preferably, the flash memory block erased by garbage collection in step S4 has an erasure count of Ci=Ci+1, where Ci represents the erasure count of the flash memory block i, ie, the number of times the flash memory block i is garbage collected.

[0013] Preferably, after the flash memory block is erased by garbage collection, it undergoes a write-erase operation of the flash memory block, and its Ci value increases, and the step S4 needs to be repeated to distinguish between cold blocks and hot blocks.

[0014] Preferably, in step S5, data of Level 1-Level 2 are placed in the free blocks with the least number of erasures, and data of Level 3 and above are placed in the free blocks with the most number of erasures.

[0015] Preferably, the condition for triggering the wear leveling operation is the difference between the block with the most erase times and the block with the least erase times.

[0016] Preferably, it is characterized in that multiple data blocks are recovered at one time when garbage collection is performed on the collection block.

[0017] Preferably, when performing garbage collection to collect blocks, as many data blocks as possible are selected.

[0018] The present invention has the following characteristics and advantages: (1) The strategy of the present invention can effectively distinguish and utilize the distribution characteristics of hot and cold data to extend the overall service life of the SSD; (2) Different from the traditional method, the present invention utilizes the multi-layer structure of LSM-Tree and the high write frequency and fast access characteristics of the low-level to store hot data preferentially in the area with less wear times of the SSD, while the high-level data is written into the cold data area of ​​the SSD, thus achieving the separation of hot and cold data and maximizing storage efficiency; (3) The method of the present invention not only avoids the waste of resources caused by cold data occupying blocks with low erase counts for a long time, but also reduces the problem of uneven wear caused by uneven hot and cold data; (4) Compared with static wear leveling's reliance on a large amount of memory resources and the limitations of dynamic wear leveling in processing cold data, this technology enables intelligent management of hot and cold data without the need to monitor the number of erases or storage time of each block, thereby extending the life of the SSD.

[0019] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a process diagram for confirming hot and cold blocks according to an embodiment of the present invention; Figure 2 It is a data placement diagram of different Levels in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0022] A wear leveling algorithm for a solid state drive based on a log structure merge tree comprises the following steps: S1. Divide the LSM-Tree into layers, and trigger a merge operation when the amount of written data increases to a threshold; S2. Differentiate between hot and cold data based on the data stratification characteristics of LSM-Tree, modify the corresponding interface to pass the stratification information to the FTL of the underlying SSD, and confirm the data placement area through the stratification information passed by the upper layer; S3, using a hot and cold block table to store the erase count C of each flash memory block, and presetting a threshold K to distinguish between hot and cold blocks; S4, for the flash memory blocks that cannot be updated in situ after the read and write operations, distinguish between hot and cold blocks. If the C value is greater than the threshold K, it will be judged as a hot block by the system, otherwise it will be judged as a cold block; S5. Organize all blocks into a bidirectional linked list according to the number of times they are worn, allocate data storage locations at different levels according to the level information of the LSM-Tree transmitted from the upper layer, and exchange data between the free block with the least number of erases and the free block with the most number of erases.

[0023] The LSM-Tree hierarchy consists of multiple levels. Level 0 is located in memory and usually uses ordered data structures (such as red-black trees, skip lists, etc.) to manage data to achieve fast read and write operations. Level 1 and above are located on disk, and each level stores sorted and merged data blocks (SSTable). As data is continuously written, the data in Level 0 will gradually increase and reach the threshold. At this time, a merge operation will be triggered to merge the data in Level 0 into Level 1, and so on, forming a bottom-up merged data structure.

[0024] The data stratification feature in LSM-Tree is naturally suitable for distinguishing between hot and cold data. Generally speaking, data at higher levels (such as L1, L2, etc.) are more "hot" because they have been frequently written or updated in the recent period. Data at lower levels (such as L3, L4, etc.) are often "cold" data because they have been merged many times and have not been accessed or modified for a long time. Therefore, by modifying the corresponding interface, the stratification information can be passed to the FTL of the underlying SSD, and the placement area of ​​this data can be confirmed through the stratification information passed by the upper layer.

[0025] In the specific implementation method, in step S3, when the solid state hard disk is in the initial state, the erasure count C of all blocks is 0, and the data written in the upper layer is written into the flash memory normally.

[0026] The flash memory blocks that cannot be updated in-place after the read and write operations in step S4 are flash memory blocks that need to be erased by garbage collection.

[0027] The erasure count of the flash memory block after being garbage collected and erased in step S4 is Ci=Ci+1, where Ci represents the erasure count of the flash memory block i, that is, the number of times the flash memory block i is garbage collected.

[0028] After the flash memory block is erased by garbage collection, it undergoes a write-erase operation of the flash memory block, and its Ci value increases. It is necessary to repeat the step S4 to distinguish between cold blocks and hot blocks.

[0029] The condition that triggers the wear leveling operation is the difference between the block with the most erase times and the block with the least erase times.

[0030] When garbage collection is performed, multiple data blocks are recovered at a time. When garbage collection is performed, as many data blocks as possible are selected.

[0031] The process diagram of the hot and cold blocks in this embodiment is as follows Figure 1As shown in the figure, when a solid-state drive is not written, all flash blocks in it can be regarded as blocks with an erase count of 0. As the number of writes gradually increases, the flash blocks will experience a write-erase cycle. In our method, we introduce a hot and cold block table, which stores the erase count C of each block, and a threshold K needs to be preset to distinguish hot and cold blocks. In the initial state of the solid-state drive, the erase count C of all blocks is 0, and the data written in the upper layer is written into the flash memory normally.

[0032] When a block of data is erased by garbage collection, Ci = Ci + 1. After a period of read and write operations, since the flash memory cannot be updated in situ, the Ci value continues to increase. If the C value is greater than the threshold K, it will be judged as a hot block by the system, otherwise it will be judged as a cold block.

[0033] In this example, a group of data storage spaces consisting of 4 blocks is defined. Assume that the threshold K = 200. After a period of writing, C0 = 210, C1 = 100, C2 = 300, and C4 = 80. Among them, the number of erase times of physical blocks Block0 and Block2 exceeds the threshold, so they are determined to be hot blocks, and the number of erase times of physical blocks Block1 and Block3 is lower than the threshold, so they are determined to be cold blocks.

[0034] The data placement diagram of different Levels in this embodiment is as follows Figure 2 As shown in the figure, all blocks are organized into a bidirectional linked list according to the number of wear times, and then the storage locations of data at different levels are allocated according to the level information of the LSM-Tree passed from the upper layer. The data of Level1-Level2 is placed in the free blocks with the least number of erasures, and the data above Level3 is placed in the free blocks with the most number of erasures. The data of Level1-Level2 is placed in the blocks Block1 and Block3 with the least number of erasures, while the data above Level3 is placed in the blocks Block0 and Block2 with the most number of erasures, because the data above Level3 is rarely updated. This naturally distinguishes cold data from hot data.

[0035] This division has the following two advantages: 1) The number of hot blocks will not increase significantly, because they will be limited to the data at the Level 1-Level 2 level, resulting in a decrease in SSD performance; 2) Hot data blocks will not contain a large number of cold data pages, avoiding unnecessary data exchange.

[0036] In addition, according to the hot data and cold data that have been distinguished in this embodiment, multiple data blocks can be recycled at one time to reduce the number of garbage collections. By selecting as many blocks as possible to complete garbage collection, effective data migration can be achieved.

[0037] In summary, this method not only avoids the waste of resources caused by cold data occupying low-erasure-count blocks for a long time, but also reduces the uneven wear problem caused by uneven hot and cold data. Compared with the reliance of static wear leveling on a large amount of memory resources and the limitations of dynamic wear leveling in cold data processing, this method realizes the intelligent management of hot and cold data without the need to monitor the erase count or storage time of each block, thereby extending the life of the SSD.

[0038] The embodiments described above are only descriptions of the preferred modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A wear leveling algorithm for solid state drives based on a log structure merge tree, characterized in that: The following steps are involved: S1. Divide the LSM-Tree into layers, and trigger a merge operation when the amount of written data increases to a threshold; S2. Differentiate between hot and cold data based on the data stratification characteristics of the LSM-Tree, modify the corresponding interface to pass the stratification information to the FTL of the underlying SSD, and confirm the data placement area through the hierarchical information passed by the upper layer; S3, using a hot and cold block table to store the number of erase times C of each flash memory block, and presetting a threshold K to distinguish between hot and cold blocks; S4, for the flash memory blocks that cannot be updated in situ after the read and write operations, distinguish between hot and cold blocks. If the C value is greater than the threshold K, the system will determine it as a hot block, otherwise it will be determined as a cold block; S5. Organize all blocks into a bidirectional linked list according to the number of times they are worn, allocate data storage locations at different levels according to the level information of the LSM-Tree transmitted from the upper layer, and exchange data between the free block with the least number of erases and the free block with the most number of erases.

2. The method according to claim 1, characterized in that In step S1, the LSM-Tree hierarchy includes multiple levels. Level 0 is located in the memory, and Level 1 and above are located on the disk. Each level stores sorted and merged data blocks (SSTable).

3. The method according to claim 1, characterized in that In step S3, when the solid state hard disk is in the initial state, the erasure count C of all blocks is 0, and the data written in the upper layer is written into the flash memory normally.

4. The method according to claim 1, characterized in that: The flash memory blocks that cannot be updated in-place after the read and write operations in step S4 are flash memory blocks that need to be erased by garbage collection.

5. The method according to claim 4, characterized in that The erasure count of the flash memory block after being garbage collected and erased in step S4 is Ci=Ci+1, where Ci represents the erasure count of the flash memory block i, that is, the number of times the flash memory block i is garbage collected.

6. The method according to claim 5, characterized in that After the flash memory block is erased by garbage collection, it undergoes a write-erase operation of the flash memory block, and its Ci value increases. It is necessary to repeat the step S4 to distinguish between cold blocks and hot blocks.

7. The method according to claim 1, characterized in that In step S5, data of Level 1-Level 2 are placed in the free blocks with the least number of erasures, and data of Level 3 and above are placed in the free blocks with the most number of erasures.

8. The method according to claim 1, characterized in that: The condition that triggers the wear leveling operation is the difference between the block with the most erase times and the block with the least erase times.

9. The method according to claim 4, characterized in that When garbage collection is performed, multiple data blocks are recovered at a time.

10. The method according to claim 9, characterized in that When garbage collecting blocks, pick as many blocks as possible.

Images