Storage management method, system and equipment and storage medium

Abstract

The invention discloses a storage management method, system and device and a storage medium, and is applied to the technical field of storage, and the method comprises the following steps: obtaining a to-be-written data block, and determining a tile partition accessed by the data block; when the accessed tile partition is a static partition, processing the data block based on a static processing rule; when the accessed tile partition is a dynamic partition, judging whether the dynamic partition is fully written or not; if not, writing the data block into the sector with the lowest number in the unused sectors of the dynamic partition; if yes, garbage collection is conducted on the dynamic partition, and after garbage collection, the data block is written into the sector with the lowest number in the unused sectors of the dynamic partition. By applying the scheme of the application, data migration caused by a write amplification problem in the shingled recording disk can be effectively reduced, so that the performance of the shingled recording disk is improved.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a storage management method, system, device and storage medium. Background Art

[0002] The shingle recording technology is based on the traditional perpendicular magnetic recording technology. It uses a specially designed write head to write data to a narrower track, which allows the disk to accommodate more data while ensuring the thermal stability of the track, thereby increasing the storage capacity of the disk. Since the width of the write head is larger than the width of the track, when the write head writes data to the track, these tracks look like tiles on a house partially overlapping, so it is called shingle recording technology.

[0003] Because the write head will overwrite the data of adjacent tracks, the shingled recording technology does not support in-place updates of data. Therefore, every time the data on a track is updated, a "read-modify-write" operation needs to be performed in the current solution, which causes write amplification of the shingled recording disk. The more frequent the data update, the more serious the write amplification problem, the more extra data is migrated, and the more the disk performance degrades.

[0004] In summary, how to effectively reduce data migration caused by the write amplification problem in a shingled recording disk and improve the performance of the shingled recording disk is a technical problem that those skilled in the art urgently need to solve. Summary of the invention

[0005] The object of the present invention is to provide a storage management method, system, device and storage medium to effectively reduce data migration caused by write amplification problems in a shingled recording disk and improve the performance of the shingled recording disk.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a storage management method, comprising:

[0008] Obtain a data block to be written, and determine a tile partition accessed by the data block;

[0009] When the accessed tile partition is a static partition, processing the data block based on a static processing rule;

[0010] When the accessed tile partition is a dynamic partition, determining whether the dynamic partition is full;

[0011] If not, writing the data block into an unused sector of the dynamic partition, the lowest numbered sector;

[0012] If yes, performing garbage collection on the dynamic partition, and writing the data block to the lowest numbered sector in the unused sectors of the dynamic partition after garbage collection;

[0013] Among them, the garbage collection means reading all valid data in the dynamic partition into the first cache, and starting with the sector with the lowest number, and writing the valid data back to the dynamic partition in sequence; for the sectors in the dynamic partition, when the logical addresses corresponding to the data in at least two sectors are the same, only the data in the sector with the highest number is used as valid data.

[0014] On the other hand, obtaining a data block to be written and determining a tile partition accessed by the data block includes:

[0015] Receive data to be written;

[0016] According to the size of a single sector, the data to be written is divided into various data blocks to be written, and for each of the data blocks, a tile partition accessed by the data block is determined.

[0017] On the other hand, it also includes:

[0018] receiving a read request, and determining a tile partition accessed by a data block to be read based on the read request;

[0019] When the tile partition accessed by the data block to be read is a static partition, reading the data block from the static partition and performing read feedback;

[0020] When the tile partition accessed by the data block to be read is a dynamic partition, the data block is read from the dynamic partition and read feedback is performed.

[0021] On the other hand, when the accessed tile partition is a static partition, it is determined whether there are unused sectors in the static partition, and the sector with the lowest number among the unused sectors corresponds to the logical address of the data block;

[0022] If yes, writing the data block to the lowest numbered sector among the unused sectors of the static partition;

[0023] If not, writing the data block into the second cache;

[0024] When the remaining space of the second cache is lower than a preset threshold, the data blocks in the second cache are written to the corresponding static partition.

[0025] On the other hand, when the remaining space of the second cache is lower than a preset threshold, writing the data block in the second cache to the corresponding static partition includes:

[0026] According to the preset weight calculation rules, the weight value of each static partition is determined;

[0027] Sorting the static partitions in order of weight values ​​from small to large to obtain a static partition sequence;

[0028] Selecting the i-th static partition in the static partition sequence; wherein i is a positive integer and its initial value is 1;

[0029] Reading the data in the i-th static partition into the first cache as data to be modified;

[0030] Based on each data block accessing the i-th static partition stored in the second cache, the data to be modified is updated to obtain updated target data;

[0031] Writing the target data into the i-th static partition, and deleting each data block stored in the second cache for accessing the i-th static partition;

[0032] Determining whether the remaining space of the second cache is lower than a preset threshold;

[0033] If yes, then i is incremented by 1 and the operation of selecting the i-th static partition in the static partition sequence is returned to be executed.

[0034] On the other hand, according to the preset weight calculation rules, the weight value of each static partition is determined, including:

[0035] Determine the write access frequency of each static partition, and determine the weight value of each static partition based on the write access frequency of each static partition;

[0036] The write access frequency of the static partition is positively correlated with the weight value of the static partition.

[0037] On the other hand, according to the preset weight calculation rules, the weight value of each static partition is determined, including:

[0038] Determine the partition size of each static partition, and determine the weight value of each static partition based on the partition size of each static partition;

[0039] The partition size of the static partition is positively correlated with the weight value of the static partition.

[0040] In a second aspect, the present invention provides a storage management system, comprising:

[0041] A tile partition determination module, used to obtain a data block to be written and determine a tile partition accessed by the data block;

[0042] A static partition processing module, used for processing the data block based on a static processing rule when the accessed tile partition is a static partition;

[0043] A dynamic partition judgment module, used for judging whether the dynamic partition is full when the accessed tile partition is a dynamic partition;

[0044] If not, triggering a write module, the write module is used to write the data block into an unused sector of the dynamic partition, the sector with the lowest number;

[0045] If yes, triggering a garbage collection module, the garbage collection module is used to perform garbage collection on the dynamic partition, and write the data block to the lowest numbered sector in the unused sectors of the dynamic partition after garbage collection;

[0046] Among them, the garbage collection means reading all valid data in the dynamic partition into the first cache, and starting with the sector with the lowest number, and writing the valid data back to the dynamic partition in sequence; for the sectors in the dynamic partition, when the logical addresses corresponding to the data in at least two sectors are the same, only the data in the sector with the highest number is used as valid data.

[0047] In a third aspect, the present invention provides a storage management device, comprising:

[0048] Memory for storing computer programs;

[0049] A processor is used to execute the computer program to implement the steps of the storage management method as described above.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the storage management method as described above are implemented.

[0051] After applying the technical solution provided by the embodiment of the present invention, obtaining the data block to be written and determining the tile partition accessed by the data block, it is necessary to perform corresponding operations according to the difference between static partition and dynamic partition. If it is a static partition, the data block can be processed based on the static processing rules. If it is a dynamic partition, it is necessary to determine whether the dynamic partition is full. If it is not full, the data block can be directly written into the unused sector of the dynamic partition, the sector with the lowest number. It can be seen that since the data block is directly written into the dynamic partition, there is no write amplification in this operation, and it should be noted that when performing a write operation, it will affect the sectors with larger sector numbers in the partition but will not affect the sectors with smaller sector numbers. Since the location of the write at this time is the sector with the lowest number in the unused sectors of the dynamic partition, this write operation will not affect the current existing data in the dynamic partition. If the tile partition accessed is a dynamic partition and is full, it means that the current sectors of the dynamic partition are all used, so it is necessary to perform garbage collection on the dynamic partition. Garbage collection means reading all valid data in the dynamic partition to the first cache, and then starting writing with the lowest-numbered sector, and writing the valid data back to the dynamic partition in a sequential manner. In other words, through garbage collection, all valid data in the dynamic partition is retained, and the rest of the data is discarded, so that the dynamic partition has unused sectors again, and after garbage collection, these valid data occupy the low-numbered sectors in the dynamic partition. Therefore, after garbage collection is completed, data blocks can continue to be written to the unused sectors of the dynamic partition, the lowest-numbered sectors. This write operation does not have write amplification and does not affect the valid data in the low-numbered sectors.

[0052] It should also be noted that the garbage collection process described above has write amplification because it needs to read all valid data in the dynamic partition to the first cache and write the valid data back to the dynamic partition. However, since garbage collection only needs to be performed when the dynamic partition is full, the impact on disk performance is very small. Compared with the traditional solution that requires the "read-modify-write" operation to be performed every time there is data to be written, the write amplification in the garbage collection process described above can be almost ignored.

[0053] In summary, the solution of the present application can effectively reduce data migration caused by the write amplification problem in the shingled recording disk, thereby improving the performance of the shingled recording disk. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A flowchart of a storage management method according to a specific embodiment of the present invention;

[0056] Figure 2 A schematic diagram of storage management of a tile recording disk in a specific implementation manner of the present invention;

[0057] Figure 3 A schematic diagram of the structure of a storage management system provided by a specific implementation mode of the present invention;

[0058] Figure 4 A schematic diagram of the structure of a storage management device provided by a specific implementation mode of the present invention;

[0059] Figure 5 The present invention is a schematic diagram of the structure of a computer-readable storage medium. DETAILED DESCRIPTION

[0060] The core of the present invention is to provide a storage management method, system, device and storage medium, which can effectively reduce data migration caused by write amplification problems in shingled recording disks, thereby improving the performance of shingled recording disks.

[0061] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0062] Please refer to Figure 1 , Figure 1 The following is a flowchart of a storage management method provided in a specific embodiment of the present invention. The storage management method may include the following steps:

[0063] Step S101: Acquire a data block to be written, and determine the tile partition accessed by the data block.

[0064] Specifically, a write request can be received to obtain the data to be written, and then the data to be written can be divided into several data blocks. Since different data blocks have their own logical addresses, the tile partition accessed by the data block can be determined based on the logical address of each data block, that is, it can be determined into which tile partition of the hard disk the data block should be written.

[0065] In a specific implementation of the present invention, step S101 may specifically include: receiving data to be written; splitting the data to be written into individual data blocks to be written according to the size of a single sector, and for each data block, determining the tile partition accessed by the data block.

[0066] This implementation method takes into account that when writing data, it is usually necessary to write with the sector as the minimum unit, that is, when performing a write operation, the data of the entire sector will be modified and part of the content in the sector cannot be modified separately. In this regard, in this implementation method, after receiving the data to be written, it is necessary to split the data to be written into individual data blocks to be written according to the size of a single sector. For example, the size of a single sector is usually set to 512B, then after the data to be written is split into individual data blocks to be written according to the size of a single sector, each data block is 512B. For each data block, based on the logical address of the data block, the tile partition accessed by the data block can be determined. For example, based on the logical address of a data block, it can be determined that the tile partition accessed by the data block is tile partition 3, which means that the data block needs to be written to a sector of tile partition 3 later.

[0067] Step S102: When the accessed tile partition is a static partition, the data block is processed based on the static processing rule.

[0068] In the present application, the disk's tile partitions are divided into static partitions and dynamic partitions. The specific ratio can be set and adjusted according to actual needs and does not affect the implementation of the present invention. For example, in one scenario, 20% of all tile partitions are used as static partitions and 80% are used as dynamic partitions.

[0069] Static partitions and dynamic partitions have different data writing rules. If the tile partition accessed by a data block is a static partition, the data block needs to be processed based on the static processing rules.

[0070] The specific content of the static processing rules can be set and adjusted according to actual needs. For example, in one occasion, the default static processing rules can be used. For example, the tile partition accessed by a data block is a static partition, and the static partition includes 200 sectors, which are conveniently described as sectors 1 to 200 in the static partition. Since it is a static partition, the correspondence between the logical address and the sector is fixed, that is, according to the logical address of the data block, it can be determined which sector the data block needs to be written into. For example, it is determined that the data block should be written into sector 150 of the static partition. Then, all the data from sectors 150 to 200 of the static partition can be read and placed in the cache, and the data content of the data block is used in the cache to replace the original data content in sector 150. After the replacement is completed, the data from sectors 150 to 200 in the cache are written back to sectors 150 to 200 of the static partition in order. It can be seen that this default static processing rule has a more obvious write amplification. In this example, the reason why the data in sector 150 of the static partition cannot be modified directly based on the content of the data block is that when the shingle recording technology is used, the write head will not only overwrite the data of the current track, but also overwrite the data of the adjacent track. From the perspective of the sector, when writing a sector, the data of the adjacent sectors of the sector will be affected. In the above example, when writing the data in sector 150 of the static partition, the data of the sector numbered plus 1 will be affected, that is, the data in sector 151 of the static partition will be affected. Similarly, when writing the data in sector 151 of the static partition, the data in sector 152 of the static partition will be affected. Therefore, in the above example, it is necessary to first read out all the data from sectors 150 to 200 of the static partition, and after making changes in the cache, write them back to sectors 150 to 200 of the static partition in the order of sectors 150 to 200, resulting in write amplification.

[0071] In a specific implementation of the present invention, step S102 may specifically include:

[0072] Step 1: When the tile partition accessed is a static partition, it is determined whether there are unused sectors in the static partition, and the sector with the lowest number among the unused sectors corresponds to the logical address of the data block;

[0073] If yes, then execute step 2: write the data block to the lowest numbered sector among the unused sectors of the static partition;

[0074] If not, execute step 3: write the data block into the second cache;

[0075] Step 4: When the remaining space of the second cache is lower than a preset threshold, the data blocks in the second cache are written to the corresponding static partition.

[0076] This implementation method takes into account that if the default static processing rule exemplified above is adopted, the write amplification is more obvious for the static partition. Therefore, in this implementation method, another static processing rule is set to effectively reduce the write amplification of the static partition.

[0077] Specifically, in this implementation, when the tile partition accessed by a data block is a static partition, it is first determined whether there are unused sectors in the static partition. For a static partition, when a sector has never been written with data, the sector is an unused sector, and conversely, when a sector has currently been written with data, the sector is a used sector.

[0078] If there are unused sectors in the static partition, it means that the static partition is not fully used and there are sectors that have not been written with data. At this time, it is necessary to further determine whether the sector with the lowest number in the unused sectors corresponds to the logical address of the data block. If so, the data needs to be written in a sequential manner, that is, the data block is written to the sector with the lowest number in the unused sectors of the static partition.

[0079] Still taking the example of a static partition including sectors 1 to 200, it can be understood that when the hard disk is first used, no data is written in each static partition, so sectors 1 to 200 of the static partition are all unused sectors. When the tile partition accessed by a data block is the static partition, and based on the logical address of the data block, it is determined that the sector corresponding to the data block is sector 1 of the static partition, that is, the data block needs to be written to sector 1 of the static partition, it can be understood that the operation of step 2 above will be triggered in this example, and the data block can be written to sector 1 of the static partition at this time. Later, for example, the tile partition accessed by another data block is the static partition, but the sector corresponding to the data block is, for example, sector 50 of the static partition. It can be understood that, although there are unused sectors in the static partition at this time (sectors 2 to sector 200 are all unused sectors at this time), since the lowest-numbered sector among the unused sectors is sector 2, it does not correspond to the logical address of the data block (sector 50 corresponds to the logical address of the data block). Therefore, the operation that needs to be performed for the data block is step three, that is, writing the data block to the second cache.

[0080] For example, as the hard disk runs, the static partition is filled up, and at this time, there is no unused sector in the static partition. After the static partition is filled up, the tile partition accessed by a data block is the static partition, and at this time, the data block needs to be written to the second cache. Later, when the remaining space of the second cache is lower than the preset threshold, the data block in the second cache needs to be written to the corresponding static partition. It can be understood that for a certain static partition, the second cache may store one or more data blocks that need to be written to the static partition. The process of writing these data blocks to the static partition will also cause write amplification. However, since these data blocks are written to the static partition at one time, it is equivalent to effectively reducing the frequency of write amplification. That is, compared with the above, each time a data block is written, a write amplification occurs. This implementation method can effectively reduce the write amplification of the static partition.

[0081] When the remaining space of the second cache is lower than a preset threshold, all data blocks in the second cache may be written to the corresponding static partition, or some of the data blocks may be written to the corresponding static partition. For example, in a specific implementation of the present invention, the above step 4 may specifically include:

[0082] According to the preset weight calculation rules, the weight value of each static partition is determined;

[0083] Sort each static partition in the order of weight values ​​from small to large, and get a static partition sequence;

[0084] Select the i-th static partition in the static partition sequence; where i is a positive integer and its initial value is 1;

[0085] Read the data in the i-th static partition into the first cache as data to be modified;

[0086] Based on each data block accessing the i-th static partition stored in the second cache, the data to be modified is updated to obtain updated target data;

[0087] Write the target data into the i-th static partition, and delete each data block stored in the second cache for accessing the i-th static partition;

[0088] Determine whether the remaining space of the second cache is lower than a preset threshold;

[0089] If so, i is incremented by 1 and the operation of selecting the i-th static partition in the static partition sequence is returned.

[0090] This implementation takes into account that the reason why the data blocks in the second cache need to be written to the corresponding static partition is that as more and more data blocks are stored in the second cache, the remaining space of the second cache will be insufficient. Whenever the data blocks in the second cache are written to the corresponding static partition, write amplification will occur. Therefore, the write frequency and the amount of data written should be reduced as much as possible. In this implementation, when the remaining space of the second cache is lower than the preset threshold, only part of the data blocks in the second cache will be written to the corresponding static partition.

[0091] Specifically, in this implementation, the weight value of each static partition is determined according to the preset weight calculation rules, and then the static partitions are sorted in the order of weight values ​​from small to large, and the resulting sequence is called a static partition sequence. In other words, in this implementation, the specific selection of which static partitions to write data can be determined according to the weight value, which ensures the flexibility of the implementation of the present application.

[0092] After sorting the static partitions, i is a positive integer and its initial value is 1. We will take i equal to 1 as an example, that is, the first static partition in the static partition sequence is selected for the first time. This static partition is the static partition with the lowest weight value among the current static partitions. At this time, all data in the first static partition needs to be read into the first cache as the data to be modified. It can be seen that the data to be modified is the original old data.

[0093] After obtaining the data to be modified, it is necessary to update the data to be modified based on the data blocks stored in the second cache that need to access the first static partition. For example, there are 60 data blocks stored in the second cache, of which 3 data blocks access the first static partition, so the data to be modified needs to be modified based on these 3 data blocks. Specifically, taking one of the 3 data blocks as an example, for example, the logical address of the data block is logical address A0, and logical address A0 corresponds to sector 13 of the first static partition. Therefore, when updating the data to be modified, it is necessary to replace the data in sector 13 of the data to be modified with this data block.

[0094] Based on the data blocks stored in the second cache accessing the ith static partition, after updating the data to be modified, the update result is called the target data. Subsequently, the target data needs to be written back to the first static partition, that is, starting from the lowest numbered sector, and writing to at least the highest numbered sector, at which point the target data is written and needs to occupy all the storage space of the first static partition.

[0095] After writing the target data to the first static partition, it is also necessary to delete the data blocks stored in the second cache for accessing the first static partition. In the above example, the three data blocks stored in the second cache for accessing the first static partition are deleted to release space in the second cache.

[0096] Afterwards, it can be determined whether the remaining space of the second cache is lower than the preset threshold. If not, it means that the remaining space of the second cache is still insufficient, and it is necessary to continue to release the storage space of the second cache. Therefore, i+1 can be returned to perform the operation of selecting the i-th static partition in the static partition sequence. In the above example, it is necessary to select the second static partition in the static partition sequence, that is, to select the static partition with the second lowest weight value, so as to write each data block accessing the second static partition into the second static partition, and continue to release the storage space of the second cache. And so on, until it is determined that the remaining space of the second cache is not lower than the preset threshold, it means that the remaining space of the second cache is sufficient, so there is no need to continue to release the storage space of the second cache, and the data writing process can be ended.

[0097] In addition, it should be noted that, in this implementation, data writing for static partitions needs to be implemented based on the first cache and the second cache. As can be seen from the above description, the second cache serves to store each data block. Therefore, in practical applications, a non-volatile storage medium can usually be used to implement the second cache of this implementation. In addition, metadata can also be selected to be stored in the second cache. The metadata can effectively reflect the correspondence between the sectors and logical addresses in the dynamic partition. That is to say, for any sector in any dynamic partition, the logical address of the data block currently stored in the sector can be determined through the metadata.

[0098] The first cache is used for temporary data storage, that is, to store the data to be modified and update the data to be modified based on the corresponding data block. Therefore, a storage medium such as RAM (Random Access Memory) can usually be used as the first cache in this implementation, which has good read and write performance. Figure 2 , Figure 2 FIG. 1 is a schematic diagram of storage management of a tile recording disk in a specific implementation mode. Figure 2 In a specific implementation manner, a first cache and a second cache are set in the tile recording disk. In addition, Figure 2 The weight calculation function and data migration function shown in the figure correspond to the process of determining the weight value of each static partition in this implementation method and the process of writing the data block in the second cache to the corresponding static partition. The garbage collection function corresponds to the process of writing data to the dynamic partition described later.

[0099] In a specific implementation of the present invention, determining the weight value of each static partition according to a preset weight calculation rule may specifically include:

[0100] Determine the write access frequency of each static partition, and determine the weight value of each static partition based on the write access frequency of each static partition;

[0101] Among them, the write access frequency of the static partition is positively correlated with the weight value of the static partition.

[0102] As described above, the weight value of each static partition is determined according to the preset weight calculation rules, so that when selecting which static partitions to write data, it can be decided according to the weight value, which ensures the implementation flexibility of the present application scheme. This implementation method further considers that the weight value of the static partition can be determined according to the hotness or coldness of the data.

[0103] Specifically, this implementation takes into account that if most of the data stored in the static partition is cold data, it means that the probability of the data in the static partition being modified is low. After the data blocks for accessing the static partition stored in the current second cache are written to the static partition, the probability of subsequent data updates to the static partition is low, which is conducive to reducing the frequency of write amplification.

[0104] On the contrary, if the static partition stores mostly hot data, it means that even if the data blocks accessing the static partition stored in the current second cache are written to the static partition, there is a high probability that the static partition will need to be updated later. Therefore, the static partition storing cold data should be given priority for data writing.

[0105] In this implementation, the write access frequency of each static partition can be determined. The lower the write access frequency, the more cold data is stored in the static partition, so the weight value of the static partition is lower. Conversely, the higher the write access frequency, the higher the weight value. Therefore, in this implementation, the write access frequency of the static partition is positively correlated with the weight value of the static partition, which is conducive to reducing the frequency of write amplification.

[0106] In a specific implementation of the present invention, the weight value of each static partition is determined according to a preset weight calculation rule, including:

[0107] Determine the partition size of each static partition, and determine the weight value of each static partition based on the partition size of each static partition;

[0108] Among them, the partition size of the static partition is positively correlated with the weight value of the static partition.

[0109] This implementation method takes into account that in some cases, the size of each tile partition is consistent, but in some cases, different tile partitions may have different sizes. When a tile partition is a static partition and the partition size is small, it can be understood that when each data block accessing the static partition stored in the current second cache is written to the static partition, the amount of data amplified by the write will not be too high. On the contrary, if a tile partition is a static partition and the partition size is large, when each data block accessing the static partition stored in the current second cache is written to the static partition, the amount of data amplified by the write is high. Therefore, the larger the partition size, the larger the weight value should be, so that static partitions with smaller partitions are written first, which is conducive to reducing the impact of write amplification.

[0110] Further, in a specific embodiment of the present invention, the weight value of the static partition can be determined based on the partition size and the write access frequency at the same time. For example, the write access frequency of each static partition can be determined, and the partition size of each static partition can be determined. Then, for each static partition, the product of the partition size and the write access frequency of the static partition is determined, and the product is used as the weight value of the static partition. For another example, the weight value of the static partition can be determined in the manner of α×X1+β×X2, where X1 and X2 are the partition size and the write access frequency of the static partition, respectively, and α and β are the preset first coefficient and second coefficient. It can be seen that by adjusting the values ​​of the first coefficient and the second coefficient, the influence of the partition size and the write access frequency of the static partition on the weight value of the static partition can be flexibly adjusted, and the flexibility is high. In specific occasions, by setting appropriate first coefficients and second coefficients, the influence of write amplification when writing data blocks to static partitions can be minimized.

[0111] Step S103: When the accessed tile partition is a dynamic partition, determine whether the dynamic partition is full. If not, execute step S104; if yes, execute step S105.

[0112] When the tile partition accessed by a data block is a dynamic partition, it is necessary to determine whether the dynamic partition is full. If it is full, garbage collection needs to be performed on the tile partition. If it is not full, the data block can be written directly.

[0113] Step S104: Write the data block into an unused sector of the dynamic partition, the sector with the lowest number.

[0114] Since the dynamic partition is not full, there is at least one unused sector in the dynamic partition. At this time, the data block needs to be written in a sequential manner, that is, the data block is written into the unused sector of the dynamic partition, the sector with the lowest number.

[0115] For example, a data block points to a dynamic partition, and the dynamic partition includes 200 sectors, which are conveniently described as sectors 1 to 200 of the dynamic partition, and, for example, sectors 1 to 100 have been used, then data block 1 (logical address A1) will be written to the unused sectors of the dynamic partition, the lowest-numbered sectors, that is, it will be written to sector 101 of the dynamic partition. Later, for example, data block 2 (logical address B1) also needs to be written to the dynamic partition, then it can be understood that, since sector 101 has been used at this time, data block 2 (logical address B1) will be written to the unused sectors of the dynamic partition, the lowest-numbered sectors, that is, it will be written to sector 102 of the dynamic partition. For another example, data block 3 (logical address A1) also needs to be written to the dynamic partition. It can be understood that since sector 102 has been used at this time, data block 3 (logical address A1) will be written to the unused sector of the dynamic partition, the sector with the lowest number, that is, it will be written to sector 103 of the dynamic partition.

[0116] And from this example, it can be seen that the logical addresses of data block 3 and data block 1 are the same, but data block 3 does not overwrite data block 1, but is written to sector 103 and sector 101 respectively. That is to say, when executing step S104, it is written sequentially, and each time the data block is written to the unused sector of the dynamic partition, the lowest-numbered sector, until the dynamic partition is full, so that for the dynamic partition, when a data block needs to be written to the dynamic partition, the specific writing location is not uniquely determined by the logical address of the data block. In other words, for the sectors in the dynamic partition, the logical address of the data block written to the sector is not limited. Therefore, it is necessary to record the correspondence between the sectors and logical addresses in the current dynamic partition through metadata, and the metadata needs to be updated and maintained.

[0117] As described above, for the static partition above, the relationship between the sector and the logical address is binding, that is, for the data block that needs to be written into the static partition, the sector into which the data block needs to be written can be determined by the logical address of the data block.

[0118] Whenever a data block is written to a sector of a dynamic partition, the correspondence between the sector number (equivalent to the physical address) and the logical address can be recorded through metadata for subsequent garbage collection. For example, in the above example, for sector 101 of the dynamic partition, the logical address of the data block stored therein is logical address A1, and the logical address of the data block stored in sector 101 is logical address B1, and the logical address of the data block stored in sector 103 is logical address A1.

[0119] Step S105: performing garbage collection on the dynamic partition, and writing the data block to the lowest numbered sector in the unused sectors of the dynamic partition after garbage collection.

[0120] Among them, garbage collection means reading all valid data in the dynamic partition into the first cache, and starting with the sector with the lowest number, and writing the valid data back to the dynamic partition in sequence; for sectors in the dynamic partition, when the logical addresses corresponding to the data in at least two sectors are the same, only the data in the sector with the highest number is used as valid data.

[0121] When a dynamic partition is full, data blocks cannot be written any more, so garbage collection is required for the dynamic partition. When garbage collection is performed, all valid data in the dynamic partition needs to be determined based on metadata, then read into the first cache, and then written back to the dynamic partition in a sequential manner.

[0122] For example, when a dynamic partition is full, it can be determined based on metadata that the logical address of the data block stored in sector 101 is logical address A1, the logical address of the data block stored in sector 103 is logical address A1, the logical address of the data block stored in sector 180 is logical address A1, and the logical addresses of the data blocks stored in the remaining sectors are not logical address A1. It can be seen that the data block stored in sector 180 is the latest data of logical address A1. Therefore, in this example, the data in sector 180 is valid data, and the data in sectors 101 and 103 are not valid data. In other words, for each sector in the dynamic partition, if the logical addresses corresponding to the data in two or more sectors are the same, then among these sectors, only the data in the sector with the highest number is valid data.

[0123] In addition, it can be understood that, for each sector in the dynamic partition, when the logical address corresponding to the data in a sector is different from the logical address corresponding to the data in other sectors, the data in the sector is valid data. For example, the logical address of the data block stored in sector 40 is logical address A4, and the logical addresses of the data blocks in other sectors in the dynamic partition are not logical address A4, then the data in sector 40 is valid data.

[0124] After reading all the valid data in the dynamic partition to the first cache, it is necessary to use the lowest-numbered sector as the start of writing, and write the valid data back to the dynamic partition in a sequential manner. For the sake of ease of description, for example, in the dynamic partition, only the data in sectors 5, 7, and 50 are valid data. In actual applications, the amount of valid data is usually not so low. Since it is a sequential write, in this example, after reading the data of sectors 5, 7, and 50 into the first cache, it is necessary to write the data of sector 5 into sector 1 of the dynamic partition, and write the data of sector 7 into sector 2, and write the data of sector 50 into sector 3, and the garbage collection is completed.

[0125] In this example, after garbage collection, sectors 4 to 200 of the dynamic partition are considered to be unused sectors of the dynamic partition. That is to say, in this example, after garbage collection, the data block will be written to sector 4. If there is a subsequent access to the data block of the dynamic partition, it will be written to sectors 5, 6, 7, and so on of the dynamic partition in sequence.

[0126] In a specific embodiment of the present invention, it may also include:

[0127] receiving a read request, and determining a tile partition accessed by a data block to be read based on the read request;

[0128] When the tile partition accessed by the data block to be read is a static partition, the data block is read from the static partition and read feedback is performed;

[0129] When the tile partition accessed by the data block to be read is a dynamic partition, the data block is read from the dynamic partition and read feedback is performed.

[0130] In this implementation, after receiving a read request, each data block to be read can be determined, for example, the data to be read can be divided into each data block of 512B to be read. For the data block to be read, if the accessed tile partition is a static partition, it is necessary to read the data block from the corresponding sector based on the metadata and the logical address of the data block and then perform read feedback. If the accessed tile partition is a dynamic partition, it is also possible to read the data block from the corresponding sector based on the metadata and the logical address of the data block and then perform read feedback.

[0131] After applying the technical solution provided by the embodiment of the present invention, obtaining the data block to be written and determining the tile partition accessed by the data block, it is necessary to perform corresponding operations according to the difference between static partition and dynamic partition. If it is a static partition, the data block can be processed based on the static processing rules. If it is a dynamic partition, it is necessary to determine whether the dynamic partition is full. If it is not full, the data block can be directly written into the unused sector of the dynamic partition, the sector with the lowest number. It can be seen that since the data block is directly written into the dynamic partition, there is no write amplification in this operation, and it should be noted that when performing a write operation, it will affect the sectors with larger sector numbers in the partition but will not affect the sectors with smaller sector numbers. Since the location of the write at this time is the sector with the lowest number in the unused sectors of the dynamic partition, this write operation will not affect the current existing data in the dynamic partition. If the tile partition accessed is a dynamic partition and is full, it means that the current sectors of the dynamic partition are all used, so it is necessary to perform garbage collection on the dynamic partition. Garbage collection means reading all valid data in the dynamic partition to the first cache, and then starting writing with the lowest-numbered sector, and writing the valid data back to the dynamic partition in a sequential manner. In other words, through garbage collection, all valid data in the dynamic partition is retained, and the rest of the data is discarded, so that the dynamic partition has unused sectors again, and after garbage collection, these valid data occupy the low-numbered sectors in the dynamic partition. Therefore, after garbage collection is completed, data blocks can continue to be written to the unused sectors of the dynamic partition, the lowest-numbered sectors. This write operation does not have write amplification and does not affect the valid data in the low-numbered sectors.

[0132] It should also be noted that the garbage collection process described above has write amplification because it needs to read all valid data in the dynamic partition to the first cache and write the valid data back to the dynamic partition. However, since garbage collection only needs to be performed when the dynamic partition is full, the impact on disk performance is very small. Compared with the traditional solution that requires the "read-modify-write" operation to be performed every time there is data to be written, the write amplification in the garbage collection process described above can be almost ignored.

[0133] In summary, the solution of the present application can effectively reduce data migration caused by the write amplification problem in the shingled recording disk, thereby improving the performance of the shingled recording disk.

[0134] Corresponding to the above method embodiment, the embodiment of the present invention further provides a storage management system, which can be referred to in correspondence with the above.

[0135] See also Figure 3 FIG. 1 is a schematic diagram of a storage management system in the present invention, comprising:

[0136] A tile partition determination module 301 is used to obtain a data block to be written and determine a tile partition accessed by the data block;

[0137] A static partition processing module 302, used for processing a data block based on a static processing rule when the accessed tile partition is a static partition;

[0138] A dynamic partition determination module 303 is used to determine whether the dynamic partition is full when the accessed tile partition is a dynamic partition;

[0139] If not, the write module 304 is triggered, and the write module 304 is used to write the data block into an unused sector of the dynamic partition, the sector with the lowest number;

[0140] If yes, the garbage collection module 305 is triggered, and the garbage collection module 305 is used to perform garbage collection on the dynamic partition and write the data block to the lowest numbered sector in the unused sectors of the dynamic partition after garbage collection;

[0141] Among them, garbage collection means reading all valid data in the dynamic partition into the first cache, and starting with the sector with the lowest number, and writing the valid data back to the dynamic partition in sequence; for sectors in the dynamic partition, when the logical addresses corresponding to the data in at least two sectors are the same, only the data in the sector with the highest number is used as valid data.

[0142] In a specific implementation of the present invention, the tile partition determination module 301 is specifically used to:

[0143] Receive data to be written;

[0144] According to the size of a single sector, the data to be written is divided into various data blocks to be written, and for each data block, the tile partition accessed by the data block is determined.

[0145] In a specific implementation of the present invention, a read request processing module is also included, which is used to:

[0146] receiving a read request, and determining a tile partition accessed by a data block to be read based on the read request;

[0147] When the tile partition accessed by the data block to be read is a static partition, the data block is read from the static partition and read feedback is performed;

[0148] When the tile partition accessed by the data block to be read is a dynamic partition, the data block is read from the dynamic partition and read feedback is performed.

[0149] In a specific implementation of the present invention, the static partition processing module 302 includes:

[0150] A static partition determination unit, used for determining whether there are unused sectors in the static partition when the accessed tile partition is a static partition, and the sector with the lowest number in the unused sectors corresponds to the logical address of the data block;

[0151] If yes, the static partition writing unit is triggered to write the data block to the lowest numbered sector among the unused sectors of the static partition;

[0152] If not, triggering a cache write unit to write the data block into the second cache;

[0153] The cache write unit is used to write the data blocks in the second cache to the corresponding static partition when the remaining space of the second cache is lower than a preset threshold.

[0154] In a specific embodiment of the present invention, the cache disk unit is specifically used for:

[0155] According to the preset weight calculation rules, the weight value of each static partition is determined;

[0156] Sort each static partition in the order of weight values ​​from small to large, and get a static partition sequence;

[0157] Select the i-th static partition in the static partition sequence; where i is a positive integer and its initial value is 1;

[0158] Read the data in the i-th static partition into the first cache as data to be modified;

[0159] Based on each data block accessing the i-th static partition stored in the second cache, the data to be modified is updated to obtain updated target data;

[0160] Write the target data into the i-th static partition, and delete each data block stored in the second cache for accessing the i-th static partition;

[0161] Determine whether the remaining space of the second cache is lower than a preset threshold;

[0162] If so, i is incremented by 1 and the operation of selecting the i-th static partition in the static partition sequence is returned.

[0163] In a specific implementation of the present invention, the weight value of each static partition is determined according to a preset weight calculation rule, including:

[0164] Determine the write access frequency of each static partition, and determine the weight value of each static partition based on the write access frequency of each static partition;

[0165] Among them, the write access frequency of the static partition is positively correlated with the weight value of the static partition.

[0166] In a specific implementation of the present invention, the weight value of each static partition is determined according to a preset weight calculation rule, including:

[0167] Determine the partition size of each static partition, and determine the weight value of each static partition based on the partition size of each static partition;

[0168] Among them, the partition size of the static partition is positively correlated with the weight value of the static partition.

[0169] Corresponding to the above method and system embodiments, the embodiments of the present invention further provide a storage management device, a computer-readable storage medium, and a computer program product, which can be referred to in correspondence with the above.

[0170] See also Figure 4 As shown, the device may include:

[0171] Memory 401, used for storing computer programs;

[0172] The processor 402 is used to execute a computer program to implement the steps of the storage management method in any of the above embodiments.

[0173] The computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the storage management method in any of the above embodiments are implemented.

[0174] See also Figure 5 The computer readable storage medium 50 stores a computer program 51, and when the computer program 51 is executed by the processor, the steps of the storage management method in any of the above embodiments are implemented. The computer readable storage medium 50 mentioned here includes RAM (Random Access Memory), memory, ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), register, hard disk, removable disk, or any other form of storage medium known in the technical field.

[0175] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0176] Professionals may further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention, and the description of the above embodiments is only used to help understand the technical solution and its core idea of ​​the present invention. It should be pointed out that for ordinary technicians in the technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A storage management method, characterized in that: include: Obtain a data block to be written, and determine a tile partition accessed by the data block; When the accessed tile partition is a static partition, processing the data block based on a static processing rule; When the accessed tile partition is a dynamic partition, determining whether the dynamic partition is full; If not, writing the data block into an unused sector of the dynamic partition, the lowest numbered sector; If yes, performing garbage collection on the dynamic partition, and writing the data block to the lowest numbered sector in the unused sectors of the dynamic partition after garbage collection; Among them, the garbage collection means reading all valid data in the dynamic partition into the first cache, and starting with the sector with the lowest number, and writing the valid data back to the dynamic partition in sequence; for the sectors in the dynamic partition, when the logical addresses corresponding to the data in at least two sectors are the same, only the data in the sector with the highest number is used as valid data.

2. The storage management method according to claim 1, characterized in that: Obtaining a data block to be written and determining a tile partition accessed by the data block includes: Receive data to be written; According to the size of a single sector, the data to be written is divided into various data blocks to be written, and for each of the data blocks, a tile partition accessed by the data block is determined.

3. The storage management method according to claim 1, characterized in that: Also includes: receiving a read request, and determining a tile partition accessed by a data block to be read based on the read request; When the tile partition accessed by the data block to be read is a static partition, reading the data block from the static partition and performing read feedback; When the tile partition accessed by the data block to be read is a dynamic partition, the data block is read from the dynamic partition and read feedback is performed.

4. The storage management method according to any one of claims 1 to 3, characterized in that: When the accessed tile partition is a static partition, processing the data block based on a static processing rule includes: When the accessed tile partition is a static partition, determining whether there are unused sectors in the static partition, and the lowest-numbered sector in the unused sectors corresponds to the logical address of the data block; If yes, writing the data block to the lowest numbered sector among the unused sectors of the static partition; If not, writing the data block into the second cache; When the remaining space of the second cache is lower than a preset threshold, the data blocks in the second cache are written to the corresponding static partition.

5. The storage management method according to claim 4, characterized in that: When the remaining space of the second cache is lower than a preset threshold, writing the data block in the second cache to the corresponding static partition includes: According to the preset weight calculation rules, the weight value of each static partition is determined; Sorting the static partitions in order of weight values ​​from small to large to obtain a static partition sequence; Selecting the i-th static partition in the static partition sequence; wherein i is a positive integer and its initial value is 1; Reading the data in the i-th static partition into the first cache as data to be modified; Based on each data block accessing the i-th static partition stored in the second cache, the data to be modified is updated to obtain updated target data; Writing the target data into the i-th static partition, and deleting each data block stored in the second cache for accessing the i-th static partition; Determining whether the remaining space of the second cache is lower than a preset threshold; If yes, then i is incremented by 1 and the operation of selecting the i-th static partition in the static partition sequence is returned to be executed.

6. The storage management method according to claim 5, characterized in that: According to the preset weight calculation rules, the weight value of each static partition is determined, including: Determine the write access frequency of each static partition, and determine the weight value of each static partition based on the write access frequency of each static partition; The write access frequency of the static partition is positively correlated with the weight value of the static partition.

7. The storage management method according to claim 5, characterized in that: According to the preset weight calculation rules, the weight value of each static partition is determined, including: Determine the partition size of each static partition, and determine the weight value of each static partition based on the partition size of each static partition; The partition size of the static partition is positively correlated with the weight value of the static partition.

8. A storage management system, characterized in that: include: A tile partition determination module, used to obtain a data block to be written and determine a tile partition accessed by the data block; A static partition processing module, used for processing the data block based on a static processing rule when the accessed tile partition is a static partition; A dynamic partition judgment module, used for judging whether the dynamic partition is full when the accessed tile partition is a dynamic partition; If not, triggering a write module, the write module is used to write the data block into an unused sector of the dynamic partition, the sector with the lowest number; If yes, triggering a garbage collection module, the garbage collection module is used to perform garbage collection on the dynamic partition, and write the data block to the lowest numbered sector in the unused sectors of the dynamic partition after garbage collection; Among them, the garbage collection means reading all valid data in the dynamic partition into the first cache, and starting with the sector with the lowest number, and writing the valid data back to the dynamic partition in sequence; for the sectors in the dynamic partition, when the logical addresses corresponding to the data in at least two sectors are the same, only the data in the sector with the highest number is used as valid data.

9. A storage management device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the storage management method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the storage management method according to any one of claims 1 to 7 are implemented.

Images