A storage management method, system, device and storage medium

Abstract

The application discloses a storage management method, system, device and storage medium, and applies to the technical field of storage, and comprises the following steps: acquiring a data block to be written, and determining a tile partition accessed by the data block; when the accessed tile partition is a static partition, processing the data block based on a static processing rule; when the accessed tile partition is a dynamic partition, judging whether the dynamic partition is full; if not, writing the data block into an unused sector of the dynamic partition, a sector with the lowest number; if yes, performing garbage collection on the dynamic partition, and writing the data block into an unused sector of the dynamic partition after the garbage collection, a sector with the lowest number. According to the scheme, data migration caused by the write amplification problem in the tile recording disk can be effectively reduced, and the performance of the tile recording disk is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a storage management method, system, device and storage medium. BACKGROUND

[0002] Shingled recording technology is based on the traditional perpendicular magnetic recording technology, which writes data to narrower tracks through a specially designed write head, ensuring track thermal stability while allowing the disk to accommodate more data, thereby increasing the storage capacity of the disk. Because the width of the write head is greater than the width of the track, when the write head writes data to the track, the tracks appear to partially overlap like roof tiles, hence the name shingled recording technology.

[0003] Because the write head covers the data of adjacent tracks, shingled recording technology does not support in-place updating of data, so every time data on a track is updated, the current solution requires a "read-change-write" operation, which causes write amplification of the shingled recording disk. The more frequent the data updates, the more serious the write amplification problem, the more data needs to be migrated, and the worse the performance of the disk.

[0004] In summary, how to effectively reduce the data migration caused by the write amplification problem in the shingled recording disk and improve the performance of the shingled recording disk is a technical problem that needs to be solved by the technical personnel in the field. SUMMARY

[0005] The purpose of the present application is to provide a storage management method, system, device and storage medium to effectively reduce the data migration caused by the write amplification problem in the shingled recording disk and improve the performance of the shingled recording disk.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a storage management method, comprising:

[0008] Obtaining a data block to be written and determining a shingled partition accessed by the data block;

[0009] When the accessed shingled partition is a static partition, processing the data block based on a static processing rule;

[0010] When the accessed shingled partition is a dynamic partition, determining whether the dynamic partition is full;

[0011] If not, write the data block to an unused sector of the dynamic partition, the lowest numbered sector;

[0012] If yes, garbage collection is performed on the dynamic partition, and after garbage collection, the data block is written into the unused sector of the dynamic partition, the sector with the lowest number;

[0013] The garbage collection means reading all valid data in the dynamic partition into a first cache, and writing the valid data back to the dynamic partition in a sequential writing manner, with the sector with the lowest number as the writing start. For the sectors in the dynamic partition, when the data in at least two sectors correspond to the same logical address, only the data in the sector with the highest number is regarded as valid data.

[0014] On the other hand, a data block to be written is obtained, and a tile partition accessed by the data block is determined, including:

[0015] A data to be written is received.

[0016] The data to be written is split into data blocks to be written according to the size of a single sector, and for each data block, a tile partition accessed by the data block is determined.

[0017] On the other hand, it further includes:

[0018] A read request is received, and a tile partition accessed by a data block to be read is determined based on the read request;

[0019] When the tile partition accessed by the data block to be read is a static partition, the data block is read from the static partition and read feedback is performed;

[0020] When the tile partition accessed by the data block to be read is a dynamic partition, the data block is read from the dynamic partition and read feedback is performed.

[0021] On the other hand, when the accessed tile partition is a static partition, it is determined whether the static partition has an unused sector, and the sector with the lowest number in the unused sector corresponds to the logical address of the data block;

[0022] If yes, the data block is written into the unused sector of the static partition, the sector with the lowest number;

[0023] If no, the data block is written into a second cache;

[0024] When the remaining space of the second cache is lower than a preset threshold, the data block in the second cache is written into the corresponding static partition.

[0025] On the other hand, when the remaining space of the second cache is lower than a preset threshold, the data block in the second cache is written into the corresponding static partition, including:

[0026] According to a preset weight calculation rule, a weight value of each static partition is determined;

[0027] According to an order from small to large of the weight values, the static partitions are sorted to obtain a static partition sequence;

[0028] The i-th static partition in the static partition sequence is selected; wherein i is a positive integer and the initial value is 1;

[0029] Data in the i-th static partition is read to the first cache as to-be-modified data;

[0030] Based on each data block in the second cache for accessing the i-th static partition, the to-be-modified data is updated to obtain target data after updating;

[0031] The target data is written into the i-th static partition, and each data block in the second cache for accessing the i-th static partition is deleted;

[0032] It is judged whether the remaining space of the second cache is lower than a preset threshold;

[0033] If yes, i is added by 1 and the operation of selecting the i-th static partition in the static partition sequence is executed.

[0034] On the other hand, according to a preset weight calculation rule, a weight value of each static partition is determined, comprising:

[0035] The write access frequency of each static partition is determined, and the weight value of each static partition is determined based on the write access frequency of each static partition;

[0036] The write access frequency of the static partition is positively correlated with the weight value of the static partition.

[0037] On the other hand, according to a preset weight calculation rule, a weight value of each static partition is determined, comprising:

[0038] The partition size of each static partition is determined, and the weight value of each static partition is determined based on the partition size of each static partition;

[0039] The partition size of the static partition is positively correlated with the weight value of the static partition.

[0040] In a second aspect, the present application provides a storage management system, comprising:

[0041] A tile partition determination module is configured to obtain a data block to be written and determine a tile partition accessed by the data block;

[0042] a static partition processing module, configured to process the data block based on a static processing rule when the accessed tile partition is a static partition;

[0043] a dynamic partition judging module, configured to judge whether the dynamic partition is full when the accessed tile partition is a dynamic partition;

[0044] if not, triggering a writing module, the writing module being configured to write the data block into an unused sector of the dynamic partition, the sector being the lowest numbered sector;

[0045] if yes, triggering a garbage collection module, the garbage collection module being configured to perform garbage collection on the dynamic partition and write the data block into an unused sector of the dynamic partition after the garbage collection, the sector being the lowest numbered sector;

[0046] wherein the garbage collection means reading all valid data in the dynamic partition into a first cache and writing the valid data back to the dynamic partition in a sequential writing manner with the lowest numbered sector as the starting writing sector; for sectors in the dynamic partition, when the data in at least two sectors correspond to the same logical address, only the data in the highest numbered sector is regarded as valid data.

[0047] In a third aspect, the present application provides a storage management device, comprising:

[0048] a memory, configured to store a computer program;

[0049] a processor, configured to execute the computer program to implement the steps of the storage management method as described above.

[0050] In a fourth aspect, the present application provides a computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the storage management method as described above.

[0051] After the data block to be written is acquired and the tile partition accessed by the data block is determined, the corresponding operation is performed according to the difference between the static partition and the dynamic partition. If it is a static partition, the data block can be processed based on the static processing rule. If it is a dynamic partition, it is determined whether the dynamic partition is full. If it is not full, the data block can be directly written into the unused sector of the dynamic partition, that is, the sector with the lowest number. It can be seen that since the data block is directly written into the dynamic partition, the write amplification does not exist in this operation. It needs to be explained that when the write operation is performed, the sector with a larger sector number in the partition is affected and the sector with a smaller sector number is not affected. Since the position of the write operation is the unused sector with the lowest number in the dynamic partition, the current data in the dynamic partition is not affected by the write operation. If the tile partition accessed is a dynamic partition and is full, it means that all sectors in the dynamic partition are used, so garbage collection needs to be performed on the dynamic partition. The garbage collection means that all valid data in the dynamic partition is read into the first cache, and then the valid data is written back to the dynamic partition in the order of the lowest sector as the starting write position. That is, through the garbage collection, all valid data in the dynamic partition is retained, and the remaining data is discarded, so that the dynamic partition has unused sectors again, and after the garbage collection, the valid data occupies the sectors with low numbers in the dynamic partition. Therefore, after the garbage collection is completed, the data block can be continuously written into the unused sector of the dynamic partition, that is, the sector with the lowest number, and the write operation does not exist in the write amplification and does not affect the valid data in the sector with a low number.

[0052] In addition, it needs to be explained that the garbage collection process needs to read all valid data in the dynamic partition into the first cache and write the valid data back to the dynamic partition, so the write amplification exists in the garbage collection process. However, since the garbage collection is only performed when the dynamic partition is full, the influence on the disk performance is very small. Compared with the operation of "read-change-write" in the traditional scheme, the write amplification in the garbage collection process can be almost ignored.

[0053] In summary, the scheme of the present application can effectively reduce the data migration caused by the write amplification in the tile recording disk, thereby improving the performance of the tile recording disk. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to make the technical scheme of the present application or prior art clearer, the accompanying drawings needed in the description of the embodiments or prior art will be briefly introduced. Obviously, the accompanying drawings described are only some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative work on the basis of the embodiments of the present application shall fall within the scope of the present application.

[0055] Figure 1 An implementation flowchart of the storage management method provided by the embodiment of the present application can include the following steps:

[0056] Figure 2 A storage management schematic diagram of the tile recording disk in the embodiment of the present application is shown in FIG. 1.

[0057] Figure 3 A structure schematic diagram of the storage management system provided by the embodiment of the present application is shown in FIG. 2.

[0058] Figure 4 A structure schematic diagram of the storage management device provided by the embodiment of the present application is shown in FIG. 3.

[0059] Figure 5 A structure schematic diagram of the computer readable storage medium of the present application is shown in FIG. 4. EMBODIMENT

[0060] The core of the present application is to provide a storage management method, system, device and storage medium, which can effectively reduce the data migration caused by the write amplification problem in the tile recording disk, thereby improving the performance of the tile recording disk.

[0061] In order to make the technical scheme of the present application, the following will be further described in detail with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the scope of the present application.

[0062] Please refer to Figure 1 , Figure 1 An implementation flowchart of the storage management method provided by the embodiment of the present application can include the following steps:

[0063] Step S101: acquiring a data block to be written, and determining a tile partition accessed by the data block.

[0064] Specifically, a write request can be received to obtain to-be-written data, and then the to-be-written data is divided into a plurality of data blocks. Since different data blocks have respective logical addresses, the tile partition accessed by each data block can be determined based on the respective logical address of each data block, that is, it can be determined which tile partition the data block should be written into.

[0065] In a specific embodiment of the present application, step S101 can specifically include: receiving to-be-written data; splitting the to-be-written data into respective data blocks to be written according to the size of a single sector, and for each data block, determining the tile partition accessed by the data block.

[0066] This embodiment takes into account that when data is written, it is usually necessary to write in the smallest unit of a sector, that is, when a write operation is performed, the data of the entire sector is modified and part of the content in the sector cannot be modified individually. In this regard, in this embodiment, after receiving the to-be-written data, the to-be-written data needs to be split into respective data blocks to be written according to the size of a single sector. For example, the size of a single sector is usually set to 512B, so after splitting the to-be-written data into respective data blocks to be written according to the size of a single sector, each data block is 512B. For each data block, based on the logical address of the data block, the tile partition accessed by the data block can be determined, for example, based on the logical address of a certain data block, it can be determined that the tile partition accessed by the data block is tile partition 3, which means that the data block needs to be written into a certain sector of tile partition 3 subsequently.

[0067] Step S102: when the accessed tile partition is a static partition, processing the data block based on a static processing rule.

[0068] In the present application, the tile partitions of the disk are divided into static partitions and dynamic partitions, and the specific proportion can be set and adjusted according to actual needs without affecting the implementation of the present application, for example, in one occasion, 20% of all tile partitions are static partitions and 80% are dynamic partitions.

[0069] The static partition and the dynamic partition have different data writing rules. If the tile partition accessed by a certain data block is a static partition, the data block needs to be processed based on a static processing rule.

[0070] The specific content of the static processing rule can be set and adjusted according to actual needs. For example, in one occasion, a default static processing rule can be used. For example, the tile partition accessed by a certain data block is a static partition, and the static partition includes 200 sectors. For the sake of description, the sectors are referred to as sector 1 to sector 200 in the static partition. Since it is a static partition, the correspondence between the logical address and the sector is fixed, that is, according to the logical address of the data block, it can be determined which sector the data block needs to be written into. For example, it is determined that the data block should be written into sector 150 of the static partition. Then, all the data of sector 150 to sector 200 of the static partition can be read and placed in the cache. The data content of the data block is used in the cache to replace the data content in the original sector 150. After the replacement is completed, the data of sector 150 to sector 200 in the cache is sequentially written back to sector 150 to sector 200 of the static partition in order. It can be seen that the default static processing rule has a relatively obvious write amplification. In the example, the data in sector 150 in the static partition cannot be directly modified based on the content of the data block because, when the tile recording technology is used, the write head will not only cover the data of the current track, but also cover the data of the adjacent track. From the perspective of the sector, when a certain sector is written, the data of the adjacent sector of the sector will be affected. In the above example, when the data in sector 150 in the static partition is written, the data in sector 151, which is numbered plus 1, will be affected, that is, the data in sector 151 in the static partition will be affected. Similarly, when the data in sector 151 in the static partition is written, the data in sector 152 in the static partition will be affected. Therefore, in the above example, all the data of sector 150 to sector 200 of the static partition needs to be read out first, and then changed in the cache. After that, the data of sector 150 to sector 200 is sequentially written back to sector 150 to sector 200 of the static partition in order, resulting in a write amplification.

[0071] In one specific embodiment of the present application, step S102 can specifically include:

[0072] Step one: when the accessed tile partition is a static partition, it is judged whether the static partition has unused sectors, and whether the lowest numbered sector in the unused sectors corresponds to the logical address of the data block;

[0073] If yes, step two is executed: the data block is written into the lowest numbered sector of the unused sectors of the static partition.

[0074] If no, step three is executed: the data block is written into the second cache.

[0075] Step four: when the remaining space of the second cache is lower than a preset threshold, the data block in the second cache is written into the corresponding static partition.

[0076] The embodiment considers that if the above-mentioned default static processing rule is adopted, the write amplification is more obvious for the static partition, and another static processing rule is set in the embodiment to effectively reduce the write amplification of the static partition.

[0077] Specifically, in the embodiment, when the tile partition accessed by a data block is a static partition, it is first determined whether there is an unused sector in the static partition. For the static partition, when a sector has never been written with data, the sector is an unused sector, and otherwise, when a sector has been written with data, the sector is a used sector.

[0078] If there is an unused sector in the static partition, it indicates that the static partition is not full and there is a sector that has never been written with data, and it is further determined whether the sector with the lowest number in the unused sector corresponds to the logical address of the data block. If yes, the data is written in sequence, that is, the data block is written into the unused sector with the lowest number in the static partition.

[0079] Still taking an example of a static partition including sectors 1 to 200, it can be understood that when the hard disk is first used, no data is written in each static partition, and therefore sectors 1 to 200 of the static partition are all unused sectors. When the tile partition accessed by a data block is the static partition, and based on the logical address of the data block, it is determined that the sector corresponding to the data block is sector 1 of the static partition, that is, the data block needs to be written into sector 1 of the static partition, it can be understood that the operation of step two in the above-mentioned example is triggered, and the data block can be written into sector 1 of the static partition. Subsequently, for example, another data block accesses the tile partition of the static partition, but the sector corresponding to the data block is, for example, sector 50 of the static partition. It can be understood that although there is an unused sector (sectors 2 to 200 are all unused sectors) in the static partition at this time, since the sector with the lowest number in the unused sector is sector 2, which does not correspond to the logical address of the data block (sector 50 corresponds to the logical address of the data block), the operation of step three needs to be performed for the data block, that is, the data block is written into the second cache.

[0080] For example, as the static partition is filled up with the running of the hard disk, there is no unused sector in the static partition. After the static partition is filled up, if a data block accesses the tile partition of the static partition, the data block needs to be written into the second cache. When the remaining space of the second cache is lower than the preset threshold, the data blocks in the second cache need to be written into the corresponding static partition. It can be understood that for a certain static partition, there can be one or more data blocks in the second cache which need to be written into the static partition. The process of writing these data blocks into the static partition will also have the situation of write amplification, but since these data blocks are written into the static partition at one time, the frequency of write amplification is effectively reduced, that is, compared with the above situation that write amplification occurs every time a data block is written, this implementation manner can effectively reduce the write amplification of the static partition.

[0081] When the remaining space of the second cache is lower than the preset threshold, all data blocks in the second cache can be selected to be written into the corresponding static partition, or part of the data blocks in the second cache can be selected to be written into the corresponding static partition. For example, in a specific embodiment of the present application, the above step four can specifically include:

[0082] According to the preset weight calculation rule, the weight value of each static partition is determined;

[0083] According to the order of the weight values from small to large, the static partitions are sorted to obtain a static partition sequence;

[0084] The i-th static partition in the static partition sequence is selected; wherein i is a positive integer and the initial value is 1;

[0085] The data in the i-th static partition is read into the first cache as the to-be-modified data;

[0086] Based on the data blocks in the second cache which access the i-th static partition, the to-be-modified data is updated to obtain target data after updating;

[0087] The target data is written into the i-th static partition, and the data blocks in the second cache which access the i-th static partition are deleted;

[0088] It is judged whether the remaining space of the second cache is lower than the preset threshold;

[0089] If yes, i is increased by 1 and the operation of selecting the i-th static partition in the static partition sequence is performed.

[0090] The implementation considers that the data blocks in the second cache need to be written to the corresponding static partition because the remaining space of the second cache is insufficient due to more and more data blocks stored in the second cache, and write amplification occurs when the data blocks in the second cache are written to the corresponding static partition, so the write frequency and the amount of data written should be reduced as much as possible. In this implementation, only part of the data blocks in the second cache are written to the corresponding static partition when the remaining space of the second cache is lower than a preset threshold.

[0091] Specifically, in this implementation, the weight value of each static partition is determined according to a preset weight calculation rule, and then the static partitions are sorted in ascending order of the weight values to obtain a sequence, which is referred to as a static partition sequence. That is, in this implementation, which static partitions are selected for data writing can be determined according to the weight values, which guarantees the flexibility of the implementation of the application scheme.

[0092] After the static partitions are sorted, i is a positive integer and the initial value is 1, and i is taken as an example, that is, the first selected is the first static partition in the static partition sequence, which is the static partition with the lowest weight value among the current static partitions. At this time, all data in the first static partition need to be read to the first cache as modified data, and it can be seen that the modified data is the original old data.

[0093] After obtaining the modified data, the modified data needs to be updated based on the data blocks in the second cache that access the first static partition, for example, there are 60 data blocks stored in the second cache, and 3 data blocks access the first static partition, so the modified data needs to be modified based on the 3 data blocks. Specifically, one of the 3 data blocks is taken as an example, for example, the logical address of the data block is logical address A0, and the logical address A0 corresponds to sector 13 of the first static partition, so when updating the modified data, the data of sector 13 in the modified data needs to be replaced with the data block.

[0094] After updating the modified data based on the data blocks in the second cache that access the i-th static partition, the update result is referred to as target data. Subsequently, the target data needs to be written back to the first static partition, that is, writing from the lowest numbered sector to at least the highest numbered sector, at which time the target data is written and needs to occupy the entire storage space of the first static partition.

[0095] After the target data is written into the first static partition, the data blocks stored in the second cache for accessing the first static partition need to be deleted, in the above example, the three data blocks stored in the second cache for accessing the first static partition need to be deleted, so as to release the space of the second cache.

[0096] Then it can be judged whether the remaining space of the second cache is lower than the preset threshold, if not, it means that the remaining space of the second cache is still insufficient, and the storage space release of the second cache needs to be continued, therefore, i+1 can be returned to execute the operation of selecting the i-th static partition in the static partition sequence. In the above example, the second static partition in the static partition sequence needs to be selected, that is, the static partition with the second lowest weight value needs to be selected, so as to write the data blocks for accessing the second static partition into the second static partition, and continue to release the storage space of the second cache. In this way, until it is judged that the remaining space of the second cache is not lower than the preset threshold, it means that the remaining space of the second cache is sufficient, therefore, the storage space release of the second cache does not need to be continued, and the current data writing process can be ended.

[0097] In addition, it needs to be explained that in this embodiment, the data writing for the static partition needs to be realized based on the first cache and the second cache, as described above, the second cache plays a role in storing the data blocks, therefore, in actual application, the second cache of this embodiment can be realized by using a non-volatile storage medium, in addition, the metadata can also be stored in the second cache, the metadata can effectively reflect the correspondence between the sectors in the dynamic partition and the logical addresses, that is, for any sector in any dynamic partition, through the metadata, the logical address of the data block currently stored in the sector can be determined.

[0098] The first cache is used for temporary data storage, that is, storing the to-be-modified data and updating the to-be-modified data based on the corresponding data block, therefore, in general, a storage medium such as RAM (Random Access Memory) can be used as the first cache in this embodiment, which has good read-write performance. For details, please refer to Figure 2 , Figure 2 is a storage management schematic diagram of a specific embodiment of a tile recording disk, in Figure 2 the specific embodiment, the first cache and the second cache are arranged in the tile recording disk. In addition, Figure 2 the functions of weight calculation and data migration shown in the above embodiment correspond to the process of determining the weight value of each static partition in this embodiment and the process of writing the data blocks in the second cache into the corresponding static partition respectively. The function of garbage collection corresponds to the process of data writing for the dynamic partition in the following.

[0099] In one specific embodiment of the present application, the weight value of each static partition is determined according to a preset weight calculation rule, which can specifically include:

[0100] The write access frequency of each static partition is determined, and the weight value of each static partition is determined based on the write access frequency of each static partition.

[0101] The write access frequency of the static partition is positively correlated with the weight value of the static partition.

[0102] As described above, the weight value of each static partition is determined according to a preset weight calculation rule, so that when specific static partitions are selected for data writing, the weight value can be used to make a decision, thereby ensuring the flexibility of the implementation of the present application. This embodiment further considers that the weight value of the static partition can be determined according to the data hotness.

[0103] Specifically, in this embodiment, if most of the data stored in the static partition is cold data, it means that the probability of modifying the data in the static partition is low. After writing each data block in the static partition stored in the second cache to the static partition, the probability of subsequently updating the data in the static partition is low, which is conducive to reducing the frequency of write amplification.

[0104] On the contrary, if most of the data stored in the static partition is hot data, it means that even if each data block in the static partition stored in the second cache is written to the static partition, there is a high probability that the data in the static partition will be updated subsequently. Therefore, the static partition storing cold data should be preferentially selected for data writing.

[0105] In this embodiment, the write access frequency of each static partition can be determined. The lower the write access frequency, the more cold data is stored in the static partition, and therefore the lower the weight value of the static partition. On the contrary, the higher the write access frequency, the higher the weight value. Therefore, in this embodiment, the write access frequency of the static partition is positively correlated with the weight value of the static partition, which is conducive to reducing the frequency of write amplification.

[0106] In one specific embodiment of the present application, the weight value of each static partition is determined according to a preset weight calculation rule, which includes:

[0107] The size of each static partition is determined, and the weight value of each static partition is determined based on the size of each static partition.

[0108] The size of the static partition is positively correlated with the weight value of the static partition.

[0109] In some cases, the size of each tile partition is consistent, but in some cases, different tile partitions can have different sizes. When a tile partition is a static partition and the size of the partition is small, it can be understood that the amount of data amplified by writing will not be too high when writing each data block stored in the second cache and accessing the static partition to the static partition. Conversely, if a tile partition is a static partition and the size of the partition is large, the amount of data amplified by writing will be high when writing each data block stored in the second cache and accessing the static partition to the static partition. Therefore, the larger the size of the partition, the larger the weight value should be, so as to preferentially write the static partition with a smaller size, which is beneficial to reduce the influence of write amplification.

[0110] Further, in a specific embodiment of the present application, the weight value of a static partition can be determined based on the size of the partition and the write access frequency. For example, the write access frequency of each static partition can be determined, and the size of each static partition can be determined. Then, for each static partition, the product of the size of the static partition and the write access frequency is determined as the weight value of the static partition. For another example, the weight value of the static partition can be determined in the form of α×X1+β×X2, where X1 and X2 are the size of the static partition and the write access frequency, respectively, and α and β are the first coefficient and the second coefficient, respectively. It can be seen that by adjusting the values of the first coefficient and the second coefficient, the influence of the size of the static partition and the write access frequency on the weight value of the static partition can be flexibly adjusted, which has high flexibility. In specific cases, by setting appropriate first and second coefficients, the influence of write amplification when writing data blocks to a static partition can be reduced as much as possible.

[0111] Step S103: When the accessed tile partition is a dynamic partition, it is determined whether the dynamic partition is full. If not, step S104 is performed, and if yes, step S105 is performed.

[0112] When the tile partition accessed by a data block is a dynamic partition, it is necessary to determine whether the dynamic partition is full. If it is full, garbage collection needs to be performed on the tile partition, and if it is not full, the writing of the data block can be directly performed.

[0113] Step S104: The data block is written to an unused sector of the dynamic partition, and the sector with the lowest number.

[0114] Since the dynamic partition is not full, there is at least one unused sector in the dynamic partition. At this time, the writing of the data block needs to be performed in the order of writing, that is, the data block is written to an unused sector of the dynamic partition, and the sector with the lowest number.

[0115] For example, a certain data block points to a certain dynamic partition, and for example, the dynamic partition includes 200 sectors, for the sake of description, sequentially recorded as sector 1 to sector 200 of the dynamic partition, and for example, sector 1 to sector 100 of which have been used, then data block 1 (logical address A1) will be written into the unused sector of the dynamic partition, the lowest numbered sector, that is, will be written into sector 101 of the dynamic partition. Subsequently, for example, data block 2 (logical address B1) also needs to be written into the dynamic partition, and it can be understood that, since sector 101 has been used at this time, data block 2 (logical address B1) will be written into the unused sector of the dynamic partition, the lowest numbered sector, that is, will be written into sector 102 of the dynamic partition. For example, data block 3 (logical address A1) also needs to be written into the dynamic partition, and it can be understood that, since sector 102 has been used at this time, data block 3 (logical address A1) will be written into the unused sector of the dynamic partition, the lowest numbered sector, that is, will be written into sector 103 of the dynamic partition.

[0116] And from the example, it can be seen that the logical addresses of data block 3 and data block 1 are the same, but data block 3 does not overwrite data block 1, but is written into sector 103 and sector 101 respectively, that is, when step S104 is executed, it is sequential writing, and each time the data block is written into the unused sector of the dynamic partition, the lowest numbered sector, until the dynamic partition is full, so that for the dynamic partition, when a certain data block needs to be written into the dynamic partition, the specific writing position is not uniquely determined by the logical address of the data block. That is, for the sectors in the dynamic partition, the logical address of the data block written into the sector is not limited. Therefore, the correspondence between the sectors in the dynamic partition and the logical addresses needs to be recorded by the metadata, and the metadata needs to be updated and maintained.

[0117] And as described above, for the static partition described above, the relationship between the sector and the logical address is bound, that is, for the data block that needs to be written into the static partition, which sector the data block needs to be written into can be determined by the logical address of the data block.

[0118] Each time a data block is written into a sector of the dynamic partition, the correspondence between the sector number (equivalent to the physical address) and the logical address can be recorded by the metadata, in order to facilitate subsequent garbage collection. For example, in the above example, for sector 101 of the dynamic partition, the logical address of the data block stored therein is logical address A1, while the logical address of the data block stored in sector 101 is logical address B1, and the logical address of the data block stored in sector 103 is logical address A1.

[0119] Step S105: Garbage collection is performed on the dynamic partition, and after garbage collection, data blocks are written into unused sectors of the dynamic partition, the sectors with the lowest number.

[0120] In the garbage collection, all valid data in the dynamic partition is read into the first cache, and the valid data is written back to the dynamic partition in a sequential writing manner, with the sector with the lowest number as the writing start. For the sectors in the dynamic partition, when the data in at least two sectors correspond to the same logical address, only the data in the sector with the highest number is regarded as valid data.

[0121] When the dynamic partition is full, data blocks cannot be continuously written into the dynamic partition, and thus garbage collection needs to be performed on the dynamic partition. In the garbage collection, based on the metadata, all valid data in the dynamic partition is determined and then read into the first cache, and then the valid data is written back to the dynamic partition in a sequential writing manner.

[0122] For example, when a certain dynamic partition is full, based on the metadata, it can be determined that the logical address of the data block stored in sector 101 is logical address Al, the logical address of the data block stored in sector 103 is logical address Al, and the logical address of the data block stored in sector 180 is logical address Al, and the logical addresses of the data blocks stored in the other sectors are not logical address Al. It can be seen that the data block stored in sector 180 is the latest data of logical address Al, and thus in this example, the data in sector 180 is valid data, and the data in sectors 101 and 103 are not valid data. That is, for each sector in the dynamic partition, if the data in two or more sectors correspond to the same logical address, only the data in the sector with the highest number belongs to valid data.

[0123] In addition, it can be understood that for each sector in the dynamic partition, when the logical address of the data in a certain sector is different from the logical addresses of the data in other sectors, the data in the sector is valid data. For example, the logical address of the data block stored in sector 40 is logical address A4, and the logical addresses of the data blocks stored in the other sectors in the dynamic partition are not logical address A4, and thus the data in sector 40 is valid data.

[0124] After reading all valid data in the dynamic partition into the first cache, the lowest numbered sector is used as the starting point for writing back the valid data to the dynamic partition in sequential order. For ease of description, for example, in the dynamic partition, only the data in sector 5, sector 7, and sector 50 are valid data, and in actual applications, the number of valid data is usually not so low. Since it is sequential writing, in this example, after reading the data in sector 5, sector 7, and sector 50 into the first cache, the data in sector 5 needs to be written to sector 1 of the dynamic partition, the data in sector 7 needs to be written to sector 2, and the data in sector 50 needs to be written to sector 3, thereby completing the garbage collection this time.

[0125] In this example, after garbage collection, sectors 4 to 200 of the dynamic partition are considered to be unused sectors of the dynamic partition, that is, in this example, after garbage collection, the data block is written to sector 4, and if there is a subsequent access to the data block of the dynamic partition, the data block is sequentially written to sector 5, sector 6, sector 7, and so on.

[0126] In one specific embodiment of the present application, the following can also be included:

[0127] The read request is received, and based on the read request, the tile partition accessed by the data block to be read is determined;

[0128] When the tile partition accessed by the data block to be read is a static partition, the data block is read from the static partition and read feedback is performed;

[0129] When the tile partition accessed by the data block to be read is a dynamic partition, the data block is read from the dynamic partition and read feedback is performed.

[0130] In this embodiment, after receiving the read request, each data block to be read can be determined, for example, the data to be read can be divided into each 512B data block to be read. For the data block to be read, if the accessed tile partition is a static partition, the data block is read from the corresponding sector based on the metadata and the logical address of the data block, and read feedback is performed. If the accessed tile partition is a dynamic partition, the data block can also be read from the corresponding sector based on the metadata and the logical address of the data block, and read feedback is performed.

[0131] After the data block to be written is obtained and the tile partition accessed by the data block is determined, the application of the technical solution provided by the embodiment of the application needs to perform corresponding operations according to the difference between the static partition and the dynamic partition. If it is a static partition, the data block can be processed based on a static processing rule. If it is a dynamic partition, it is needed to determine whether the dynamic partition is full. If it is not full, the data block can be directly written into an unused sector of the dynamic partition, that is, a sector with the lowest number. It can be seen that, since the data block is directly written into the dynamic partition, the operation does not exist in the case of write amplification, and it needs to be noted that, when the write operation is performed, the sectors with larger numbers in the partition are affected but the sectors with smaller numbers are not affected. Since the position of the write at this time is the sector with the lowest number in the unused sector of the dynamic partition, the current existing data in the dynamic partition is not affected by the write operation. If the tile partition accessed is a dynamic partition and is full, it indicates that all the sectors of the dynamic partition are used, and therefore garbage collection needs to be performed on the dynamic partition. The garbage collection means that all the valid data in the dynamic partition is read into the first cache, and then the valid data is written back to the dynamic partition in the order of the lowest number of the sector as the starting write position. That is, through the garbage collection, all the valid data in the dynamic partition is retained, and the remaining data is discarded, so that the dynamic partition reacquires the unused sectors, and after the garbage collection, the valid data occupies the sectors with low numbers in the dynamic partition. Therefore, after the garbage collection is completed, the data block can be continuously written into the unused sector of the dynamic partition, that is, the sector with the lowest number, and the write operation does not exist in the case of write amplification and does not affect the valid data in the sectors with low numbers.

[0132] In addition, it also needs to be noted that, since the garbage collection needs to read all the valid data in the dynamic partition into the first cache and write the valid data back to the dynamic partition, the garbage collection process exists in the case of write amplification. However, since the garbage collection needs to be performed only when the dynamic partition is full, the influence on the performance of the disk is very small. Compared with the operation of "read-change-write" that needs to be performed whenever there is data to be written in the traditional scheme, the write amplification existing in the garbage collection process can be almost ignored.

[0133] In summary, the scheme of the application can effectively reduce the data migration caused by the write amplification problem in the tile recording disk, thereby improving the performance of the tile recording disk.

[0134] Corresponding to the above method embodiment, the embodiment of the application further provides a storage management system, which can be mutually corresponding and referred to above.

[0135] Referring to Figure 3 FIG. 1 shows a structure schematic diagram of a storage management system in the application, which comprises:

[0136] The tile partition determination module 301 is configured to acquire a data block to be written and determine a tile partition accessed by the data block.

[0137] The static partition processing module 302 is configured to, when the tile partition accessed is a static partition, process the data block based on a static processing rule.

[0138] The dynamic partition judgment module 303 is configured to, when the tile partition accessed is a dynamic partition, judge whether the dynamic partition is full.

[0139] If not, the write module 304 is triggered to write the data block into an unused sector of the dynamic partition, i.e., a sector with the lowest number.

[0140] If yes, the garbage collection module 305 is triggered to perform garbage collection on the dynamic partition and write the data block into an unused sector of the dynamic partition, i.e., a sector with the lowest number, after the garbage collection.

[0141] The garbage collection means reading all valid data in the dynamic partition into a first cache and writing the valid data back to the dynamic partition in a sequential writing manner with a sector with the lowest number as a starting writing sector. For sectors in the dynamic partition, when the data in at least two sectors correspond to the same logical address, only the data in a sector with the highest number is regarded as valid data.

[0142] In one specific embodiment of the present application, the tile partition determination module 301 is specifically configured to:

[0143] receive data to be written;

[0144] split the data to be written into data blocks to be written according to the size of a single sector, and for each data block, determine a tile partition accessed by the data block.

[0145] In one specific embodiment of the present application, the read request processing module is further configured to:

[0146] receive a read request and determine a tile partition accessed by a data block to be read based on the read request;

[0147] when the tile partition accessed by the data block to be read is a static partition, read the data block from the static partition and perform read feedback;

[0148] when the tile partition accessed by the data block to be read is a dynamic partition, read the data block from the dynamic partition and perform read feedback.

[0149] In one specific embodiment of the present application, the static partition processing module 302 includes:

[0150] a static partition determination unit configured to determine whether an unused sector exists in the static partition when the accessed tile partition is the static partition, and the lowest-numbered sector in the unused sector corresponds to a logical address of the data block;

[0151] If yes, triggering a static partition writing unit configured to write the data block into the lowest-numbered sector in the unused sector of the static partition.

[0152] If no, triggering a cache writing unit configured to write the data block into the second cache.

[0153] a cache writing to disk unit configured to write the data block in the second cache to a corresponding static partition when a remaining space of the second cache is lower than a preset threshold.

[0154] In one specific embodiment of the present application, the cache writing to disk unit is specifically configured to:

[0155] determine a weight value of each static partition according to a preset weight calculation rule;

[0156] sort the static partitions according to the weight values from small to large to obtain a static partition sequence;

[0157] select an i-th static partition in the static partition sequence; wherein i is a positive integer and the initial value is 1;

[0158] read data in the i-th static partition to the first cache as to-be-modified data;

[0159] update the to-be-modified data based on each data block stored in the second cache for accessing the i-th static partition to obtain target data after updating;

[0160] write the target data into the i-th static partition and delete each data block stored in the second cache for accessing the i-th static partition;

[0161] determine whether a remaining space of the second cache is lower than a preset threshold;

[0162] If yes, increase i by 1 and return to execute the operation of selecting the i-th static partition in the static partition sequence.

[0163] In one specific embodiment of the present application, the determination of the weight value of each static partition according to the preset weight calculation rule comprises:

[0164] determining a write access frequency of each static partition and determining the weight value of each static partition based on the write access frequency of each static partition;

[0165] wherein the write access frequency of the static partition is positively correlated with the weight value of the static partition.

[0166] In one specific embodiment of the present application, the weight value of each static partition is determined according to a preset weight calculation rule, including:

[0167] The size of each static partition is determined, and the weight value of each static partition is determined based on the size of each static partition;

[0168] The size of the static partition is positively correlated with the weight value of the static partition.

[0169] Corresponding to the above method and system embodiments, the embodiments of the present application also provide a storage management device, a computer readable storage medium and a computer program product, which can be mutually corresponding and referred to above.

[0170] Referring to Figure 4 The device can include:

[0171] The memory 401 is configured to store a computer program.

[0172] The processor 402 is configured to execute the computer program to implement the steps of the storage management method in any of the above embodiments.

[0173] The computer program product includes computer programs / instructions, which, when executed by the processor, implement the steps of the storage management method in any of the above embodiments.

[0174] Referring to Figure 5 The computer readable storage medium 50 stores a computer program 51, which, when executed by the processor, implements the steps of the storage management method in any of the above embodiments. The computer readable storage medium 50 mentioned herein includes RAM (Random Access Memory, Random Access Memory), memory, ROM (Read-Only Memory, Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory, Electrically Erasable Programmable Read-Only Memory), register, hard disk, removable disk, or any other form of storage medium known in the technical field.

[0175] It is also important to note that the use of relational terms such as first and second, and the like, are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0176] Those skilled in the art will further appreciate that the units and algorithms described in connection with the examples disclosed herein can be embodied directly in hardware, in software, or in a combination of the two. For the sake of brevity, descriptions of these conventional implementations are not included herein. Those skilled in the art will appreciate that the embodiments described herein can be practiced with the aid of one or more computers or processors, which can be hand-held, notebook, desktop, or mainframe. In general, the computer or processor will have a memory that can include, without limitation, local memory, one or more disk drives, and can have removable media storage. The memory stores program modules, which are implemented in software, and which are executed by the computer or processor. The program modules include routines, objects, components, data structures, and the like, that have the particular processing instructions used by the processor for implementing the herein described processes. A user can enter commands and information into the computer or processor through one or more wireless input devices, for example, a keyboard and a pointing device such as a mouse. These and other input devices are connected to the processor through an input device interface. Alternatively, the input devices can be connected by other interfaces, such as parallel ports, serial ports, game ports, and the like. A monitor or other type of display device is also connected to the processor through an interface, such as a video adapter. In addition to the monitor, computers typically include other peripheral output devices (not shown), such as speakers and printers.

Claims

1. A storage management method, characterized in that: include: Obtaining a data block to be written, and determining a tile partition accessed by the data block; When the accessed tile partition is a static partition, processing the data block based on a static processing rule; When the accessed tile partition is a dynamic partition, determining whether the dynamic partition is full; If not, writing the data block into an unused sector of the dynamic partition, the lowest numbered sector; If yes, performing garbage collection on the dynamic partition, and writing the data block to the lowest numbered sector in the unused sectors of the dynamic partition after garbage collection; Among them, the garbage collection means reading all valid data in the dynamic partition to the first cache, and starting with the sector with the lowest number, and writing the valid data back to the dynamic partition in a sequential manner; for the sectors in the dynamic partition, when the logical addresses corresponding to the data in at least two sectors are the same, only the data in the sector with the highest number is used as valid data.

2. The storage management method according to claim 1, wherein: Obtaining a data block to be written and determining a tile partition accessed by the data block includes: Receive data to be written; The data to be written is divided into data blocks to be written according to the size of a single sector, and for each data block, a tile partition accessed by the data block is determined.

3. The storage management method according to claim 1, wherein: Also includes: receiving a read request, and determining, based on the read request, a tile partition accessed by a data block to be read; When the tile partition accessed by the data block to be read is a static partition, reading the data block from the static partition and performing read feedback; When the tile partition accessed by the data block to be read is a dynamic partition, the data block is read from the dynamic partition and read feedback is performed.

4. The storage management method according to any one of claims 1 to 3, characterized in that: When the accessed tile partition is a static partition, processing the data block based on a static processing rule includes: When the accessed tile partition is a static partition, determining whether there are unused sectors in the static partition, and the lowest-numbered sector in the unused sectors corresponds to the logical address of the data block; If yes, writing the data block to the lowest numbered sector among the unused sectors of the static partition; If not, writing the data block into the second cache; When the remaining space of the second cache is lower than a preset threshold, the data blocks in the second cache are written to the corresponding static partition.

5. The storage management method according to claim 4, wherein: When the remaining space of the second cache is lower than a preset threshold, writing the data block in the second cache to the corresponding static partition includes: According to the preset weight calculation rules, the weight value of each static partition is determined; Sort the static partitions according to the order of weight values ​​from smallest to smallest to obtain a static partition sequence; Selecting the i-th static partition in the static partition sequence; wherein i is a positive integer and its initial value is 1; Reading the data in the i-th static partition into the first cache as data to be modified; Based on each data block accessing the i-th static partition stored in the second cache, the data to be modified is updated to obtain updated target data; Writing the target data into the i-th static partition, and deleting each data block stored in the second cache for accessing the i-th static partition; Determining whether the remaining space of the second cache is lower than a preset threshold; If yes, then i is incremented by 1 and the process returns to execute the operation of selecting the i-th static partition in the static partition sequence.

6. The storage management method according to claim 5, characterized in that: According to the preset weight calculation rules, the weight value of each static partition is determined, including: Determine the write access frequency of each static partition, and determine the weight value of each static partition based on the write access frequency of each static partition; The write access frequency of the static partition is positively correlated with the weight value of the static partition.

7. The storage management method according to claim 5, characterized in that: According to the preset weight calculation rules, the weight value of each static partition is determined, including: Determine a partition size of each static partition, and determine a weight value of each static partition based on the partition size of each static partition; The partition size of the static partition is positively correlated with the weight value of the static partition.

8. A storage management system, characterized in that: include: a tile partition determination module, configured to obtain a data block to be written and determine a tile partition accessed by the data block; a static partition processing module, configured to process the data block based on a static processing rule when the accessed tile partition is a static partition; A dynamic partition judgment module, configured to judge whether the dynamic partition is full when the accessed tile partition is a dynamic partition; If not, triggering a writing module, the writing module is used to write the data block into an unused sector of the dynamic partition, namely, the sector with the lowest number; If yes, triggering a garbage collection module, the garbage collection module is used to perform garbage collection on the dynamic partition, and write the data block to the lowest numbered sector in the unused sectors of the dynamic partition after garbage collection; Among them, the garbage collection means reading all valid data in the dynamic partition to the first cache, and starting with the sector with the lowest number, and writing the valid data back to the dynamic partition in a sequential manner; for the sectors in the dynamic partition, when the logical addresses corresponding to the data in at least two sectors are the same, only the data in the sector with the highest number is used as valid data.

9. A storage management device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the storage management method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the storage management method according to any one of claims 1 to 7 are implemented.

Images