Storage space management method and device, equipment, medium and program product

Abstract

The invention discloses a storage space management method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring a first storage state of a storage unit set in a target storage system; wherein the storage unit set comprises a plurality of data storage units; the data storage unit is a minimum storage unit in which the target storage system responds to a single-time data writing request to execute a single-time data writing operation so as to store data; the first storage state comprises a state of valid data stored in a data storage unit in the storage unit set and / or a garbage rate of the data storage unit; and based on the first storage state, recycling at least part of data storage units in the storage unit set to manage a target storage space corresponding to the target storage system.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a storage space management method, device, equipment, medium and program product. Background Art

[0002] In actual applications, a storage system that stores data in an append-only manner usually recycles Plog according to the garbage rate of Plog in the storage system to manage the storage space of the storage system. However, the above storage space management method cannot meet the actual storage space management needs. Summary of the invention

[0003] Based on the above technical problems, the embodiments of the present application provide a storage space management method, device, equipment, medium and program product.

[0004] The technical solution provided by the embodiment of the present application is as follows:

[0005] The present application embodiment first provides a storage space management method, the method comprising:

[0006] Acquire a first storage state of a storage unit set in a target storage system; wherein the storage unit set includes a plurality of data storage units; the data storage unit is a minimum storage unit for the target storage system to perform a single data write operation to store data in response to a single data write request; the first storage state includes a state of valid data stored in the data storage unit in the storage unit set and / or a garbage rate of the data storage unit;

[0007] Based on the first storage state, at least part of the data storage units in the storage unit set are recycled to manage the target storage space corresponding to the target storage system.

[0008] In some embodiments, the recycling of at least some of the data storage units in the storage unit set based on the first storage state includes:

[0009] Obtaining a first garbage rate of a kth slice in a data storage slice set of the target storage system; wherein the data storage slice set includes K data storage slices; the data storage slices are used for the target storage system to perform a data write operation to store data in response to a data write request; the data storage slices include a plurality of the data storage units, and the data storage units in the storage unit set are distributed in the data storage slices in the data storage slice set; the data write operation includes at least one single data write operation; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1;

[0010] If the first garbage rate of the kth slice is greater than or equal to a first threshold, based on the first storage status of the data storage unit included in the kth slice, at least part of the data storage unit included in the kth slice is recycled.

[0011] In some embodiments, the method further comprises:

[0012] Obtaining a second garbage rate of the kth slice after recycling at least part of the storage unit;

[0013] If the second garbage rate is greater than or equal to the first threshold, the first storage unit in the kth slice is recycled; wherein the garbage rate of the first storage unit is greater than or equal to the second threshold.

[0014] In some embodiments, the step of reclaiming the first storage unit in the kth slice includes:

[0015] determining a recycling priority for the first storage unit based on the second garbage rate;

[0016] Based on the recycling priority, the data storage units in the first storage unit are recycled in sequence.

[0017] In some embodiments, the recycling of at least part of the data storage units included in the kth slice based on the first storage state of the data storage units included in the kth slice includes:

[0018] Based on the first storage state, determining a second storage state of the data storage unit included in the kth slice; wherein the second storage state includes a state and / or a garbage rate of valid data stored in the data storage unit included in the kth slice;

[0019] A second storage unit is determined and recycled based on the second storage state; wherein the second storage unit includes a data storage unit in the kth slice area that is in a data deleted state.

[0020] In some embodiments, reclaiming at least part of the data storage units in the storage unit set includes:

[0021] Acquire the first data stored in the third storage unit in the at least part of the storage units;

[0022] storing the first data in a fourth storage unit;

[0023] Determine the first storage area currently associated with the third storage unit;

[0024] Switch the first storage area associated with the third storage unit to the second storage area; wherein the first storage area and the second storage area are data storage areas in the target storage space; the data storage area is used for the target storage system to respond to data write requests and perform data write operations to store data; the data storage area includes multiple data storage units in the storage unit set; the data write operation includes at least one single data write operation.

[0025] In some embodiments, reclaiming at least part of the data storage units in the storage unit set includes:

[0026] Acquire the first data stored in the third storage unit in the at least part of the storage units;

[0027] Acquire the second data stored in the fifth storage unit in the at least part of the storage units;

[0028] storing the first data and the second data in a fourth storage unit;

[0029] Switch the data storage area associated with the third storage unit and the fifth storage unit; wherein the data storage area is used for the target storage system to respond to data write requests and perform data write operations to store data; the data storage area includes a plurality of the data storage units in the storage unit set; the data write operation includes at least one single data write operation.

[0030] In some embodiments, reclaiming at least part of the data storage units in the storage unit set includes:

[0031] If the third garbage rate of the sixth storage unit in at least part of the storage units is less than or equal to the second threshold, switch the data storage area associated with the sixth storage unit; wherein the data storage area is used for the target storage system to respond to a data write request and perform a data write operation to store data; the data storage area includes a plurality of the data storage units in the storage unit set; the data write operation includes at least one single data write operation.

[0032] The present application also provides a storage space management device, the storage space management device comprising:

[0033] An acquisition module is used to acquire a first storage state of a storage unit set in a target storage system; wherein the storage unit set includes a plurality of data storage units; the data storage unit is a minimum storage unit for the target storage system to perform a single data write operation in response to a single data write request; the first storage state includes a state of valid data stored in the data storage unit in the storage unit set and / or a garbage rate of the data storage unit;

[0034] A management module is used to recycle at least part of the data storage units in the storage unit set based on the first storage state, so as to manage the target storage space corresponding to the target storage system.

[0035] An embodiment of the present application also provides an electronic device, comprising a processor and a memory; wherein a computer program is stored in the memory; when the computer program is executed by the processor, it can implement any of the storage space management methods described above.

[0036] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored; when the computer program is executed by a processor of an electronic device, the storage space management method as described above can be implemented.

[0037] An embodiment of the present application also provides a computer program product, which includes a computer program; when the computer program is executed by a processor of an electronic device, it can implement any of the storage space management methods described above.

[0038] The storage space management method provided in the embodiment of the present application obtains the first storage state of the storage unit set in the target storage system, and the multiple data storage units included in the storage unit set are the minimum storage units for the target storage system to perform a single data write operation to store data in response to a single data write request, and the first storage state includes the state of valid data stored in the data storage unit in the storage unit set and / or the garbage rate of the data storage unit. Thus, through the above operation, real-time monitoring of the first storage state of the minimum storage unit in the target storage system in the valid data dimension and the garbage rate dimension is realized; and, by recycling at least part of the data storage units in the storage unit set based on the first storage state, the target storage is realized. Fine-grained recovery of the smallest storage unit in the storage system for responding to a single data write operation. When the target storage system stores data in an Append-only manner and the data storage unit is a Stripe, the above operation can reduce the probability of a decrease in the data processing bandwidth of business data due to frequent migration of Plog, and reduce the negative impact on the data processing process of business data. On this basis, by recovering at least part of the storage unit based on the first storage state, the accuracy and robustness of the recovery of at least part of the storage unit can be improved, thereby improving the pertinence and flexibility of managing the target storage space corresponding to the target storage system, thereby meeting the management requirements of the target storage space of the target storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the principle of Plog migration for a storage system that writes data in the append-only mode;

[0040] Figure 2 A flowchart of a storage space management method provided in an embodiment of the present application;

[0041] Figure 3 This is a schematic diagram of the structure of a disk group;

[0042] Figure 4 It is a schematic diagram of the structure of Plog;

[0043] Figure 5 This is a schematic diagram of the Stripe structure;

[0044] Figure 6 This is a schematic diagram of the structure between Plog and Stripe;

[0045] Figure 7 A schematic diagram of the principle of recycling Stripe in Plog provided in an embodiment of the present application;

[0046] Figure 8 A schematic diagram of the structure of a storage space management device provided in an embodiment of the present application;

[0047] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0049] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] A storage system that stores data in an append-only manner changes random writes to the storage system into sequential writes to a solid-state drive (SSD) or a hard disk drive (HDD) by appending new data to the storage system without modifying the data already stored in the storage system, thereby improving the data write bandwidth of the storage system data. On the other hand, the data writing process does not involve data overwrite write operations, thereby improving the consistency and integrity of the data in the storage system, and enabling the storage system to achieve short-time data rollback.

[0051] However, although the above data writing operation can be executed sequentially, the data deletion operation for the above storage system is random, so some data stored in the Plog of the storage system that writes data in the append-only mode may be invalid on the business processing side. Therefore, in order to improve the utilization rate of storage space in the storage system, the relevant system provides a technical solution to migrate the valid data stored in the current Plog to a new Plog to release the current Plog, so as to realize the garbage collection (GC) processing of the storage space corresponding to the above storage system.

[0052] Figure 1 The schematic diagram is a schematic diagram of the principle of Plog migration for a storage system that writes data in the append-only mode. Figure 1As shown, according to the records of the index system of the storage system, it is determined that the garbage rate of the first Plog 101 is 40%, and valid data is stored in Stripe1 to Stripe3, while garbage data is stored in Stripe4 to Stripe5. At this time, the storage system can migrate Stripe1 to Stripe3 in the first Plog 101 to the second Plog 102, and then GC the storage space corresponding to the first Plog 101; and, if the garbage rate of the third Plog 103 is 60%, and valid data is stored in Stripe6 to Stripe7, and garbage data is stored in Stripe8 to Stripe10, at this time, Stripe6 to Stripe7 can be migrated to the second Plog 102, and the storage space corresponding to the third Plog 103 can be GCed; and, after the above two migration operations are completed, the relationship between the data object in the index system and the offset address of the Plog can also be changed to complete the release and recovery of the storage space corresponding to the first Plog 101 and the third Plog 103.

[0053] According to the above description, the management of the storage space of the storage system that stores data in the append-only mode in the related art is performed in units of Plog (Persistent Log). This storage space management method is not flexible enough. In addition, since the Plog space is generally large, when the system capacity is relatively high, the garbage rate of all Plogs may not be high. In this case, the bandwidth consumption generated by relocating the valid data stored in the Plog to perform GC is very large, which will inevitably lead to a conflict between the above data migration operation and the business processing process, and then lead to a decline in business bandwidth and the number of input / output operations per second (IOPS). Among them, Plog is the append-only storage unit that the storage system that writes data in the append-only mode can use externally. In order to maintain the sequential write characteristics of the disk, Plog is generally large, usually above 64MB or 128MB.

[0054] Moreover, as the storage system capacity increases, the above-mentioned decline phenomenon will become more prominent, resulting in the compression of the bandwidth used to perform business processing, thereby affecting the execution of the business processing process. Therefore, the GC mechanism based on Plog in the related art cannot meet the actual storage space management needs.

[0055] Based on the above technical problems, the embodiments of the present application provide a storage space management method, device, equipment, medium and program product.

[0056] Figure 2Schematic flowchart of the storage space management method provided by the embodiments of the present application. As Figure 2 shown, the method may include the following steps:

[0057] Step 201: Obtain the first storage status of the storage unit set in the target storage system.

[0058] Among them, the storage unit set includes multiple data storage units; the data storage unit is the smallest storage unit for the target storage system to execute a single data write operation to store data in response to a single data write request; the first storage status includes the status of the valid data stored in the data storage units in the storage unit set and / or the garbage rate of the data storage units.

[0059] In one implementation, the target storage system may store data in an Append-only manner.

[0060] In one implementation, the target storage space corresponding to the target storage system may include the storage space of the disks in the disk group.

[0061] Figure 3 Schematic diagram of the structure of the disk group. As Figure 3 shown, the first disk group 301 to the fifth disk group 305 may each include multiple disks, and the disks in the first disk group 301 to the fifth disk group 305 may be evenly distributed in the first node 306 to the sixth node 311 respectively.

[0062] Among them, a disk group usually includes a set of disks that meet the requirements of fault redundancy in a distributed system. Physically speaking, the disks within a disk group are usually distributed on different hosts or nodes; logically speaking, externally, the disk group can provide the function of creating a Plog, and internally, it can implement data recovery through Erasure Code (EC). In some cases, it can also implement a regional group within the disk group to further optimize certain characteristics.

[0063] Among them, EC is a data recovery technology that can perform data verification and restoration operations when data is damaged, thereby improving the reversibility and stability of data; the mathematical principle on which the above data verification and restoration is based is: for any determinant of M rows and N columns in a row-column matrix and M < N, any MxM submatrix thereof is invertible.

[0064] Figure 4 Schematic diagram of the structure of the Plog. As Figure 4As shown, the first disk 401 to the sixth disk 406 may constitute a disk group, and multiple Chunks in the first disk 401 to the sixth disk 406 constitute a Plog 407. Chunk is the smallest physical allocation unit of Plog on the disk, and its size is usually 1MB.

[0065] In one embodiment, the storage space corresponding to the data storage unit may be smaller than the storage space of Plog; illustratively, the data storage unit may include a Stripe in Plog, and a Stripe may include multiple Chunks; wherein a Stripe is also called a stripe, which is the minimum storage unit that can be provided when a data output device sends a data write request and a target storage system responds to the data write request and performs a single data write operation, and its size is usually 4MB; in actual applications, it is usually required that the business system aggregates before performing the data write operation to simplify the usage model of Plog.

[0066] Figure 5 This is a schematic diagram of the Stripe structure. Figure 5 As shown, the first Stripe 501 to the third Stripe 503 can respectively include multiple Chunks; wherein, the index system of the storage system can determine the mapping relationship between the offset address of the Plog and the Chunk, and when the storage system responds to the read request, the index system can determine the target Chunk in the Plog to perform a data reading operation; and, data blocks and check blocks can be stored in the Chunk.

[0067] Figure 6 This is a schematic diagram of the structure between Plog and Stripe, such as Figure 6 As shown, the first Plog 101 may include Stripe1 to Stripe5, and the third Plog 103 may include Stripe6 to Stripe10.

[0068] In one embodiment, the first storage status may include whether the Stripe contained in the target storage system stores data, the amount of stored data, the type of data stored, the business associated with the stored data, and at least one of the storage time of the stored data.

[0069] In one embodiment, valid data may include data that has been written into a data storage unit but has not been read out or has been set to a deleted state; accordingly, the state of the valid data stored in the data storage unit may include the size of the storage space occupied by the valid data in the data storage unit, the address index of the above storage space, and the ratio between the size of the above storage space and the storage space capacity of the data storage unit.

[0070] In one embodiment, the garbage rate of the data storage unit may include the ratio of the size of the storage space occupied by the garbage data in the data storage unit to the storage space capacity of the data storage unit; wherein the garbage data may include data stored in the data storage space that has been read out or set to a deleted state.

[0071] In one implementation, the first storage state may be obtained in the following manner:

[0072] The data storage status of Stripe contained in the index system is obtained from the index system, and the above data storage status is integrated to determine a first storage state; exemplarily, the data storage status may include the amount of valid data stored in Stripe and / or the amount of junk data, i.e., invalid data.

[0073] Step 202: Based on the first storage state, reclaim at least part of the data storage units in the storage unit set to manage the target storage space corresponding to the target storage system.

[0074] In one implementation, at least some of the storage units may belong to the same or different Plogs.

[0075] In one implementation, recycling at least part of the storage units in the storage unit set may be achieved by any of the following methods:

[0076] At least part of the storage units that do not store valid data are recycled, and at the same time, the storage space corresponding to the at least part of the storage units is released.

[0077] Reclaim at least part of the storage units that do not store valid data, thereby achieving dynamic adjustment of the target Plog space, and associate at least part of the recovered storage units with other Plogs; wherein the target Plog may include the Plog associated with at least part of the storage units.

[0078] In one embodiment, managing the target storage space corresponding to the target storage system may include dynamically expanding or reducing the storage space corresponding to multiple Plogs contained in the target storage space, and may also include releasing at least part of the storage units in real time so that the target storage system can update the correspondence or association relationship between the above-mentioned at least part of the storage units and Plog in real time; illustratively, the above-mentioned real-time update of the association relationship or correspondence relationship can be achieved by adjusting the relationship between Plog and Stripe in the index system.

[0079] As can be seen from the above, the storage space management method provided in the embodiment of the present application obtains the first storage state of the storage unit set in the target storage system, and the multiple data storage units included in the storage unit set are the minimum storage units for the target storage system to perform a single data write operation to store data in response to a single data write request, and the first storage state includes the state of valid data stored in the data storage unit in the storage unit set and / or the garbage rate of the data storage unit. In this way, through the above operation, real-time monitoring of the first storage state of the minimum storage unit in the target storage system in the valid data dimension and the garbage rate dimension is realized; and by recycling at least part of the storage units in the storage unit set based on the first storage state, the target storage system is realized. Fine-grained recovery of the smallest storage unit in the target storage system for responding to a single data write operation. When the target storage system stores data in an Append-only manner and the data storage unit is a Stripe, the above operation can reduce the probability of a decrease in the data processing bandwidth of business data due to frequent migration of Plog, and reduce the negative impact on the data processing process of business data. On this basis, by recovering at least part of the storage unit based on the first storage state, the accuracy and robustness of the recovery of at least part of the storage unit can be improved, thereby improving the pertinence and flexibility of managing the target storage space corresponding to the target storage system, thereby meeting the management requirements of the target storage space of the target storage system.

[0080] Based on the foregoing embodiment, in the storage space management method provided in the embodiment of the present application, based on the first storage state, reclaiming at least part of the data storage units in the storage unit set can be achieved by the following steps:

[0081] Step A1: Obtain a first garbage rate of a kth slice in a data storage slice set of a target storage system.

[0082] Among them, the data storage area set includes K data storage areas; the data storage area is used for the target storage system to respond to the data write request and perform the data write operation to store data; the data storage area includes multiple data storage units, and the data storage units in the storage unit set are distributed in the data storage areas in the data storage area set; the data write operation includes at least one single data write operation; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1.

[0083] In one implementation, the data storage area may include multiple Stripes; illustratively, the data storage area may include a Plog.

[0084] In one implementation, the first junk rate is used to represent the percentage of junk data carried by the kth slice to all data carried by the kth slice.

[0085] In one implementation, the first garbage rate may be obtained in the following manner:

[0086] The first garbage rate is obtained by statistically integrating the data storage status of the Chunks in the kth slice in the index system.

[0087] Step A2: If the first garbage rate of the kth slice is greater than or equal to the first threshold, based on the first storage state of the data storage unit included in the kth slice, recycle at least part of the data storage unit included in the kth slice.

[0088] Exemplarily, if the first garbage rate of the kth slice is less than the first threshold, the operation of reclaiming at least part of the data storage units included in the kth slice based on the first storage state may not be performed.

[0089] In one implementation, the first threshold may be determined or adjusted according to at least one of a business type associated with data stored in the target storage system, business processing efficiency, and a priority or importance level of the business.

[0090] As can be seen from the above, in the storage space management method provided in the embodiment of the present application, the first garbage rate of the kth area in the data storage area set in the target storage system is obtained, and the multiple data storage areas included in the data storage area set are used for the target storage system to perform a data write operation including at least one single data write operation in response to a data write request to store data, and the data storage units in the storage unit set are distributed in the data storage areas in the data storage area set. In this way, when the data storage area is Plog and the data storage unit is Stripe, through the above operation, real-time monitoring of the first garbage rate of Plog in the target storage system can be achieved; and if the first garbage rate of the kth area is greater than or equal to the first threshold, at least part of the data storage units included in the kth area are recovered based on the first storage state. In this way, not only is strict control of the operation of recovering at least part of the data storage units included in the kth area achieved, but also the pertinence and accuracy of recovering at least part of the data storage units included in the kth area can be improved.

[0091] Based on the above embodiments, the storage space management method provided in the embodiments of the present application may further perform the following steps:

[0092] Step B1, obtaining a second garbage rate of the kth slice after recycling at least part of the storage units.

[0093] In one implementation, during the process of reclaiming at least part of the storage units in the kth slice, the association relationship between the kth slice and at least part of the storage units in the index system may be synchronously adjusted.

[0094] In one implementation, the second garbage rate may be obtained in the following manner:

[0095] The second garbage rate is determined by collecting statistics on the data storage status corresponding to the Chunk of the kth slice in the index system.

[0096] Step B2: If the second garbage rate is greater than or equal to the first threshold, reclaim the first storage unit in the kth slice.

[0097] The garbage rate of the first storage unit is greater than or equal to the second threshold.

[0098] Exemplarily, if the second garbage rate is less than the first threshold, the recycling operation on the storage units included in the kth slice may be stopped.

[0099] In one implementation, the first storage unit may be determined in the following manner:

[0100] By integrating and counting the data storage status associated with the Stripe in the kth area in the index system, a set of unit garbage rates of the data storage units contained in the kth area is determined, and then the data storage unit in the set of unit garbage rates that is greater than or equal to the second threshold is determined as the first storage unit.

[0101] From the above, it can be seen that in the storage space management method provided in the embodiment of the present application, by obtaining the second garbage rate of the kth area after recycling at least part of the storage units, real-time tracking of the second garbage rate of the kth area is achieved during the recycling of at least part of the storage units in the kth area; and if the second garbage rate is greater than or equal to the first threshold, the first storage unit in the kth area whose garbage rate is greater than or equal to the second threshold is recycled. In this way, through the above operations, continuous, recursive and automatic recycling of the data storage units in the kth area is achieved, thereby improving the efficiency of recycling the data storage units in the kth area.

[0102] Based on the above embodiments, in the storage space management method provided in the embodiments of the present application, recycling the first storage unit in the kth slice can be achieved by the following methods:

[0103] Determine a recycling priority for the first storage unit based on the second garbage rate; and sequentially recycle the data storage units in the first storage unit based on the recycling priority.

[0104] In one implementation, the recycling priority may represent a priority order in which the data storage unit is released or recycled.

[0105] In one implementation, the recycling priority may be determined as follows:

[0106] The recycling priority is determined according to the second garbage rate; illustratively, the recycling priority may be proportional to the second garbage rate, that is, the recycling priority may increase as the second garbage rate increases.

[0107] In one embodiment, the data storage unit in the first storage unit can be recycled in the following manner:

[0108] The recycling order of each data storage unit in the first storage unit is determined according to the recycling priority level, and then the data storage units included in the first storage unit are recycled in sequence according to the recycling order.

[0109] From the above, it can be seen that the storage space management method provided in the embodiment of the present application realizes the association between the recycling priority and the second garbage rate of the data storage unit in the first storage unit by determining the recycling priority for the first storage unit based on the second garbage rate, thereby improving the accuracy of the recycling priority; and, by recycling the data storage units in the first storage unit in turn based on the recycling priority, targeted recycling of the data storage units in the first storage unit can be achieved.

[0110] Based on the foregoing embodiment, in the storage space management method provided in the embodiment of the present application, based on the first storage state of the data storage unit included in the kth slice, reclaiming at least part of the data storage unit included in the kth slice can be achieved by the following steps:

[0111] Step C1: Based on the first storage state, determine the second storage state of the data storage unit included in the kth slice.

[0112] The second storage state includes the state and / or garbage rate of valid data stored in the data storage unit included in the kth slice.

[0113] In one implementation, the second storage state may include at least one of a garbage rate of all data storage units included in the kth slice, whether the data is in a complete deletion state, and whether the data is in a partial deletion state.

[0114] In one implementation, the second storage state may be determined by:

[0115] The data storage states of the data storage units included in the first storage state are integrated to obtain the second storage state of all the data storage units in the kth slice.

[0116] Step C2: determining and recycling the second storage unit based on the second storage state.

[0117] The second storage unit includes a data storage unit in the kth slice area in a data deleted state.

[0118] In one implementation, being in a data deleted state may include a state in which all data is deleted.

[0119] In one embodiment, the second storage unit may be recycled by any of the following methods:

[0120] The data storage units in the second storage unit are recycled according to a preset recycling order.

[0121] The recycling priority of the second storage unit is determined according to the second storage state, and then the data storage unit in the second storage unit is recycled according to the recycling priority of the second storage unit.

[0122] From the above, it can be seen that the storage space management method provided in the embodiment of the present application determines the second storage state of the data storage unit contained in the kth slice based on the first storage state, and the second storage state includes the state of valid data stored in the data storage unit contained in the kth slice and / or the garbage rate. In this way, through the above operation, it is possible to achieve real-time tracking and monitoring of the second storage state of the data storage unit in any storage slice in the target storage space corresponding to the target storage system in the state of valid data and / or the garbage rate dimension; and, based on the second storage state, determine the data storage unit in the kth slice that is in the data deleted state, that is, the second storage unit, so that the data storage unit in the kth slice that is in the data deleted state can be recovered in real time, thereby improving the management efficiency of the target storage space.

[0123] Based on the above embodiments, in the storage space management method provided in the embodiments of the present application, reclaiming at least part of the data storage units in the storage unit set can be achieved by the following steps:

[0124] Step D1, obtaining first data stored in a third storage unit in at least part of the storage units.

[0125] In one implementation, the third storage unit may include at least one storage unit whose garbage rate among at least some of the storage units is greater than or equal to a garbage rate threshold.

[0126] In one implementation, the first data may include valid data stored in a data storage unit of the third storage unit.

[0127] Step D2, storing the first data in the fourth storage unit.

[0128] In one implementation, the data storage areas associated with the fourth storage unit and the first storage unit may be the same or different.

[0129] In one implementation, the remaining available storage space in the fourth storage unit may be greater than or equal to the data volume of the first data, that is, the fourth storage unit may store no valid data or may store valid data.

[0130] In one implementation, the first data may be migrated to a fourth storage unit.

[0131] Step D3: Determine the first storage area currently associated with the third storage unit.

[0132] In one implementation, the first storage slice may include a Plog currently associated with the third storage unit in the indexing system.

[0133] Step D4: Switch the first storage area associated with the third storage unit to the second storage area.

[0134] Among them, the first storage area and the second storage area are data storage areas in the target storage space; the data storage area is used for the target storage system to respond to data write requests and perform data write operations to store data; the data storage area includes multiple data storage units in a storage unit set; the data write operation includes at least one single data write operation.

[0135] In one implementation, the first storage slice associated with the third storage unit can be switched to the second storage slice by modifying the identifier of the storage slice associated with the third storage unit in the index system.

[0136] As can be seen from the above, the storage space management method provided in the embodiment of the present application, after acquiring the first data stored in the third storage unit in at least part of the storage units, stores the first data in the fourth storage unit, and switches the first storage area associated with the third storage unit to the second storage area, and the first storage area and the second storage area are data storage areas in the target storage space, the data storage area is used for the target storage system to respond to the data write request and perform the data write operation to store data, the data storage area includes multiple data storage units in the storage unit set, and the data write operation includes at least one single data write operation. In this way, through the above operation, the first data stored in the third storage unit in the first storage area of ​​the target storage space can be migrated to the fourth storage unit, thereby realizing the fine-grained migration of the data stored in the data storage unit in the target storage space, and also realizing the real-time update of the storage area associated with the third storage unit, thereby providing a data basis for the accurate management and status tracking of the target storage space.

[0137] Based on the above embodiments, in the storage space management method provided in the embodiments of the present application, reclaiming at least part of the data storage units in the storage unit set can also be achieved by the following steps:

[0138] Step E1, obtaining first data stored in a third storage unit in at least some of the storage units; obtaining second data stored in a fifth storage unit in at least some of the storage units.

[0139] In one implementation, the process of acquiring the first data and the second data may be executed in parallel, or the order of the processes may be adjusted, which is not limited in this embodiment of the present application.

[0140] In one implementation, the data storage areas associated with the third storage unit and the fifth storage unit may be the same or different.

[0141] In one implementation, the garbage rates of the third storage unit and the fifth storage unit may both be greater than or equal to a garbage rate threshold.

[0142] Step E2: storing the first data and the second data in a fourth storage unit.

[0143] In one embodiment, the data storage areas associated with the fourth storage unit, the third storage unit, and the fifth storage unit are the same as or different from each other.

[0144] In one implementation, the fourth storage unit may store no valid data or may store valid data, but the remaining available storage space of the fourth storage unit may be greater than or equal to the sum of the data amounts of the first data and the second data.

[0145] Step E3: switch the data storage areas associated with the third storage unit and the fifth storage unit.

[0146] Among them, the data storage area is used for the target storage system to respond to data write requests and perform data write operations to store data; the data storage area includes multiple data storage units in a storage unit set; the data write operation includes at least one single data write operation.

[0147] In one implementation, when the target storage system determines to reclaim the third storage unit and the fifth storage unit, the data storage slices associated with the third storage unit and the fifth storage unit may be switched.

[0148] From the above, it can be seen that in the storage space management method provided in the embodiment of the present application, after obtaining the first data stored in the third storage unit and the second data stored in the fifth storage unit in at least part of the storage units, the first data and the second data are stored in the fourth storage unit. In this way, in the dimension of the data storage unit, the data respectively stored in the two storage units are merged and stored in one storage unit. In this way, not only the fine-grained management of the target storage space in the target storage system is achieved, but also the redundancy of the data storage unit after the target storage space management can be reduced; and the data storage areas associated with the third storage unit and the fifth storage unit are switched, so as to realize real-time switching of the data storage areas associated with the third storage unit and the fifth storage unit, thereby providing a data basis for the subsequent storage space management of the third storage unit and the fifth storage unit.

[0149] Based on the above embodiments, in the storage space management method provided in the embodiments of the present application, recycling at least part of the data storage units in the storage unit set can also be achieved in the following manner:

[0150] If the third garbage rate of the sixth storage unit in at least some of the storage units is less than or equal to the second threshold, the data storage area associated with the sixth storage unit is switched.

[0151] Among them, the data storage area is used for the target storage system to respond to the data write request and perform the data write operation to store data; the data storage area includes multiple data storage units in the storage unit set; the data write operation includes at least one single data write operation.

[0152] Correspondingly, if the third garbage rate of the sixth storage unit is greater than the second threshold, the sixth storage unit can be recycled by processing the first data stored in the third storage unit and recycling the third storage unit in the aforementioned embodiment.

[0153] Correspondingly, if the third garbage rate of the sixth storage unit is greater than the second threshold, the operation of switching the data storage area associated with the sixth storage unit may not be performed.

[0154] In one implementation, the third garbage rate of the sixth storage unit may be determined based on the identifier of the sixth storage unit and the first storage state.

[0155] Figure 7 The schematic diagram of the principle of recycling Stripe in Plog provided in the embodiment of the present application is as follows: Figure 7As shown, the garbage rate of the first Plog101 may be greater than or equal to the first threshold. At this time, the garbage rate corresponding to each of the five Stripes in the first Plog may be determined. If the garbage rates of Stripe3 to Stripe4 are greater than or equal to the garbage rate threshold, the valid data stored in Stripe3 to Stripe4 may be respectively migrated to Stripe6 of the second Plog 102, and Stripe3 to Stripe4 may be recycled at the same time; exemplarily, if the garbage rates of Stripe1 and Stripe2 are less than the garbage rate threshold, Stripe1 to Stripe2 may be retained and still associated with the first Plog; exemplarily, Stripe5 may be in a state where all data is deleted, and therefore, Stripe5 may be directly recycled, thereby obtaining a new first Plog701 after Stripe is recycled.

[0156] Through the above process, targeted processing of Stripes with different garbage rates and different data deletion states in the first Plog is achieved, thereby achieving flexible and targeted recycling processing of different Stripes in the first Plog.

[0157] As can be seen from the above, the storage space management method provided in the embodiment of the present application switches the data storage area associated with the sixth storage unit if the third garbage rate of the sixth storage unit in at least some storage units is less than or equal to the second threshold value. In this way, not only is strict control of the operation of recycling the sixth storage unit achieved, but also, by switching the data storage area associated with the sixth storage unit based on the third garbage rate of the sixth storage unit, the pertinence of the operation of switching the data storage area associated with the sixth storage unit can be improved.

[0158] Based on the above embodiments, the present application also provides a storage space management device. Figure 8 A schematic diagram of the structure of the storage space management device provided in the embodiment of the present application is shown in FIG. Figure 8 As shown, the storage space management device 8 may include:

[0159] The acquisition module 801 is used to acquire a first storage state of a storage unit set in a target storage system; wherein the storage unit set includes a plurality of data storage units; a data storage unit is a minimum storage unit for the target storage system to perform a single data write operation to store data in response to a single data write request; the first storage state includes a state of valid data stored in the data storage unit in the storage unit set and / or a garbage rate of the data storage unit;

[0160] The management module 802 is configured to reclaim at least part of the data storage units in the storage unit set based on the first storage state, so as to manage the target storage space corresponding to the target storage system.

[0161] In some embodiments, the acquisition module 801 is used to acquire a first garbage rate of a kth slice in a data storage slice set of a target storage system; wherein the data storage slice set includes K data storage slices; the data storage slice is used for the target storage system to perform a data write operation to store data in response to a data write request; the data storage slice includes a plurality of data storage units, and the data storage units in the storage unit set are distributed in the data storage slices in the data storage slice set; the data write operation includes at least one single data write operation; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1 and is used to characterize the number of data storage slices included in the data storage slice set;

[0162] The management module 802 is configured to recycle at least part of the data storage units included in the kth slice based on the first storage status of the data storage units included in the kth slice if the first garbage rate of the kth slice is greater than or equal to the first threshold.

[0163] In some embodiments, the acquisition module 801 is used to acquire a second garbage rate of the kth slice after reclaiming at least part of the storage unit;

[0164] The management module 802 is used to recycle the first storage unit in the kth slice if the second garbage rate is greater than or equal to the first threshold; wherein the garbage rate of the first storage unit is greater than or equal to the second threshold.

[0165] In some embodiments, the management module 802 is used to determine a recycling priority for the first storage unit based on the second garbage rate; and to sequentially recycle the data storage units in the first storage unit based on the recycling priority.

[0166] In some embodiments, the management module 802 is used to determine the second storage state of the data storage unit contained in the kth area based on the first storage state; determine and recycle the second storage unit based on the second storage state; wherein the second storage state includes the state and / or garbage rate of valid data stored in the data storage unit contained in the kth area.

[0167] In some embodiments, the acquisition module 801 is used to acquire the first data stored in the third storage unit in at least part of the storage units;

[0168] Management module 802 is used to store the first data in the fourth storage unit; determine the first storage area currently associated with the third storage unit; switch the first storage area associated with the third storage unit to the second storage area; wherein the first storage area and the second storage area are data storage areas in the target storage space; the data storage area is used for the target storage system to respond to data write requests and perform data write operations to store data; the data storage area includes multiple data storage units in the storage unit set; the data write operation includes at least one single data write operation.

[0169] In some embodiments, the acquisition module 801 is used to acquire the first data stored in the third storage unit in at least some of the storage units; acquire the second data stored in the fifth storage unit in at least some of the storage units;

[0170] Management module 802 is used to store the first data and the second data in the fourth storage unit; switch the data storage area associated with the third storage unit and the fifth storage unit; wherein the data storage area is used for the target storage system to respond to the data write request and perform the data write operation to store the data; the data storage area includes multiple data storage units in the storage unit set; the data write operation includes at least one single data write operation.

[0171] In some embodiments, the management module 802 is used to switch the data storage area associated with the sixth storage unit if the third garbage rate of the sixth storage unit in at least some of the storage units is less than or equal to the second threshold; wherein the data storage area is used for the target storage system to respond to the data write request and perform the data write operation to store data; the data storage area includes multiple data storage units in the storage unit set; the data write operation includes at least one single data write operation.

[0172] Based on the above embodiments, the present application also provides an electronic device, Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Fig. 9 As shown, the electronic device 9 may include a processor 901 and a memory 902; wherein the memory 902 stores a computer program, and when the computer program is executed by the processor 901, it can implement the storage space management method provided in any of the previous embodiments.

[0173] Based on the foregoing embodiments, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored; when the computer program is executed by a processor of an electronic device, it can implement a storage space management method provided in any of the previous embodiments.

[0174] Based on the foregoing embodiments, an embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor of an electronic device, it can implement a storage space management method as provided in any of the previous embodiments.

[0175] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0176] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0177] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0178] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0179] It should be noted that the above-mentioned computer-readable storage medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM) and other memories; it can also be various electronic devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0180] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0181] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0182] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus necessary general hardware nodes, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0183] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0184] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0186] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A storage space management method, characterized in that: The method comprises: Acquire a first storage state of a storage unit set in a target storage system; wherein the storage unit set includes a plurality of data storage units; the data storage unit is a minimum storage unit for the target storage system to perform a single data write operation to store data in response to a single data write request; the first storage state includes a state of valid data stored in the data storage unit in the storage unit set and / or a garbage rate of the data storage unit; Based on the first storage state, at least part of the data storage units in the storage unit set are recycled to manage the target storage space corresponding to the target storage system.

2. The method according to claim 1, characterized in that The recycling of at least part of the data storage units in the storage unit set based on the first storage state includes: Obtaining a first garbage rate of a kth slice in a data storage slice set of the target storage system; wherein the data storage slice set includes K data storage slices; the data storage slices are used for the target storage system to perform a data write operation to store data in response to a data write request; the data storage slices include a plurality of the data storage units, and the data storage units in the storage unit set are distributed in the data storage slices in the data storage slice set; the data write operation includes at least one single data write operation; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1; If the first garbage rate of the kth slice is greater than or equal to a first threshold, based on the first storage status of the data storage unit included in the kth slice, at least part of the data storage unit included in the kth slice is recycled.

3. The method according to claim 2, characterized in that The method further comprises: Obtaining a second garbage rate of the kth slice after recycling at least part of the storage unit; If the second garbage rate is greater than or equal to the first threshold, the first storage unit in the kth slice is recycled; wherein the garbage rate of the first storage unit is greater than or equal to the second threshold.

4. The method according to claim 3, characterized in that The step of reclaiming the first storage unit in the kth slice area comprises: determining a recycling priority for the first storage unit based on the second garbage rate; Based on the recycling priority, the data storage units in the first storage unit are recycled in sequence.

5. The method according to claim 2, characterized in that: The step of recycling at least part of the data storage units contained in the kth slice area based on the first storage state of the data storage units contained in the kth slice area comprises: Based on the first storage state, determining a second storage state of the data storage unit included in the kth slice; wherein the second storage state includes a state and / or a garbage rate of valid data stored in the data storage unit included in the kth slice; A second storage unit is determined and recycled based on the second storage state; wherein the second storage unit includes a data storage unit in the kth slice area that is in a data deleted state.

6. The method according to claim 1, characterized in that The reclaiming at least part of the data storage units in the storage unit set comprises: Acquire the first data stored in the third storage unit in the at least part of the storage units; storing the first data in a fourth storage unit; Determine the first storage area currently associated with the third storage unit; Switch the first storage area associated with the third storage unit to the second storage area; wherein the first storage area and the second storage area are data storage areas in the target storage space; the data storage area is used for the target storage system to respond to data write requests and perform data write operations to store data; the data storage area includes multiple data storage units in the storage unit set; the data write operation includes at least one single data write operation.

7. The method according to claim 1, characterized in that The reclaiming at least part of the data storage units in the storage unit set comprises: Acquire the first data stored in the third storage unit in the at least part of the storage units; Acquire the second data stored in the fifth storage unit in the at least part of the storage units; storing the first data and the second data in a fourth storage unit; Switch the data storage area associated with the third storage unit and the fifth storage unit; wherein the data storage area is used for the target storage system to respond to data write requests and perform data write operations to store data; the data storage area includes a plurality of the data storage units in the storage unit set; the data write operation includes at least one single data write operation.

8. The method according to claim 1, characterized in that The reclaiming at least part of the data storage units in the storage unit set comprises: If the third garbage rate of the sixth storage unit in at least part of the storage units is less than or equal to the second threshold, switch the data storage area associated with the sixth storage unit; wherein the data storage area is used for the target storage system to respond to a data write request and perform a data write operation to store data; the data storage area includes a plurality of the data storage units in the storage unit set; the data write operation includes at least one single data write operation.

9. A storage space management device, characterized in that: The storage space management device comprises: An acquisition module is used to acquire a first storage state of a storage unit set in a target storage system; wherein the storage unit set includes a plurality of data storage units; the data storage unit is a minimum storage unit for the target storage system to perform a single data write operation in response to a single data write request; the first storage state includes a state of valid data stored in the data storage unit in the storage unit set and / or a garbage rate of the data storage unit; A management module is used to recycle at least part of the data storage units in the storage unit set based on the first storage state, so as to manage the target storage space corresponding to the target storage system.

10. An electronic device, characterized in that: The electronic device includes a processor and a memory; wherein the memory stores a computer program; when the computer program is executed by the processor, it can implement the storage space management method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The storage medium stores a computer program; when the computer program is executed by a processor of an electronic device, it can implement the storage space management method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that The program product includes a computer program; when the computer program is executed by a processor of an electronic device, it can implement the storage space management method as described in any one of claims 1 to 8.

Images