Space recovery method, device, equipment and medium

Abstract

The invention provides a space recovery method and device, equipment and a medium, and relates to the technical field of computers.The method comprises the steps that a data area in a storage system is divided into a plurality of data blocks, and the utilization rate of each data block contained in the data blocks is obtained, the utilization rate of the data block is determined according to mapping of a logic address and a physical address of data contained in the data block; according to the interval where the utilization rate is located, determining a space recovery strategy corresponding to each data block; and according to the space recovery strategy, carrying out space recovery on the data block. And effective utilization of space resources of the storage system is ensured. And the space recovery strategy of each data block is determined according to the utilization rate, so that the storage space in the data blocks can be effectively and efficiently recovered, and the data management of each data block in the storage system can be optimized.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a space recovery method, device, equipment and medium. Background Art

[0002] Storage space garbage collection refers to releasing storage space that is no longer in use, preventing space leaks, and effectively using available storage space. Its principle is mainly to track objects in the storage space when the program is running, determine which objects are no longer referenced by the program, and then mark the storage space of these objects as recyclable. Finally, the system reclaims these spaces so that they can be reused.

[0003] Current storage systems are generally implemented based on a log structure, and data is quickly written by appending writes. However, due to operations such as data modification, erasure, and deletion, the data in the actual hard disk is not deleted synchronously. Only the corresponding physical space mapping is deleted. This results in a lot of invalid data being stored in the used space in the storage system. In order to improve disk utilization, the space occupied by these invalid data needs to be reclaimed. In related technologies, blocks in the storage system are sequentially reclaimed, which cannot improve the efficiency of space recovery. Summary of the invention

[0004] In view of this, embodiments of the present application provide a space recovery method, apparatus, device and medium to overcome the above-mentioned problems or at least partially solve the above-mentioned problems.

[0005] A first aspect of an embodiment of the present application provides a space recovery method, the method comprising:

[0006] Dividing a data area in a storage system into a plurality of data blocks, and obtaining a utilization rate of each data block contained in the plurality of data blocks, wherein the utilization rate of the data block is determined according to a mapping between a logical address and a physical address of data contained in the data block;

[0007] Determining a space recovery strategy corresponding to each data block according to the utilization interval;

[0008] According to the space recovery strategy, space recovery is performed on the data block.

[0009] Optionally, before obtaining the utilization rate of each data block included in the multiple data blocks, the method further includes:

[0010] Recording the usage of each data block included in the plurality of data blocks through a bitmap, wherein the used data block is recorded as a first value;

[0011] The obtaining the utilization rate of each data block included in the plurality of data blocks includes:

[0012] For each data block recorded as the first value, determine valid data, where the valid data is data having a mapping between a logical address and a physical address among a plurality of data included in the data block;

[0013] For each data block recorded as the first value, the utilization rate of the data block is calculated based on the sum of the physical address corresponding to the valid data and the address of the data block where the valid data is located.

[0014] Optionally, the performing space reclamation on the data block according to the space reclamation strategy includes:

[0015] Copy valid data in multiple data blocks that need to be reclaimed into data blocks corresponding to consecutive physical addresses;

[0016] Initialize the physical address space of multiple data blocks that need space reclamation.

[0017] Optionally, determining the space recovery strategy corresponding to each data block according to the interval in which the utilization rate is located includes:

[0018] When the utilization rate of the data block is less than a first threshold, determining that the space recovery strategy corresponding to the data block is direct recovery;

[0019] When the utilization rate of the data block is not less than the first threshold and less than the second threshold, determining that the space recovery strategy corresponding to the data block is waiting for recovery;

[0020] When the utilization rate of the data block is not less than the second threshold, determining that the space recovery strategy corresponding to the data block is not to recover;

[0021] The first threshold is smaller than the second threshold.

[0022] Optionally, before determining the space recovery strategy corresponding to each data block according to the interval where the utilization rate is located, the method further includes:

[0023] Dividing a plurality of data blocks contained in a data area in the storage system into a plurality of data block groups, each of the data block groups containing a plurality of data blocks;

[0024] Allocating a corresponding recycling queue to each of the data block groups, and configuring a corresponding sliding window for each recycling queue;

[0025] Set the sliding window length and sliding step size of each sliding window;

[0026] The determining, according to the interval in which the utilization rate is located, the space recovery strategy corresponding to each of the data blocks, comprises: when the sliding window performs a sliding operation, determining, according to the utilization rate corresponding to each of the data blocks currently included in the sliding window, the space recovery strategy corresponding to each of the data blocks currently included in the sliding window;

[0027] The method further comprises:

[0028] When the space recovery strategy corresponding to the data block in the sliding window is direct recovery, directly recovering the space of the data block;

[0029] When the space recovery strategy corresponding to the data block in the sliding window is waiting for recovery, the data block is added to the sorting queue, and the data blocks in the sorting queue are sorted in ascending order according to utilization rate;

[0030] When the space recovery strategy corresponding to the data block in the sliding window is not to recover, the space of the data block is not recovered;

[0031] The method further comprises:

[0032] After the data blocks whose utilization is less than the first threshold are recovered, the data blocks in the sorting queue are added back into the recovery queue in ascending order of utilization to recover space.

[0033] Optionally, the method further comprises:

[0034] In the process of the sliding window performing the sliding operation, the theoretical recycling speed and the theoretical CPU usage of the system corresponding to the recycling queue are calculated, and the theoretical recycling speed and the theoretical CPU usage of the system corresponding to the recycling queue are determined according to the theoretical recycling speeds and the theoretical CPU usage of the system pre-set by the multiple data block models corresponding to the multiple data blocks currently contained in the sliding window in the recycling queue;

[0035] When the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient, increasing the sliding window length of the sliding window and increasing the sliding step size of the sliding window;

[0036] When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient, reducing the sliding window length of the sliding window and reducing the sliding step size of the sliding window;

[0037] The actual recycling speed of the recycling queue is determined according to the space size of the data blocks recycled per unit time.

[0038] Optionally, the space recovery mode is divided into a first mode and a second mode, and determining the space recovery strategy corresponding to each data block according to the interval where the utilization rate is located includes:

[0039] When the space recovery mode is the first mode, determining a corresponding space recovery strategy for the data block according to the first threshold and the second threshold;

[0040] When the space recovery mode is the second mode, determining a speed gear of the space recovery, adjusting the first threshold, the second threshold, the first coefficient, and the second system to a first threshold, a second threshold, and a first coefficient and a second coefficient corresponding to the speed gear according to the speed gear; determining a corresponding space recovery strategy for the data block according to the first threshold and the second threshold corresponding to the speed gear;

[0041] The method further comprises: when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient corresponding to the speed gear, increasing the sliding window length of the sliding window and increasing the sliding step length of the sliding window;

[0042] When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient corresponding to the speed gear, the sliding window length of the sliding window is reduced and the sliding step size of the sliding window is reduced.

[0043] A second aspect of an embodiment of the present application provides a space recovery device, the device comprising:

[0044] an acquisition module, configured to divide a data area in a storage system into a plurality of data blocks, and acquire a utilization rate of each data block contained in the plurality of data blocks, wherein the utilization rate of the data block is determined based on a mapping between a logical address and a physical address of data contained in the data block;

[0045] A determination module, used to determine the space recovery strategy corresponding to each data block according to the interval of the utilization rate;

[0046] The recycling module is used to reclaim space for the data block according to the space recycling strategy.

[0047] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the space recovery method as described in the first aspect.

[0048] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the space recovery method as described in the first aspect is implemented.

[0049] Beneficial effects of this application:

[0050] The embodiments of the present application provide a space recovery method, apparatus, device and medium, the method comprising: dividing a data area in a storage system into a plurality of data blocks, and obtaining the utilization rate of each data block contained in the plurality of data blocks, wherein the utilization rate of the data block is determined based on the mapping between the logical address and the physical address of the data contained in the data block; determining the space recovery strategy corresponding to each data block according to the interval where the utilization rate is located; and performing space recovery on the data block according to the space recovery strategy. This ensures that the space resources of the storage system are effectively utilized. Determining the space recovery strategy for each data block according to the utilization rate can not only effectively and efficiently recover the storage space in the data block, but also optimize the data management of each data block in the storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application.

[0052] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 It is a flowchart of a space recovery method provided by an embodiment of the present application;

[0054] Figure 2 It is a schematic diagram of a mapping change between a logical address and a physical address in a space recovery method provided in an embodiment of the present application;

[0055] Figure 3 is a schematic diagram of a recycling queue and a sliding window in a space recycling method provided in an embodiment of the present application;

[0056] Figure 4 It is a schematic diagram of a framework of a space recovery device provided by an embodiment of the present application;

[0057] Figure 5 It is a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0058] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] First, the technical solution of the present application is applied to the SSR (Storage space recycling) management module. The SSR management module is located on the board and mainly uses programmable logic devices such as ARM. The module monitors the working status of all SSR operations in the cluster in real time and optimizes the handling measures for abnormal scenarios. Among them, space recovery is an important function in the SSR management module. The SSR management module monitors the data area of ​​the storage system. The space recovery method of the present application is further explained below through various embodiments.

[0061] Figure 1 is a flow chart of a space recovery method provided by an embodiment of the present application, such as Figure 1 shown.

[0062] A first aspect of an embodiment of the present application provides a space recovery method, the method comprising:

[0063] Step S101, dividing a data area in a storage system into a plurality of data blocks, and obtaining a utilization rate of each data block contained in the plurality of data blocks, wherein the utilization rate of the data block is determined according to a mapping between a logical address and a physical address of data contained in the data block;

[0064] Step S102, determining a space recovery strategy corresponding to each data block according to the interval of the utilization rate;

[0065] Step S103: reclaim space for the data block according to the space reclaiming strategy.

[0066] In step S101, the data area in the storage system is divided into multiple data blocks (i.e., blocks). The data area refers to the area in the storage system used to store actual data. The data can be a user's file, database record, or other form of information. The division of the data area is usually to optimize storage management, improve read and write efficiency, and ensure data security. The data block is the basic unit of the data area in the storage system. Different data blocks may contain data of different types and sizes. After the data blocks are divided, the data can be stored, accessed, and managed through the data blocks.

[0067] During the use of data blocks, the utilization rate of each data block included in multiple data blocks in the storage system can be obtained in real time. The utilization rate refers to the ratio of the storage space actually storing data in the data block to the total storage space of the data block. The calculation of the utilization rate of each data block is determined based on the mapping between the logical address and the physical address of the data contained in the data block, that is, the utilization rate of each data block is determined by analyzing the mapping relationship between the logical address (i.e., the address of the data in the storage system) and the physical address (i.e., the actual storage location of the data in the hardware) corresponding to the data contained in the data block. For example, if the total capacity of a data block is 100MB, but 30MB of data is actually stored, the utilization rate is 30%.

[0068] Furthermore, in step S102, in order to efficiently reclaim space in the storage system, the previous method of sequentially reclaiming each data block is no longer used, but it is necessary to determine the space recovery strategy based on the utilization rate of each data block. Therefore, after obtaining the utilization rate of each data block, it is necessary to determine the space recovery strategy corresponding to each data block based on the utilization rate of each data block. Different space recovery strategies are determined according to the interval in which the utilization rate of the data block is located. For example, a utilization rate of less than 20% is a utilization rate interval, a utilization rate of 20% to 80% is a utilization rate interval, and a utilization rate of more than 80% is a utilization rate interval. Different utilization rate intervals correspond to different space recovery strategies.

[0069] Finally, in step S103, once the space recovery strategy for each data block is determined, the actual space recovery operation can be performed on the data block according to these space recovery strategies. For example, for a data block with low utilization, its content may be completely deleted or archived. For a data block with high utilization, the measures that may be taken are to reorganize the data and optimize the storage method instead of recycling. At the same time, after the space recovery is completed, the impact of the space recovery operation on the storage system should be evaluated, including performance improvement, change in space utilization, etc.

[0070] Through the above embodiments, it is ensured that the space resources of the storage system are effectively utilized. Determining the space recovery strategy of each data block according to the utilization rate can not only effectively and efficiently recover the storage space in the data block, but also optimize the data management of each data block in the storage system.

[0071] Optionally, before obtaining the utilization rate of each data block included in the multiple data blocks, the method further includes:

[0072] Recording the usage of each data block included in the plurality of data blocks through a bitmap, wherein the used data block is recorded as a first value;

[0073] The obtaining the utilization rate of each data block included in the plurality of data blocks includes:

[0074] For each data block recorded as the first value, determine valid data, where the valid data is data having a mapping between a logical address and a physical address among a plurality of data included in the data block;

[0075] For each data block recorded as the first value, the utilization rate of the data block is calculated based on the sum of the physical address corresponding to the valid data and the address of the data block where the valid data is located.

[0076] Specifically, in one embodiment, in order to facilitate the management of the usage of each data block in the storage system, so as to determine which data blocks need to be used for utilization calculation later, the usage of each data block included in the multiple data blocks in the storage system can be recorded through a bitmap, and the bitmap is a data structure that uses bits to represent the state of a data set. By identifying the usage of the storage block. Each bit in the bitmap represents a data block, and a first value (for example, a value of 1) indicates that the data block is a used data block, and a second value (for example, a value of 0) indicates that the data block is an unused data block.

[0077] Regarding the process of obtaining the utilization rate of each data block contained in the data area of ​​the storage system, under this embodiment, each data block recorded as the first value can be determined based on the bitmap. Each data block recorded as the first value needs to have its utilization rate calculated, while each data block recorded as the second value does not need to have its utilization rate calculated. In this way, the data blocks that need to have their utilization rate calculated are screened out in a convenient manner, and there is no need to calculate the utilization rate of each data block, thereby improving the calculation efficiency of the utilization rate.

[0078] After filtering and obtaining the data blocks recorded as the first value, for each data block recorded as the first value, determine the valid data in the data block, wherein three areas contained in a used data block respectively store valid data, invalid data, and no data, the valid data being data with a mapping between a logical address and a physical address among the multiple data contained in the data block, the invalid data being data that was written historically but has expired, and the invalid data being data with no mapping between a logical address and a physical address.

[0079] Space recovery for data blocks is essentially the recovery of space occupied by invalid data.

[0080] After the data blocks recorded as the first value are filtered out, for each data block recorded as the first value, the utilization rate of the data block is calculated based on the sum of the physical address corresponding to the valid data stored in the data block and the address of the data block where the valid data is located (i.e., the capacity of the data block).

[0081] For example, data block A is any data block in the storage system whose utilization needs to be calculated. The calculation formula for the utilization of data block A can be expressed by the following formula:

[0082] Utilization rate = physical address corresponding to valid data in data block A / total address of data block A.

[0083] Through the above embodiment, the usage of each data block is recorded through a bitmap, and combined with the logical address and physical address mapping of valid data, the utilization rate of each data block in the storage system can be effectively calculated. This process provides strong support for storage management, and can quickly identify used data blocks and unused data blocks, and then quickly filter out the used data blocks that need to calculate the utilization rate, and then calculate the utilization rate according to the proportion of the total physical address of the valid data, thereby improving the calculation efficiency of the utilization rate.

[0084] Figure 2 It is a schematic diagram of the mapping changes between logical addresses and physical addresses in a space recovery method provided in an embodiment of the present application.

[0085] Optionally, the step S103 includes:

[0086] Copy valid data in multiple data blocks that need to be reclaimed into data blocks corresponding to consecutive physical addresses;

[0087] Initialize the physical address space of multiple data blocks that need space reclamation.

[0088] Specifically, in one embodiment, storage space is released by cleaning invalid data and organizing valid data to optimize the use of storage resources in the storage system.

[0089] For the space recovery strategy, it indicates that multiple data blocks need to be reclaimed, and the valid data is the data that needs to be retained in the storage system. Therefore, it is necessary to transfer the valid data in the multiple data blocks that need to be reclaimed first. The transfer process is: copy the valid data in the multiple data blocks to the data blocks corresponding to the continuous physical addresses. In this process, it is necessary to ensure that the utilization rate of the data blocks where the valid data is located is as high as possible after the copy. Therefore, according to the utilization rate of the multiple data blocks, the valid data in the multiple data blocks can be transferred to the same free data block. For example, the valid data in multiple data blocks with utilization rates of 20%, 30%, 20%, and 30% can be copied to the continuous physical addresses corresponding to the same data block. At the same time, the mapping of the logical address and the physical address of the transferred valid data is modified to point to the location where the valid data is actually stored.

[0090] like Figure 2 As shown, data blocks 1 to 3 all contain valid data and invalid data, as well as space without data, and the utilization rate is not high. At this time, the valid data in data blocks 1 to 3 need to be transferred to consecutive physical addresses, and space recovery is performed on data blocks 1 to 3. Through space recovery, the valid data in data blocks 1 to 3 are copied to the free data block 4, and after data blocks 1 to 3 are initialized, there is no data in data blocks 1 to 3, and the valid data is transferred to data block 4. The mapping of the logical address of the valid data to the physical address before space recovery is modified to the mapping of the logical address to the physical address after space recovery.

[0091] Optionally, in the process of extracting and copying valid data one by one to the selected new continuous physical addresses, it is necessary to ensure the integrity of the valid data before and after copying, and the consistency of the valid data during the copying process can be verified by means of checksum and the like.

[0092] After valid data is copied, the data block needs to be reclaimed. The valid data that has been copied in the data block that needs to be reclaimed also becomes invalid data. Therefore, the data block can be initialized directly to completely delete the invalid data in the data block, and the record of the initialized data block in the bitmap is modified to the second value to indicate that the data block can now be reallocated to new data writing. This helps the storage system better manage and allocate available storage space.

[0093] For example, assume that in a storage system, there are three data blocks (B1, B2, B3) containing the following data:

[0094] B1: valid data A, invalid data X;

[0095] B2: valid data B, invalid data Y;

[0096] B3: valid data C, invalid data Z;

[0097] According to the space recovery strategy, perform the following steps:

[0098] Step 1: Determine the data block:

[0099] The data blocks that need to be recovered are B1, B2, and B3.

[0100] Step 2: Data Copy:

[0101] Valid data A is extracted from B1, valid data B is extracted from B2, and valid data C is extracted from B3.

[0102] Choose a new physical address, say D1 (assumed to be empty), and copy the data to this new location:

[0103] D1 = A + B + C (placed in order).

[0104] Step 3: Initialize the original data space:

[0105] Initialize the physical address space storing invalid data in B1, B2, and B3, and mark these three data blocks (B1, B2, and B3) as 0 in the bitmap.

[0106] Through the above-mentioned embodiments, the existing valid data is extracted from the data blocks that need to reclaim space for transfer, and the storage system will significantly optimize the structure of data storage and release the space originally occupied by invalid data. In this way, the valid data is usually still useful data, which will not cause the loss of valid data and maintain the accessibility of valid data. Furthermore, selecting a continuous physical address in the storage system can not only ensure that the available space is sufficient to accommodate the valid data that needs to be copied, but also use continuous physical addresses to help improve the access efficiency of the storage system, because continuous physical addresses can reduce the addressing process and increase the possibility of cache hits. The storage system can achieve more efficient data management, reduce fragmentation, and improve data access efficiency.

[0107] Optionally, in order to ensure the consistency and integrity of data during the transfer of valid data, the following strategies can be used:

[0108] 1. In the process of copying valid data from the original data block to the new data block, an event is predefined. After the copy process fails, the rollback mechanism of the event is used to undo the related operations of the copy process.

[0109] 2. Calculate the checksum of valid data before and after copying. After the valid data is copied, perform checksum comparison operations on the copied valid data blocks to verify the integrity of the valid data during transmission. If the check fails, it means that the data is incorrect, and you can choose to copy it again or notify the system administrator.

[0110] 3. When performing data replication or initialization operations, a monitoring mechanism can be used to detect whether anomalies or interruptions occur. A timeout mechanism can be set. If the operation is not completed within the time limit, it will automatically end and roll back. In the event of a power outage or system crash during the replication of valid data, the storage system can automatically detect the incomplete replication of valid data at the next restart and automatically restore to the state before the replication, thereby avoiding the confusion of partially written and partially unwritten valid data.

[0111] 4. Before copying and initializing any valid data, generate a temporary backup of the data blocks that need space reclamation. Even if problems occur during operation, you can use the backed-up data for quick recovery. For example, before copying valid data, create a snapshot of the data block. If subsequent operations fail, you only need to recover from the snapshot without having to manually find the original data.

[0112] Through the above strategies and examples, the consistency and integrity of valid data can be effectively ensured during the replication and initialization operations. This multi-level protection mechanism can greatly reduce the risk of data loss due to interruptions or erroneous operations. Especially for large and small data systems, the implementation of the above strategies not only improves the security of data management, but also improves the stability and reliability of the storage system.

[0113] Optionally, in order to prevent the space reclamation process from weakening the performance of the storage system and affecting the normal operation of business, the space reclamation process can be monitored and corresponding monitoring indicators can be set to deal with the processing plan after the monitoring parameters exceed the threshold corresponding to the monitoring indicator.

[0114] Monitoring indicators include:

[0115] Read and write latency: Monitor the read and write latency of data blocks to assess the impact of space reclamation. If the space reclamation process causes increased latency, the system may need to adjust the strategy, such as reducing the concurrency of data replication or performing reclamation operations during off-peak hours.

[0116] IOPS (Input and Output Operations Per Second): Monitor IOPS during space reclamation to ensure that the workload of input and output operations does not drop significantly. It is ideal to perform reclamation operations during idle time periods.

[0117] CPU and memory usage: Monitors the system's CPU and memory usage. When the resources used during space reclamation exceed a certain threshold, it may affect the performance of other services.

[0118] Storage bandwidth: Monitor the bandwidth usage of storage media to ensure that no bottleneck is reached during space reclamation. Dynamic bandwidth adjustment policies can be used to optimize performance during space reclamation.

[0119] Through dedicated performance monitoring software (such as Prometheus, Grafana), various indicator data are collected in real time and visual charts are generated to facilitate timely detection of problems. When the monitoring indicator exceeds the preset threshold, an alarm is automatically triggered and the administrator is notified so that adjustments can be made in time.

[0120] For example, suppose a large database storage system performs space reclamation operations during idle time periods. The monitoring system records the following data during this process:

[0121] Read and write latency: increased from 5ms to 8ms.

[0122] IOPS: decreased from 3000 to 2500.

[0123] CPU usage: increased from 40% to 70%.

[0124] Storage bandwidth: Utilization reaches 90%.

[0125] After monitoring these indicators, you can use the following solutions:

[0126] 1. Pause space recovery operations.

[0127] 2. Optimize the performance of space recovery to reduce its occupation of system resources.

[0128] 3. Change the recycling to designated off-peak hours at night and increase the available IOPS.

[0129] Optionally, for the space recovery operation of the storage system, a periodic recovery method can be adopted, and different space recovery cycles can be determined according to the change frequency, business requirements and usage scenarios of different types of data. For example, for log files, recovery may be required once a week, while for databases, it can be selected once a month.

[0130] At the same time, based on the historical usage of system resources of the storage system, you can choose to reclaim the resources during a period of low storage system load, such as weekends or nights, to reduce the impact on user business operations.

[0131] By implementing performance monitoring and regular recycling strategies, the storage system can achieve efficient data management, avoid unnecessary storage waste and performance degradation, and ensure maximum utilization of storage resources while ensuring stability. Regular space recycling can not only maintain the stability of the storage system, but also extend the life of the storage hardware and improve the overall business operation efficiency.

[0132] Optionally, the step S102 includes:

[0133] When the utilization rate of the data block is less than a first threshold, determining that the space recovery strategy corresponding to the data block is direct recovery;

[0134] When the utilization rate of the data block is not less than the first threshold and less than the second threshold, determining that the space recovery strategy corresponding to the data block is waiting for recovery;

[0135] When the utilization rate of the data block is not less than the second threshold, determining that the space recovery strategy corresponding to the data block is not to recover;

[0136] The first threshold is smaller than the second threshold.

[0137] Specifically, in one embodiment, before determining the space recovery strategy according to the utilization interval, it is necessary to first set thresholds corresponding to different intervals, where the first threshold is less than the second threshold, wherein the first threshold is a lower percentage value, such as 20%, which is usually used to indicate that space recovery is required immediately. The second threshold is a higher percentage value, but lower than 100%, such as 80%, which is usually used to indicate that recovery is pending. The main purpose of setting these two thresholds is to optimize the recovery and use of storage space through hierarchical management.

[0138] When the utilization rate of a data block is less than the first threshold, it means that the data block has very little valid data, which makes it easy to recycle. Therefore, for quantitative space recovery tasks, giving priority to recovering data blocks with a utilization rate less than the first threshold can make space recovery more efficient. At this time, you can choose to directly recycle the data blocks with a utilization rate less than the first threshold to free up space.

[0139] When the utilization of a data block is between the first threshold and the second threshold, it means that the data block still has a certain amount of valid data. At this time, you can choose not to recycle it immediately, but enter the waiting state for recycling. At this stage, you can monitor the data block regularly to make decisions in the future.

[0140] When the utilization rate is higher than the second threshold, it indicates that the valid data of the data block accounts for a large proportion and the overall utilization rate is good. In this case, there is no need to recycle the data block to avoid affecting system performance and data access efficiency.

[0141] When the utilization rate is higher than the second threshold, it means that the valid data of the data block accounts for a large proportion, the overall utilization rate is good, and the time for space recovery is relatively long. At this time, there is no need to recover the space to avoid affecting the system performance and data access efficiency. Therefore, the data block is not reclaimed.

[0142] At the same time, as the storage system usage and data characteristics change, the first and second thresholds can be regularly evaluated and adjusted to ensure more efficient space reclamation.

[0143] Through the above embodiments, by setting different utilization thresholds and formulating corresponding space recovery strategies based on these thresholds, the storage system can achieve more efficient space management. The strategies of direct recovery, waiting for recovery, and no recovery help to avoid waste of storage space and ensure that system performance is not affected. This method of preferentially recovering low-utilization data blocks can effectively improve the overall performance and resource utilization of the storage system.

[0144] Optionally, before determining the space recovery strategy corresponding to each data block according to the interval where the utilization rate is located, the method further includes:

[0145] Dividing a plurality of data blocks contained in a data area in the storage system into a plurality of data block groups, each of the data block groups containing a plurality of data blocks;

[0146] Allocating a corresponding recycling queue to each of the data block groups, and configuring a corresponding sliding window for each recycling queue;

[0147] Set the sliding window length and sliding step size of each sliding window;

[0148] The step S102 comprises: when the sliding window performs a sliding operation, determining a space recovery strategy corresponding to each of the multiple data blocks currently included in the sliding window according to the utilization rate corresponding to each of the multiple data blocks currently included in the sliding window;

[0149] The method further comprises:

[0150] When the space recovery strategy corresponding to the data block in the sliding window is direct recovery, directly recovering the space of the data block;

[0151] When the space recovery strategy corresponding to the data block in the sliding window is waiting for recovery, the data block is added to the sorting queue, and the data blocks in the sorting queue are sorted in ascending order according to utilization rate;

[0152] When the space recovery strategy corresponding to the data block in the sliding window is not to recover, the space of the data block is not recovered;

[0153] The method further comprises:

[0154] After the data blocks whose utilization is less than the first threshold are recovered, the data blocks in the sorting queue are added back into the recovery queue in ascending order of utilization to recover space.

[0155] Figure 3 It is a schematic diagram of a recycling queue and a sliding window in a space recycling method provided in an embodiment of the present application.

[0156] Specifically, Figure 3 As shown, in one embodiment, space recovery can be performed based on a recovery queue and a sliding window to improve space recovery efficiency.

[0157] Before determining the space recovery strategy corresponding to each data block according to the utilization interval, it is necessary to group multiple data blocks to obtain multiple data block groups, each data block group contains multiple data blocks, for example, 3000 data blocks are contained, and each 1000 data blocks are grouped into three data block groups, including data block group 1: data block 1 to data block 1000, data block group 2: data block 1001 to data block 2000, and data block group 3: data block 2001 to data block 3000.

[0158] Furthermore, a corresponding recycling queue is allocated to each data block group, and a corresponding sliding window is configured for each recycling queue, and the recycling queue is used to manage space recycling operations of multiple data blocks included in the data block group.

[0159] A corresponding sliding window is configured for each recycling queue, including setting the sliding window length and sliding step of the sliding window. The sliding window length and sliding step have the same value, and the unit is pieces. The sliding window is used to calculate the utilization of the data blocks contained in the sliding window and determine the interval in which the utilization is located.

[0160] Sliding window length: refers to the number of data blocks contained in the sliding window in one sliding operation.

[0161] Sliding step size: refers to the number of blocks that the sliding window moves backward each time it slides.

[0162] When the sliding window performs a sliding operation, the utilization of multiple data blocks currently contained in the sliding window is checked. According to the utilization interval of each data block, the space recovery strategy of each data block can be determined.

[0163] When the space recovery strategy corresponding to the data block in the sliding window is direct recovery, the data block is directly reclaimed; when the space recovery strategy corresponding to the data block in the sliding window is waiting for recovery, the data block is transferred from the recovery queue to the sorting queue, and the multiple data blocks transferred to the sorting queue are sorted in ascending order according to utilization, and wait to be rejoined to the recovery queue for space recovery, wherein the sorting queue contains the ID of each data block and the corresponding utilization; when the space recovery strategy corresponding to the data block in the sliding window is not to recover, the data block is not reclaimed.

[0164] For example, assume that the utilization of data blocks D1 to D5 in a recycling queue is as follows:

[0165] D1: 10%;

[0166] D2: 30%;

[0167] D3: 50%;

[0168] D4: 80%;

[0169] D5: 90%;

[0170] Assume that the first threshold is 25% and the second threshold is 75%. The space recovery strategies in the sliding windows (D1, D2, D3) are:

[0171] D1: Direct recycling;

[0172] D2: Waiting for recycling;

[0173] D3: No recycling;

[0174] After space is reclaimed for multiple data blocks whose utilization is less than a first threshold, if the total space of the reclaimed data blocks is still insufficient, the multiple data blocks that have been arranged in ascending order of utilization in the sorting queue are added to the end of the recovery queue, and the added data blocks are traversed and space is reclaimed through a sliding window until the reclaimed space meets the required total space, and space recovery is stopped.

[0175] Through the above embodiment, by setting multiple recycling queues and multiple sliding windows, each recycling queue performs space recycling in parallel, thereby improving the space recycling efficiency of data blocks. Compared with a simple utilization threshold strategy, it can be more flexibly adjusted for different data block usage. The mechanism of regular inspection and reordering helps to optimize storage resources and reduce storage space waste, ensuring high efficiency and sustainability in space management.

[0176] Optionally, the method further comprises:

[0177] In the process of the sliding window performing the sliding operation, the theoretical recycling speed and the theoretical CPU usage of the system corresponding to the recycling queue are calculated, and the theoretical recycling speed and the theoretical CPU usage of the system corresponding to the recycling queue are determined according to the theoretical recycling speeds and the theoretical CPU usage of the system pre-set by the multiple data block models corresponding to the multiple data blocks currently contained in the sliding window in the recycling queue;

[0178] When the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient, increasing the sliding window length of the sliding window and increasing the sliding step size of the sliding window;

[0179] When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient, reducing the sliding window length of the sliding window and reducing the sliding step size of the sliding window;

[0180] The actual recycling speed of the recycling queue is determined according to the space size of the data blocks recycled per unit time.

[0181] Specifically, in one embodiment, the sliding window is a sampling technique that analyzes the utilization of data blocks and performs space recovery by "sliding" within a certain range (quantity and size). Whether the sliding window size and sliding step length of the sliding window can meet the efficiency of space recovery and load conditions, the sliding window can be adjusted according to the theoretical recovery speed and the theoretical usage of the system CPU.

[0182] The theoretical recovery speed is the recovery speed under ideal conditions calculated based on the characteristics of the data blocks in the recovery queue and the pre-set model. It is a predicted value that indicates how fast the storage system can complete the recovery of data blocks under the best conditions to achieve the best space recovery efficiency. The theoretical recovery speed can be reflected by the maximum size of space recovered per unit time.

[0183] The theoretical system CPU usage refers to the amount of system CPU resources required to perform space reclamation operations under normal conditions. This is also a predicted value obtained through the model.

[0184] The data block model is a model pre-established with a 0-1 distribution based on the usage of the data blocks. The sorting dimension of the data block model is the number of consecutive 0s, that is, the number of unused data blocks in the data area. Each data block has a corresponding data block model, and the theoretical recovery speed and system CPU theoretical usage pre-set for different data block models are different. Therefore, it is necessary to calculate the current corresponding theoretical recovery speed and system CPU theoretical usage of the sliding window in real time during the sliding operation of the sliding window to judge whether the actual recovery speed and system CPU theoretical usage of the multiple data blocks currently contained in the sliding window meet the current corresponding theoretical recovery speed and system CPU theoretical usage of the sliding window.

[0185] The theoretical recovery speed and system CPU theoretical usage currently corresponding to the sliding window are average values ​​of the theoretical recovery speeds and system CPU theoretical usage of data block models corresponding to multiple data blocks currently included in the sliding window.

[0186] For example, the calculation formula for the current theoretical recovery speed of the sliding window is: v = (a1+a2+...+am) / m, where v represents the current theoretical recovery speed of the sliding window, a1+a2+...+am represents the sum of the theoretical recovery speeds of the data block models corresponding to multiple (m) data blocks currently contained in the sliding window, and m is the sliding window size of the sliding window (in pieces, that is, the number of data blocks contained).

[0187] The calculation formula for the current theoretical system CPU usage of the sliding window is: s = (b1+b2+...+bm) / m, where s represents the current theoretical system CPU usage of the sliding window, b1+b2+...+bm represents the sum of the theoretical system CPU usage of the data block models corresponding to the multiple (m) data blocks currently contained in the sliding window, and m is the sliding window size of the sliding window (in pieces, that is, the number of data blocks contained).

[0188] For example, when the sliding window currently contains data block A, data block B, and data block C, the data block model corresponding to data block A is A1, the data block model corresponding to data block B is B1, and the data block model corresponding to data block C is C1. At this time, the theoretical recovery speed of data block model A1 is a1 and the theoretical system CPU usage is b1, the theoretical recovery speed of data block model B1 is a2 and the theoretical system CPU usage is b2, and the theoretical recovery speed of data block model C1 is a3 and the theoretical system CPU usage is b3. Then, the current theoretical recovery speed of the sliding window is (a1+a2+a3) / 3, and the current theoretical system CPU usage of the sliding window is (b1+b2+b3) / 3.

[0189] During the sliding operation of the sliding window, it is necessary to determine whether the current actual recovery speed of the sliding window is less than the product of the current theoretical recovery speed of the sliding window and the first coefficient based on the calculated current theoretical CPU usage of the sliding window. When the current actual recovery speed of the sliding window is less than the product of the current theoretical recovery speed of the sliding window and the first coefficient, it means that the actual recovery speed is too slow. At this time, the sliding window length and the sliding step size of the sliding window can be increased to improve the actual recovery speed. For example, the sliding window length can be increased by 1, and the sliding step size can be increased by 1.

[0190] At the same time, during the sliding operation of the sliding window, it is also necessary to judge whether the current actual system CPU usage of the sliding window is less than the product of the current theoretical system CPU usage of the sliding window and the second coefficient based on the calculated current theoretical system CPU usage of the sliding window. When the current actual system CPU usage of the sliding window is less than the product of the current theoretical system CPU usage of the sliding window and the second coefficient, it means that the space recovery operation has a certain impact on the hierarchical system. At this time, the sliding window length and the sliding step size of the sliding window can be reduced to reduce the pressure on the storage system. For example, the sliding window length can be reduced by 1, and the sliding step size can be reduced by 1.

[0191] Assume that at a certain moment, the recycling queue reclaims space, the theoretical recycling speed of the current sliding window is 2000MB / hour, and the theoretical usage of the system CPU is 100%.

[0192] The first coefficient is 0.8. Therefore, when the actual recovery rate is lower than 1600 MB / hour, the sliding window length and sliding step size need to be increased.

[0193] The actual recovery speed is 1500MB / hour, which is lower than 1600MB. Therefore, the system decides to increase the sliding window length and step size, and again determine whether the actual recovery speed is lower than 1600MB / hour.

[0194] The second coefficient is 1.2. When the actual system CPU usage exceeds 120%, the system will shorten the sliding window. If the actual system CPU usage is 135% higher than 120%, such an adjustment will also be triggered.

[0195] Through the above embodiments, by monitoring the actual operation status and theoretical expectations, and adjusting the sliding window parameters according to the actual performance of recovery and CPU load, optimal resource utilization and more efficient data recovery are achieved, while providing flexibility in space recovery.

[0196] Optionally, the space recovery mode is divided into a first mode and a second mode, and determining the space recovery strategy corresponding to each data block according to the interval where the utilization rate is located includes:

[0197] When the space recovery mode is the first mode, determining a corresponding space recovery strategy for the data block according to the first threshold and the second threshold;

[0198] When the space recovery mode is the second mode, determining a speed gear of the space recovery, adjusting the first threshold, the second threshold, the first coefficient, and the second system to a first threshold, a second threshold, and a first coefficient and a second coefficient corresponding to the speed gear according to the speed gear; determining a corresponding space recovery strategy for the data block according to the first threshold and the second threshold corresponding to the speed gear;

[0199] The method further comprises: when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient corresponding to the speed gear, increasing the sliding window length of the sliding window and increasing the sliding step length of the sliding window;

[0200] When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient corresponding to the speed gear, the sliding window length of the sliding window is reduced and the sliding step size of the sliding window is reduced.

[0201] Specifically, in one embodiment, the space recovery modes can be divided into two types, one is a first mode and the other is a second mode, wherein the first mode can be understood as a normal mode and the second mode can be understood as a high-speed mode.

[0202] In the first mode, the utilization interval of each data block is judged according to a pre-set first threshold and a pre-set second threshold. For example, the first threshold is 20% and the second threshold is 80%. Then, the corresponding space recovery strategy is determined based on the judged utilization interval.

[0203] Meanwhile, the first coefficient and the second coefficient are also preset, and the first threshold, the second threshold, the first coefficient and the second coefficient will not change in the first model.

[0204] In the second mode, it means that the speed of space recovery needs to be accelerated. At this time, the first threshold and the second threshold can be adjusted accordingly according to the different speed gears of space recovery set by the user to obtain the first threshold and the second threshold corresponding to the speed gear. When the space recovery speed needs to be accelerated, the first threshold is reduced and the second threshold is reduced. For example, the first threshold is reduced from 20% to 10%, and the second threshold is reduced from 80% to 50%. The corresponding space recovery strategy of the data block is determined according to the first threshold and the second threshold corresponding to the speed gear, and the space of the data block is recovered according to the determined space recovery strategy.

[0205] Since the first threshold is lowered, the space recovery strategy is to directly recover data blocks with a utilization rate of 1-20% and reduce it to 1-10%. By immediately recovering the data blocks with a utilization rate of 1-10%, the actual recovery speed will be improved due to the reduction in the average utilization rate. At this time, it can quickly respond to the higher actual recovery speed required by users. By lowering the second threshold, the data blocks with higher utilization rates can be screened out. In this way, the average utilization rate of the data blocks with a utilization rate from the first threshold to the second threshold is also reduced, and the actual recovery speed of this part of the data blocks will also be improved.

[0206] At the same time, the value of the first coefficient and the value of the second coefficient are increased at the same time. In this way, the product of the theoretical recovery speed and the first coefficient will increase. In order to increase the actual recovery speed, the sliding window length and the sliding step length in the recovery queue will increase, and more data block objects will be processed at one time. The product of the theoretical system CPU usage and the second coefficient will increase. In this way, when the actual system CPU usage increases slightly, it will not cause the sliding window length and the sliding step length in the recovery queue to decrease, thereby avoiding a reduction in the number of data blocks processed at one time.

[0207] The first threshold, the second threshold, the first coefficient and the second coefficient are all system preset parameters, and can be adjusted in a storage system or through a serial port module.

[0208] Through the above embodiments, in different modes, by adjusting the space recovery mode and speed gear, the data block recovery strategy is flexibly managed. By continuously monitoring the actual recovery speed and CPU usage, the length and step size of the sliding window are dynamically adjusted, so that storage resources can be effectively utilized while ensuring system performance. This method greatly improves the intelligence of storage management and ensures maximum resource utilization in different situations.

[0209] Figure 4 is a schematic diagram of a space recovery device provided by an embodiment of the present application. Figure 4 shown.

[0210] Based on the same inventive concept, another embodiment of the present application further provides a space recovery device, the device comprising:

[0211] An acquisition module 11 is used to divide a data area in a storage system into a plurality of data blocks, and acquire a utilization rate of each data block contained in the plurality of data blocks, wherein the utilization rate of the data block is determined according to a mapping between a logical address and a physical address of data contained in the data block;

[0212] A determination module 12, configured to determine a space recovery strategy corresponding to each data block according to the interval of the utilization rate;

[0213] The recycling module 13 is used to reclaim space for the data block according to the space recycling strategy.

[0214] Optionally, the device further comprises:

[0215] a recording module, configured to record the usage of each data block included in the plurality of data blocks through a bitmap before obtaining the utilization rate of each data block included in the plurality of data blocks, wherein the used data blocks are recorded as the first value;

[0216] The acquisition module 11 includes:

[0217] A valid data determination module, configured to determine valid data for each data block recorded as the first value, wherein the valid data is data having a mapping between a logical address and a physical address among a plurality of data included in the data block;

[0218] The utilization calculation module is used to calculate the utilization of each data block recorded as the first value based on the total physical address corresponding to the valid data and the total address corresponding to the data block where the valid data is located.

[0219] Optionally, the recycling module 13 comprises:

[0220] A copy unit, used for copying valid data in a plurality of data blocks that need to be space-reclaimed to data blocks corresponding to continuous physical addresses;

[0221] The initialization unit is used to initialize the physical address space of multiple data blocks that need to be reclaimed.

[0222] Optionally, the determining module 12 includes:

[0223] A first determining unit, configured to determine that a space recovery strategy corresponding to the data block is direct recovery when the utilization rate of the data block is less than a first threshold;

[0224] A second determining unit, configured to determine that the space recovery strategy corresponding to the data block is waiting for recovery when the utilization rate of the data block is not less than the first threshold and less than a second threshold;

[0225] a third determining unit, configured to determine, when the utilization rate of the data block is not less than the second threshold, that the space recovery strategy corresponding to the data block is not to recover the data block;

[0226] The first threshold is smaller than the second threshold.

[0227] Optionally, the device further comprises:

[0228] A data block group division module, used for dividing a plurality of data blocks included in a data area of ​​the storage system into a plurality of data block groups, each of which includes a plurality of data blocks, before determining a space recovery strategy corresponding to each data block according to the interval where the utilization rate is located;

[0229] A recycling queue allocation module, used to allocate a corresponding recycling queue to each of the data block groups, and configure a corresponding sliding window for each recycling queue;

[0230] A setting module, used to set the sliding window length and sliding step size of each sliding window;

[0231] The determination module 12 includes: a fourth determination unit, configured to determine, when the sliding window performs a sliding operation, a space recovery strategy corresponding to each of the multiple data blocks currently included in the sliding window according to the utilization rate corresponding to each of the multiple data blocks currently included in the sliding window;

[0232] The device also includes:

[0233] A first execution module, configured to directly reclaim space for the data block when the space reclaim strategy corresponding to the data block in the sliding window is direct reclaim;

[0234] A second execution module is used to add the data block to a sorting queue when the space recovery strategy corresponding to the data block in the sliding window is waiting for recovery, and the data blocks in the sorting queue are sorted in ascending order according to utilization rate;

[0235] A third execution module is used for not reclaiming space for the data block when the space reclaiming strategy corresponding to the data block in the sliding window is not to reclaim;

[0236] The device also includes:

[0237] A fourth execution module is configured to, after the data blocks whose utilization is less than the first threshold are recovered, add the data blocks in the sorting queue back into the recovery queue in ascending order of utilization for space recovery.

[0238] Optionally, the method further comprises:

[0239] A calculation module, used for calculating the theoretical recycling speed and system CPU theoretical usage currently corresponding to the recycling queue during the sliding operation of the sliding window, wherein the theoretical recycling speed and system CPU theoretical usage currently corresponding to the recycling queue are determined according to the theoretical recycling speed and system CPU theoretical usage pre-set by multiple data block models corresponding to multiple data blocks currently contained in the sliding window in the recycling queue;

[0240] A first adjustment module, configured to increase the sliding window length of the sliding window and increase the sliding step size of the sliding window when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient;

[0241] A second adjustment module is used to reduce the sliding window length of the sliding window and reduce the sliding step size of the sliding window when the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient;

[0242] The actual recycling speed of the recycling queue is determined according to the space size of the data blocks recycled per unit time.

[0243] Optionally, the space recovery mode is divided into a first mode and a second mode, and the determination module 12 includes:

[0244] a fourth determining unit, configured to determine, when the space reclamation mode is the first mode, a corresponding space reclamation strategy for the data block according to the first threshold and the second threshold;

[0245] a fifth determining unit, configured to determine, when the space recovery mode is the second mode, a speed gear of the space recovery, and adjust the first threshold, the second threshold, the first coefficient, and the second system to a first threshold, a second threshold, and a first coefficient and a second coefficient corresponding to the speed gear according to the speed gear; and determine a corresponding space recovery strategy for the data block according to the first threshold and the second threshold corresponding to the speed gear;

[0246] The first adjustment module further includes: a first adjustment unit, configured to increase the sliding window length of the sliding window and increase the sliding step length of the sliding window when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient corresponding to the speed gear;

[0247] The second adjustment module also includes: a second adjustment unit, which is used to reduce the sliding window length of the sliding window and reduce the sliding step size of the sliding window when the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient corresponding to the speed gear.

[0248] Based on the same inventive concept, another embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the space recovery method as described in any of the above embodiments.

[0249] Among them, electronic equipment refers to Figure 5 , Figure 5 is a schematic diagram of an electronic device provided by an embodiment of the present application. Figure 5 As shown, the electronic device 500 includes: a memory 510 and a processor 520. The memory 510 and the processor 520 are connected via a bus communication. A computer program is stored in the memory 510. The computer program can be run on the processor 520 to implement the steps in the space recovery method disclosed in the above embodiment of the present application.

[0250] Based on the same inventive concept, another embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to perform the space recovery method described in any of the above embodiments.

[0251] Based on the same inventive concept, another embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, wherein when the program is executed by a processor, the space recovery method as described in any of the above embodiments is implemented.

[0252] As for the device, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0253] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0254] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0255] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0256] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0257] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0258] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0259] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0260] The above is a detailed introduction to a space recovery method, device, equipment and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A space recovery method, characterized in that: The method comprises: Dividing a data area in a storage system into a plurality of data blocks, and obtaining a utilization rate of each data block contained in the plurality of data blocks, wherein the utilization rate of the data block is determined according to a mapping between a logical address and a physical address of data contained in the data block; Determining a space recovery strategy corresponding to each data block according to the interval of the utilization rate; According to the space recovery strategy, space recovery is performed on the data block.

2. The space recovery method according to claim 1, characterized in that: Before obtaining the utilization rate of each data block included in the plurality of data blocks, the method further includes: Recording the usage of each data block included in the plurality of data blocks through a bitmap, wherein the used data block is recorded as a first value; The obtaining the utilization rate of each data block included in the plurality of data blocks includes: For each data block recorded as the first value, determine valid data, where the valid data is data having a mapping between a logical address and a physical address among a plurality of data included in the data block; For each data block recorded as the first value, the utilization rate of the data block is calculated based on the sum of the physical address corresponding to the valid data and the address of the data block where the valid data is located.

3. The space recovery method according to claim 2, characterized in that: The performing space reclamation on the data block according to the space reclamation strategy includes: Copy valid data in multiple data blocks that need to be reclaimed into data blocks corresponding to consecutive physical addresses; Initialize the physical address space of multiple data blocks that need space reclamation.

4. The space recovery method according to claim 1, characterized in that: The determining, according to the interval in which the utilization rate is located, a space recovery strategy corresponding to each data block, includes: When the utilization rate of the data block is less than a first threshold, determining that the space recovery strategy corresponding to the data block is direct recovery; When the utilization rate of the data block is not less than the first threshold and less than the second threshold, determining that the space recovery strategy corresponding to the data block is waiting for recovery; When the utilization rate of the data block is not less than the second threshold, determining that the space recovery strategy corresponding to the data block is not to recover; The first threshold is smaller than the second threshold.

5. The space recovery method according to claim 4, characterized in that: Before determining the space recovery strategy corresponding to each data block according to the interval where the utilization rate is located, the method further includes: Dividing a plurality of data blocks contained in a data area in the storage system into a plurality of data block groups, each of the data block groups containing a plurality of data blocks; Allocating a corresponding recycling queue to each of the data block groups, and configuring a corresponding sliding window for each recycling queue; Set the sliding window length and sliding step size of each sliding window; The determining, according to the interval in which the utilization rate is located, the space recovery strategy corresponding to each of the data blocks, comprises: when the sliding window performs a sliding operation, determining, according to the utilization rate corresponding to each of the data blocks currently included in the sliding window, the space recovery strategy corresponding to each of the data blocks currently included in the sliding window; The method further comprises: When the space recovery strategy corresponding to the data block in the sliding window is direct recovery, directly recovering the space of the data block; When the space recovery strategy corresponding to the data block in the sliding window is waiting for recovery, the data block is added to the sorting queue, and the data blocks in the sorting queue are sorted in ascending order according to utilization rate; When the space recovery strategy corresponding to the data block in the sliding window is not to recover, the space of the data block is not recovered; The method further comprises: After the data blocks whose utilization is less than the first threshold are recovered, the data blocks in the sorting queue are added back to the recovery queue in ascending order of utilization to recover space.

6. The space recovery method according to claim 5, characterized in that: The method further comprises: In the process of the sliding window performing the sliding operation, the theoretical recycling speed and the theoretical CPU usage of the system corresponding to the recycling queue are calculated, and the theoretical recycling speed and the theoretical CPU usage of the system corresponding to the recycling queue are determined according to the theoretical recycling speeds and the theoretical CPU usage of the system pre-set by the multiple data block models corresponding to the multiple data blocks currently contained in the sliding window in the recycling queue; When the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient, increasing the sliding window length of the sliding window and increasing the sliding step size of the sliding window; When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient, reducing the sliding window length of the sliding window and reducing the sliding step size of the sliding window; The actual recycling speed of the recycling queue is determined according to the space size of the data blocks recycled per unit time.

7. The space recovery method according to claim 6, characterized in that: The space recovery mode is divided into a first mode and a second mode, and the space recovery strategy corresponding to each data block is determined according to the interval where the utilization rate is located, including: When the space recovery mode is the first mode, determining a corresponding space recovery strategy for the data block according to the first threshold and the second threshold; When the space recovery mode is the second mode, determining a speed gear of the space recovery, adjusting the first threshold, the second threshold, the first coefficient, and the second system to a first threshold, a second threshold, and a first coefficient and a second coefficient corresponding to the speed gear according to the speed gear; determining a corresponding space recovery strategy for the data block according to the first threshold and the second threshold corresponding to the speed gear; The method further comprises: when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient corresponding to the speed gear, increasing the sliding window length of the sliding window and increasing the sliding step length of the sliding window; When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient corresponding to the speed gear, the sliding window length of the sliding window is reduced and the sliding step size of the sliding window is reduced.

8. A space recovery device, characterized in that: The device comprises: an acquisition module, configured to divide a data area in a storage system into a plurality of data blocks, and acquire a utilization rate of each data block contained in the plurality of data blocks, wherein the utilization rate of the data block is determined based on a mapping between a logical address and a physical address of data contained in the data block; A determination module, used to determine the space recovery strategy corresponding to each data block according to the interval of the utilization rate; The recycling module is used to reclaim space for the data block according to the space recycling strategy.

9. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the space recovery method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the space recovery method as described in any one of claims 1-7 is implemented.

Images