Space recycling method, apparatus, device, and medium

Abstract

The application provides a space recycling method and device, equipment and medium, and relates to the technical field of computers.The method comprises the following steps: dividing a data area in a storage system into a plurality of data blocks, and obtaining the utilization rate of each data block contained in the plurality of data blocks, wherein the utilization rate of the data block is determined according to the mapping of the logical address and the physical address of the data contained in the data block; determining the respective space recycling strategy of each data block according to the interval of the utilization rate; and recycling the space of the data block according to the space recycling strategy. The space resource of the storage system is effectively utilized. The space recycling strategy of each data block is determined according to the utilization rate, which not only effectively and efficiently recycles the storage space in the data block, but also optimizes the data management of each data block in the storage system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a space reclamation method, apparatus, device and medium. Background Technology

[0002] Garbage collection refers to the process of releasing unused storage space, preventing space leaks, and effectively utilizing available storage space. Its principle is mainly to track objects in the storage space during program runtime, determine which objects are no longer referenced by the program, mark the storage space of these objects as reclaimable, and finally have the system reclaim this space so that it can be reused.

[0003] Current storage systems are generally based on log structures, using append-only writes to achieve fast data writing. However, due to data modification, erasure, and deletion operations, the actual data on the hard drive is not deleted synchronously; only the corresponding physical space mapping is deleted. This results in a lot of invalid data being stored in the used space of the storage system. To improve disk utilization, the space occupied by this invalid data needs to be reclaimed. However, related technologies reclaim the blocks in the storage system sequentially, which cannot improve the efficiency of space reclamation. Summary of the Invention

[0004] In view of this, embodiments of this application provide a space recycling method, apparatus, device, and medium to overcome or at least partially solve the above problems.

[0005] A first aspect of this application provides a space reclamation method, the method comprising:

[0006] The data area in the storage system is divided into multiple data blocks, and the utilization rate of each data block contained in the multiple data blocks is obtained. The utilization rate of the data block is determined based on the mapping between the logical address and the physical address of the data contained in the data block.

[0007] Based on the range in which the utilization rate falls, determine the space reclamation strategy corresponding to each data block;

[0008] The data block is reclaimed according to the space reclamation strategy.

[0009] Optionally, before obtaining the utilization rate of each data block contained in the plurality of data blocks, the method further includes:

[0010] The usage of each data block contained in the plurality of data blocks is recorded using a bitmap, wherein the used data blocks are recorded as a first value;

[0011] The step of obtaining the utilization rate of each data block contained in the plurality of data blocks includes:

[0012] For each data block recorded as the first value, valid data is determined. The valid data is the data in the data block that contains a mapping between logical address and physical address.

[0013] For each data block recorded as the first value, the utilization rate of the data block is calculated based on the sum of the physical address corresponding to the valid data and the address of the data block containing the valid data.

[0014] Optionally, the step of reclaiming space from the data block according to the space reclamation strategy includes:

[0015] Copy the valid data from multiple data blocks that need to be reclaimed to the data blocks corresponding to consecutive physical addresses;

[0016] Initialize the physical address space of multiple data blocks that need to be reclaimed.

[0017] Optionally, determining the space reclamation strategy corresponding to each data block based on the interval where the utilization rate falls includes:

[0018] When the utilization rate of the data block is less than the first threshold, the space reclamation strategy corresponding to the data block is determined to be direct reclamation;

[0019] When the utilization rate of the data block is not less than the first threshold and less than the second threshold, the space reclamation strategy corresponding to the data block is determined to be waiting for reclamation;

[0020] When the utilization rate of the data block is not less than the second threshold, the space reclamation strategy corresponding to the data block is determined to be not to reclaim it;

[0021] Wherein, the first threshold is less than the second threshold.

[0022] Optionally, before determining the space reclamation strategy corresponding to each data block based on the interval where the utilization rate falls, the method further includes:

[0023] The data area in the storage system is divided into multiple data block groups, and each data block group contains multiple data blocks;

[0024] Assign a corresponding recycling queue to each of the data block groups, and configure a corresponding sliding window for each recycling queue;

[0025] Set the sliding window length and sliding step size for each sliding window;

[0026] The step of determining the space reclamation strategy corresponding to each data block based on the interval in which the utilization rate is located includes: when the sliding window performs a sliding operation, determining the space reclamation strategy corresponding to each data block in the multiple data blocks currently contained in the sliding window based on the utilization rate of each data block in the multiple data blocks currently contained in the sliding window.

[0027] The method further includes:

[0028] When the space reclamation strategy corresponding to the data block in the sliding window is direct reclamation, the space of the data block is directly reclaimed;

[0029] When the space reclamation strategy corresponding to the data block in the sliding window is to wait for reclamation, the data block is added to the sorting queue, and the data blocks in the sorting queue are sorted in ascending order of utilization rate;

[0030] When the space reclamation policy corresponding to the data block in the sliding window is not to reclaim space, the space of the data block is not reclaimed.

[0031] The method further includes:

[0032] After the data blocks with utilization rates less than the first threshold are reclaimed, the data blocks in the sorting queue are added back to the reclamation queue in ascending order of utilization rate for space reclamation.

[0033] Optionally, the method further includes:

[0034] During the sliding operation of the sliding window, the theoretical recycling speed and theoretical CPU usage of the system corresponding to the current recycling queue are calculated. The theoretical recycling speed and theoretical CPU usage of the system corresponding to the current recycling queue are determined based on the theoretical recycling speed and theoretical CPU usage of the system pre-set according to the multiple data block models corresponding to the multiple data blocks currently contained in the sliding window of the recycling queue.

[0035] When the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient, the sliding window length and the sliding step size of the sliding window are increased.

[0036] When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient, the sliding window length and the sliding step size of the sliding window are reduced.

[0037] The actual recycling speed of the recycling queue is determined based on the size of the data blocks recycled per unit time.

[0038] Optionally, the space reclamation mode is divided into a first mode and a second mode. The step of determining the space reclamation strategy corresponding to each data block based on the utilization rate interval includes:

[0039] When the space reclamation mode is the first mode, the corresponding space reclamation strategy for the data block is determined according to the first threshold and the second threshold;

[0040] When the space reclamation mode is the second mode, the speed level of the space reclamation is determined, and the first threshold, the second threshold, the first coefficient, and the second system are adjusted to the first threshold, the second threshold, the first coefficient, and the second coefficient corresponding to the speed level according to the speed level; the corresponding space reclamation strategy for the data block is determined according to the first threshold and the second threshold corresponding to the speed level.

[0041] The method further includes: when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient corresponding to the speed level, increasing the sliding window length and the sliding step size of the sliding window;

[0042] When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient corresponding to the speed level, the sliding window length and the sliding step size of the sliding window are reduced.

[0043] A second aspect of this application provides a space recycling device, the device comprising:

[0044] The acquisition module is used to divide the data area in the storage system into multiple data blocks and acquire the utilization rate of each data block contained in the multiple data blocks, wherein the utilization rate of the data block is determined based on the mapping between the logical address and the physical address of the data contained in the data block;

[0045] The determination module is used to determine the space reclamation strategy corresponding to each data block based on the interval in which the utilization rate is located;

[0046] The recycling module is used to reclaim space from the data block according to the space reclamation strategy.

[0047] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the space reclamation method as described in the first aspect.

[0048] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the space reclamation method as described in the first aspect.

[0049] The beneficial effects of this application are:

[0050] This application provides a space reclamation method, apparatus, device, and medium. The method includes: dividing a data area in a storage system into multiple data blocks, and obtaining the utilization rate of each data block contained in the multiple data blocks, wherein the utilization rate of the data block is determined based on the mapping between the logical address and physical address of the data contained in the data block; determining a space reclamation strategy corresponding to each data block based on the interval in which the utilization rate falls; and reclaiming space from the data blocks according to the space reclamation strategy. This ensures that the space resources of the storage system are effectively utilized. Determining the space reclamation strategy for each data block according to the utilization rate not only effectively and efficiently reclaims storage space in the data blocks but also optimizes the data management of each data block in the storage system. Attached Figure Description

[0051] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0052] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic flowchart of a space reclamation method provided in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram illustrating the mapping change between logical addresses and physical addresses in a space reclamation method provided in an embodiment of this application;

[0055] Figure 3 This is a schematic diagram of a recycling queue and a sliding window in a space recycling method provided in an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of the frame of a space recycling device provided in one embodiment of this application;

[0057] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0058] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] First, the technical solution of this application is applied to the SSR (Storage Space Recycling) management module. The SSR management module is located on a board and primarily utilizes programmable logic devices such as ARM. This module monitors the real-time operating status of all SSRs within the cluster and optimizes handling measures for abnormal scenarios. Space recycling is a crucial function of the SSR management module. The SSR management module monitors the data area of ​​the storage system. The space recycling method of this application will be further explained below through various embodiments.

[0061] Figure 1 This is a schematic flowchart of a space reclamation method provided in an embodiment of this application, as shown below. Figure 1 As shown.

[0062] A first aspect of this application provides a space reclamation method, the method comprising:

[0063] Step S101: Divide the data area in the storage system into multiple data blocks and obtain the utilization rate of each data block contained in the multiple data blocks, wherein the utilization rate of the data block is determined based on the mapping between the logical address and the physical address of the data contained in the data block;

[0064] Step S102: Determine the space reclamation strategy corresponding to each data block based on the range of utilization rate.

[0065] Step S103: Reclaim space from the data block according to the space reclamation strategy.

[0066] In step S101, the data area in the storage system is divided into multiple data blocks. A data area refers to the region in the storage system used to store actual data. Data can be user files, database records, or other forms of information. The division of data areas is typically for optimizing storage management, improving read / write efficiency, and ensuring data security. A data block is the basic unit of the data area in the storage system; different data blocks may contain data of different types and sizes. After the data blocks are obtained, data can be stored, accessed, and managed through these blocks.

[0067] During data block usage, the utilization rate of each data block within multiple data blocks in the storage system can be obtained in real time. Utilization rate refers to the ratio of the actual storage space of the data in a data block to the total storage space of that data block. The utilization rate of each data block is calculated based on the mapping between the logical addresses and physical addresses of the data contained within that data block. In other words, the utilization rate of each data block is determined by analyzing the mapping relationship between the logical addresses (i.e., the addresses of the data in the storage system) and the physical addresses (i.e., the actual storage location of the data in the hardware) of the data contained within the data block. For example, if a data block has a total capacity of 100MB and actually stores 30MB of data, then the utilization rate is 30%.

[0068] Furthermore, in step S102, in order to efficiently reclaim space from the storage system, the previous method of sequentially reclaiming each data block is no longer followed. Instead, a space reclamation strategy needs to be determined based on the utilization rate of each data block. Therefore, after obtaining the utilization rate of each data block, the next step is to determine the space reclamation strategy corresponding to each data block based on its utilization rate. Different space reclamation strategies are determined according to the range of data block utilization rates. For example, a utilization rate of less than 20% constitutes one utilization rate range, a utilization rate between 20% and 80% constitutes another, and a utilization rate above 80% constitutes yet another. Different utilization rate ranges correspond to different space reclamation strategies.

[0069] Finally, in step S103, once the space reclamation strategy for each data block is determined, the actual space reclamation operation can be performed on the data blocks according to these strategies. For example, for low-utilization data blocks, their contents may be completely deleted or archived. For high-utilization data blocks, measures such as reorganizing the data and optimizing the storage method may be taken instead of reclamation. At the same time, after the space reclamation is completed, the impact of the space reclamation operation on the storage system should be evaluated, including performance improvement and changes in space utilization.

[0070] The above embodiments ensure that the storage system's space resources are used effectively. Determining the space reclamation strategy for each data block based on utilization rate not only effectively and efficiently reclaims storage space within data blocks but also optimizes data management for each data block in the storage system.

[0071] Optionally, before obtaining the utilization rate of each data block contained in the plurality of data blocks, the method further includes:

[0072] The usage of each data block contained in the plurality of data blocks is recorded using a bitmap, wherein the used data blocks are recorded as a first value;

[0073] The step of obtaining the utilization rate of each data block contained in the plurality of data blocks includes:

[0074] For each data block recorded as the first value, valid data is determined. The valid data is the data in the data block that contains a mapping between logical address and physical address.

[0075] For each data block recorded as the first value, the utilization rate of the data block is calculated based on the sum of the physical address corresponding to the valid data and the address of the data block containing the valid data.

[0076] Specifically, in one embodiment, to facilitate the management of the usage of each data block in the storage system and to subsequently determine which data blocks require utilization calculations, a bitmap can be used to record the usage of each data block within a plurality of data blocks in the storage system. A bitmap is a data structure that uses bits to represent the state of a data set. The usage of storage blocks is identified by these bits. Each bit in the bitmap represents a data block, with a first value (e.g., 1) indicating that the data block is used, and a second value (e.g., 0) indicating that the data block is unused.

[0077] In this embodiment, the process of obtaining the utilization rate of each data block contained in the data area of ​​the storage system can be specifically determined based on the bitmap. The data blocks with the first value need to be used for utilization rate calculation, while the data blocks with the second value do not need to be used for utilization rate calculation. This allows for a convenient way to filter out the data blocks that need to be used for utilization rate calculation, without having to calculate the utilization rate of each data block, thus improving the efficiency of utilization rate calculation.

[0078] After filtering out the data blocks that are recorded as the first value, for each data block recorded as the first value, the valid data in this data block is determined. Among them, the three areas contained in a used data block store valid data, invalid data, and no data. Valid data is the data in the data block that has a mapping between logical address and physical address. Invalid data is the data that was written in the past but has expired. Invalid data is the data that does not have a mapping between logical address and physical address.

[0079] The reclamation of data block space is essentially the reclamation of space occupied by invalid data.

[0080] After filtering out the data blocks recorded as the first value, for each data block recorded as the first value, the utilization rate of the data block is calculated based on the sum of the physical address corresponding to the valid data stored in the data block and the address of the data block containing the valid data (i.e., the capacity of the data block).

[0081] For example, data block A is any data block in the storage system whose utilization needs to be calculated. The formula for calculating the utilization of data block A can be expressed as follows:

[0082] Utilization = Physical address of valid data in data block A / Total address of data block A.

[0083] Through the above embodiments, by recording the usage of each data block using a bitmap and combining the mapping between the logical and physical addresses of valid data, the utilization rate of each data block in the storage system can be effectively calculated. This process provides strong support for storage management, enabling rapid identification of used and unused data blocks, quickly filtering out used data blocks for which utilization calculations are needed, and then calculating the utilization rate based on the proportion of the total physical addresses of valid data, thus improving the efficiency of utilization calculation.

[0084] Figure 2 This is a schematic diagram illustrating the mapping changes between logical addresses and physical addresses in a space reclamation method provided in an embodiment of this application.

[0085] Optionally, step S103 includes:

[0086] Copy the valid data from multiple data blocks that need to be reclaimed to the data blocks corresponding to consecutive physical addresses;

[0087] Initialize the physical address space of multiple data blocks that need to be reclaimed.

[0088] Specifically, in one embodiment, storage space is freed up by cleaning up invalid data and organizing valid data, thereby optimizing the use of storage resources in the storage system.

[0089] For a space reclamation strategy, multiple data blocks that need to be reclaimed must have valid data that needs to be retained in the storage system. Therefore, the valid data in the multiple data blocks that need to be reclaimed must first be transferred. The transfer process is as follows: the valid data in the multiple data blocks is copied to the data blocks corresponding to consecutive physical addresses. In this process, it is necessary to ensure that the utilization rate of the data block containing the valid data is as high as possible after copying. Therefore, the valid data in the multiple data blocks can be transferred to the same free data block according to the utilization rate of the multiple data blocks. For example, the valid data in multiple data blocks with utilization rates of 20%, 30%, 20%, and 30% can be copied to the consecutive physical addresses corresponding to the same data block. At the same time, the mapping between the logical address and physical address of the transferred valid data is modified to point to the actual storage location of the valid data.

[0090] like Figure 2 As shown, data blocks 1 through 3 contain valid data, invalid data, and empty space, resulting in low utilization. Therefore, it is necessary to move the valid data from data blocks 1 through 3 to contiguous physical addresses and reclaim the space in these blocks. Through space reclamation, the valid data from data blocks 1 through 3 is copied to the free data block 4. After initializing data blocks 1 through 3, they will be empty, and the valid data will be moved to data block 4. The mapping between the logical address of the valid data and the physical address before space reclamation will be changed to a mapping between the logical address and the physical address after space reclamation.

[0091] Optionally, during the process of extracting and copying valid data one by one to a selected new contiguous physical address, it is necessary to ensure the integrity of the valid data before and after copying. This can be achieved by verifying the consistency of the valid data during the copying process through methods such as checksums.

[0092] After valid data is copied, space needs to be reclaimed from the data blocks. The valid data already copied within these blocks becomes invalid. Therefore, the data blocks can be directly initialized to completely delete the invalid data and modify the bitmap record of the initialized data blocks to a second value, indicating that these blocks can now be reallocated for new data writing. This helps the storage system better manage and allocate available storage space.

[0093] For example, suppose there are three data blocks (B1, B2, B3) in a certain storage system, containing the following data:

[0094] B1: Valid data A, invalid data X;

[0095] B2: Valid data B, invalid data Y;

[0096] B3: Valid data C, invalid data Z;

[0097] Based on the space reclamation strategy, the following steps are performed:

[0098] Step 1: Identify the data block:

[0099] The data blocks that need to be reclaimed are B1, B2, and B3.

[0100] Step 2: Data Copying

[0101] Extract valid data A from B1, extract valid data B from B2, and extract valid data C from B3.

[0102] Choose a new physical address, such as D1 (assuming it's empty), and copy the data to this new location:

[0103] D1 = A + B + C (placed in order).

[0104] Step 3: Initialize the original data space:

[0105] The physical address space storing invalid data in B1, B2, and B3 is initialized, and these three data blocks (B1, B2, and B3) are marked as 0 in the bitmap.

[0106] Through the above embodiments, valid data is extracted and transferred from data blocks that require space reclamation. The storage system significantly optimizes the data storage structure and releases the space previously occupied by invalid data. In this way, valid data is usually still useful data, and there is no loss of valid data, maintaining the accessibility of valid data. Furthermore, selecting a contiguous physical address in the storage system not only ensures that the available space is sufficient to accommodate the valid data that needs to be copied, but also helps improve the access efficiency of the storage system. This is because contiguous physical addresses can reduce the addressing process and increase the probability of cache hits. The storage system can achieve more efficient data management, reduce fragmentation, and improve the efficiency of data access.

[0107] Alternatively, to ensure data consistency and integrity during the transfer of valid data, the following strategies can be used:

[0108] 1. During the process of copying valid data from the original data block to the new data block, a predefined event is used. If the copying process fails, the rollback mechanism of this event is used to undo the relevant operations of the copying process.

[0109] 2. Calculate the checksum of the valid data before and after copying. After copying the valid data, perform a checksum comparison operation on the copied valid data block to verify the integrity of the valid data during transmission. If the checksum fails, it indicates that the data is incorrect, and you can choose to recopy it or notify the system administrator.

[0110] 3. During data replication or initialization operations, a monitoring mechanism can be employed to detect any anomalies or interruptions. A timeout mechanism can be set so that if the operation is not completed within the specified time, it will automatically terminate and roll back. In the event of a power outage or system crash during the replication of valid data, the storage system can automatically detect the incomplete replication of valid data upon the next restart and automatically restore the system to its state before replication, thereby avoiding the chaos of partially written valid data and partially unwritten data.

[0111] 4. Before copying and initializing any valid data, generate a temporary backup of the data blocks that require space reclamation. Even if problems occur during operation, the backup data can be used for quick recovery. For example, create a snapshot of the data block before copying valid data. If subsequent operations fail, simply restore from the snapshot; there's no need to manually search for the original data.

[0112] The strategies and examples described above effectively ensure the consistency and integrity of valid data during replication and initialization operations. This multi-layered protection mechanism significantly reduces the risk of data loss due to interruptions or erroneous operations. Especially for large and small data systems, implementing these strategies not only enhances data management security but also improves the stability and reliability of the storage system.

[0113] Optionally, to prevent the space reclamation process from weakening the performance of the storage system and affecting the operation of normal business, the space reclamation process can be monitored, and corresponding monitoring indicators can be set to deal with the handling scheme when the monitoring parameters exceed the threshold corresponding to the monitoring indicators.

[0114] Monitoring indicators include:

[0115] Read / write latency: Monitor the read / write latency of data blocks to assess the impact of space reclamation. If the space reclamation process causes increased latency, the system may need to adjust its strategies, such as reducing the concurrency of data replication or performing reclamation operations during off-peak hours.

[0116] IOPS (Input / Output Operations Per Second): IOPS is monitored during space reclamation to ensure that the workload of input / output operations does not decrease significantly. Reclamation during idle periods is ideal.

[0117] CPU and memory usage: Monitors the system's CPU and memory usage. When resources used during space reclamation exceed a certain threshold, it may impact the performance of other services.

[0118] Storage bandwidth: Monitor the bandwidth usage of the storage media to ensure that a bottleneck is not reached during space reclamation. Dynamic bandwidth adjustment strategies can be employed to optimize performance during space reclamation.

[0119] Dedicated performance monitoring software (such as Prometheus and Grafana) is used to collect various metric data in real time and generate visual charts to facilitate timely problem identification. When monitored metrics exceed preset thresholds, alarms are automatically triggered and administrators are notified for timely adjustments.

[0120] For example, suppose a large database storage system performs space reclamation operations during idle periods, and the monitoring system records the following data during this process:

[0121] Read / write latency: increased from 5ms to 8ms.

[0122] IOPS: decreased from 3000 to 2500.

[0123] CPU utilization increased from 40% to 70%.

[0124] Storage bandwidth: Utilization reaches 90%.

[0125] After monitoring these metrics, the following processing methods can be used:

[0126] 1. Pause space reclamation operations.

[0127] 2. Optimize the performance of space reclamation to reduce its consumption of system resources.

[0128] 3. Change the recycling to the specified nighttime off-peak period and increase the available IOPS.

[0129] Optionally, for storage system space reclamation operations, a periodic reclamation method can be adopted. Different space reclamation cycles can be determined based on the change frequency of different types of data, business needs, and usage scenarios. For example, log files may need to be reclaimed once a week, while databases can be reclaimed once a month.

[0130] Additionally, based on the historical system resource usage data, data reclamation can be performed during periods of lower storage system load, such as weekends or nighttimes, thus minimizing disruption to user operations.

[0131] By implementing performance monitoring and regular space reclamation strategies, storage systems can achieve efficient data management, avoiding unnecessary storage waste and performance degradation. This ensures that storage resources are utilized to the maximum extent while maintaining stability. Regular space reclamation not only maintains the stability of the storage system but also extends the lifespan of storage hardware and improves overall business operational efficiency.

[0132] Optionally, step S102 includes:

[0133] When the utilization rate of the data block is less than the first threshold, the space reclamation strategy corresponding to the data block is determined to be direct reclamation;

[0134] When the utilization rate of the data block is not less than the first threshold and less than the second threshold, the space reclamation strategy corresponding to the data block is determined to be waiting for reclamation;

[0135] When the utilization rate of the data block is not less than the second threshold, the space reclamation strategy corresponding to the data block is determined to be not to reclaim it;

[0136] Wherein, the first threshold is less than the second threshold.

[0137] Specifically, in one embodiment, before determining the space reclamation strategy based on utilization intervals, thresholds corresponding to different intervals need to be set. The first threshold is lower than the second threshold. The first threshold is a lower percentage value, such as 20%, typically used to indicate situations requiring immediate space reclamation. The second threshold is a higher percentage value, but below 100%, such as 80%, typically used to indicate situations where reclamation is pending. The main purpose of setting these two thresholds is to optimize storage space reclamation and usage through hierarchical management.

[0138] When the utilization rate of a data block is less than the first threshold, it means that there is very little effective data in the data block. This situation is conducive to recycling. Therefore, for a quantitative space reclamation task, prioritizing the reclamation of data blocks with a utilization rate less than the first threshold can make the space reclamation more efficient. At this time, you can choose to directly reclaim the space of data blocks with a utilization rate less than the first threshold to release space.

[0139] When the utilization rate of a data block is between the first and second thresholds, it indicates that the data block still contains a certain amount of valid data. At this point, it can be chosen not to reclaim the data immediately, but instead to enter a waiting-for-reclaim state. During this stage, the data block can be monitored periodically to inform future decisions.

[0140] When the utilization rate is higher than the second threshold, it indicates that the effective data ratio of the data block is relatively large and the overall utilization rate is good. At this time, there is no need to reclaim the data to avoid affecting system performance and data access efficiency.

[0141] When the utilization rate is higher than the second threshold, it indicates that the effective data ratio of the data block is relatively large, the overall utilization rate is good, and the time for space reclamation is relatively long. At this time, there is no need to reclaim space to avoid affecting system performance and data access efficiency. Therefore, space reclamation is not performed on this data block.

[0142] Furthermore, as storage system usage and data characteristics change, the first and second thresholds can be periodically evaluated and adjusted to ensure more efficient space reclamation.

[0143] Through the above embodiments, by setting different utilization thresholds and formulating corresponding space reclamation strategies based on these thresholds, the storage system can achieve more efficient space management. Strategies such as direct reclamation, waiting for reclamation, and no reclamation help avoid wasting storage space and ensure that system performance is not affected. This approach of prioritizing the reclamation of low-utilization data blocks can effectively improve the overall performance and resource utilization of the storage system.

[0144] Optionally, before determining the space reclamation strategy corresponding to each data block based on the interval where the utilization rate falls, the method further includes:

[0145] The data area in the storage system is divided into multiple data block groups, and each data block group contains multiple data blocks;

[0146] Assign a corresponding recycling queue to each of the data block groups, and configure a corresponding sliding window for each recycling queue;

[0147] Set the sliding window length and sliding step size for each sliding window;

[0148] Step S102 includes: when the sliding window performs a sliding operation, determining the space reclamation strategy corresponding to each data block in the multiple data blocks currently contained in the sliding window based on the utilization rate of each data block in the multiple data blocks currently contained in the sliding window;

[0149] The method further includes:

[0150] When the space reclamation strategy corresponding to the data block in the sliding window is direct reclamation, the space of the data block is directly reclaimed;

[0151] When the space reclamation strategy corresponding to the data block in the sliding window is to wait for reclamation, the data block is added to the sorting queue, and the data blocks in the sorting queue are sorted in ascending order of utilization rate;

[0152] When the space reclamation policy corresponding to the data block in the sliding window is not to reclaim space, the space of the data block is not reclaimed.

[0153] The method further includes:

[0154] After the data blocks with utilization rates less than the first threshold are reclaimed, the data blocks in the sorting queue are added back to the reclamation queue in ascending order of utilization rate for space reclamation.

[0155] Figure 3 This is a schematic diagram of a recycling queue and a sliding window in a space recycling method provided in an embodiment of this application.

[0156] Specifically, such as Figure 3 As shown, in one embodiment, space reclamation can be performed based on a reclamation queue and a sliding window to improve space reclamation efficiency.

[0157] Before determining the space reclamation strategy for each data block based on the utilization interval, it is necessary to group the multiple data blocks into multiple data block groups. Each data block group contains multiple data blocks. For example, if there are 3000 data blocks, they can be divided into three data block groups of 1000 each: Data block group 1: data blocks 1 to 1000, Data block group 2: data blocks 1001 to 2000, and Data block group 3: data blocks 2001 to 3000.

[0158] Furthermore, a corresponding reclamation queue is assigned to each data block group, and a corresponding sliding window is configured for each reclamation queue. The reclamation queue is used to manage the space reclamation operations of multiple data blocks contained in the data block group.

[0159] Configure a corresponding sliding window for each recycling queue, including setting the sliding window length and sliding step size. The values ​​of the sliding window length and sliding step size are equal, and the unit is: number of blocks. The sliding window is used to calculate the utilization rate of the data blocks contained in the sliding window, and to determine the interval in which the utilization rate falls.

[0160] Sliding window length: refers to the number of data blocks contained within the sliding window in a single sliding operation.

[0161] Sliding step size: refers to the number of blocks that the sliding window moves backward each time it slides.

[0162] When the sliding window performs a sliding operation, it checks the utilization rate of the multiple data blocks currently contained within the sliding window. Based on the utilization range of each data block, the space reclamation strategy for each data block can be determined.

[0163] When the space reclamation policy corresponding to a data block in the sliding window is direct reclamation, the data block is directly reclaimed. When the space reclamation policy corresponding to a data block in the sliding window is waiting for reclamation, the data block is transferred from the reclamation queue to the sorting queue. Multiple data blocks transferred to the sorting queue are sorted according to their utilization rate from smallest to largest, and wait to be added back to the reclamation queue for space reclamation. The sorting queue contains the ID of each data block and its corresponding utilization rate. When the space reclamation policy corresponding to a data block in the sliding window is not to reclamation, the data block is not reclaimed.

[0164] For example, suppose the utilization of data blocks D1 to D5 in a certain recycling queue is as follows:

[0165] D1: 10%;

[0166] D2: 30%;

[0167] D3: 50%;

[0168] D4: 80%;

[0169] D5: 90%;

[0170] Assuming the first threshold is 25% and the second threshold is 75%, the space reclamation strategies in the sliding window (D1, D2, D3) are as follows:

[0171] D1: Direct recycling;

[0172] D2: Awaiting recycling;

[0173] D3: Not recycled;

[0174] If the total space of the reclaimed data blocks is still insufficient after multiple data blocks with utilization rates less than the first threshold have been reclaimed, then multiple data blocks that have been sorted in the sorting queue in ascending order of utilization rate are added to the end of the reclamation queue. The added data blocks are then traversed and reclaimed using a sliding window until the reclaimed space meets the required total space, at which point the space reclamation stops.

[0175] Through the above embodiments, by setting up multiple reclamation queues and multiple sliding windows, each reclamation queue performs space reclamation in parallel, improving the space reclamation efficiency of data blocks. Compared with a simple utilization threshold strategy, it can be adjusted more flexibly for different data block usage scenarios. The mechanism of periodic checking and reordering helps optimize storage resources and reduce storage space waste, ensuring efficiency and sustainability in space management.

[0176] Optionally, the method further includes:

[0177] During the sliding operation of the sliding window, the theoretical recycling speed and theoretical CPU usage of the system corresponding to the current recycling queue are calculated. The theoretical recycling speed and theoretical CPU usage of the system corresponding to the current recycling queue are determined based on the theoretical recycling speed and theoretical CPU usage of the system pre-set according to the multiple data block models corresponding to the multiple data blocks currently contained in the sliding window of the recycling queue.

[0178] When the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient, the sliding window length and the sliding step size of the sliding window are increased.

[0179] When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient, the sliding window length and the sliding step size of the sliding window are reduced.

[0180] The actual recycling speed of the recycling queue is determined based on the size of the data blocks recycled per unit time.

[0181] Specifically, in one embodiment, a sliding window is a sampling technique that analyzes the utilization of data blocks and performs space reclamation by "sliding" within a certain range (number and size). Whether the sliding window size and sliding step size can meet the efficiency of space reclamation and the load conditions can be adjusted according to the theoretical reclamation speed and the theoretical CPU usage of the system.

[0182] The theoretical reclamation speed is the ideal reclamation speed calculated based on the characteristics of the data blocks in the reclamation queue and a pre-defined model. It is a predicted value that indicates how quickly the storage system can reclaim data blocks under optimal conditions to achieve the best space reclamation efficiency. The theoretical reclamation speed can be represented by the maximum amount of space reclaimed per unit time.

[0183] The theoretical CPU usage of a system refers to the amount of system CPU resources required to perform a space reclamation operation under normal conditions. This is also a predicted value derived from a model.

[0184] The data block model is a pre-built model based on the usage of data blocks, distributed in a 0-1 pattern. The sorting dimension of the data block model is the number of consecutive 0s, which represents the number of unused data blocks in the data area. Each data block has a corresponding data block model, and the pre-set theoretical recycling speed and theoretical system CPU usage are different for different data block models. Therefore, during the sliding window operation, it is necessary to calculate the theoretical recycling speed and theoretical system CPU usage corresponding to the current sliding window in real time to evaluate whether the actual recycling speed and theoretical system CPU usage of the multiple data blocks currently contained in the sliding window meet the theoretical recycling speed and theoretical system CPU usage corresponding to the current sliding window.

[0185] The theoretical recycling speed and theoretical system CPU usage corresponding to the current sliding window are the average of the theoretical recycling speed and theoretical system CPU usage of the data block models corresponding to the multiple data blocks currently contained in the sliding window.

[0186] For example, the formula for calculating the current theoretical recycling speed of the sliding window is: v = (a1 + a2 + ... + am) / m, where v represents the current theoretical recycling speed of the sliding window, a1 + a2 + ... + am represents the sum of the theoretical recycling speeds of the data block models corresponding to the multiple (m) data blocks currently contained in the sliding window, and m is the size of the sliding window (in units, that is, the number of data blocks contained).

[0187] The formula for calculating the theoretical CPU usage of the current sliding window is: s = (b1 + b2 + ... + bm) / m, where s represents the theoretical CPU usage of the current sliding window, b1 + b2 + ... + bm represents the sum of the theoretical CPU usage of the data block models corresponding to the multiple (m) data blocks currently contained in the sliding window, and m is the sliding window size (in units, that is, the number of data blocks contained).

[0188] For example, when the sliding window currently contains data block A, data block B, and data block C, the data block model corresponding to data block A is A1, the data block model corresponding to data block B is B1, and the data block model corresponding to data block C is C1. At this time, the theoretical recycling speed a1 and the theoretical system CPU usage b1 of data block model A1, the theoretical recycling speed a2 and the theoretical system CPU usage b2 of data block model B1, and the theoretical recycling speed a3 and the theoretical system CPU usage b3 of data block model C1 are as follows: The current theoretical recycling speed of the sliding window is (a1+a2+a3) / 3, and the current theoretical system CPU usage of the sliding window is (b1+b2+b3) / 3.

[0189] During the sliding window operation, it is necessary to determine whether the actual recycling speed of the sliding window is less than the product of the theoretical recycling speed and the first coefficient, based on the calculated theoretical CPU usage of the sliding window. If the actual recycling speed is less than the product of the theoretical recycling speed and the first coefficient, it means that the actual recycling speed is too slow. In this case, the sliding window length and the sliding step size can be increased to improve the actual recycling speed. For example, the sliding window length can be increased by 1, and the sliding step size can be increased by 1.

[0190] Meanwhile, during the sliding window's sliding operation, it is also necessary to determine whether the actual system CPU usage of the sliding window is less than the product of the theoretical system CPU usage and the second coefficient, based on the calculated theoretical system CPU usage of the sliding window. When the actual system CPU usage is less than the product of the theoretical system CPU usage and the second coefficient, it indicates that the space reclamation operation has a certain impact on the hierarchical system. In this case, the sliding window length and sliding step size can be reduced to reduce the pressure on the storage system. For example, the sliding window length can be reduced by 1, and the sliding step size can be reduced by 1.

[0191] Assume that at a certain moment, the reclamation queue performs space reclamation, the theoretical reclamation speed of the current sliding window is 2000MB / hour, and the theoretical CPU usage of the system is 100%.

[0192] First coefficient: 0.8. Therefore, when the actual recycling rate is below 1600 MB / hour, it is necessary to increase the sliding window length and sliding step size.

[0193] Actual recycling speed: 1500MB / hour, which is lower than 1600MB / hour. Therefore, the system decides to increase the sliding window length and step size, and then re-evaluate whether the actual recycling speed is lower than 1600MB / hour.

[0194] The second coefficient, 1.2, means that when the actual system CPU usage exceeds 120%, the system will shorten the sliding window. This adjustment will also be triggered if the actual system CPU usage is 135%, which is higher than 120%.

[0195] Through the above embodiments, by monitoring the actual operation and theoretical expectations, and adjusting the sliding window parameters according to the actual performance of the data recovery and the CPU load, optimal resource utilization and more efficient data recovery are achieved, while providing flexibility in space recovery.

[0196] Optionally, the space reclamation mode is divided into a first mode and a second mode. The step of determining the space reclamation strategy corresponding to each data block based on the utilization rate interval includes:

[0197] When the space reclamation mode is the first mode, the corresponding space reclamation strategy for the data block is determined according to the first threshold and the second threshold;

[0198] When the space reclamation mode is the second mode, the speed level of the space reclamation is determined, and the first threshold, the second threshold, the first coefficient, and the second system are adjusted to the first threshold, the second threshold, the first coefficient, and the second coefficient corresponding to the speed level according to the speed level; the corresponding space reclamation strategy for the data block is determined according to the first threshold and the second threshold corresponding to the speed level.

[0199] The method further includes: when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient corresponding to the speed level, increasing the sliding window length and the sliding step size of the sliding window;

[0200] When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient corresponding to the speed level, the sliding window length and the sliding step size of the sliding window are reduced.

[0201] Specifically, in one embodiment, the space reclamation mode can be divided into two types: a first mode and a second mode. The first mode can be understood as the normal mode, and the second mode can be understood as the high-speed mode.

[0202] In the first mode, the utilization rate of each data block is determined according to a pre-set first threshold and a pre-set second threshold. For example, the first threshold is 20% and the second threshold is 80%. Then, based on the determined utilization rate range, the corresponding space reclamation strategy is determined.

[0203] Meanwhile, the first and second coefficients are also preset, and the first threshold, second threshold, first coefficient, and second coefficient will not change in the first model.

[0204] In the second mode, it indicates that the space reclamation speed needs to be accelerated. In this mode, the first and second thresholds are adjusted according to the different space reclamation speed levels set by the user. When a faster space reclamation speed is needed, the first and second thresholds are lowered; for example, the first threshold is lowered from 20% to 10%, and the second threshold is lowered from 80% to 50%. The corresponding space reclamation strategy for the data block is then determined according to the first and second thresholds for the speed level, and space reclamation is performed on the data block according to the determined strategy.

[0205] Because the first threshold is lowered, the space reclamation strategy reduces the utilization rate of data blocks that are directly reclaimed from 1-20% to 1-10%. By immediately reclaiming data blocks with a utilization rate of 1-10%, the actual reclamation speed will increase due to the reduced average utilization rate. This allows for a faster response to the higher actual reclamation speed required by users. Lowering the second threshold can filter out data blocks with higher utilization rates. In this way, the average utilization rate of data blocks with a utilization rate between the first and second thresholds is also reduced, and the actual reclamation speed of these data blocks will also increase.

[0206] At the same time, the values ​​of the first coefficient and the second coefficient are increased. This will increase the product of the theoretical recycling speed and the first coefficient. In order to increase the actual recycling speed, the sliding window length and sliding step size in the recycling queue will be increased, and more data block objects will be processed at once. The theoretical system CPU usage and the product of the second coefficient will also increase. In this way, when the actual system CPU usage increases slightly, it will not cause the sliding window length and sliding step size in the recycling queue to decrease, thus avoiding a decrease in the number of data blocks processed at once.

[0207] The first threshold, the second threshold, the first coefficient, and the second coefficient are all preset parameters of the system and can be adjusted in the storage system or serial port module, etc.

[0208] Through the above embodiments, data block reclamation strategies can be flexibly managed by adjusting space reclamation modes and speed levels under different conditions. By continuously monitoring actual reclamation speed and CPU usage, the length and step size of the sliding window are dynamically adjusted, ensuring both effective utilization of storage resources and guaranteed system performance. This method greatly improves the intelligence of storage management, ensuring maximum resource utilization under different circumstances.

[0209] Figure 4 This is a schematic diagram of the frame of a space recycling device provided in one embodiment of this application, as shown below. Figure 4 As shown.

[0210] Based on the same inventive concept, another embodiment of this application also provides a space recycling device, the device comprising:

[0211] The acquisition module 11 is used to divide the data area in the storage system into multiple data blocks and acquire the utilization rate of each data block contained in the multiple data blocks, wherein the utilization rate of the data block is determined based on the mapping between the logical address and the physical address of the data contained in the data block;

[0212] The determining module 12 is used to determine the space reclamation strategy corresponding to each data block based on the interval in which the utilization rate is located;

[0213] The recycling module 13 is used to reclaim space from the data block according to the space reclamation strategy.

[0214] Optionally, the device further includes:

[0215] A recording module is used to record the usage of each data block contained in the plurality of data blocks via a bitmap before obtaining the utilization rate of each data block contained in the plurality of data blocks, wherein the used data blocks are recorded as a first value;

[0216] The acquisition module 11 includes:

[0217] The valid data determination module is used to determine valid data for each data block recorded as a first value. The valid data is data in the data block that contains a mapping between logical address and physical address.

[0218] The utilization calculation module is used to calculate the utilization rate of each data block recorded as the first value, based on the sum of the physical addresses corresponding to the valid data and the sum of the addresses corresponding to the data block where the valid data is located.

[0219] Optionally, the recycling module 13 includes:

[0220] The copying unit is used to copy valid data from multiple data blocks that need to be reclaimed to data blocks corresponding to consecutive physical addresses;

[0221] The initialization unit is used to initialize the physical address space of multiple data blocks that need to be reclaimed.

[0222] Optionally, the determining module 12 includes:

[0223] The first determining unit is used to determine that the space reclamation strategy corresponding to the data block is direct reclamation when the utilization rate of the data block is less than a first threshold.

[0224] The second determining unit is used to determine the space reclamation strategy corresponding to the data block as waiting for reclamation when the utilization rate of the data block is not less than the first threshold and less than the second threshold.

[0225] The third determining unit is used to determine that the space reclamation strategy corresponding to the data block is not to reclaim it when the utilization rate of the data block is not less than the second threshold.

[0226] Wherein, the first threshold is less than the second threshold.

[0227] Optionally, the device further includes:

[0228] The data block group partitioning module is used to divide the multiple data blocks contained in the data area of ​​the storage system into multiple data block groups before determining the space reclamation strategy corresponding to each data block according to the interval where the utilization rate is located. Each data block group contains multiple data blocks.

[0229] The recycling queue allocation module is used to allocate a corresponding recycling queue to each of the data block groups and configure a corresponding sliding window for each recycling queue;

[0230] The settings module is used to set the sliding window length and sliding step size for each sliding window;

[0231] The determining module 12 includes: a fourth determining unit, used to determine the space reclamation strategy corresponding to each data block in the multiple data blocks currently contained in the sliding window based on the utilization rate of each data block in the multiple data blocks currently contained in the sliding window when the sliding window performs a sliding operation;

[0232] The device further includes:

[0233] The first execution module is used to directly reclaim the space of the data block when the space reclamation strategy corresponding to the data block in the sliding window is direct reclamation;

[0234] The second execution module is used to add the data block to the sorting queue when the space reclamation strategy corresponding to the data block in the sliding window is waiting for reclamation. The data blocks in the sorting queue are sorted in ascending order of utilization rate.

[0235] The third execution module is used to not reclaim the space of the data block when the space reclamation strategy corresponding to the data block in the sliding window is not to reclaim it.

[0236] The device further includes:

[0237] The fourth execution module is used to re-add the data blocks in the sorting queue to the recycling queue in ascending order of utilization after the data blocks with utilization rates less than the first threshold have been recycled.

[0238] Optionally, the method further includes:

[0239] The calculation module is used to calculate the theoretical recycling speed and theoretical CPU usage of the recycling queue during the sliding operation of the sliding window. The theoretical recycling speed and theoretical CPU usage of the recycling queue are determined based on the theoretical recycling speed and theoretical CPU usage of the multiple data blocks corresponding to the multiple data blocks currently contained in the sliding window of the recycling queue, which are pre-set according to the theoretical recycling speed and theoretical CPU usage of the multiple data blocks.

[0240] The first adjustment module is used to increase the sliding window length and the sliding step size of the sliding window when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient.

[0241] The second adjustment module is used to reduce the sliding window length and the sliding step size of the sliding window when the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient.

[0242] The actual recycling speed of the recycling queue is determined based on the size of the data blocks recycled per unit time.

[0243] Optionally, the space reclamation mode is divided into a first mode and a second mode, and the determining module 12 includes:

[0244] The fourth determining unit is used to determine the corresponding space reclamation strategy of the data block according to the first threshold and the second threshold when the space reclamation mode is the first mode;

[0245] The fifth determining unit is configured to, when the space reclamation mode is the second mode, determine the speed level of the space reclamation, adjust the first threshold, the second threshold, the first coefficient, and the second system according to the speed level to the first threshold, the second threshold, and the first coefficient and the second coefficient corresponding to the speed level; and determine the corresponding space reclamation strategy for the data block according to the first threshold and the second threshold corresponding to the speed level.

[0246] The first adjustment module further includes: a first adjustment unit, used to increase the sliding window length and the sliding step size of the sliding window when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient corresponding to the speed level;

[0247] The second adjustment module further includes: a second adjustment unit, used to reduce the sliding window length and the sliding step size of the sliding window when the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient corresponding to the speed level.

[0248] Based on the same inventive concept, another embodiment of this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the space reclamation method as described in any of the above embodiments.

[0249] Among them, electronic devices refer to Figure 5 , Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 includes a memory 510 and a processor 520. The memory 510 and the processor 520 are connected via a bus for communication. The memory 510 stores a computer program that can run on the processor 520 to implement the steps in the space reclamation method disclosed in the above embodiments of this application.

[0250] Based on the same inventive concept, another embodiment of this application also provides a computer program product, including a computer program that is executed by a processor as the space reclamation method described in any of the above embodiments.

[0251] Based on the same inventive concept, another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the space reclamation method as described in any of the above embodiments.

[0252] As the apparatus is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment.

[0253] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0254] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0255] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0256] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0257] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0258] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0259] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0260] The above provides a detailed description of a space recycling method, apparatus, device, and medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A space reclamation method, characterized in that, The method includes: The data area in the storage system is divided into multiple data blocks, and the utilization rate of each data block contained in the multiple data blocks is obtained. The utilization rate of the data block is determined based on the mapping between the logical address and the physical address of the data contained in the data block. Based on the range in which the utilization rate falls, determine the space reclamation strategy corresponding to each data block; According to the space reclamation strategy, the data block is reclaimed. Before determining the space reclamation strategy corresponding to each data block based on the interval where the utilization rate falls, the method further includes: The data area in the storage system is divided into multiple data block groups, and each data block group contains multiple data blocks; Assign a corresponding recycling queue to each of the data block groups, and configure a corresponding sliding window for each of the recycling queues; Set the sliding window length and sliding step size for each sliding window; The step of determining the space reclamation strategy corresponding to each data block based on the interval in which the utilization rate is located includes: when the sliding window performs a sliding operation, determining the space reclamation strategy corresponding to each data block in the multiple data blocks currently contained in the sliding window based on the utilization rate of each data block in the multiple data blocks currently contained in the sliding window.

2. The space reclamation method according to claim 1, characterized in that, Before obtaining the utilization rate of each data block contained in the plurality of data blocks, the method further includes: The usage of each data block contained in the plurality of data blocks is recorded using a bitmap, wherein the used data blocks are recorded as a first value; The step of obtaining the utilization rate of each data block contained in the plurality of data blocks includes: For each data block recorded as the first value, valid data is determined. The valid data is the data in the data block that contains a mapping between logical address and physical address. For each data block recorded as the first value, the utilization rate of the data block is calculated based on the sum of the physical address corresponding to the valid data and the address of the data block containing the valid data.

3. The space reclamation method according to claim 2, characterized in that, The step of reclaiming space from the data block according to the space reclamation strategy includes: Copy the valid data from multiple data blocks that need to be reclaimed to the data blocks corresponding to consecutive physical addresses; Initialize the physical address space of multiple data blocks that need to be reclaimed.

4. The space reclamation method according to claim 1, characterized in that, The step of determining the space reclamation strategy corresponding to each data block based on the interval in which the utilization rate falls includes: When the utilization rate of the data block is less than the first threshold, the space reclamation strategy corresponding to the data block is determined to be direct reclamation; When the utilization rate of the data block is not less than the first threshold and less than the second threshold, the space reclamation strategy corresponding to the data block is determined to be waiting for reclamation; When the utilization rate of the data block is not less than the second threshold, the space reclamation strategy corresponding to the data block is determined to be not to reclaim it; Wherein, the first threshold is less than the second threshold.

5. The space reclamation method according to claim 4, characterized in that, The step of determining the space reclamation strategy corresponding to each data block based on the interval in which the utilization rate falls also includes: When the space reclamation strategy corresponding to the data block in the sliding window is direct reclamation, the space of the data block is directly reclaimed; When the space reclamation strategy corresponding to the data block in the sliding window is to wait for reclamation, the data block is added to the sorting queue, and the data blocks in the sorting queue are sorted in ascending order of utilization rate; When the space reclamation policy corresponding to the data block in the sliding window is not to reclaim space, the space of the data block is not reclaimed. The method further includes: After the data blocks with utilization rates less than the first threshold are reclaimed, the data blocks in the sorting queue are added back to the reclamation queue in ascending order of utilization rate for space reclamation.

6. The space reclamation method according to claim 5, characterized in that, The method further includes: During the sliding operation of the sliding window, the theoretical recycling speed and theoretical CPU usage of the system corresponding to the current recycling queue are calculated. The theoretical recycling speed and theoretical CPU usage of the system corresponding to the current recycling queue are determined based on the theoretical recycling speed and theoretical CPU usage of the system pre-set according to the multiple data block models corresponding to the multiple data blocks currently contained in the sliding window of the recycling queue. When the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient, the sliding window length and the sliding step size of the sliding window are increased. When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient, the sliding window length and the sliding step size of the sliding window are reduced. The actual recycling speed of the recycling queue is determined based on the size of the data blocks recycled per unit time.

7. The space reclamation method according to claim 6, characterized in that, The space reclamation mode is divided into a first mode and a second mode. The space reclamation strategy for each data block is determined based on the utilization rate range, including: When the space reclamation mode is the first mode, the corresponding space reclamation strategy for the data block is determined according to the first threshold and the second threshold; When the space reclamation mode is the second mode, the speed level of the space reclamation is determined, and the first threshold, the second threshold, the first coefficient, and the second coefficient are adjusted to the first threshold, the second threshold, the first coefficient, and the second coefficient corresponding to the speed level according to the speed level; the corresponding space reclamation strategy for the data block is determined according to the first threshold and the second threshold corresponding to the speed level. The method further includes: when the actual recycling speed of the recycling queue is less than the product of the theoretical recycling speed and the first coefficient corresponding to the speed level, increasing the sliding window length and increasing the sliding step size of the sliding window; When the actual system CPU usage of the recycling queue is greater than the product of the theoretical system CPU usage and the second coefficient corresponding to the speed level, the sliding window length and the sliding step size of the sliding window are reduced.

8. A space recycling device, characterized in that, The device includes: The acquisition module is used to divide the data area in the storage system into multiple data blocks and acquire the utilization rate of each data block contained in the multiple data blocks, wherein the utilization rate of the data block is determined based on the mapping between the logical address and the physical address of the data contained in the data block; The determination module is used to determine the space reclamation strategy corresponding to each data block based on the interval in which the utilization rate is located; The recycling module is used to reclaim space from the data block according to the space reclamation strategy. The device further includes: The data block group partitioning module is used to divide the multiple data blocks contained in the data area of ​​the storage system into multiple data block groups before determining the space reclamation strategy corresponding to each data block according to the interval where the utilization rate is located. Each data block group contains multiple data blocks. The recycling queue allocation module is used to allocate a corresponding recycling queue to each of the data block groups and configure a corresponding sliding window for each of the recycling queues; The settings module is used to set the sliding window length and sliding step size for each sliding window; The determining module includes: a fourth determining unit, used to determine the space reclamation strategy corresponding to each data block in the multiple data blocks currently contained in the sliding window based on the utilization rate of each data block in the multiple data blocks currently contained in the sliding window when the sliding window performs a sliding operation.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the space reclamation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, wherein the computer program, when executed by a processor, implements the space reclamation method as described in any one of claims 1-7.

Images