Data space recovery method, device, computer equipment and storage medium

Abstract

The present application discloses a data space recovery method, apparatus, computer equipment, and storage medium, relating to the field of computer technology. The data space recovery method includes, in response to a situation in which the total length of data blocks to be recovered in a first recovery queue is greater than a first threshold and the first recovery queue includes data blocks to be recovered with a data length greater than a second threshold, splitting the data blocks to be recovered with a data length greater than the second threshold and storing them in a second recovery queue; selecting split data blocks from the second recovery queue and adding them to a third recovery queue; and in response to the number of split data blocks in the third recovery queue being greater than a third threshold, aggregating the split data blocks in the third recovery queue and sending them to a storage area network array for data space recovery. This method solves the technical problem in related technologies that directly processing block device unload requests can lead to fragmented storage space release, affecting subsequent allocation efficiency, thereby achieving the technical effect of improving the reliability of data space recovery.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data space recovery method, apparatus, computer equipment, and storage medium. Background Art

[0002] In distributed file systems, space reclamation is a key mechanism for efficiently managing storage resources, avoiding space waste, and ensuring storage system performance and stability. In a SAN (Storage Area Network) storage environment, when the file system deletes files, traditional SAN storage does not automatically release the underlying physical space (the logical volume remains marked as "used"). Unmap (block device unmount) notifies the storage array which data blocks are no longer in use, allowing the array to reclaim this space for use by other volumes.

[0003] In related technologies, when a distributed file system issues an Unmap command, it typically does so directly through its own block device management interface or by sending the command to the kernel driver via a virtual machine SCSI command (Small Computer Interface command). This results in a large number of small-grained Unmap requests, each corresponding to one or more logical block addresses. Frequent execution of small-grained Unmap requests results in frequent metadata updates, increasing backend load and leading to fragmented storage space. Summary of the Invention

[0004] The present application provides a data space recovery method, apparatus, computer equipment and storage medium to at least solve the problem in related technologies that directly processing block device unloading requests will lead to fragmentation of storage space release, affecting subsequent allocation efficiency.

[0005] This application provides a data space recovery method, including:

[0006] In response to the total length of the to-be-reclaimed data blocks in the first recycling queue being greater than a first threshold, determining whether the first recycling queue includes a to-be-reclaimed data block having a data length greater than a second threshold;

[0007] In response to the first recycling queue including a to-be-reclaimed data block whose data length is greater than a second threshold, splitting the to-be-reclaimed data block whose data length is greater than the second threshold to obtain a plurality of split data blocks, and storing the plurality of split data blocks in the second recycling queue;

[0008] Split data blocks are selected from the second recycling queue and added to the third recycling queue. In response to the number of split data blocks in the third recycling queue being greater than a third threshold, the split data blocks in the third recycling queue are aggregated and sent to the storage area network array for data space recycling.

[0009] The present application also provides a data space recovery device, comprising:

[0010] A judging module configured to judge, in response to a total length of the to-be-reclaimed data blocks in the first recycling queue being greater than a first threshold, whether the first recycling queue includes a to-be-reclaimed data block having a data length greater than a second threshold;

[0011] a splitting module, configured to, in response to the first recycling queue including a to-be-reclaimed data block having a data length greater than a second threshold, split the to-be-reclaimed data block having a data length greater than the second threshold to obtain a plurality of split data blocks, and store the plurality of split data blocks in the second recycling queue;

[0012] The recycling module is configured to select split data blocks from the second recycling queue and add them to the third recycling queue. In response to the number of split data blocks in the third recycling queue being greater than a third threshold, the module aggregates the split data blocks in the third recycling queue and sends them to the storage area network array for data space recycling.

[0013] The present application also provides a computer device, comprising: a memory for storing a computer program; and a processor for implementing the steps of the data space recovery method in the following embodiment when executing the computer program.

[0014] In response to the total length of the to-be-reclaimed data blocks in the first recycling queue being greater than a first threshold, determining whether the first recycling queue includes a to-be-reclaimed data block having a data length greater than a second threshold;

[0015] In response to the first recycling queue including a to-be-reclaimed data block whose data length is greater than a second threshold, splitting the to-be-reclaimed data block whose data length is greater than the second threshold to obtain a plurality of split data blocks, and storing the plurality of split data blocks in the second recycling queue;

[0016] Split data blocks are selected from the second recycling queue and added to the third recycling queue. In response to the number of split data blocks in the third recycling queue being greater than a third threshold, the split data blocks in the third recycling queue are aggregated and sent to the storage area network array for data space recycling.

[0017] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the data space recovery method in any of the following embodiments are implemented.

[0018] In response to the total length of the to-be-reclaimed data blocks in the first recycling queue being greater than a first threshold, determining whether the first recycling queue includes a to-be-reclaimed data block having a data length greater than a second threshold;

[0019] In response to the first recycling queue including a to-be-reclaimed data block whose data length is greater than a second threshold, splitting the to-be-reclaimed data block whose data length is greater than the second threshold to obtain a plurality of split data blocks, and storing the plurality of split data blocks in the second recycling queue;

[0020] Split data blocks are selected from the second recycling queue and added to the third recycling queue. In response to the number of split data blocks in the third recycling queue being greater than a third threshold, the split data blocks in the third recycling queue are aggregated and sent to the storage area network array for data space recycling.

[0021] The data space recovery method provided by the present application can determine whether the first recovery queue includes data blocks to be recovered whose data length is greater than a second threshold value when the total length of the data blocks to be recovered in the first recovery queue is greater than a first threshold value; when the first recovery queue includes data blocks to be recovered whose data length is greater than the second threshold value, the data blocks to be recovered whose data length is greater than the second threshold value are split to obtain multiple split data blocks, and the multiple split data blocks are stored in the second recovery queue; split data blocks are selected from the second recovery queue and added to the third recovery queue; when the number of split data blocks in the third recovery queue is greater than the third threshold value, the split data blocks in the third recovery queue are aggregated and sent to the storage area network array for data space recovery.

[0022] In this way, multiple small-grained data blocks to be recovered can be aggregated, and the data space corresponding to the data blocks to be recovered can be recovered in batches, reducing the number of recovery times and reducing the pressure on the back-end load caused by multiple data recovery. It is also considered to set a data length threshold for the data blocks to be recovered and a data volume threshold for splitting the data blocks to adapt to the data length requirements and data number requirements of the storage area network array for block device unloading, so as to solve the technical problem in the related technology that directly processing block device unloading requests will lead to fragmentation of storage space release and affect the subsequent allocation efficiency, thereby achieving the technical effect of improving the reliability of data space recovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 A flowchart of a data space recovery method provided in one embodiment of the present application;

[0025] Figure 2 A schematic diagram of data block aggregation provided in an embodiment of the present application;

[0026] Figure 3 A flowchart of a data space recovery method provided in another embodiment of the present application;

[0027] Figure 4 A flowchart of a data space recovery method provided in yet another embodiment of the present application;

[0028] Figure 5 A schematic diagram of splitting the data blocks to be recovered provided in an embodiment of the present application;

[0029] Figure 6 A flowchart of a data space recovery method provided in yet another embodiment of the present application;

[0030] Figure 7 A structural block diagram of a data space recovery device provided in an embodiment of the present application;

[0031] Figure 8 This is a diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0033] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0034] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0035] like Figure 1 As shown, an embodiment of the present application provides a data space recovery method, which specifically includes the following steps:

[0036] Step 101: In response to the total length of the to-be-reclaimed data blocks in the first recycling queue being greater than a first threshold, it is determined whether the first recycling queue includes a to-be-reclaimed data block whose data length is greater than a second threshold.

[0037] Step 102: In response to the first recycling queue including the to-be-reclaimed data blocks whose data length is greater than the second threshold, split the to-be-reclaimed data blocks whose data length is greater than the second threshold to obtain multiple split data blocks, and store the multiple split data blocks in the second recycling queue.

[0038] Specifically, in response to the total length of the data blocks to be recycled in the first recycling queue being greater than the first threshold, the method includes: determining whether the block device unloading instruction has been executed; in response to the block device unloading instruction not being executed, selecting the data blocks to be recycled from the cache queue in sequence and adding them to the first recycling queue until the total length of the data blocks to be recycled in the first recycling queue is greater than the first threshold.

[0039] The Unmap RBD block device command is a key command used by the client to notify the storage system to release unused disk blocks. It is used to efficiently manage storage space, especially in thinly provisioned storage systems. Its core function is to release the kernel's occupation of RBD devices, freeing up the network connection and memory cache between the client and the OSD.

[0040] The data block to be reclaimed refers to the data block whose data space is about to be reclaimed, and the data length refers to the size of the data contained in the data block.

[0041] The total length of the data blocks to be recycled in the first recycling queue refers to the sum of the data amounts contained in all the data blocks to be recycled in the first recycling queue.

[0042] First, when the block device uninstall instruction is not executed, the data blocks to be recycled are selected from the cache queue (the data block queue released by the cache in the metadata) and added to the first recycling queue in sequence until the total length of the data blocks to be recycled in the first recycling queue is greater than the first threshold.

[0043] Among them, selecting the data blocks to be recovered from the cache queue in sequence and adding them to the first recovery queue includes: selecting any data block to be recovered in the cache queue as the target data block in sequence; obtaining the target data block capacity and the used space size of the target data block corresponding to the target data block, and calculating the target recovery benefit of the target data block based on the target data block capacity and the used space size of the target data block; obtaining the target data block access frequency, access attenuation factor and new access count in the historical period corresponding to the target data block, and calculating the target access frequency of the target data block based on the target data block access frequency, access attenuation factor and new access count in the historical period; obtaining the current time corresponding to the target data block and the most recent access time of the target data block, and calculating the target data age of the target data block based on the current time corresponding to the target data block and the most recent access time of the target data block; calculating the recovery priority of the target data block based on the target recovery benefit, target access frequency, target data age and the priority calculation formula, and selecting the target data block with the largest recovery priority in sequence according to the size of the recovery priority to add to the first recovery queue.

[0044] The priority calculation formula is as follows:

[0045] ;

[0046] Where P represents the recycling priority of the target data block, a represents the recycling benefit coefficient, X represents the capacity of the target data block, Y represents the size of the used space of the target data block, b represents the data age coefficient, T1 represents the current time, T2 represents the last access time of the target data block, c represents the access frequency coefficient, N represents the access frequency of the target data block in the historical period, A represents the access decay factor, and C represents the new access count.

[0047] The target data block capacity refers to the maximum amount of data that can be accommodated in the target data block. The used space of the target data block refers to the amount of data currently stored in the target data block. The target data block access frequency within the historical period corresponding to the target data block refers to the number of times the target data block was accessed within a certain historical period. The access attenuation factor is a constant that can be set based on actual experience. The access attenuation factor is used to characterize the impact of the number of accesses on performance attenuation. New access calculation refers to the number of new accesses to the data block. The target data block corresponds to the current access time, and the target data block's last access time refers to the time when the target data block was last accessed.

[0048] The higher the recovery benefit, the better the data space recovery operation is for the target data block. The higher the data age, the more likely the target data block is no longer in use and is inactive, so it is better to perform the data space recovery operation. The lower the access frequency, the more likely the target data block is cold data, so it is better to perform the data space recovery operation.

[0049] The recovery benefit coefficient, data age coefficient, and access frequency coefficient here can be set according to the impact of recovery benefit, data age, and access frequency on data space recovery. For example, if recovery benefit has the greatest impact on data space recovery, data age has the second greatest impact, and access frequency has little impact, then the recovery benefit coefficient is set to the maximum value, the data age coefficient to the second greatest value, and the access frequency coefficient to the minimum value. Here, the sum of the recovery benefit coefficient, data age coefficient, and access frequency coefficient is 1.

[0050] In this way, the recovery priority of each data block to be recovered is calculated based on the target recovery income, access frequency, data age and priority calculation formula of the data block to be recovered, and the data blocks to be recovered with high recovery priority are added to the first recovery queue for processing first, which optimizes the multi-dimensional quantitative evaluation of the performance, benefit and resource utilization of the storage system, and can improve the reliability of data space recovery for the data blocks to be recovered.

[0051] The first threshold here is a pre-set threshold for the total length of aggregated data blocks. When the total length of the data blocks to be recovered in the first recovery queue is greater than the first threshold, the data blocks to be recovered in the first recovery queue can be considered for aggregation. The first threshold here can be set based on actual experience and is used to characterize the data length critical value of the data blocks of data aggregation.

[0052] When the total length of the to-be-reclaimed data blocks in the first recycling queue is greater than the first threshold, it can be determined whether there is a to-be-reclaimed data block in the first recycling queue whose data length is greater than the second threshold.

[0053] The second threshold here refers to a set upper threshold of the length of a single data block, and the second threshold is used to distinguish whether the data block to be processed is a data block of large granularity or a data block of small granularity.

[0054] If the first recycling queue contains no pending data blocks with a length greater than the second threshold, the queue is considered to contain only small-sized pending data blocks. The pending data blocks in the first recycling queue can be aggregated and sent to the storage area network array for data space reclamation. This allows for the aggregation of scattered pending data blocks and the overall reclamation of their corresponding space, avoiding the problem of multiple space reclamations for multiple small-sized data blocks, which would result in frequent metadata updates and increased backend load.

[0055] When the first recycling queue includes data blocks to be recycled whose data length is greater than the second threshold, the data blocks to be recycled whose data length is greater than the second threshold are split and processed to obtain multiple split data blocks, that is, large-granularity data blocks are converted into small-granularity data blocks. The data block lengths of the multiple split data blocks meet the requirements of the SAN array for the length of a single data block, and the multiple split data blocks are stored in the second recycling queue.

[0056] In one embodiment, splitting a to-be-reclaimed data block having a data length greater than a second threshold to obtain a plurality of split data blocks includes: obtaining a data block processing length threshold set by a storage area network array; and splitting the to-be-reclaimed data block having a data length greater than the second threshold using the data block processing length threshold as a splitting unit to obtain a plurality of split data blocks.

[0057] The data block processing length threshold set by the storage area network array here refers to the maximum length threshold set by the storage area network array for processing a single data block. The data block processing length threshold is used to split the to-be-reclaimed data blocks whose data length is greater than a second threshold. This can split the large-grained to-be-reclaimed data blocks into multiple split data blocks that are suitable for the storage area network array, flexibly adapt to the requirements of different storage area network array environments, and improve the reliability of data space recovery.

[0058] Step 103: Select split data blocks from the second recycling queue and add them to the third recycling queue. In response to the number of split data blocks in the third recycling queue being greater than a third threshold, aggregate the split data blocks in the third recycling queue and send them to the storage area network array for data space recovery.

[0059] Specifically, the total amount of data blocks recovered in a single block device unloading operation is obtained; the number of data blocks recovered in a single block device unloading operation is calculated based on the total amount of data blocks recovered in the single block device unloading operation and a data block processing length threshold; in response to the number of split data blocks in the third recycling queue being greater than the number of data blocks recovered in the single block device unloading operation, the split data blocks in the third recycling queue are aggregated and sent to a storage area network array for data space recovery.

[0060] The total amount of data blocks recovered in a single block device unload operation refers to the maximum capacity of data space that can be reclaimed by executing a single block device unload operation command. The quotient is calculated using the total amount of data blocks recovered in a single block device unload operation as the dividend and the maximum length threshold for processing a single data block set by the storage area network array as the divisor, and the resulting quotient is used as the number of data blocks recovered in a single block device unload operation. The number of data blocks recovered in a single block device unload operation is referred to as the third threshold. The third threshold refers to the maximum number of split data blocks that can be processed, and is used to determine whether the number of split data blocks in the third reclaim queue meets the SAN array's requirement for the number of data blocks required for a single data space reclaim operation.

[0061] Split data blocks are selected from the second recycling queue and added to a third recycling queue until the number of split data blocks in the third recycling queue exceeds a third threshold. When the number of split data blocks in the third recycling queue exceeds the third threshold, the split data blocks in the third recycling queue are aggregated and sent to a storage area network array for data space reclamation.

[0062] In one embodiment, the data space recovery method further includes: after aggregating the split data blocks in the third recovery queue and sending them to the storage area network array for data space recovery, determining whether the split data blocks in the third recovery queue have been successfully recovered based on the returned data after aggregating and sending them to the storage area network array for data space recovery; in response to unsuccessful recovery of the split data blocks in the third recovery queue, reaggregating the split data blocks in the third recovery queue and sending them to the storage area network array for data space recovery; in response to successful recovery of the split data blocks in the third recovery queue, clearing the data in the third recovery queue, continuing to select split data blocks from the second recovery queue and adding them to the third recovery queue until all the split data blocks in the second recovery queue have been added to the third recovery queue. In response to all the split data blocks in the second recovery queue being added to the third recovery queue, clearing the data in the first recovery queue, and continuing to select data blocks to be recovered from the cache queue and adding them to the first recovery queue.

[0063] The present application is also provided with a method for obtaining the return data after the SAN array reclaims the data space, and judging whether the split data blocks in the third recycling queue are successfully recycled based on the return data. When the split data blocks in the third recycling queue are not successfully recycled, the split data blocks in the third recycling queue are re-aggregated and sent to the storage area network array for data space recycling. When the split data blocks in the third recycling queue are successfully recycled, the data in the third recycling queue is cleared, and split data blocks are continuously selected from the second recycling queue and added to the third recycling queue until all the split data blocks in the second recycling queue are added to the third recycling queue. Similarly, when all the split data blocks in the second recycling queue are added to the third recycling queue, the data in the first recycling queue is cleared, and data blocks to be recycled are continuously selected from the cache queue and added to the first recycling queue until all the data blocks to be recycled in the cache queue are added to the first recycling queue.

[0064] In this way, it is verified whether the data block to be recovered sent by each recovery queue has successfully released the data space, and the problem of space leakage recovery is avoided by retrying, thereby improving the reliability of data space recovery.

[0065] This application proactively aggregates small-scale block device unload requests (Unmap requests), reducing the pressure on backend workloads, addressing the fragmentation of freed storage space, and improving subsequent allocation efficiency. Furthermore, by setting a data block processing length threshold and the number of data blocks reclaimed per block device unload operation, the application flexibly adapts to the requirements of different storage area network array environments, achieving optimal data aggregation.

[0066] In a feasible implementation, before performing a data space recovery operation on a data block to be recovered / split data block based on a SAN array, the size of the available space corresponding to the data block to be recovered / split data block can be recorded. After performing the data space recovery operation on the data block to be recovered / split data block based on the SAN array, the size of the available space of the recovered data block is obtained, and the available space sizes of the data block before and after the data space recovery operation are compared to see whether they are consistent. If the available space sizes of the data block before and after the data space recovery operation are consistent, the data space recovery operation for the data block to be recovered / split data block is considered successful. If the available space sizes of the data block before and after the data space recovery operation are inconsistent, the data space recovery operation for the data block to be recovered / split data block is considered unsuccessful, and an alarm operation is performed. In this way, the effectiveness of the data space recovery operation can be ensured.

[0067] In a specific implementation, the data space recovery method may be as follows:

[0068] S1: Set the total length of the data block aggregation data block (first threshold), for example 512k.

[0069] In the Ceph system (an open source, distributed, and scalable storage system), files are divided into multiple blobs (binary large objects) for storage. Blobs are physical storage units for file data in the file system, and they can be managed and stored independently of the file system. Data blocks (extents) represent continuous byte ranges within a file. When a file is divided into multiple blobs, each blob corresponds to one or more extents in the file. Extents are used to describe the layout of file data in physical storage, and specify the starting position (Lba offset) and length (length) of the file data in the blob, as shown in the attached figure. Figure 2 The data structure shown in the figure. Common Extent length specifications include: 4k, 128k, 2m, 4m, etc.

[0070] S2: Create a thread (discard_thread) to process the data block information released from the MDS (metadata server) cache after the file is deleted. Each data block is recorded by LBA offset and length.

[0071] S3: When the block device unloading operation sent by the metadata is not completed, a data block to be recycled is taken from the extent queue (cache queue) released by the MDS cache, and the data block to be recycled is added to the first recycling queue.

[0072] S4: Determine whether the total length of all to-be-reclaimed data blocks in the first recycling queue is greater than a first threshold.

[0073] If the first threshold is exceeded, the data blocks to be recovered in the first recovery queue are aggregated and sent to the SAN array for data space recovery. Otherwise, step S3 is executed to continue taking the next data block to be recovered and putting it into the first recovery queue until the total length of all data blocks to be recovered in the first recovery queue exceeds the first threshold. Assuming that the data blocks to be recovered taken from the first recovery queue are all small-granularity 4k in length, the number of data blocks to be recovered must reach 128 to exceed the first threshold and meet the total length requirement of data block aggregation, and then a batch processing will be triggered, thereby achieving the effect of data aggregation for frequent small-granularity Unmap requests. The processing flow of steps S1-S4 is shown in the attached figure. Figure 3 shown.

[0074] Assume that the first recycling queue includes data blocks to be recycled whose data length is greater than the second threshold, that is, large-granularity data blocks to be recycled, for example, a length of 2M, which exceeds the set upper limit length threshold of a single data block (the second threshold), then execute steps S5-S8. For details, see Figure 4 .

[0075] S5: Set the upper limit of the length of a single data block to EXTENT_MAX_SIZE, for example, 2M.

[0076] S6: traverse the first recycling queue in step S4, take out a data block to be recycled from the first recycling queue, and if the data block length of the data block to be recycled is greater than the second threshold, split the data block to be recycled into multiple split data blocks, otherwise do not split it.

[0077] Since most SAN arrays can set a length threshold (second threshold) for processing a single data block, such as 2MB, the large-granularity data block to be recycled is split into multiple split data blocks that are suitable for the storage area network array, flexibly adapting to the requirements of different storage area network array environments. The data block splitting process can be as follows: Figure 5 As shown, the data block to be recovered is split into multiple split data blocks (as shown in the figure, split data block 0, split data block 1, split data block 2, ... split data block N) based on the data block processing length threshold. The starting position and length of the data block to be recovered and the split data blocks are recorded. The starting length of the split data block is the starting position of the previous split data block plus the data block processing length threshold. The length of the split data block is usually equal to the data block processing length threshold. The length of the data block to be recovered is the sum of the lengths of the multiple split data blocks corresponding to it. For example, a data block to be recovered with a length of 4MB can be split into two split data blocks of 2MB. A data block to be recovered with a length of 2.128MB can be split into a split data block of 2MB and a split data block of 128kB.

[0078] S7: Store the multiple split data blocks obtained in step S6 into the second recycling queue.

[0079] S8: Perform batch processing on the multiple split data blocks in the second recycling queue.

[0080] S9: Set the upper limit (third threshold) of the number of data blocks released by a single block device unload operation, for example, 64.

[0081] S10: traverse the second recycling queue, take out a split data block from the second recycling queue and insert it into the third recycling queue.

[0082] S11: Determine whether the number of split data blocks currently in the third recycling queue is greater than a third threshold. If so, the split data blocks in the third recycling queue are sent to the SAN array for space reclamation. If less than the third threshold, return to step S10 and continue to remove the next split data block and insert it into the third recycling queue until the number of split data blocks in the third recycling queue exceeds the third threshold.

[0083] Since most SAN arrays can set the total amount of data blocks recovered in a single block device unload operation, the present invention calculates the number of data blocks recovered in a single block device unload operation (a third threshold) based on the total amount of data blocks recovered in a single block device unload operation and the data block processing length threshold. The third threshold is used to determine whether the number of split data blocks in the third recovery queue meets the SAN array's requirement for the number of data blocks for a single data space recovery process, thereby flexibly adapting to the configuration of different SAN array environments.

[0084] S12: Obtain the return data after the SAN array reclaims the data space, and determine whether the split data blocks in the third recycling queue are successfully recycled based on the returned data. If failed, re-issue the split data blocks in the third recycling queue in step 11 to avoid the problem of space recycling leakage by retrying.

[0085] S13: If the split data blocks in the third recycling queue are successfully recycled according to the returned data in step S12, the third recycling queue is cleared, and the process returns to step 10 to process the next batch of data blocks. Figure 6 shown.

[0086] S14: If step S13 completes processing of the multiple split data blocks to be recycled in the second recycling queue, it indicates that the first recycling queue is no longer required to re-split the data blocks to be recycled whose data length is greater than the second threshold. Multiple split data blocks are generated and sent to the first recycling queue. The first recycling queue is then cleared. The process returns to step S3 to retrieve new data blocks to be recycled from the queue released from the MDS cache. Data space recycling continues for the unprocessed data blocks to be recycled in the cache queue according to the aforementioned steps.

[0087] This application intelligently identifies unmap requests sent by the MDS, distinguishing between small- and large-grained unmap requests, that is, small- and large-grained data blocks to be recovered. It then actively aggregates small-grained unmap requests, thereby reducing the pressure on the backend load from small-grained unmap requests, resolving the fragmentation problem of storage space release, and improving subsequent allocation efficiency. Furthermore, by setting a data block processing length threshold and the number of data blocks recovered in a single block device unloading operation, the application flexibly adapts to the requirements of different storage area network array environments, achieving better data aggregation results.

[0088] The embodiment of the present application provides a data space recovery device, the data space recovery device is specifically as follows Figure 7 As shown, the data space recovery device includes: a judgment module 20, a splitting module 21 and a recovery module 22.

[0089] A judging module configured to judge, in response to a total length of the to-be-reclaimed data blocks in the first recycling queue being greater than a first threshold, whether the first recycling queue includes a to-be-reclaimed data block having a data length greater than a second threshold;

[0090] a splitting module, configured to, in response to the first recycling queue including a to-be-reclaimed data block having a data length greater than a second threshold, split the to-be-reclaimed data block having a data length greater than the second threshold to obtain a plurality of split data blocks, and store the plurality of split data blocks in the second recycling queue;

[0091] The recycling module is configured to select split data blocks from the second recycling queue and add them to the third recycling queue. In response to the number of split data blocks in the third recycling queue being greater than a third threshold, the module aggregates the split data blocks in the third recycling queue and sends them to the storage area network array for data space recycling.

[0092] For the description of the features in the embodiment corresponding to the data space recovery device, reference can be made to the relevant description of the embodiment corresponding to the data space recovery method, which will not be repeated here.

[0093] The embodiment of the present application also provides a computer device, such as Figure 8 As shown, it includes a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above data space recovery method embodiments.

[0094] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned data space recovery method embodiments when running.

[0095] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0096] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0097] The above is a detailed introduction to a data space recovery method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A data space recovery method, characterized in that: include: Select any data block to be recycled in the cache queue as the target data block in turn; Obtaining a target data block capacity and a used space size of the target data block corresponding to the target data block, and calculating a target recycling benefit of the target data block based on the target data block capacity and the used space size of the target data block; Obtaining a target data block access frequency, an access attenuation factor, and a new access count within a historical period corresponding to the target data block, and calculating a target access frequency of the target data block based on the target data block access frequency within the historical period, the access attenuation factor, and the new access count; Obtaining a current time corresponding to a target data block and a last access time of the target data block, and calculating a target data age of the target data block based on the current time and the last access time; Calculating the recycling priority of the target data block based on the target recycling income, the target access frequency, the target data age, and the priority calculation formula, and selecting the target data block with the largest recycling priority in order of recycling priority to add it to the first recycling queue; In response to the total length of the to-be-reclaimed data blocks in the first recycling queue being greater than a first threshold, determining whether the first recycling queue includes a to-be-reclaimed data block having a data length greater than a second threshold; In response to the first recycling queue including a to-be-reclaimed data block whose data length is greater than a second threshold, splitting the to-be-reclaimed data block whose data length is greater than the second threshold to obtain a plurality of split data blocks, and storing the plurality of split data blocks in the second recycling queue; Split data blocks are selected from the second recycling queue and added to the third recycling queue. In response to the number of split data blocks in the third recycling queue being greater than a third threshold, the split data blocks in the third recycling queue are aggregated and sent to the storage area network array for data space recycling.

2. The data space recovery method according to claim 1, characterized in that: The step of sequentially selecting any to-be-reclaimed data block in the cache queue as the target data block includes: Determine whether the block device uninstall instruction has been executed; In response to the block device unloading instruction not being executed completely, any to-be-reclaimed data block in the cache queue is selected in sequence as the target data block.

3. The data space recovery method according to claim 1, characterized in that: The step of splitting the to-be-reclaimed data block whose data length is greater than the second threshold to obtain a plurality of split data blocks comprises: Get the data block processing length threshold set by the storage area network array; The data block processing length threshold is used as a splitting unit, and the to-be-reclaimed data block whose data length is greater than the second threshold is split to obtain a plurality of split data blocks.

4. The data space recovery method according to claim 1, characterized in that: In response to the number of split data blocks in the third recycling queue being greater than a third threshold, aggregating the split data blocks in the third recycling queue and sending them to the storage area network array for data space recycling includes: Get the total amount of data blocks recovered in a single block device unmount operation; Calculating the number of data blocks recovered in a single block device unloading operation based on the total amount of data blocks recovered in the single block device unloading operation and a data block processing length threshold; In response to the number of split data blocks in the third recycling queue being greater than the number of data blocks recycled in a single block device unloading operation, the split data blocks in the third recycling queue are aggregated and sent to the storage area network array for data space recycling.

5. The data space recovery method according to claim 1, characterized in that: The method further includes: obtaining return data from a data space recovery operation in a storage area network array, and determining whether the split data blocks in the third recovery queue are successfully recovered based on the return data; In response to unsuccessful recycling of the split data blocks in the third recycling queue, re-aggregating the split data blocks in the third recycling queue and sending them to a storage area network array for data space recycling; In response to successful recycling of the split data blocks in the third recycling queue, the data in the third recycling queue is cleared, and split data blocks are continuously selected from the second recycling queue and added to the third recycling queue until all the split data blocks in the second recycling queue are added to the third recycling queue.

6. The data space recovery method according to claim 1, characterized in that: The method further includes: in response to all split data blocks in the second recycling queue being added to the third recycling queue, clearing the data in the first recycling queue, and continuing to select data blocks to be recycled from the cache queue and add them to the first recycling queue.

7. The data space recovery method according to claim 1, characterized in that: The priority calculation formula is: Where P represents the recycling priority of the target data block, a represents the recycling benefit coefficient, X represents the capacity of the target data block, Y represents the size of the used space of the target data block, b represents the data age coefficient, T1 represents the current time, T2 represents the last access time, c represents the access frequency coefficient, N represents the access frequency of the target data block in the historical period, A represents the access attenuation factor, and C represents the new access count.

8. A data space recovery device, characterized in that: include: A judgment module is used to sequentially select any to-be-reclaimed data block in the cache queue as a target data block; Obtaining a target data block capacity and a used space size of the target data block corresponding to the target data block, and calculating a target recycling benefit of the target data block based on the target data block capacity and the used space size of the target data block; Obtaining a target data block access frequency, an access attenuation factor, and a new access count within a historical period corresponding to the target data block, and calculating a target access frequency of the target data block based on the target data block access frequency within the historical period, the access attenuation factor, and the new access count; Obtaining the current time corresponding to the target data block and the last access time of the target data block, and calculating the target data age of the target data block based on the current time and the last access time; calculating the recycling priority of the target data block based on the target recycling income, the target access frequency, the target data age, and a priority calculation formula, and selecting the target data block with the highest recycling priority in order of recycling priority to add it to the first recycling queue; In response to the total length of the to-be-reclaimed data blocks in the first recycling queue being greater than a first threshold, determining whether the first recycling queue includes a to-be-reclaimed data block having a data length greater than a second threshold; a splitting module, configured to, in response to the first recycling queue including a to-be-reclaimed data block having a data length greater than a second threshold, split the to-be-reclaimed data block having a data length greater than the second threshold to obtain a plurality of split data blocks, and store the plurality of split data blocks in the second recycling queue; The recycling module is configured to select split data blocks from the second recycling queue and add them to the third recycling queue. In response to the number of split data blocks in the third recycling queue being greater than a third threshold, the module aggregates the split data blocks in the third recycling queue and sends them to the storage area network array for data space recycling.

9. A computer device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the data space recovery method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data space recovery method according to any one of claims 1 to 7.

Images