Data block recovery method and device, computer equipment and storage medium
By using an automated data block reclamation method, the problem of long processing times when manually performing compressed volume data block reclamation is solved, and a more efficient data block reclamation process is achieved.
Patent Information
- Application Number
- CN202411151595.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-29
AI Technical Summary
In existing technologies, manually performing compressed volume data block reclamation is time-consuming, resulting in low efficiency in compressed volume data block reclamation.
The system uses an automated data block reclamation method to obtain the total storage resources of the compressed volume, identify target data blocks and migrate them to free data blocks. After certain conditions are met, a shrinkage request is sent to the storage pool to reclaim free data blocks, including filtering invalid data blocks, migrating valid data blocks and managing their addresses.
It improves the efficiency of compressed volume data block reclamation, reduces manual intervention time, and increases the speed and efficiency of data block reclamation.
Smart Images

Figure CN119045737A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data storage, and in particular to a data block recovery method, device, computer equipment and storage medium. Background Art
[0002] With the continuous development and progress of storage system technology, compressed volumes are being used more and more widely. At the same time, the capacity management technology of compressed volumes has also been continuously developed and improved, especially the shrinking technology of compressed volumes, which allows users to dynamically adjust the capacity of volumes according to actual needs, so as to manage storage space more flexibly. The so-called shrinking is to recycle the data blocks of compressed volumes and release the redundant data blocks to the storage pool.
[0003] In the related art, some storage systems allow users to return the data blocks of the compressed volume that are no longer needed to the storage pool through specific commands or interface options. Alternatively, when there is no longer needed data in the compressed volume, the user can migrate the no longer needed data to other storage devices or cloud storage, and the data blocks of the compressed volume originally occupied can be released for reuse by the storage pool. These methods all require manual execution of the compressed volume data block recovery, which is time-consuming and results in low compressed volume data block recovery efficiency. Summary of the invention
[0004] In view of this, the present invention provides a data block recovery method, apparatus, computer equipment and storage medium to solve the problem that manual recovery of compressed volume data blocks is time-consuming and leads to low recovery efficiency of compressed volume data blocks.
[0005] In a first aspect, the present invention provides a data block recovery method, the method comprising: obtaining the total storage resources of a compressed volume, the total storage resources being composed of M data blocks, M≥1 and being a positive integer; determining M1 target data blocks among the M data blocks, each target data block being a data block storing valid data, M1≤M; obtaining the address of each target data block among the M1 target data blocks; when the addresses of the M1 target data blocks meet a preset condition, storing all the data stored in the M1 target data blocks into M2 free data blocks, so that M-M2 free data blocks remain in the compressed volume, the preset condition being that there are free data blocks whose addresses are less than the addresses of the M1 target data blocks among the M2 free data blocks, M2≤M1. When the ratio between M-M2 and M is greater than a first threshold value, and the number of continuous free data blocks at the end of the compressed volume is greater than a second threshold value, sending a reduction request to a storage pool, the reduction request being used to request the storage pool to recycle some or all of the data blocks among the M-M2 free data blocks.
[0006] Based on the method of the first aspect above, M1 target data blocks can be determined among the M data blocks of the compressed volume, and the address of each target data block in the M1 target data blocks can be obtained; when the addresses of the M1 target data blocks meet the preset conditions, all data stored in the M1 target data blocks are stored in M2 free data blocks, so that M-M2 free data blocks remain in the compressed volume, and when the ratio between M-M2 and M is greater than a first threshold and the number of continuous free data blocks at the end of the compressed volume is greater than a second threshold, a reduction request is sent to the storage pool.
[0007] Since the tail of the compressed volume is a high-address data block, when there is a free data block with an address smaller than the address of the M1 target data block in the M2 free data blocks, all data stored in the M1 target data blocks are stored in the M2 free data blocks (for example, the data stored in the target data block with the highest address among the M1 target data blocks is stored in the free data with the lowest address among the M2 free data blocks). Therefore, the data stored in the M1 target data blocks can be migrated as a whole from the high-address data blocks to the low-address data blocks, thereby increasing the number of consecutive free data blocks at the tail of the compressed volume.
[0008] In addition, when the ratio between M-M2 and M is greater than the first threshold, and the number of the tail continuous free data blocks of the compressed volume is greater than the second threshold, that is, when there are too many free data blocks remaining in the compressed volume and the tail continuous free data blocks are excessive, a reduction request is sent to the storage pool to request the storage pool to recycle the excessive or surplus free data blocks. Compared with the method of manually executing the recovery of compressed volume data blocks, it takes less time and is more efficient.
[0009] In an optional implementation, M1 target data blocks are determined from M data blocks, including: screening out N data blocks from the M data blocks, each of the N data blocks storing invalid data and valid data, N≤M; determining N1 data blocks from the N data blocks, the invalid data ratio of each data block in the N1 data blocks being less than or equal to a preset ratio, the invalid data ratio being the ratio of invalid data stored in a data block to all data, N1≤N; when N1 is greater than a preset threshold, extracting the valid data stored in each data block in the N1 data blocks to form a valid data set corresponding to the N1 data blocks; storing the valid data set in N2 free data blocks to obtain N2 data blocks storing valid data, N2≤N1; screening out M3 data blocks from the M data blocks, each of the M3 data blocks being a data block full of valid data, M3≤M; determining M1 target data blocks based on the N2 data blocks and M3 data blocks storing valid data, M1 being the sum of N2 and M3.
[0010] Based on the above method, N data blocks that are not fully stored with valid data can be screened out from the M data blocks, and N1 data blocks with a relatively high proportion of invalid data can be determined from the N data blocks, and then the valid data stored in the N1 data blocks can be extracted, and the valid data can be merged and stored in the N2 free data blocks, so that the N1 data blocks are released. Obviously, N2 is smaller than N1, so the number of free data blocks in the compressed volume can be increased, which makes it easier to recycle the free data blocks in the subsequent situation where there are too many free data blocks remaining in the compressed volume and there are too many continuous free data blocks at the end.
[0011] In an optional embodiment, when the addresses of the M1 target data blocks meet a preset condition, all data stored in the M1 target data blocks are stored in M2 free data blocks, including: when the addresses of the M1 target data blocks meet a preset condition, M2 free data blocks are determined among the M data blocks; all data stored in the M1 target data blocks are extracted in order from high address to low address, and all data are stored in the M2 free data blocks, and the storage order of the M2 free data blocks is from low address to high address.
[0012] Based on the above method, all data stored in the M1 target data blocks can be stored in the M2 free data blocks in order from high address to low address, and the storage order of the M2 free data blocks is from low address to high address, that is, the data stored in the target data block with the highest address among the M1 target data blocks can be stored in the free data block with the lowest address among the M2 free data blocks in sequence. Therefore, the data stored in the M1 target data blocks can be migrated as a whole from the high-address data blocks to the low-address data blocks, thereby increasing the number of continuous free data blocks at the end of the compressed volume.
[0013] In an optional embodiment, when the addresses of M1 target data blocks meet a preset condition, M2 free data blocks are determined among the M data blocks, including: when the addresses of the M1 target data blocks meet the preset condition, metadata of each data block in the M data blocks is obtained, the metadata including the storage status of each data block, the storage status including a state of storing full valid data, or a state of storing invalid data and valid data, or a state of storing invalid data or no data; from the M data blocks, data blocks whose storage status is a state of storing invalid data or no data are stored are screened out, and determined as M2 free data blocks.
[0014] Based on the above method, the metadata of each data block can be obtained, and then the storage status of each data block can be obtained, and based on the storage status, M2 free data blocks can be screened out from the M data blocks.
[0015] In an optional implementation, the metadata also includes the storage location of the valid data stored in each data block; extracting all the data stored in M1 target data blocks in order from high address to low address, and storing all the data in M2 free data blocks, including: generating M1 read requests, each of the M1 read requests carries the storage location of the valid data stored in each of the M1 target data blocks; sending M1 read requests to the M1 target data blocks respectively, reading the valid data of each of the M1 target data blocks from the storage location, and extracting all the data; generating M2 write requests based on all the data, each of the M2 write requests carries the address of each of the M2 free data blocks; sending M2 write requests to the M2 free data blocks respectively, and storing all the data in the M2 free data blocks.
[0016] Based on the above method, the data stored in the M1 target data blocks can be migrated as a whole from the data blocks with high addresses to the data blocks with low addresses, thereby increasing the number of continuous free data blocks at the end of the compressed volume.
[0017] In an optional implementation, after all data stored in the M1 target data blocks are extracted, the storage status of the M1 target data blocks is marked as storing invalid data or not storing data.
[0018] Based on the above method, after all data stored in the M1 target data blocks are released, the M1 target data blocks can be marked as free data blocks, so as to facilitate the subsequent determination of the number of free data blocks.
[0019] In an optional embodiment, when the ratio between M-M2 and M is less than or equal to a first threshold, or the number of consecutive free data blocks at the end of the compressed volume is less than or equal to a second threshold, a stop request is sent to the storage pool, and the stop request is used to request the storage pool to stop reclaiming data blocks.
[0020] Based on the above method, when there are fewer free data blocks in the compressed volume or fewer continuous free data blocks at the end, the storage pool can stop reclaiming data blocks.
[0021] In a second aspect, the present invention provides a data block recovery device, which includes: an acquisition module, used to acquire the total storage resources of the compressed volume, the total storage resources are composed of M data blocks, M ≥ 1 and is a positive integer; a processing module, used to determine M1 target data blocks in the M data blocks, each target data block is a data block storing valid data, M1 ≤ M; the acquisition module is also used to acquire the address of each target data block in the M1 target data blocks; the processing module is also used to convert the M1 target data blocks into a new data block when the addresses of the M1 target data blocks meet a preset condition. All data stored in the target data block are stored in M2 free data blocks, so that M-M2 free data blocks remain in the compressed volume. The preset condition is that there is a free data block with an address smaller than the address of the M1 target data blocks among the M2 free data blocks, M2≤M1; the processing module is also used to send a reduction request to the storage pool when the ratio between M-M2 and M is greater than a first threshold and the number of continuous free data blocks at the end of the compressed volume is greater than a second threshold. The reduction request is used to request the storage pool to reclaim some or all of the M-M2 free data blocks.
[0022] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the data block recovery method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0023] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data block recovery method of the first aspect or any corresponding embodiment thereof.
[0024] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the data block recovery method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0026] Figure 1 is a flow chart of a data block recovery method according to an embodiment of the present invention;
[0027] Figure 2is a schematic diagram of a block data append writing process according to an embodiment of the present invention;
[0028] Figure 3 is a schematic diagram of data migration of a Full block data block according to an embodiment of the present invention;
[0029] Figure 4 is a structural block diagram of a data block recovery device according to an embodiment of the present invention;
[0030] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0032] The physical space of a compressed volume is allocated from the storage pool. The virtual capacity of a compressed volume is the capacity that the volume provides to the outside world and the host. The physical space is the physical space actually allocated from the storage pool. In general, due to the existence of compressed volumes, the physical space allocated to compressed volumes is definitely smaller than the virtual capacity. Due to the different compression rates of different data, the actual physical space occupied by writing to a compressed volume is also different. Therefore, as data is written repeatedly, the actual physical space of the compressed volume continues to change.
[0033] For example, when writing data with a low compression ratio at the beginning, the actual physical space occupied by the compressed volume is larger. Then, when writing data with a higher compression ratio, the old data will be overwritten because of data overwriting, so the actual physical space occupied by the compressed volume will decrease. The compression ratio of the compressed volume will be significantly different.
[0034] As described in the background technology, manually executing the recovery of compressed volume data blocks takes a long time, resulting in low recovery efficiency of compressed volume data blocks. In order to solve the above technical problems, an embodiment of the present invention provides a data block recovery method, which can solve the problem that manually executing the recovery of compressed volume data blocks takes a long time, resulting in low recovery efficiency of compressed volume data blocks.
[0035] In an embodiment of the present invention, the above data block recovery method can be performed by a data block recovery device. The data block recovery device can be any device with computing and communication functions. For example, the data block recovery device can be a server or a cloud server.
[0036] According to an embodiment of the present invention, an embodiment of a method for recovering a data block is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0037] In this embodiment, a data block recovery method is provided, which can be used in the above-mentioned data block recovery device. Figure 1 is a flow chart of a data block recovery method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:
[0038] S101: Acquire the total storage resources of the compressed volume.
[0039] The total storage resource is composed of M data blocks, where M≥1 and is a positive integer. In the embodiment of the present invention, a data block is also referred to as a block.
[0040] S102: Determine M1 target data blocks among the M data blocks.
[0041] Wherein, each target data block is a data block storing valid data, M1≤M.
[0042] In some optional embodiments, a data block recovery device selects N data blocks from M data blocks, and determines N1 data blocks from the N data blocks; when N1 is greater than a preset threshold, extracts valid data stored in each of the N1 data blocks to form a valid data set corresponding to the N1 data blocks; stores the valid data set in N2 free data blocks to obtain N2 data blocks storing valid data; selects M3 data blocks from the M data blocks; and determines M1 target data blocks based on the N2 data blocks and M3 data blocks storing valid data.
[0043] Each of the N data blocks stores invalid data and valid data, where N≤M. A data block in the N data blocks is also called a partial block.
[0044] In the embodiment of the present invention, the invalid data ratio of each data block in the N1 data blocks is less than or equal to the preset ratio, N1≤N. The invalid data ratio is the ratio of the invalid data stored in a data block to the total data.
[0045] In the embodiment of the present invention, each of the M3 data blocks is a data block fully storing valid data, M3≤M. The data block fully storing valid data is also referred to as a Full block.
[0046] In the embodiment of the present invention, M1 is the sum of N2 and M3, and N2≤N1.
[0047] In a possible design, the preset threshold value can be set according to actual needs without limitation. For example, the preset threshold value can be 100.
[0048] In one example, a data block recovery device obtains metadata of each data block among M data blocks.
[0049] The metadata may be the BlockInfo information and LogInfo information of the data block. The metadata includes the address, storage status, invalid data ratio, and storage location of valid data of each data block.
[0050] The storage state includes a state in which valid data is fully stored, or a state in which invalid data and valid data are stored, or a state in which invalid data is stored or no data is stored.
[0051] In some optional implementations, the data block recovery device selects data blocks whose invalid data proportion is less than or equal to a preset proportion from N data blocks, and determines them as N1 free data blocks.
[0052] Among them, the free data block is also called invalid block.
[0053] The preset ratio can be set according to actual needs without limitation. For example, the preset ratio can be 10%.
[0054] It is understandable that during the data appending process, the updated data and the original data are always stored in different blocks. After the data is updated, the original data becomes invalid data, and its corresponding block becomes a partial block. The more data is updated, the more invalid data there is. Figure 2 As shown, Figure 2 FIG. 1 is a schematic diagram of a block data append writing process according to an embodiment of the present invention. Figure 2 In the example, the data block recovery device first sends requests (Request1, Request2, Request3) to write 1 and 2 in the first block and 3 in the second block; in the process of data appending, it sends requests (Request1', Request3') to update "1" to "1'" and "3" to "3'". Therefore, the idle data blocks full of invalid data can be recycled to save storage costs.
[0055] S103: Obtain the address of each target data block in the M1 target data blocks.
[0056] It can be understood that the metadata includes the address of each data block. The data block recovery device obtains the metadata of each data block in the M data blocks, and can obtain the address of each target data block in the M1 target data blocks.
[0057] S104: When the addresses of the M1 target data blocks meet a preset condition, all data stored in the M1 target data blocks are stored in the M2 free data blocks, so that M-M2 free data blocks remain in the compressed volume.
[0058] The preset condition is that there is an idle data block with an address smaller than the address of the M1 target data blocks among the M2 idle data blocks, M2≤M1.
[0059] In the embodiment of the present invention, M2 idle data blocks are used to store all the data stored in M1 target data blocks.
[0060] In some optional embodiments, the data block recovery device determines M2 free data blocks among the M data blocks when the addresses of the M1 target data blocks meet a preset condition; extracts all data stored in the M1 target data blocks in order from high address to low address, and stores all data in the M2 free data blocks.
[0061] The storage order of the M2 free data blocks is from low address to high address.
[0062] In some optional embodiments, the data block recovery device obtains metadata of each data block in the M data blocks when the addresses of the M1 target data blocks meet preset conditions; from the M data blocks, the data blocks whose storage status is storing invalid data or not storing data are screened out and determined as M2 free data blocks.
[0063] In some optional implementations, a data block recovery device generates M1 read requests; sends the M1 read requests to M1 target data blocks respectively, reads valid data of each of the M1 target data blocks from a storage location, and extracts all data; generates M2 write requests based on all the data; sends the M2 write requests to M2 free data blocks respectively, and stores all data in the M2 free data blocks.
[0064] Each of the M1 read requests carries the storage location of valid data stored in each of the M1 target data blocks.
[0065] In the embodiment of the present invention, each of the M2 write requests carries the address of each of the M2 free data blocks.
[0066] In some optional implementations, after extracting all data stored in the M1 target data blocks, the data block recovery device marks the storage status of the M1 target data blocks as storing invalid data or not storing data.
[0067] Optionally, the data block recycling device may also update the LogInfo information and BlockInfo information of each data block.
[0068] It can be understood that the data block recycling device marks the storage status of M1 target data blocks to facilitate the subsequent determination of the number of free data blocks.
[0069] S105: When the ratio of M-M2 to M is greater than a first threshold, and the number of consecutive free data blocks at the end of the compressed volume is greater than a second threshold, a shrink request is sent to the storage pool.
[0070] The shrink request is used to request the storage pool to reclaim some or all of the M-M2 free data blocks.
[0071] In the embodiment of the present invention, the first threshold and the second threshold can be set according to actual needs without limitation. For example, the first threshold can be 10% and the second threshold can be 10000.
[0072] In the embodiment of the present invention, the number of the trailing continuous free data blocks of the compressed volume may be determined according to the block with the highest address storing valid data in the compressed volume and the block with the highest address in the compressed volume.
[0073] Optionally, when the number of M-M2 is greater than a preset number and the ratio of the tail continuous free data blocks of the compressed volume to all data blocks is greater than a preset ratio, a reduction request is sent to the storage pool.
[0074] The preset number and the preset ratio can be set according to actual needs without limitation. For example, the preset number can be 40000 and the preset ratio can be 40%.
[0075] In some optional embodiments, the data block recovery device sends a stop request to the storage pool when the ratio between M-M2 and M is less than or equal to a third threshold, or the number of consecutive free data blocks at the end of the compressed volume is less than or equal to a fourth threshold.
[0076] The stop request is used to request the storage pool to stop reclaiming data blocks.
[0077] In the embodiment of the present invention, the third threshold and the fourth threshold can be set according to actual needs without limitation. For example, the third threshold can be 5% and the second threshold can be 5000.
[0078] It is understandable that during the data appending process, due to data update, invalid data in some blocks will increase until the entire block can be recycled. In order to maintain the sequential writing in the physical space, the algorithm migrates the data of the full block with valid data at the high address of the space to the free block at the low address, such as Figure 3 As shown, Figure 3 It is a schematic diagram of data migration of a Full block data block according to an embodiment of the present invention.
[0079] Based on the method of the first aspect above, M1 target data blocks can be determined among the M data blocks of the compressed volume, and the address of each target data block in the M1 target data blocks can be obtained; when the addresses of the M1 target data blocks meet the preset conditions, all the data stored in the M1 target data blocks are stored in M2 free data blocks, so that M-M2 free data blocks remain in the compressed volume, and the preset condition is that there are free data blocks with addresses smaller than the addresses of the M1 target data blocks among the M2 free data blocks, so that M-M2 free data blocks remain in the compressed volume, and when the ratio between M-M2 and M is greater than a first threshold value, and the number of consecutive free data blocks at the end of the compressed volume is greater than a second threshold value, a reduction request is sent to the storage pool.
[0080] Since the tail of the compressed volume is a high-address data block, when the addresses of the M1 target data blocks meet the preset conditions, all data stored in the M1 target data blocks can be stored in M2 free data blocks in order from high address to low address (for example, the data stored in the target data block with the highest address among the M1 target data blocks is stored in the free data with the lowest address in the compressed volume). Therefore, the data stored in the M1 target data blocks can be migrated as a whole from the high-address data blocks to the low-address data blocks, thereby increasing the number of consecutive free data blocks at the tail of the compressed volume.
[0081] In addition, when the ratio between M-M2 and M is greater than the first threshold, and the number of the tail continuous free data blocks of the compressed volume is greater than the second threshold, that is, when there are too many free data blocks remaining in the compressed volume and the tail continuous free data blocks are excessive, a reduction request is sent to the storage pool to request the storage pool to recycle the excessive or surplus free data blocks. Compared with the method of manually executing the recovery of compressed volume data blocks, it takes less time and is more efficient.
[0082] In the present embodiment, a data block recovery device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.
[0083] This embodiment provides a data block recovery device, such as Figure 4 As shown, the device comprises:
[0084] The acquisition module 401 is used to acquire the total storage resources of the compressed volume, where the total storage resources consist of M data blocks, where M≥1 and is a positive integer.
[0085] The processing module 402 is used to determine M1 target data blocks among the M data blocks, each target data block is a data block storing valid data, and M1≤M.
[0086] The acquisition module 401 is further used to acquire the address of each target data block in the M1 target data blocks.
[0087] The processing module 402 is also used to store all the data stored in the M1 target data blocks into M2 free data blocks when the addresses of the M1 target data blocks meet a preset condition, so that M-M2 free data blocks remain in the compressed volume. The preset condition is that there is a free data block in the M2 free data blocks whose address is smaller than the address of the M1 target data blocks, and M2≤M1.
[0088] Processing module 402 is also used to send a reduction request to the storage pool when the ratio between M-M2 and M is greater than a first threshold and the number of continuous free data blocks at the end of the compressed volume is greater than a second threshold. The reduction request is used to request the storage pool to reclaim some or all of the M-M2 free data blocks.
[0089] In some optional embodiments, the processing module 402 is specifically used to screen out N data blocks from M data blocks, each of the N data blocks stores invalid data and valid data, N≤M; the processing module 402 is also specifically used to determine N1 data blocks from the N data blocks, the proportion of invalid data in each of the N1 data blocks is less than or equal to a preset proportion, and the invalid data proportion is the ratio of invalid data stored in a data block to all data, N1≤N.
[0090] The processing module 402 is also specifically used to extract the valid data stored in each of the N1 data blocks to form a valid data set corresponding to the N1 data blocks when N1 is greater than a preset threshold; the processing module 402 is also specifically used to store the valid data set in N2 free data blocks to obtain N2 data blocks storing valid data, N2≤N1; the processing module 402 is also specifically used to screen out M3 data blocks from the M data blocks, each of the M3 data blocks is a data block full of valid data, M3≤M; the processing module 402 is also specifically used to determine M1 target data blocks based on the N2 data blocks and M3 data blocks storing valid data, M1 being the sum of N2 and M3.
[0091] In some optional embodiments, the processing module 402 is further specifically used to determine M2 free data blocks in the M data blocks when the addresses of the M1 target data blocks meet a preset condition; the processing module 402 is further specifically used to extract all data stored in the M1 target data blocks in order from high address to low address, and store all data in the M2 free data blocks, and the storage order of the M2 free data blocks is from low address to high address.
[0092] In some optional embodiments, the processing module 402 is further specifically used to obtain metadata of each data block in the M data blocks when the addresses of the M1 target data blocks meet a preset condition, the metadata including the storage status of each data block, the storage status including a full valid data state, or a state of storing invalid data and valid data, or a state of storing invalid data or no data; the processing module 402 is further specifically used to filter out data blocks whose storage status is storing invalid data or no data from the M data blocks, and determine them as M2 free data blocks.
[0093] In some optional implementations, the metadata also includes the storage location of the valid data stored in each data block; the processing module 402 is also specifically used to generate M1 read requests, each of the M1 read requests carries the storage location of the valid data stored in each of the M1 target data blocks; the processing module 402 is also specifically used to send M1 read requests to the M1 target data blocks respectively, read the valid data of each of the M1 target data blocks from the storage location, and extract all the data; the processing module 402 is also specifically used to generate M2 write requests based on all the data, each of the M2 write requests carries the address of each of the M2 free data blocks; the processing module 402 is also specifically used to send M2 write requests to the M2 free data blocks respectively, and store all the data in the M2 free data blocks.
[0094] In some optional implementations, the processing module 402 is further configured to mark the storage status of the M1 target data blocks as storing invalid data or not storing data after extracting all data stored in the M1 target data blocks.
[0095] In some optional embodiments, the processing module 402 is also used to send a stop request to the storage pool when the ratio between M-M2 and M is less than or equal to a third threshold, or the number of consecutive free data blocks at the end of the compressed volume is less than or equal to a fourth threshold, and the stop request is used to request the storage pool to stop reclaiming data blocks.
[0096] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0097] The data block recovery device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0098] The embodiment of the present invention also provides a computer device having the above Figure 5 The recycling device of the data block shown.
[0099] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.
[0100] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware integrated circuit. The hardware integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.
[0101] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.
[0102] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0103] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0104] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.
[0105] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0106] Part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes but is not limited to source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.
[0107] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for recovering a data block, It is characterized in that The method comprises: Obtaining total storage resources of a compressed volume, where the total storage resources consist of M data blocks, where M ≥ 1 and is a positive integer; Determine M1 target data blocks from the M data blocks, each target data block is a data block storing valid data, M1≤M; Obtaining the address of each target data block in the M1 target data blocks; When the addresses of the M1 target data blocks meet a preset condition, all data stored in the M1 target data blocks are stored in M2 free data blocks, so that M-M2 free data blocks remain in the compressed volume, and the preset condition is that there is a free data block with an address smaller than the address of the M1 target data block in the M2 free data blocks, and M2≤M1; When the ratio between the M-M2 and the M is greater than a first threshold, and the number of continuous free data blocks at the end of the compressed volume is greater than a second threshold, a reduction request is sent to the storage pool, and the reduction request is used to request the storage pool to reclaim some or all of the M-M2 free data blocks.
2. The method according to claim 1, It is characterized in that The determining M1 target data blocks from the M data blocks comprises: Selecting N data blocks from the M data blocks, each of the N data blocks storing invalid data and valid data, N≤M; Determine N1 data blocks from the N data blocks, wherein the invalid data ratio of each data block in the N1 data blocks is less than or equal to a preset ratio, and the invalid data ratio is a ratio of the invalid data stored in a data block to all the data, and N1≤N; When N1 is greater than a preset threshold, extracting valid data stored in each of the N1 data blocks to form a valid data set corresponding to the N1 data blocks; The valid data set is stored in N2 idle data blocks to obtain N2 data blocks storing valid data, where N2≤N1; M3 data blocks are selected from the M data blocks, each of the M3 data blocks is a data block fully storing valid data, and M3≤M; The M1 target data blocks are determined according to the N2 data blocks storing valid data and the M3 data blocks, where M1 is the sum of the N2 and the M3.
3. The method according to claim 1, It is characterized in that When the addresses of the M1 target data blocks meet a preset condition, storing all the data stored in the M1 target data blocks into M2 free data blocks includes: In a case where the addresses of the M1 target data blocks meet a preset condition, determining the M2 idle data blocks from the M data blocks; All data stored in the M1 target data blocks are extracted in order from high address to low address, and all data are stored in the M2 free data blocks, and the storage order of the M2 free data blocks is from low address to high address.
4. The method according to claim 3, It is characterized in that The step of determining M2 idle data blocks from the M data blocks when the addresses of the M1 target data blocks meet a preset condition comprises: In the case where the addresses of the M1 target data blocks meet the preset condition, obtaining metadata of each data block in the M data blocks, the metadata including the storage status of each data block, the storage status including a state of storing full valid data, or a state of storing invalid data and valid data, or a state of storing invalid data or no data; From the M data blocks, the data blocks whose storage status is storing invalid data or not storing data are screened out and determined as the M2 free data blocks.
5. The method according to claim 4, It is characterized in that The metadata also includes the storage location of valid data stored in each data block; extracting all data stored in the M1 target data blocks in order from high address to low address, and storing all data in the M2 free data blocks, including: Generate M1 read requests, each of the M1 read requests carries a storage location of valid data stored in each of the M1 target data blocks; Sending the M1 read requests to the M1 target data blocks respectively, reading valid data of each target data block in the M1 target data blocks from the storage location, and extracting all the data; Based on all the data, generate M2 write requests, each of the M2 write requests carrying the address of each free data block in the M2 free data blocks; The M2 write requests are sent to the M2 free data blocks respectively, and all the data are stored in the M2 free data blocks.
6. The method according to any one of claims 1 to 5, It is characterized in that The method further comprises: After all data stored in the M1 target data blocks are extracted, the storage status of the M1 target data blocks is marked as a state of storing invalid data or no data.
7. The method according to claim 1, It is characterized in that The method further comprises: When the ratio of the M-M2 to the M is less than or equal to a third threshold, or when the number of the trailing continuous free data blocks of the compressed volume is less than or equal to a fourth threshold, a stop request is sent to the storage pool, wherein the stop request is used to request the storage pool to stop reclaiming data blocks.
8. A data block recovery device, It is characterized in that The device comprises: An acquisition module, used for acquiring total storage resources of a compressed volume, wherein the total storage resources are composed of M data blocks, where M≥1 and is a positive integer; A processing module, configured to determine M1 target data blocks from the M data blocks, each target data block being a data block storing valid data, M1≤M; The acquisition module is further used to acquire the address of each target data block in the M1 target data blocks; The processing module is further configured to store all data stored in the M1 target data blocks into M2 free data blocks, so that M-M2 free data blocks remain in the compressed volume, when the addresses of the M1 target data blocks meet a preset condition, wherein the preset condition is that there is a free data block in the M2 free data blocks whose address is smaller than the address of the M1 target data blocks, and M2≤M1; The processing module is also used to send a reduction request to the storage pool when the ratio between the M-M2 and the M is greater than a first threshold and the number of continuous free data blocks at the end of the compressed volume is greater than a second threshold. The reduction request is used to request the storage pool to reclaim some or all of the M-M2 free data blocks.
9. A computer device, It is characterized in that include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data block recovery method according to any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data block recycling method according to any one of claims 1 to 7.
