Data recovery method and device

Abstract

The invention provides a data recovery method and device, and belongs to the technical field of storage. According to the scheme provided by the invention, the data blocks in the to-be-recycled archiving storage medium in the first storage region can be written into the cache region, and the to-be-archived data is adopted to replace the junk data blocks in the cache region. And then, writing the data block in the cache region into an idle archiving storage medium, and updating a verification data block corresponding to the junk data block in the first storage region. According to the data recovery scheme, the to-be-archived data can be adopted to replace the junk data blocks, and the verification data blocks corresponding to the junk data blocks are updated, so that the single archived storage medium can be recovered. Therefore, the migration amount of the effective data blocks in the data recovery process is effectively reduced, the data recovery efficiency is improved, and the recovery overhead is reduced.

Description

[0001] This application claims priority to Chinese patent application No. 202310735915.6, filed on June 20, 2023, and entitled “A method, device and other equipment for data recovery”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of storage technology, and in particular to a data recovery method and device. Background Art

[0003] An archival storage system is a storage system dedicated to storing archival data. Among them, archival data has the characteristics of long storage time, low access frequency and large data volume. In order to reduce storage costs, improve storage security and extend storage time, archival storage systems usually use low-cost storage media such as tapes or optical disks as archival storage media. In addition, archival storage media also have the characteristics of high storage density, overall erasure (that is, the minimum erasure unit is the entire storage medium), offline storage and long storage life.

[0004] In order to improve the disk acquisition rate, the archival storage system generally uses erasure coding to encode the archived data. The erasure coding scheme divides the archived data to be stored into multiple data blocks, and uses the coding matrix to perform matrix operations on the multiple data blocks to obtain multiple check blocks. After that, the archival storage system will concurrently write the multiple data blocks and multiple check blocks to multiple archival storage media, and the multiple archival storage media can be called an archival media group.

[0005] Data recycling, also known as garbage collection (GC), is a storage system management mechanism. When a storage system no longer needs certain data, it needs to delete the data to free up storage space. When performing data recycling on an archive storage system that uses erasure coding, the entire group of data stored in the archive media group needs to be recycled, resulting in low data recycling efficiency and high recycling overhead. Summary of the invention

[0006] The present application provides a data recovery method and device, which can solve the technical problems of low data recovery efficiency and high recovery overhead in the archiving storage system of the related art.

[0007] In a first aspect, a data recovery method is provided, which is applied to an archival storage system, wherein the archival storage system includes a first storage area, the first storage area is used to store archived data blocks encoded with erasure codes, and the first storage area includes multiple archival storage media. The method includes: determining a first archival storage medium to be recovered from the first storage area, and writing a target data block set located in the first archival storage medium into a first cache area, wherein the target data block set in the first archival storage medium includes garbage data blocks. Afterwards, the garbage data blocks in the target data block set in the first cache area are replaced with the data to be archived, wherein the data to be archived is the archived data to be stored in the first storage area. The target data block set located in the first cache area is written into a second archival storage medium in the first storage area, and based on the target data block set in the first cache area, the check data blocks corresponding to the garbage data blocks are updated in the first storage area. Wherein, the second archival storage medium is an idle archival storage medium.

[0008] The solution provided by the present application can replace the garbage data block with the data to be archived, and update the check data block corresponding to the garbage data block, so that the recycling of a single archival storage medium can be realized. Thus, the amount of relocation of valid data blocks in the data recycling process is effectively reduced, the efficiency of data recycling is improved, and the recycling overhead is reduced.

[0009] As a possible implementation, the target verification data block set corresponding to the target data block set in the first archival storage medium can be stored in a third archival storage medium in the first storage area. That is, both the data blocks and the verification data blocks are stored in the archival storage medium. Accordingly, based on the target data block set in the first cache area, the process of updating the verification data blocks corresponding to the garbage data blocks in the first storage area may include: writing the target verification data block set in the third archival storage medium into the second cache area, and updating the verification data blocks corresponding to the garbage data blocks in the target verification data block set in the second cache area based on the target data block set in the first cache area. Afterwards, the target verification data block set in the second cache area is written to the fourth archival storage medium in the first storage area, and the fourth archival storage medium is an idle archival storage medium.

[0010] In this implementation, since the check data blocks are also stored in the archival storage medium, it is necessary to update the check data blocks corresponding to the junk data blocks in the target check data block set, and then store the target check data block set in the idle fourth archival storage medium. Based on the above scheme, the valid data blocks can be moved to the second archival storage medium, and the data to be archived can be archived and stored in the second archival storage medium. In addition, the check data blocks corresponding to the valid data blocks can be moved to the fourth archival storage medium, and the check data blocks corresponding to the data to be archived can be archived and stored in the fourth archival storage medium.

[0011] As another possible implementation, the first storage area may also include: a verification storage medium, in which a target verification data block set corresponding to the target data block set in the first archival storage medium is stored. The verification storage medium is a storage medium that supports partial erasure and writing. That is, in this implementation, the data blocks and verification data blocks in the first storage area are stored in different types of storage media.

[0012] Correspondingly, the process of updating the check data blocks corresponding to the garbage data blocks in the first storage area based on the target data block set in the first cache area may include: updating the check data blocks corresponding to the garbage data blocks stored in the check storage medium based on the target data block set in the first cache area.

[0013] In this implementation, since the check data block is stored in the check storage medium that supports partial erasure and writing, the check data block corresponding to the garbage data block can be directly updated in the check storage medium without storing the updated check data block in a new archival storage medium. As a result, the number of drives of the archival storage medium required to be occupied during the data recovery process is effectively reduced, and the amount of check data blocks written is effectively reduced, thereby effectively simplifying the data recovery process and improving the efficiency of data recovery.

[0014] Optionally, based on the target data block set in the first cache area, the process of updating the check data block corresponding to the garbage data block in the first storage area may include: determining the check data block corresponding to the data to be archived based on the data to be archived, the garbage data block and the check data block corresponding to the garbage data block; and updating the check data block corresponding to the garbage data block in the first storage area using the check data block corresponding to the data to be archived.

[0015] Based on the encoding principle of erasure codes, it can be known that when a garbage data block is updated to data to be archived, the verification data block corresponding to the data to be archived can be determined based on the data to be archived, the garbage data block and the verification data block corresponding to the garbage data block, without obtaining other data blocks (i.e., data blocks stored in other archiving storage media in the same archiving media group).

[0016] Optionally, after writing the target data block set in the first cache area to the second archival storage medium in the first storage area, the method may further include: updating metadata of the data blocks stored in the first storage area, and marking the first archival storage medium as an idle archival storage medium. The metadata may include attribute information of the data blocks (such as file name and creator, etc.), and may also include layout information of the data blocks, and the layout information is used to indicate the storage location of the archived data.

[0017] It is understandable that after the first archival storage medium is marked as an idle archival storage medium, the first archival storage medium can be used to write new data blocks in the subsequent process of recycling other archival storage media.

[0018] It can also be understood that the idle archival storage medium in the present application may refer to an archival storage medium that can be overwritten, which may include an archival storage medium newly added to the archival storage system without written data, and may also include a recycled archival storage medium.

[0019] Optionally, the archival storage system may further include: a second storage area for storing the data to be archived; accordingly, the process of replacing the junk data blocks in the target data block set in the first cache area with the data to be archived may include: based on the data amount of the junk data blocks in the target data block set in the first cache area, reading the data amount of the data to be archived from the second storage area, and replacing the junk data blocks in the target data block set in the first cache area with the data to be archived.

[0020] The second storage area can be used to temporarily store data to be archived, and its storage medium can be an archival storage medium or other high-performance storage medium. In addition, the encoding method used for the data to be archived stored in the second storage area can be an erasure code or other encoding methods such as a copy. Since the solution provided by the present application can replace the garbage data blocks with the data to be archived in the second storage area, it can not only realize the recovery of garbage data blocks, but also realize the archival storage of the data to be archived.

[0021] Optionally, the archiving storage system may further include: a second storage area for storing metadata of the data to be archived. Accordingly, the process of replacing the junk data blocks in the target data block set in the first cache area with the data to be archived may include: reading the data of the amount of junk data blocks in the target data block set in the first cache area based on the metadata and the data amount of the junk data blocks in the target data block set in the first cache area; and replacing the junk data blocks in the target data block set in the first cache area with the data to be archived.

[0022] The third storage area may be a storage medium, storage device, storage cluster or storage pool for storing the original data to be archived. The second storage area is used to store metadata of the data to be archived in the third storage area, that is, the second storage area does not need to store the actual data to be archived. Thus, the storage space and network bandwidth required by the second storage area can be effectively reduced.

[0023] Optionally, the process of determining the first archival storage medium to be recycled from the first storage area may include: determining the recycling values ​​of the multiple archival storage media, and determining the first archival storage medium to be recycled based on the sorting of the recycling values ​​of the multiple archival storage media. The recycling value is related to the amount of junk data blocks in the archival storage medium. For example, the more the amount of junk data blocks, the higher the recycling value of the archival storage medium.

[0024] The solution provided in this application can give priority to recycling archival storage media with higher recycling value, thereby timely increasing the proportion of valid data blocks in the archival storage system.

[0025] Optionally, the recycling value may also be related to the storage time and / or type of the valid data block in the archival storage medium. The longer the storage time of the valid data block, the higher the possibility of its subsequent recycling, so the recycling value may be negatively correlated with the storage time of the valid data block, that is, the longer the storage time, the lower the recycling value. The type of valid data block may include data type and storage type. The data type may include text, audio, and video, etc.; the storage type may include standard storage, archival storage, and deep archival storage, etc. The recycling value corresponding to video data may be higher, and the recycling value corresponding to deep archival data stored for a long time may be higher.

[0026] Optionally, the first storage area includes a plurality of archival media groups, each archival media group includes at least two archival storage media, and the check data blocks corresponding to the archival data blocks stored in the at least two archival storage media are stored in the same storage medium. The recycling value may also be related to the total amount of junk data blocks in the archival media group to which the archival storage medium belongs.

[0027] The greater the total amount of garbage data blocks in the archiving medium group to which the archiving storage medium belongs, the higher the recycling value of the archiving storage medium.

[0028] Optionally, the process of determining the first archival storage medium to be recycled based on the sorting of the recycling values ​​of the plurality of archival storage media may include: determining the first archival storage medium and the fifth archival storage medium to be recycled based on the sorting of the recycling values ​​of the plurality of archival storage media, wherein the first archival storage medium and the fifth archival storage medium belong to the same archival medium group. Furthermore, the method may further include: in the process of recycling the junk data blocks in the first archival storage medium, recycling the junk data blocks in the fifth archival storage medium.

[0029] Since the verification data blocks corresponding to the archived data blocks in the same archived media group are stored in the same storage medium, and the verification data blocks need to be updated when each archived storage medium is recycled, in order to improve the efficiency of data recovery and avoid frequent updates of the verification data blocks, the solution provided in the present application can also simultaneously recover data from multiple recyclable archived storage media in an archived media group.

[0030] Optionally, the capacity of the first cache area may be smaller than the capacity of the first archival storage medium, and the data volume of the target data block set is smaller than or equal to the capacity of the first cache area. After writing the target data block set in the first cache area to the second archival storage medium in the first storage area, the method may further include: writing the next data block set in the first archival storage medium after the target data block set as a new target data block set to the first cache area until the garbage data blocks in the first archival storage medium are recovered.

[0031] It is understandable that the first buffer area can be constructed by the memory of the data recovery device, so its capacity is usually much smaller than the capacity of the first archival storage medium. Accordingly, the data recovery device needs to write the data blocks in the first archival storage medium to the first buffer area in batches, and replace the garbage data blocks in the target data block set written each time.

[0032] Optionally, the process of writing the target data block set located in the first archival storage medium into the first cache area may include: if the amount of data to be archived is greater than the amount of garbage data blocks in the first archival storage medium, writing the target data block set located in the first archival storage medium into the first cache area.

[0033] Since the junk data blocks need to be replaced with the data to be archived during the data recovery process, the data recovery device can start the data recovery process after determining that the amount of the data to be archived is greater than the amount of the junk data blocks in the first archival storage medium. In this way, the effective recovery of the junk data blocks in the first archival storage medium can be ensured.

[0034] In a second aspect, a data recovery device is provided. The data recovery device may include at least one module. The at least one module may be used to implement the data recovery method provided in the above aspect.

[0035] In a third aspect, a computing device is provided, the computing device comprising a processor and a memory. The processor is used to execute instructions stored in the memory to implement the data recovery method provided in the above aspect.

[0036] In a fourth aspect, a computing device cluster is provided, the computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory. The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the data recovery method provided by the above aspect.

[0037] In a fifth aspect, a computer-readable storage medium is provided, in which computer program instructions are stored. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the data recovery method provided in the above aspects.

[0038] In a sixth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the data recovery method provided in the above aspects.

[0039] In summary, the present application provides a data recovery method and device. The scheme provided by the present application can write the data blocks in the archive storage medium to be recovered in the first storage area into the cache area, and replace the garbage data blocks in the cache area with the data to be archived. After that, the data blocks in the cache area are written to the idle archive storage medium, and the verification data blocks corresponding to the garbage data blocks are updated in the first storage area. Since the above-mentioned data recovery scheme can replace the garbage data blocks with the data to be archived, and update the verification data blocks corresponding to the garbage data blocks, the recovery of a single archive storage medium can be realized. As a result, the amount of relocation of valid data blocks in the data recovery process is effectively reduced, the efficiency of data recovery is improved, and the recovery overhead is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the encoding principle of an erasure code provided in an embodiment of the present application;

[0041] Figure 2 It is a schematic diagram of the principle of erasure code encoding using a Vandermonde matrix provided in an embodiment of the present application;

[0042] Figure 3 is a schematic diagram of a data recovery process provided in an embodiment of the present application;

[0043] Figure 4 It is a schematic diagram of concurrently writing erasure coded data into a magnetic tape provided by an embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of an application scenario of a data recovery method provided in an embodiment of the present application;

[0045] Figure 6 is a schematic diagram of an application scenario of another data recovery method provided in an embodiment of the present application;

[0046] Figure 7 is a flow chart of a data recovery method provided in an embodiment of the present application;

[0047] Figure 8 is a schematic diagram of a data recovery process provided in an embodiment of the present application;

[0048] Fig. 9 It is a flow chart of a method for updating a verification data block provided by an embodiment of the present application;

[0049] Fig.10 is a schematic diagram of a storage medium included in a first storage area provided in an embodiment of the present application;

[0050] Fig.11 is a schematic diagram of another data recovery process provided in an embodiment of the present application;

[0051] Fig.12 is a schematic diagram of another data recovery process provided in an embodiment of the present application;

[0052] Fig.13 It is a structural schematic diagram of a data recovery device provided in an embodiment of the present application;

[0053] Fig.14 is a structural schematic diagram of another data recovery device provided in an embodiment of the present application;

[0054] Fig.15 is a structural schematic diagram of another data recovery device provided in an embodiment of the present application;

[0055] Fig.16 is a structural schematic diagram of another data recovery device provided in an embodiment of the present application;

[0056] Fig.17 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0057] Fig.18 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0058] Fig.19It is a structural diagram of a network connection between computing devices provided by the present application. DETAILED DESCRIPTION

[0059] The data recovery method and device provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0060] In order to make the technical solutions provided by the embodiments of the present application clearer, relevant terms are first explained before the technical solutions provided by the embodiments of the present application are described in detail.

[0061] (1) Erasure coding (EC): EC is a data protection method that divides data into fragments, expands and encodes redundant data blocks, and stores them in different locations, such as disks, storage nodes, or other geographical locations. Erasure coding technology mainly uses an erasure coding algorithm to encode the original data to obtain redundancy, and stores the data and redundancy together to achieve fault tolerance. Its basic idea is to obtain m blocks of redundant elements (also called check blocks or check data blocks) from n blocks of original data elements (also called data blocks or original data blocks) through certain calculations. For these n+m blocks of elements, when any of the m blocks of elements (including original data and redundant data) is wrong, the original n blocks of data can be restored through the corresponding reconstruction algorithm. The process of generating the check is called encoding, and the process of restoring the lost data blocks is called decoding. The disk utilization of the erasure coding scheme is n / (n+m). Compared with the multi-copy method, the method based on erasure coding has the advantages of low redundancy and high disk utilization.

[0062] (2) Reed-Solomon codes (RS codes): are a type of forward error-corrected channel coding that is based on the effective polynomial generated by correcting the oversampled data. The encoding process first finds redundancy in these polynomials at multiple points before transmitting or storing them. This sampling of the polynomial beyond the necessary value makes the polynomial overdetermined (overdefined). When the receiver correctly receives enough points, it can recover the original polynomial, even if many points on the received polynomial are distorted by noise interference.

[0063] (3) Vandermonde matrix: A Vandermonde matrix is ​​a matrix whose columns are geometric series. In error correction coding, the Vandermonde matrix is ​​used to encode redundant blocks in the commonly used erasure code Reed-Solomon coding.

[0064] (4) Locally repairable codes (LRC): This is a local repairable coding method. The core idea is to divide the parity block into a global parity block and a local reconstruction parity block, and perform group calculations during fault recovery. When any data block error occurs, only the data and parity blocks in the local group need to be used for calculation to recover the original data.

[0065] (5) Garbage collection: refers to an automatic memory management mechanism. When a part of the memory space occupied by a program is no longer accessed by the program, the program will return this part of the memory space to the operating system with the help of the garbage collection algorithm. The garbage collector can reduce the burden on programmers and reduce errors in programs.

[0066] (6) Linear tape file system (LTFS): A new type of ultra-dense tape was developed in 2012, called the "Linear Tape File System". This storage system has higher storage density and lower energy consumption, and can replace hard disks. This prototype ultra-dense tape is covered with a layer of barium ferrite particles. The tape box used is 10 cm long, 10 cm wide, and 2 cm high. It can store 35TB of data, which is approximately equivalent to the information contained in 35 million books. LTFS tape uses a file system similar to that used in disk-based storage, allowing files to be dragged and dropped onto the tape in the same way as they are dragged and dropped onto a disk.

[0067] (7) Total cost of ownership (TCO) is a technique that helps organizations assess, manage, and reduce all costs associated with acquiring an asset over a certain period of time. These assets may be: buildings, vehicles, or software systems. TCO can be described as the sum of the cost of acquiring an asset and the costs incurred throughout its life cycle.

[0068] (8) Terabit (TB): A unit of information measurement, now commonly used to indicate the total capacity of a hard drive or the storage capacity of a large-capacity storage medium.

[0069] (9) Non-volatile memory express (NVMe): is a communication interface and driver that takes advantage of the higher bandwidth provided by the peripheral component interconnect express (PCIe) bus. It is designed to improve performance and efficiency while enabling interoperability between a wide range of enterprise and client systems. NVMe is designed specifically for solid state drives (SSDs) and uses high-speed PCIe slots to communicate between the storage interface and the system central processing unit (CPU), without any form factor restrictions.

[0070] (10) Solid-state drive (SSD): Also known as a solid-state drive, it is a hard disk made of a solid-state electronic storage chip array. Compared with traditional hard disks, solid-state drives have the characteristics of low power consumption, no noise, vibration resistance, and low heat. These characteristics not only make data more securely stored, but also extend the continuous operation time of battery-powered devices. The main disadvantage of solid-state drives is that data is difficult to repair after being damaged.

[0071] (11) Hard disk drive (HDD): Also known as a hard disk drive or mechanical hard disk, it is a traditional hard disk used in most desktop and laptop computers. These drives are electromechanical storage devices that use magnetic storage on a series of physical platters for storing and retrieving digital data. Compared to solid-state drives, mechanical hard disks support higher storage capacities but have slower read / write speeds. Mechanical hard disks contain moving components inside.

[0072] (12) Solid state hybrid drive (SSHD): A computer storage device that has a similar structure to a regular hard drive, but also has built-in flash memory particles of a solid state drive, which enables the drive to achieve read speeds nearly as fast as a solid state drive.

[0073] Since the archived data stored in the archive storage system has the characteristics of long storage time, low access frequency and large data volume, the storage media (including but not limited to tapes and optical disks) used by the archive storage system generally meet the following characteristics:

[0074] 1. High storage density: A single unit can provide several TB or more of storage space. For example, the capacity of LTO9 standard tapes can reach 18TB, and the capacity of some commercial optical disc cartridges can reach 5.5TB. LTO refers to the linear tape open protocol.

[0075] 2. Low storage cost: Compared with storage media such as disks, the TCO of long-term data storage using archival storage media used by archival storage systems is about 1 / 3 or even lower.

[0076] 3. Overall erase: It does not support partial erase or in-place data update. New data can only be written after overall erase or formatting. In other words, the minimum erase and write unit of archival storage media is a single storage medium. For example, when a tape in the tape library of some commercial archival storage systems is full, new data can only be written by reformatting the tape.

[0077] 4. Offline storage: Data is not lost after power failure, and the reading or writing of storage media requires the use of a drive (such as a tape drive or optical drive). In addition, the number of drives is much smaller than the number of storage media. For example, in some commercial archive-level optical disk libraries, 3 drives correspond to 532 optical disk boxes.

[0078] 5. Long storage life: The storage media used in the archiving storage system can have a storage life of up to decades under appropriate storage conditions.

[0079] In order to reduce storage costs, the industry usually uses erasure codes to encode data in archive storage systems. The more mature and widely used erasure codes include Reed-Solomon codes, local check codes, and low-density parity check codes (LDPC). The disk success rate of these commercial EC coding schemes is generally about 75%. Compared with multi-copy storage solutions, erasure code storage solutions take up less space under the premise of the same reliability and availability, but after data corruption, erasure code storage solutions need to complete data recovery through calculations.

[0080] In order to achieve lower storage costs in archival storage systems, a large proportion of erasure codes are usually used to encode data. Figure 1 As shown, the RS (4,2) encoder needs to split the data to be stored into 4 data blocks of equal length d1, d2, d3 and d4, and obtain the check data blocks c1 and c2 by performing matrix operations through a pre-designed encoding matrix. Figure 1 It can be seen that the upper half of the encoding matrix is ​​a unit matrix, and the lower half (including elements b11 to b14, and elements b21 to b24) can be a Vandermonde matrix or a Cauchy matrix.

[0081] After substituting the Vandermonde matrix into the encoding matrix, we can get Figure 2 The encoding process shown in Figure 2. When data block d2 needs to be updated, in addition to updating data block d2 itself, check blocks c1 and c2 also need to be recalculated. Figure 2 It can be seen that the check data blocks c1 and c2 before the update respectively satisfy: c1=d1+d2+d3+d4, c2=d1+2×d2+3×d3+4×d4; the check data blocks c1' and c2' after the update respectively satisfy: c1'=d1+d2'+d3+d4, c2'=d1+2×d2'+3×d3+4×d4. It can be obtained that c1'=c1+d2'-d2, c2'=c2+2×(d2'-d2). Based on the above derivation, it can be seen that the process of updating the check data blocks c1 and c2 does not require reading other data blocks. That is, there is no need to read data blocks d1, d3 and d4. It is only necessary to know d2, c1 and c2 before the update to calculate the updated check data blocks c1' and c2'.

[0082] When some data blocks stored in the archive storage system are no longer needed, the storage space occupied by the data blocks (i.e., garbage data blocks) must be released, which means that garbage collection is required for the archive storage system. The garbage collection process can generally be divided into three stages: garbage identification, data copying, and garbage removal. The following article takes garbage collection of SSD as an example to introduce the garbage collection process. Figure 3 As shown in the figure, the small rectangular blocks in the figure are storage pages, and the rectangle composed of 8 pages is a storage block. In each storage block, the white small squares represent unused pages, and the small squares filled with diagonal lines represent used pages. In SSD, the minimum read and write unit is page, and the minimum erase unit is block. After the page is written with data, new data cannot be written before it is erased.

[0083] refer to Figure 3 In the garbage identification phase of garbage collection, garbage pages in storage block b1 and storage block b2 that are full of data blocks are identified (represented by small squares filled with checkered patterns in the figure). By comparing the amount of garbage data, it is determined that storage block b1 should be recycled. In the data copy phase, the data blocks in the non-garbage pages in storage block b1 are copied to storage block b3 containing unused pages. In the garbage removal phase, all data blocks in storage block b1 are erased to complete the recycling of storage block b1.

[0084] The garbage collection process of an archive storage system is similar to that of an SSD. However, since the storage media used by an archive storage system is usually larger in capacity and the smallest unit of data erased is the entire storage medium, the garbage collection process is more difficult than that of an SSD.

[0085] For example, in LTFS, which is widely used in archival storage systems, after writing data linearly to a tape medium, if the full tape space needs to be reused, the entire tape can only be overwritten. LTFS itself does not provide EC encoding, but only abstracts the tape as LTFS capacity (volume) for use by the storage system. It is necessary to use a distributed file system (Hadoop distributed file system, HDFS) or a distributed storage system such as Ceph on it to have distributed storage capabilities. Among them, Ceph is a distributed storage system that uses RS (10,3), a typical erasure code. In order to ensure reliability, Ceph distributes the data blocks and check blocks of the erasure code as dispersedly as possible in different storage nodes. RS (10,3) used by Ceph has a disk acquisition rate of about 76.9%, which is much higher than the 33.3% disk acquisition rate of three replicas, greatly reducing storage costs.

[0086] Taking the archival storage system using LTO9 tapes and RS (12,2) encoding as an example, in order to ensure the writing and recycling performance and avoid frequent tape switching during the writing and recycling process (a single tape switching takes tens of seconds), Figure 4 As shown, after mounting the tape according to the EC stripe width, the 12 encoded data blocks (data blocks d1 to d12) and 2 check data blocks (check data blocks c1 and c2) are written to 14 tapes concurrently. For the convenience of description, this article refers to the archival storage media written simultaneously as the archival media group. The stripe width can refer to the total number of data blocks and check data blocks obtained after RS ​​encoding. For example, using the RS (n, m) encoding scheme will obtain n data blocks and m check data blocks, and the stripe width is n+m.

[0087] For an archival storage system that adopts an erasure code scheme, if the garbage data blocks in a single archival storage medium in an archival medium group are recovered, the valid data blocks in the archival storage medium to be recovered can be written to a new archival storage medium, and then the data blocks in the archival storage medium to be recovered can be erased. However, the above recovery method will destroy the encoding scheme of the erasure code, resulting in the inability to effectively recover the lost data blocks. Based on this, when performing garbage recovery on an archival storage system that adopts an erasure code scheme, it is necessary to recover the data of the entire archival medium group as a whole, that is, the number of archival storage media to be recovered each time is equal to the stripe width of the erasure code. For example, the valid data blocks in the entire archival medium group can be written to an intermediate storage medium (such as an HDD) first, and then the data blocks (including valid data blocks and garbage data blocks) in the entire archival medium group are deleted. Afterwards, the valid data blocks in the intermediate storage medium are re-encoded with an erasure code and re-concurrently written to the archival medium group.

[0088] Since the capacity of a single archive storage medium is relatively high, it is difficult to ensure that the data blocks in each archive media group need to be deleted at the same time in the actual system. If the garbage data blocks in the archive media group are forcibly recycled, it will lead to the relocation of unacceptable valid data blocks. If the garbage data blocks in the archive media group are recycled only when the data volume exceeds the threshold, the proportion of valid data blocks in the archive media group will be greatly reduced, thereby weakening the cost advantage of the archive storage system. For example, the capacity of a single LTO9 tape is as high as 18TB. RS (12,2) encoding will make the data block capacity in the archive media group that does not contain the check data block as high as 12×18TB, that is, 216TB. In the actual system, it is difficult to ensure that these 216TB data blocks need to be deleted at the same time.

[0089] Based on the above analysis, in order to achieve a higher disk utilization rate and a higher cost-effectiveness, archival storage systems usually use a large proportion of erasure codes to encode data. However, since archival storage media such as optical disks and tapes require the entire storage medium to be erased and overwritten, garbage collection is difficult to perform and the cost is huge. In addition, the proportion of valid data blocks in an archival storage system built with erasure codes is much lower than that of storage media such as disks and SSDs, which greatly weakens the cost advantage of the archival storage system. As the capacity of storage media such as tapes and optical disks continues to increase, the above problems have become more serious and have significantly limited the application scenarios of archival storage systems.

[0090] Figure 5 Schematic diagram of an application scenario of a data recovery method provided in an embodiment of the present application. Figure 5 As shown, the application scenario may be a small or medium-sized archival storage system, and the archival storage system may include multiple storage nodes, and multiple archival storage media may be deployed in each storage node. The client may connect to the storage node through the access node, and write archived data to the storage node through the access node.

[0091] Optionally, refer to Figure 5 The application scenario may also include a cluster or node for storing metadata. The metadata may include attribute information (such as file name, suffix, creator information, etc.) and layout information of the archived data, and the layout information is used to indicate the storage location of the archived data.

[0092] Figure 6 Schematic diagram of another application scenario of the data recovery method provided in the embodiment of the present application. Figure 6 As shown in , the application scenario can be a large-scale cloud archive storage system. Figure 6As shown, the application scenario may include a client, a metadata cluster or node, a standard storage pool, and an archive storage pool. Among them, the standard storage pool may include an access cluster (which may be composed of multiple access nodes) and multiple standard storage nodes. The client can write data to the standard storage node through the access cluster, and the data may include archive data and non-archive data (i.e., data that does not need to be archived). The archive storage pool may include an access cluster and multiple archive storage nodes, and the standard storage pool may write archive data to the archive storage node through the access cluster in the archive storage pool.

[0093] It is understandable that each archive storage node in the archive storage pool may be deployed with multiple archive storage media, and each standard storage node in the standard storage pool may be deployed with multiple standard storage media. The standard storage medium may be a high-performance storage medium such as HDD, SDD or SSHD.

[0094] It is also understandable that the data recovery method provided in the embodiment of the present application can be applied to Figure 5 or Figure 6 The distributed storage system (also referred to as a storage pool) shown can also be applied to a single-machine storage system, that is, the archiving storage system can be deployed in a storage node.

[0095] Figure 7 is a flow chart of a data recovery method provided in an embodiment of the present application. The method can be applied to an archival storage system. The archival storage system can be, for example, Figure 5 or Figure 6 The distributed storage system shown in the figure may also be a stand-alone storage system. In addition, the archival storage system includes a first storage area, which is used to store archival data blocks encoded with erasure codes, and the first storage area includes multiple archival storage media. The archival storage medium may be a storage medium such as a magnetic tape or an optical disk that needs to be erased and overwritten as a whole. Figure 7 As shown, the method includes:

[0096] Step 101: Determine the recycling values ​​of a plurality of archival storage media included in a first storage area.

[0097] In an embodiment of the present application, the data recovery device can periodically monitor the data volume (i.e., garbage volume) of the garbage data blocks stored in each archival storage medium in the first storage area, and can determine the recovery value of each archival storage medium based on the garbage volume. The recovery value can be positively correlated with the garbage volume stored in the archival storage medium, that is, the more garbage volume, the higher the recovery value of the archival storage medium. Among them, the garbage data block can refer to a data block in the archival storage medium that does not need to be further stored, that is, a data block that needs to be recovered. Correspondingly, other data blocks in the archival storage medium except the garbage data blocks can be called valid data blocks.

[0098] Optionally, the recycling value may also be related to the storage time and / or type of the valid data blocks in the archival storage medium. It is understandable that the longer the storage time of the valid data blocks, the higher the possibility of their subsequent recycling. If the valid data blocks are relocated (i.e., copied to new archival storage media), the data blocks to be recycled will occupy more recycling bandwidth. Based on this, the recycling value may be negatively correlated with the storage time of the valid data blocks, that is, the longer the storage time of the valid data blocks, the lower the recycling value of the archival storage medium.

[0099] The type of valid data block may include data type and storage type. The data type may include text, audio, and video, etc. The storage type may include standard storage, archive storage, and deep archive storage, etc. The recycling value corresponding to video data may be higher, and the recycling value corresponding to deep archive data stored for a long time may be higher.

[0100] Step 102: Determine a first archival storage medium to be recycled based on the sorting of the recycling values ​​of the multiple archival storage media.

[0101] The data recovery device may sort the multiple archival storage media based on the recovery values ​​of the multiple archival storage media, and may determine the first archival storage medium to be recovered according to the sorting result. That is, the data recovery device may determine the archival storage medium with a higher recovery value as the first archival storage medium to be recovered. For example, the data recovery device may determine the archival storage medium with the highest recovery value as the first archival storage medium to be recovered.

[0102] Optionally, a value threshold may be set in the data recovery device. Furthermore, the data recovery device may first determine the archival storage medium with a recovery value higher than the value threshold as the recoverable archival storage medium. Afterwards, the recoverable archival storage medium is sorted in descending order according to the recovery value, and the first archival storage medium to be recovered is determined.

[0103] Since the data recovery device can determine the archival storage media to be recovered based on the ranking of the recovery values ​​of each archival storage medium, the archival storage media with higher recovery values ​​can be recovered first, thereby timely increasing the proportion of valid data blocks in the archival storage system.

[0104] Step 103: If the amount of data to be archived is greater than the amount of garbage data blocks in the first archival storage medium, write the target data block set in the first archival storage medium into the first cache area.

[0105] In the embodiment of the present application, since the garbage data blocks need to be replaced with the data to be archived, before starting the data recovery process, the data recovery device can first determine whether the data volume of the data to be archived is greater than the data volume of the garbage data blocks in the first archival storage medium. If the data volume of the data to be archived is greater than the data volume of the garbage data blocks in the first archival storage medium, the data recovery device can determine that the recovery condition is met and start the data recovery process.

[0106] In the data recovery process, the data recovery device can first construct a first cache area, and select a new archival storage medium for storing valid data blocks from the first storage area, that is, a second archival storage medium. It is understandable that the data recovery device can construct the first cache area in the memory, and can select any free archival storage medium in the first storage area as the second archival storage medium. Afterwards, the data recovery device can write the data blocks in the first archival storage medium into the first cache area.

[0107] It is understandable that the capacity of the first cache area is usually smaller than the capacity of the first archival storage medium, so the data recovery device can write the data blocks in the first archival storage medium into the first cache area in batches based on the capacity of the first cache area. The data recovery device can write the target data block set in the first archival storage medium into the first cache area each time, and the data volume of the target data block set is less than or equal to the capacity of the first cache area.

[0108] For example, the capacity of the first archival storage medium may be N times the capacity of the first cache area, where N is an integer greater than 1. Accordingly, the data recovery device may write the data in the first archival storage medium to the first cache area N times, and the amount of data written to the first cache area each time (i.e., the amount of data in the target data block set) may be equal to the capacity of the first cache area.

[0109] Figure 8 Schematic diagram of a data recovery process provided by an embodiment of the present application. Figure 8As shown, assuming that the first archive storage medium to be recycled is D1, the data recovery device can first create a first cache area B1 corresponding to D1. Afterwards, the data blocks in D1 can be read into the first cache area B1 in batches based on the recycling sliding window. The data blocks in D1 that are within the recycling sliding window are the target data block set, and the size of the recycling sliding window can be determined based on the capacity of the first cache area B1.

[0110] Step 104: Use the data to be archived to replace the garbage data blocks in the target data block set in the first cache area.

[0111] Among them, the data to be archived is archived data to be stored in the first storage area. For example, the data to be archived can be newly added archived data. In an embodiment of the present application, in order to achieve effective replacement of the garbage data blocks in the first cache area, the data recovery device can read an equal amount of data to be archived based on the data amount of the garbage data blocks in the first cache area, and use the read data to be archived to overwrite the garbage data blocks. Based on this, the storage space of the garbage data blocks in the first cache area can be filled with the data to be archived, or it can be understood that: the data recovery device can replace (or backfill) the garbage data blocks in the first cache area with the data to be archived to achieve the recovery of the garbage data blocks.

[0112] In the first optional implementation, refer to Figure 8 The archiving storage system may further include a second storage area, the second storage area being used to store the data to be archived. Figure 8 As shown, the data recovery device can read the amount of data to be archived from the second storage area based on the amount of garbage data blocks in the first cache area B1, and use the read data to be archived to replace the garbage data blocks in the first cache area B1.

[0113] In this implementation, the data to be archived stored in the second storage area may be data encoded using erasure codes or data in the form of copies, which is not limited in this embodiment of the present application. It is understandable that if the data to be archived stored in the second storage area is encoded using erasure codes, the data to be archived used to replace the garbage data block in step 104 may be a data block, rather than a check data block.

[0114] In a second optional implementation, the archival storage system may further include a second storage area, and the archival storage system may establish a communication connection with a third storage area. The third storage area may be a storage medium, a storage device, a storage cluster or a storage pool for storing the original data to be archived. The second storage area may be used to store metadata of the data to be archived in the third storage area, that is, the second storage area does not need to store the actual data to be archived. Thus, the storage space and network bandwidth required by the second storage area can be effectively reduced.

[0115] Accordingly, in step 104, the data recovery device may read the amount of data to be archived from the third storage area based on the metadata of the second storage area and the amount of garbage data blocks in the first cache area, and use the data to be archived to replace the garbage data blocks in the first cache area. Figure 6 As shown, the third storage area may be a standard storage pool, and the archiving storage system may pull the data to be archived from the standard storage pool by accessing the cluster.

[0116] It is understandable that the third storage area may also store data that does not need to be archived, that is, the archiving storage system may be used to transfer the archived data that needs to be archived in the third storage area. The process of transferring the archived data may also be called archive transfer.

[0117] It can also be understood that the second storage area can be constructed by at least one storage medium, and the at least one storage medium can be an archival storage medium (such as a tape or an optical disk, etc.), or a storage medium that supports partial erasure and writing (such as an SSD or an HDD, etc.). In addition, the at least one storage medium can be deployed on a storage node, or can be distributedly deployed on different storage nodes, that is, the second storage area can be a storage pool. The embodiment of the present application does not limit the storage medium type and deployment method of the second storage area.

[0118] Step 105: Write the target data block set in the first cache area into the second archival storage medium in the first storage area.

[0119] After replacing the garbage data blocks in the first cache area with the data to be archived, the data recovery device can write the target data block set in the first cache area into the second archival storage medium. The second archival storage medium is an idle archival storage medium in the first storage area. For example, refer to Figure 8 , the data recovery device may write the target data block set in the first cache area into the second archival storage medium D1'.

[0120] It is understandable that the target data block set in the first cache area may include not only the newly written data to be archived, but also the valid data blocks that have not been replaced. Of course, if the target data block set read by the data recovery device from the first archival storage medium through the recovery sliding window at a certain time is all garbage data blocks, then the target data block set in the first cache area may all be the newly written data to be archived.

[0121] Based on the above analysis, it can be known that after the garbage data blocks of the target data block set in the first cache area are replaced, the target data block set in the first cache area is written to the second archival storage medium, so that the relocation of the valid data blocks and the archival storage of the data to be archived can be achieved. Correspondingly, the data blocks stored in the second archival storage medium are archived data blocks that do not contain garbage data blocks.

[0122] Step 106: Based on the target data block set in the first cache area, update the check data block corresponding to the garbage data block in the first storage area.

[0123] In the embodiment of the present application, since the archived data stored in the archive storage system are all encoded with erasure codes, the first storage area also stores check data blocks corresponding to the garbage data blocks. Accordingly, after the garbage data blocks are replaced with the data to be archived, the check data blocks corresponding to the garbage data blocks need to be updated based on the target data block set in the first cache area.

[0124] Based on the encoding principle of the erasure code described above, it can be known that when a data block in the original archived data is updated (i.e., the garbage data block is updated to the data to be archived), the corresponding verification data block needs to be recalculated and updated. Moreover, in the process of updating the verification data block, it is only necessary to obtain the data block and the verification data block before the update (i.e., the garbage data block and the verification data block corresponding to the garbage data block), as well as the updated data block (i.e., the data to be archived), without obtaining other data blocks. That is, when updating the verification data block corresponding to the garbage data block, there is no need to obtain the data blocks stored in other archive storage media in the same archive media group.

[0125] It can be seen from this that the data recovery device can determine the verification data block corresponding to the data to be archived based on the data to be archived, the garbage data block and the verification data block corresponding to the garbage data block, and then use the verification data block corresponding to the data to be archived to update the verification data block corresponding to the garbage data block stored in the first storage area.

[0126] In a first optional implementation, the target verification data block set corresponding to the target data block set in the first archival storage medium is stored in a third archival storage medium in the first storage area. That is, the storage media included in the first storage area can all be archival storage media, and the erasure coded data blocks and verification data blocks are all stored in the archival storage medium. Fig. 9 As shown, the above step 106 may include:

[0127] Step 1061: Write the target verification data block set in the third archival storage medium into the second cache area.

[0128] In this implementation, the data recovery device can construct a second cache area corresponding to the third archival storage medium while constructing the first cache area, and the size of the second cache area can be the same as the size of the first cache area. Afterwards, the data recovery device can write the target data block set in the first archival storage medium into the first cache area while writing the target verification data block set in the third archival storage medium into the second cache area. The target verification data block set corresponds to the target data block set, that is, the verification data blocks in the target verification data block set are obtained by encoding the data blocks in the target data block set.

[0129] It is understandable that one garbage data block may correspond to multiple check data blocks, and the multiple check data blocks may be correspondingly stored in multiple archival storage media, that is, the number of third archival storage media may be multiple. Accordingly, the data recovery device may construct multiple second cache areas corresponding to the multiple third archival storage media, and write the target check data block set in each third archival storage medium into a corresponding second cache area.

[0130] It can also be understood that the capacity of the second cache area is also smaller than the capacity of the third archival storage medium, so the data recovery device can write the check data blocks in the third archival storage medium into the second cache area in batches based on the capacity of the second cache area. The data recovery device can write the target check data block set in the third archival storage medium into the second cache area each time, and the data volume of the target check data block set is less than or equal to the capacity of the second cache area.

[0131] For example, Figure 8As shown, assuming that the check data blocks corresponding to the data blocks stored in the first archival storage medium D1 are stored in the third archival storage mediums C1 and C2, the data recovery device can create a first cache area B1 corresponding to D1, and create a second cache area B2 corresponding to the third archival storage medium C1, and a second cache area B3 corresponding to the third archival storage medium C2. Afterwards, the data recovery device can read the data blocks and check data blocks in D1, C1 and C2 to the corresponding cache areas in batches based on the recovery sliding window. It can be understood that the check data blocks in C1 and C2 located within the recovery sliding window are the target check data block set.

[0132] Step 1062: Based on the target data block set in the first cache area, update the check data blocks corresponding to the garbage data blocks in the target check data block set in the second cache area.

[0133] As described above, the data recovery device can calculate the check data block corresponding to the data to be archived based on the data to be archived, the garbage data block and the check data block corresponding to the garbage data, and then use the check data block corresponding to the data to be archived to replace the check data block corresponding to the garbage data block in the second cache area. Figure 8 As shown, the data recovery device can use the calculated new check data blocks to replace the check data blocks corresponding to the garbage data blocks in the second buffer areas B2 and B3 respectively.

[0134] Step 1063: Write the target verification data block set in the second cache area to the fourth archival storage medium in the first storage area.

[0135] Among them, the fourth archival storage medium is an idle archival storage medium in the first storage area. It can be understood that the target verification data block set in the second cache area may include verification data blocks corresponding to valid data blocks that have not been replaced, in addition to the new verification data blocks that are replaced and written. Of course, if the verification data blocks read from the third storage medium by the data recovery device through the recovery sliding window at a certain time are all verification data blocks corresponding to garbage data blocks, then the target verification data block set in the second cache area may all be new verification data blocks that are replaced and written. Correspondingly, after the target verification data block set in the second cache area is written into the fourth archival storage medium in the first storage area, the verification data blocks corresponding to the valid data blocks can be moved from the third archival storage medium to the fourth archival storage medium, and the verification data blocks corresponding to the data to be archived can be archived and stored in the fourth archival storage medium.

[0136] It is understandable that step 1063 and step 105 may be performed synchronously, that is, the data recovery device may concurrently write the data in the first cache area and the second cache area to the corresponding archival storage medium.

[0137] Example, reference Figure 8 , the data recovery device may write the target verification data block set in the second buffer area B2 to the fourth archival storage medium C1', and may write the target verification data block set in the second buffer area B3 to the fourth archival storage medium C2'. For example, the data recovery device may concurrently write the contents of the first buffer area B1, the second buffer areas B2 and B3 to the corresponding archival storage media D1', C1' and C2'.

[0138] In the first implementation, since the check data block is also stored in the archival storage medium, and data recycling is to recycle the garbage data block, in the above steps 101 and 102, the data recycling device can only determine the recycling value of the archival storage medium used to store the data block, and determine the first archival storage medium to be recycled from the archival storage medium used to store the data block. For the archival storage medium used to store the check data block, it is not necessary to determine its recycling value.

[0139] In the second alternative implementation, refer to Fig.10 , the first storage area may also include: a verification storage medium, in which a target verification data block set corresponding to the target data block set in the first archival storage medium is stored, and the verification storage medium supports partial erasure and writing. For example, the verification storage medium may be a storage medium with a long erasure life such as an SSD, HDD or SSHD. It can be seen that, if Fig.10 As shown, in the second implementation, the first storage area includes two types of storage media: an archival storage medium for storing data blocks, and a verification storage medium for storing verification data blocks.

[0140] Since the verification storage medium supports partial erasure and writing, in step 106, after the data recovery device determines the verification data block corresponding to the data to be archived, it can directly use the verification data block corresponding to the data to be archived to update the verification data block corresponding to the garbage data block stored in the verification storage medium. That is, in the second implementation, the data recovery device does not need to write the verification data block corresponding to the data to be archived into a new storage medium, but can directly implement in-situ update of the verification data block in the verification storage medium.

[0141] Example, reference Fig.11 , the data recovery device can also construct second cache areas B2 and B3 corresponding to the verification storage medium, and read the verification data blocks corresponding to the garbage data blocks stored in the verification storage medium into the second cache areas B2 and B3. Afterwards, the data recovery device can replace the verification data blocks in the second cache areas B2 and B3 with the verification data blocks corresponding to the data to be archived, and update the verification data blocks corresponding to the garbage data blocks in the verification storage medium based on the verification data blocks in the second cache areas B2 and B3.

[0142] It is understandable that in the second implementation, the size of each second buffer area constructed by the data recovery device can be the same as the size of the first buffer area. Alternatively, the data recovery device can also determine the data volume of the check data block to be updated according to the data volume of the garbage data block in the target data block set in the first archival storage medium, and construct a second buffer area of ​​corresponding capacity according to the data volume of the check data block to be updated. Fig.11 As shown, the capacity of the second cache area may be less than or equal to the capacity of the first cache area.

[0143] Based on the coding principle of erasure codes, it is known that when each archival storage medium in the archival medium group is recycled, the verification data block needs to be updated, that is, the verification data block needs to be repeatedly erased. If the verification data block is stored in the archival storage medium, the drive of the scarce archival storage medium needs to be occupied each time the verification data block is updated. Therefore, in the above-mentioned second implementation method, a verification storage medium with a longer erasure life and that can support partial erasure and writing (i.e., supporting in-situ updates) can be used to store the verification data block. Based on this, during the data recovery process, the verification data block corresponding to the garbage data block can be directly updated in the verification storage medium without the need to store the updated verification data block in the new archival storage medium. As a result, the number of drives required to be occupied during the data recovery process is effectively reduced, and the write amount of the verification data block is effectively reduced, thereby effectively simplifying the data recovery process and improving the efficiency of data recovery.

[0144] Step 107: Write the next data block set after the target data block set in the first archival storage medium into the first cache area as a new target data block set.

[0145] It can be understood that, based on the above steps 104 to 106, the garbage data blocks in a target data block set can be recovered. Accordingly, the data recovery device can write the next data block set after the target data block set in the first archival storage medium as a new target data block set into the first cache area, and continue to execute the above steps 104 to 106 until the recovery of all garbage data blocks in the first archival storage medium is completed.

[0146] Example, reference Figure 8 and Fig.11 After the data recovery device completes the recovery of garbage data blocks within a recovery sliding window based on the above steps 104 to 106, it can slide the recovery sliding window and execute the method shown in the above steps 104 to 106 again until the recovery of garbage data blocks in the first archival storage medium is completed.

[0147] Step 108: Update the metadata of the data block stored in the first storage area.

[0148] In the embodiment of the present application, after the data recovery device completes the recovery of the first archival storage medium, since the junk data blocks in the first archival storage medium have been replaced by the data to be archived and stored in the second archival storage medium, and the valid data blocks in the first archival storage medium have been moved to the second archival storage medium, it is necessary to update the metadata of the data blocks stored in the first storage area. The metadata may include attribute information of the data blocks (such as file name, suffix, creator, etc.), and may also include layout information of the data blocks (i.e., storage location of the data blocks).

[0149] For example, after the data recovery device completes the recovery of the first archival storage medium, it can modify the mapping relationship between the archival storage medium and the abstract layer. Based on this, the metadata of the valid data blocks stored in the first archival storage medium does not need to be changed, and only the metadata of the newly written valid data blocks (i.e., the data to be archived) need to be newly created, and then the metadata of the garbage data blocks need to be deleted. Alternatively, the data recovery device can create the metadata of the data to be archived, update the metadata of the valid data blocks in the first archival storage medium, and delete the metadata of the garbage data blocks.

[0150] Step 109: Mark the first archival storage medium as an idle archival storage medium.

[0151] After the data recovery device completes the recovery of the first archival storage medium, it can mark the first archival storage medium as an idle archival storage medium. Accordingly, when other archival storage media are subsequently recovered, the first archival storage medium can be used to store new data blocks or verification data blocks.

[0152] It is understandable that if the check data blocks corresponding to the garbage data blocks in the first archival storage medium are stored in the third archival storage medium, then after completing the recycling of the first archival storage medium, the data recycling device can also mark the third archival storage medium as an idle archival storage medium. Figure 8 , the data recovery device can mark the archival storage media D1, C2 and C2 as idle archival storage media.

[0153] It can also be understood that after the data recovery device completes the recovery of the first archival storage medium, it is not necessary to erase the data blocks in the first archival storage medium, nor to erase the verification data blocks in the third archival storage medium, but only to mark the above archival storage medium as idle archival storage medium. When the archival storage medium is needed later, new data can be written into the archival storage medium by overwriting.

[0154] Based on the above analysis, it can be known that the idle archival storage medium described in the embodiment of the present application may refer to an archival storage medium that can be overwritten and written. In addition, the idle archival storage medium may be an archival storage medium newly added to the first storage area with no data written thereto, or an archival storage medium recovered by a data recovery device.

[0155] In the embodiment of the present application, the multiple archival storage media included in the first storage area can be divided into multiple archival media groups, each archival media group can include at least two archival storage media, and the verification data blocks corresponding to the archival data blocks stored in the at least two archival storage media can be stored in the same storage medium. That is, at least two archival storage media in each archival media group can share the same verification media group, and the verification media group includes multiple storage media for storing verification data blocks.

[0156] Since the verification data block needs to be updated during the process of recycling each archival storage medium, in order to improve the efficiency of data recycling and avoid frequent updating of the verification data block, the data recycling device can also simultaneously recycle data from multiple recyclable archival storage media in an archival medium group.

[0157] Accordingly, in the above step 101, the recovery value of the archival storage medium determined by the data recovery device may also be related to the total amount of junk data blocks in the archival medium group to which it belongs. Moreover, the more junk data blocks there are in the archival medium group to which the archival storage medium belongs, the higher the recovery value of the archival storage medium. Based on this, in the above step 102, the data recovery device may not only determine the first archival storage medium to be recovered, but also determine the fifth archival storage medium to be recovered, and the first archival storage medium and the fifth archival storage medium belong to the same archival medium group.

[0158] Accordingly, in the process of recycling the garbage data blocks in the first archival storage medium as shown in the above steps 103 to 107, the data recycling device can synchronously recycle the garbage data blocks in the fifth archival storage medium. That is, the data recycling device can synchronously recycle the garbage data blocks in the fifth archival storage medium based on the method shown in the above steps 103 to 107. Moreover, after completing the recycling of the fifth archival storage medium, the data recycling device can mark the fifth archival storage medium as an idle archival storage medium.

[0159] Example, reference Fig.12, the archival storage media to be recovered determined by the data recovery device may include D1 and D2 belonging to the same archival media group, and the check data blocks corresponding to the data blocks in the two archival storage media are stored in C1 and C2. Accordingly, the data recovery device can create a first cache area B1 corresponding to D1, a first cache area B2 corresponding to D2, a second cache area B3 corresponding to C1, and a second cache area B4 corresponding to C2. Afterwards, the data recovery device can read the target data block sets in D1 and D2 into the first cache areas B1 and B2 respectively, and read the target check data block sets in C1 and C2 into the second cache areas B3 and B4 respectively through the recovery sliding window. Then, the data recovery device can use the data to be archived in the second storage area to replace the garbage data blocks in B1 and B2 respectively, and update the check data blocks corresponding to the garbage data blocks in the second cache areas B3 and B4 accordingly. Finally, the data recovery device can write the target data block sets in the first cache areas B1 and B2 into the archival storage media D1' and D2' respectively, and write the target verification data block sets in the second cache areas B3 and B4 into the archival storage media D3' and D4' respectively.

[0160] The above description is based on the example of a data recovery device recovering two archive storage media synchronously. It is understandable that the data recovery device can also recover more archive storage media in the same archive media group synchronously. Since the method provided in the embodiment of the present application can recover garbage data blocks in multiple archive storage media at the same time, it can avoid frequent updates of verification data blocks, thereby effectively improving the efficiency of data recovery.

[0161] Optionally, the data recovery device may determine the multiple archive storage media to be recovered in the following ways:

[0162] As a first possible implementation, the data recovery device may first calculate the recovery value of each archival storage medium in the first storage area, and determine the archival storage medium with the highest recovery value as the first archival storage medium to be recovered. Afterwards, the data recovery device may determine all other recoverable archival storage media in the archival medium group to which the first archival storage medium belongs as archival storage media to be recovered.

[0163] As a second possible implementation, the data recovery device may first calculate the recovery value of each archival medium group in the first storage area, and the recovery value of the archival medium group is positively correlated with the recovery value of each archival storage medium included therein. Afterwards, the data recovery device may determine the archival medium group with the highest recovery value as the archival medium group to be recovered, and determine all the recyclable archival storage media in the archival medium group to be recovered as the archival storage media to be recovered.

[0164] As a third possible implementation, the data recovery device may first calculate the recovery value of each archival storage medium in the first storage area, and determine the recoverable archival storage medium in the first storage area. Afterwards, the data recovery device may determine the archival media group containing the most recoverable archival storage media as the archival media group to be recovered, and determine all the recoverable archival storage media in the archival media group to be recovered as the archival storage media to be recovered.

[0165] It can be understood that, for the scenario in which the data recovery device synchronously recovers garbage data blocks in multiple archival storage media, in the above step 103, the data recovery device can start the garbage recovery process when it determines that the amount of data to be archived is greater than the total amount of garbage data blocks in the multiple archival storage media.

[0166] It can also be understood that after the above step 109, the data recovery device can continue to perform steps 102 to 109, that is, continue to determine the next archival storage medium to be recovered based on the sorting of recovery values, and continue to recover the garbage data blocks in the archival storage medium. In addition, the data recovery device can repeatedly perform the method shown in the above steps 102 to 109 until all recyclable archival storage media are recovered, or the amount of data to be archived in the second storage area (or the third storage area) is less than the amount of garbage data blocks in the archival storage medium to be recovered, and then stop the data recovery operation.

[0167] It can also be understood that the order of the steps of the data recovery method provided in the embodiment of the present application can be appropriately adjusted, and the steps can also be increased or decreased accordingly according to the situation. For example, the above steps 101 and 102 can be deleted according to the situation, such as the data recovery device can determine any archival storage medium whose data volume of the garbage data block is greater than the threshold as the first archival storage medium to be recovered. Alternatively, the above step 106 can be performed before step 105. Alternatively, the above step 109 can be performed before step 108.

[0168] Based on the above analysis, it can be known that compared with the traditional solution of recycling the entire media group according to the stripe width of the erasure code, the method provided in the embodiment of the present application can be centered on the archival storage medium, and replace re-encoding with data replacement and update, thereby breaking through the limitation that the entire storage medium group needs to be recycled after the erasure code is encoded. Based on this, the method provided in the embodiment of the present application can recycle data for any number of archival storage media according to the amount of garbage in the archival storage medium, that is, a single archival storage medium or an archival storage medium less than the stripe width of the erasure code can be recycled each time. For example, assuming that the erasure code used by the archival storage system is RS (12, 2), the number of archival storage media recycled at a single time can be less than 14. If the archival storage medium is a magnetic tape, less than 14 tapes can be loaded into the tape drive for data recovery in a single recycling.

[0169] In summary, an embodiment of the present application provides a data recovery method, which can write data blocks in an archive storage medium to be recovered in a first storage area into a cache area, and replace garbage data blocks in the cache area with the data to be archived. Afterwards, the data blocks in the cache area are written to an idle archive storage medium, and the verification data blocks corresponding to the garbage data blocks are updated in the first storage area. Since the above-mentioned data recovery scheme can replace garbage data blocks with data to be archived, and update the verification data blocks corresponding to the garbage data blocks, the recovery of a single archive storage medium can be achieved. As a result, the amount of relocation of valid data blocks during the data recovery process is effectively reduced, the efficiency of data recovery is improved, and the recovery overhead is reduced. Among them, the recovery overhead can refer to the number of drives and system bandwidth occupied during the data recovery process.

[0170] Moreover, since the method provided in the embodiment of the present application designs a unique local recycling strategy, it abandons the practice of recycling by erasure code stripes in traditional solutions and realizes the recycling of part or even a single archive storage medium. Therefore, it can also effectively increase the proportion of valid data blocks in the archive storage system and reduce the overall cost of the archive storage system.

[0171] In addition, the method provided in the embodiment of the present application can also store the verification data block in a high-performance verification storage medium. Based on the idea of ​​hybrid storage, the number of drives required for the data recovery process is further reduced, and the problem of low erase life of the archival storage medium is avoided, thereby achieving efficient recovery. Moreover, for the archival transfer scenario, the method provided in the embodiment of the present application can store metadata in the second storage area, thereby further optimizing the recovery process and reducing resource consumption in the data recovery process.

[0172] The present application also provides a data recovery device, which can be applied to Figure 5 or Figure 6The archival storage system shown in the figure can be used to implement the data recovery method provided by the above method embodiment. The archival storage system includes a first storage area, the first storage area is used to store archived data blocks encoded with erasure codes, and the first storage area includes multiple archival storage media. Fig.13 As shown, the data recovery device comprises:

[0173] The data management module 201 is used to determine the first archival storage medium to be recycled from the first storage area. The function of the data management module 201 can be realized by referring to the relevant description of step 102 in the above method embodiment.

[0174] The garbage collection module 202 is used to write the target data block set located in the first archive storage medium into the first cache area, where the target data block set in the first archive storage medium includes garbage data blocks; replace the garbage data blocks in the target data block set in the first cache area with data to be archived, where the data to be archived is archived data to be stored in the first storage area; and write the target data block set located in the first cache area into a second archive storage medium in the first storage area, where the second archive storage medium is an idle archive storage medium.

[0175] The functional implementation of the garbage collection module 202 may refer to the relevant descriptions of steps 103 to 105 in the above method embodiment.

[0176] The data encoding module 203 is used to update the check data block corresponding to the garbage data block in the first storage area based on the target data block set in the first buffer area. The function of the data encoding module 203 can be realized by referring to the relevant description of step 106 in the above method embodiment.

[0177] As a possible implementation, the target verification data block set corresponding to the target data block set in the first archival storage medium is stored in a third archival storage medium in the first storage area. Fig.14 As shown, the data blocks and the verification data blocks in the first storage area are stored in the same type of storage medium (ie, archival storage medium).

[0178] The garbage collection module 202 may also be configured to write the target verification data block set in the third archival storage medium into the second cache area.

[0179] The data encoding module 203 may be used to update the check data blocks corresponding to the junk data blocks in the target check data block set in the second cache area based on the target data block set in the first cache area.

[0180] The garbage collection module 202 may also be configured to write the target verification data block set in the second cache area to a fourth archival storage medium in the first storage area, where the fourth archival storage medium is an idle archival storage medium.

[0181] The implementation of the garbage collection module 202 can also refer to the relevant description of step 1061 and step 1063 in the above method embodiment. The implementation of the data encoding module 203 can also refer to the relevant description of step 1062 in the above method embodiment.

[0182] As another possible implementation, Fig.15 As shown, the first storage area may further include: a verification storage medium, in which a target verification data block set corresponding to the target data block set in the first archival storage medium is stored.

[0183] Correspondingly, the data encoding module 203 can be used to update the verification data blocks corresponding to the garbage data blocks stored in the verification storage medium based on the target data block set in the first cache area.

[0184] Optionally, the data encoding module 203 can be used to determine the verification data block corresponding to the data to be archived based on the data to be archived, the junk data block and the verification data block corresponding to the junk data block, and use the verification data block corresponding to the data to be archived to update the verification data block corresponding to the junk data block in the first storage area.

[0185] Optionally, the data management module 201 may also be used to update metadata of the data blocks stored in the first storage area and mark the first archival storage medium as an idle archival storage medium after the garbage collection module 202 writes the target data block set in the first cache area into the second archival storage medium in the first storage area. The function of the data management module 201 may also refer to the relevant description of step 108 and step 109 in the above method embodiment.

[0186] Optionally, the archiving storage system may further include: a second storage area for storing the data to be archived. The garbage collection module 202 may be configured to read the data to be archived of the amount of garbage data blocks in the target data block set in the first cache area from the second storage area, and replace the garbage data blocks in the target data block set in the first cache area with the data to be archived.

[0187] Alternatively, if Fig.16As shown, the archiving storage system may also include: a second storage area for storing metadata of the data to be archived; the garbage collection module 202 may be used to read the data amount of the data to be archived from the third storage area based on the metadata and the data amount of the garbage data blocks in the target data block set in the first cache area, and use the data to be archived to replace the garbage data blocks in the target data block set in the first cache area.

[0188] Optionally, the data management module 201 can be used to determine the recycling values ​​of the multiple archival storage media, and determine the first archival storage medium to be recycled based on the sorting of the recycling values ​​of the multiple archival storage media. The recycling value is related to the amount of garbage data blocks in the archival storage medium. The function implementation of the data management module 201 can also refer to the relevant description of step 101 in the above method embodiment.

[0189] Optionally, the recycling value may also be related to the storage time and / or type of the valid data block in the archival storage medium.

[0190] Optionally, the first storage area may include a plurality of archival media groups, each archival media group includes at least two archival storage media, and the check data blocks corresponding to the archival data blocks stored in the at least two archival storage media are stored in the same storage medium. The recycling value may also be related to the total amount of junk data blocks in the archival media group to which the archival storage medium belongs.

[0191] Optionally, the data management module 201 may be used to determine a first archival storage medium and a fifth archival storage medium to be recycled based on the sorting of recycling values ​​of the multiple archival storage media, where the first archival storage medium and the fifth archival storage medium belong to the same archival medium group.

[0192] Correspondingly, the garbage collection module 202 and the data encoding module 203 can also be used to synchronously recycle the garbage data blocks in the fifth archiving storage medium during the process of reclaiming the garbage data blocks in the first archiving storage medium.

[0193] Optionally, the capacity of the first cache area is smaller than the capacity of the first archival storage medium, and the data volume of the target data block set in the first archival storage medium is smaller than or equal to the capacity of the first cache area.

[0194] The data management module 201 can also be used to write the next data block set located after the target data block set located in the first cache area into the second archive storage medium in the first storage area as a new target data block set, after the garbage collection module 202 writes the target data block set located in the first cache area into the second archive storage medium in the first storage area, until the garbage collection module 202 completes the recovery of garbage data blocks in the first archive storage medium.

[0195] Optionally, the data management module 201 can also be used to call the garbage collection module 202 to write the target data block set in the first archive storage medium into the first cache area if the amount of data to be archived is greater than the amount of garbage data blocks in the first archive storage medium.

[0196] Alternatively, if Figures 13 to 16 As shown, the data recovery device may further include: a data interface module 204. The data interface module 204 is used to provide a standard data access interface to obtain the data to be archived. Fig.16 The data interface module 204 can obtain the data to be archived from the third storage area.

[0197] It is understandable that the data management module 201 can monitor the status of each storage area (e.g., the first storage area and the second storage area), determine the data writing mode, and trigger the garbage collection process. The data encoding module 203 can provide various types of data encoding and verification update capabilities. The garbage collection module 202 can be used to perform garbage collection actions.

[0198] In summary, an embodiment of the present application provides a data recovery device, which can write data blocks in an archive storage medium to be recovered in a first storage area into a cache area, and replace garbage data blocks in the cache area with the data to be archived. Afterwards, the data blocks in the cache area are written to an idle archive storage medium, and the check data blocks corresponding to the garbage data blocks are updated in the first storage area. Since the above-mentioned data recovery scheme can replace garbage data blocks with data to be archived, and update the check data blocks corresponding to the garbage data blocks, it is possible to achieve the recovery of a single archive storage medium. As a result, the amount of relocation of valid data blocks in the data recovery process is effectively reduced, the efficiency of data recovery is improved, and the recovery overhead is reduced.

[0199] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the data recovery device and each module described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0200] It is understandable that each module in the data recovery device (such as the data management module 201, the garbage collection module 202, the data encoding module 203 and the data interface module 204, etc.) can be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the data management module 201 is introduced below by taking the data management module 201 as an example. Similarly, the implementation of the garbage collection module 202, the data encoding module 203 and the data interface module 204 can refer to the implementation of the data management module 201.

[0201] As an example of a software functional unit, the data management module 201 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the data management module 201 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0202] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0203] As an example of a hardware functional unit, the data management module 201 may include at least one computing device, such as a server, etc. Alternatively, the data management module 201 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0204] The multiple computing devices included in the data management module 201 can be distributed in the same region or in different regions. The multiple computing devices included in the data management module 201 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the data management module 201 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0205] It is understandable that in other embodiments, the data management module 201 can be used to execute any step in the above data recovery method, and the garbage collection module 202, the data encoding module 203 and the data interface module 204 can also be used to execute any step in the above data recovery method. The steps that the data management module 201, the garbage collection module 202, the data encoding module 203 and the data interface module 204 are responsible for implementing can be specified as needed, and the above modules are used to implement different steps in the data recovery method to realize the full functions of the data recovery device.

[0206] It can also be understood that if the archival storage system provided in the embodiment of the present application is a stand-alone storage system, then each module in the data recovery device can be deployed in one storage node. Figure 5 or Figure 6 For a distributed storage system as shown, the various modules in the data recovery device can be distributedly deployed in different nodes. For example, the data interface module 204 can be deployed in an access node or access cluster, the data management module 201 can be deployed in a metadata cluster or node, and the garbage collection module 202 and the data encoding module 203 can be deployed in a storage node. Alternatively, for a small-scale distributed storage system, the various modules in the data recovery device can be centrally deployed in one node (e.g., an access node).

[0207] Fig.17 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. Fig.17 , the computing device may include: a bus 302, a processor 304, a memory 306, and a communication interface 308. The processor 304, the memory 306, and the communication interface 308 communicate with each other via the bus 302. The computing device may be a server or a terminal device. It should be understood that the embodiments of the present application do not limit the number of processors and memories in the computing device.

[0208] The bus 302 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.17 The fact that only one line is used in the figure does not mean that there is only one bus or one type of bus. Bus 304 may include a path for transmitting information between various components of the computing device (for example, memory 306, processor 304, and communication interface 308).

[0209] The processor 304 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0210] The memory 306 may include a volatile memory, such as a random access memory (RAM). The processor 304 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0211] The memory 306 stores executable program codes, and the processor 304 executes the executable program codes to respectively implement the functions of the aforementioned data management module 201, garbage collection module 202, data encoding module 203, and data interface module 204, thereby implementing the data recovery method provided by the above method embodiment. That is, the memory 306 stores instructions for executing the data recovery method.

[0212] The communication interface 303 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device and other devices or a communication network.

[0213] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0214] like Fig.18 As shown, the computing device cluster includes at least one computing device. The memory 306 in one or more computing devices in the computing device cluster may store the same instructions for executing the above data recovery method.

[0215] In some possible implementations, the memory 306 of one or more computing devices in the computing device cluster may also store partial instructions for executing the above data recovery method. In other words, a combination of one or more computing devices may jointly execute instructions for executing the above data recovery method.

[0216] It is understandable that the memory 306 in different computing devices in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the data recovery device. That is, the instructions stored in the memory 306 in different computing devices can implement the functions of one or more modules of the data management module 201, the garbage collection module 202, the data encoding module 203 and the data interface module 204.

[0217] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.19 A possible implementation is shown in FIG. Fig.19 As shown, two computing devices 300A and 300B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device. In this type of possible implementation, the memory 306 in different computing devices in the computing device cluster can store different instructions for executing some functions of the data recovery device. That is, the instructions stored in the memory 306 in different computing devices can implement the functions of one or more modules in the data recovery device.

[0218] The present application embodiment also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the data recovery method provided by the above method embodiment.

[0219] The present application embodiment also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the data recovery method provided in the above method embodiment.

[0220] In the embodiments of the present application, the terms "first", "second" and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In the embodiments of the present application, the term "at least one" refers to one or more, and "multiple" refers to two or more. In the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previously associated objects are in an "or" relationship.

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, rather than to limit them. Although the embodiments of the present application have been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A data recovery method, characterized in that: Applied to an archival storage system, the archival storage system comprises a first storage area, the first storage area is used to store archival data blocks encoded with erasure codes, and the first storage area comprises a plurality of archival storage media; the method comprises: determining a first archival storage medium to be recycled from the first storage area; Writing a target data block set in the first archival storage medium into a first cache area, wherein the target data block set in the first archival storage medium includes garbage data blocks; Replacing the garbage data blocks in the target data block set in the first buffer area with data to be archived, the data to be archived being archived data to be stored in the first storage area; Writing the target data block set located in the first cache area into a second archival storage medium in the first storage area, where the second archival storage medium is an idle archival storage medium; Based on the target data block set in the first cache area, the check data blocks corresponding to the garbage data blocks are updated in the first storage area.

2. The method according to claim 1, characterized in that The target verification data block set corresponding to the target data block set in the first archival storage medium is stored in a third archival storage medium in the first storage area; The updating, in the first storage area, of the check data blocks corresponding to the garbage data blocks based on the target data block set in the first cache area includes: Writing the target verification data block set in the third archival storage medium into a second buffer area; Based on the target data block set in the first cache area, updating the check data block corresponding to the garbage data block in the target check data block set in the second cache area; The target verification data block set in the second cache area is written into a fourth archival storage medium in the first storage area, where the fourth archival storage medium is an idle archival storage medium.

3. The method according to claim 1, characterized in that The first storage area further includes: a verification storage medium, in which a target verification data block set corresponding to the target data block set in the first archival storage medium is stored; The updating, in the first storage area, of the check data block corresponding to the garbage data block based on the target data block set in the first cache area includes: Based on the target data block set in the first cache area, the check data block corresponding to the garbage data block stored in the check storage medium is updated.

4. The method according to any one of claims 1 to 3, characterized in that: The updating, in the first storage area, of the check data block corresponding to the garbage data block based on the target data block set in the first cache area includes: Determine the check data block corresponding to the data to be archived based on the data to be archived, the garbage data block and the check data block corresponding to the garbage data block; The check data block corresponding to the to-be-archived data is used to update the check data block corresponding to the garbage data block in the first storage area.

5. The method according to any one of claims 1 to 4, characterized in that: After writing the target data block set in the first cache area into the second archival storage medium in the first storage area, the method further includes: Updating metadata of the data blocks stored in the first storage area; The first archival storage medium is marked as a free archival storage medium.

6. The method according to any one of claims 1 to 5, characterized in that: The archiving storage system further includes: a second storage area for storing the data to be archived; and the replacing of the garbage data blocks in the target data block set in the first buffer area with the data to be archived includes: Based on the data amount of the junk data blocks in the target data block set in the first cache area, reading the to-be-archived data of the data amount from the second storage area; The to-be-archived data is used to replace the garbage data blocks in the target data block set in the first buffer area.

7. The method according to any one of claims 1 to 5, characterized in that: The archiving storage system further comprises: a second storage area for storing metadata of the data to be archived; The step of replacing the garbage data blocks in the target data block set in the first buffer area with the data to be archived includes: Based on the metadata and the data amount of the junk data blocks in the target data block set in the first cache area, reading the data amount of the to-be-archived data from the third storage area; The to-be-archived data is used to replace the garbage data blocks in the target data block set in the first buffer area.

8. The method according to any one of claims 1 to 7, characterized in that: The determining of the first archival storage medium to be recycled from the first storage area includes: determining recycling values ​​of the plurality of archival storage media, the recycling values ​​being related to the amount of data of junk data blocks in the archival storage media; Based on the ranking of the recycling values ​​of the plurality of archival storage media, a first archival storage medium to be recycled is determined.

9. The method according to claim 8, characterized in that The recycling value is also related to the storage time and / or type of the valid data blocks in the archival storage medium.

10. The method according to claim 8 or 9, characterized in that: The first storage area includes a plurality of archival medium groups, each archival medium group includes at least two archival storage media, and the verification data blocks corresponding to the archival data blocks stored in the at least two archival storage media are stored in the same storage medium; The recycling value is also related to the total amount of garbage data blocks in the archiving medium group to which the archiving storage medium belongs.

11. The method according to claim 10, characterized in that The step of determining a first archival storage medium to be recycled based on the sorting of the recycling values ​​of the plurality of archival storage media comprises: Determining a first archival storage medium and a fifth archival storage medium to be recycled based on the sorting of the recycling values ​​of the plurality of archival storage media, wherein the first archival storage medium and the fifth archival storage medium belong to the same archival medium group; The method further comprises: In the process of reclaiming the garbage data blocks in the first archival storage medium, the garbage data blocks in the fifth archival storage medium are reclaimed.

12. The method according to any one of claims 1 to 11, characterized in that: The capacity of the first cache area is smaller than the capacity of the first archival storage medium, and the data volume of the target data block set is smaller than or equal to the capacity of the first cache area; After writing the target data block set in the first cache area into the second archival storage medium in the first storage area, the method further includes: The next data block set in the first archival storage medium that is located after the target data block set is written into the first cache area as a new target data block set until the garbage data blocks in the first archival storage medium are completely recycled.

13. The method according to any one of claims 1 to 12, characterized in that: Writing the target data block set located in the first archival storage medium into the first cache area includes: If the data volume of the data to be archived is greater than the data volume of the garbage data blocks in the first archival storage medium, the target data block set in the first archival storage medium is written into the first cache area.

14. A data recovery device, characterized in that: Applied to an archival storage system, the archival storage system comprises a first storage area, the first storage area is used to store archival data blocks encoded with erasure codes, and the first storage area comprises a plurality of archival storage media; the device comprises: A data management module, configured to determine a first archival storage medium to be recycled from the first storage area; a garbage collection module, configured to write a target data block set in the first archival storage medium into a first cache area, wherein the target data block set in the first archival storage medium includes garbage data blocks; The garbage collection module is further configured to replace the garbage data blocks in the target data block set in the first buffer area with data to be archived, wherein the data to be archived is archived data to be stored in the first storage area; The garbage collection module is further configured to write the target data block set in the first cache area into a second archival storage medium in the first storage area, where the second archival storage medium is an idle archival storage medium; A data encoding module is used to update the check data block corresponding to the garbage data block in the first storage area based on the target data block set in the first cache area.

15. The device according to claim 14, characterized in that The target verification data block set corresponding to the target data block set in the first archival storage medium is stored in a third archival storage medium in the first storage area; The garbage collection module is further configured to write the target verification data block set in the third archival storage medium into a second cache area; The data encoding module is used to update the check data block corresponding to the junk data block in the target check data block set in the second cache area based on the target data block set in the first cache area; The garbage collection module is further configured to write the target verification data block set in the second cache area into a fourth archival storage medium in the first storage area, where the fourth archival storage medium is an idle archival storage medium.

16. The device according to claim 14, characterized in that The first storage area further includes: a verification storage medium, in which a target verification data block set corresponding to the target data block set in the first archival storage medium is stored; The data encoding module is used to update the verification data block corresponding to the garbage data block stored in the verification storage medium based on the target data block set in the first cache area.

17. The device according to any one of claims 14 to 16, characterized in that The data encoding module is used to: Determine the check data block corresponding to the data to be archived based on the data to be archived, the garbage data block and the check data block corresponding to the garbage data block; The check data block corresponding to the to-be-archived data is used to update the check data block corresponding to the garbage data block in the first storage area.

18. The device according to any one of claims 14 to 17, characterized in that The data management module is also used for: After the garbage collection module writes the target data block set located in the first cache area into the second archival storage medium in the first storage area, updating metadata of the data blocks stored in the first storage area; The first archival storage medium is marked as a free archival storage medium.

19. The device according to any one of claims 14 to 18, characterized in that The archiving storage system further includes: a second storage area for storing the data to be archived; and a garbage collection module for: Based on the data amount of the junk data blocks in the target data block set in the first cache area, reading the to-be-archived data of the data amount from the second storage area; The to-be-archived data is used to replace the garbage data blocks in the target data block set in the first buffer area.

20. The device according to any one of claims 14 to 18, characterized in that The archiving storage system further includes: a second storage area for storing metadata of the data to be archived; and the garbage collection module for: Based on the metadata and the data amount of the junk data blocks in the target data block set in the first cache area, reading the data amount of the to-be-archived data from the third storage area; The to-be-archived data is used to replace the garbage data blocks in the target data block set in the first buffer area.

21. The device according to any one of claims 14 to 20, characterized in that The data management module is used to: determining recycling values ​​of the plurality of archival storage media, the recycling values ​​being related to the amount of data of junk data blocks in the archival storage media; Based on the ranking of the recycling values ​​of the plurality of archival storage media, a first archival storage medium to be recycled is determined.

22. The device according to claim 21, characterized in that The recycling value is also related to the storage time and / or type of the valid data blocks in the archival storage medium.

23. The device according to claim 21 or 22, characterized in that The first storage area includes a plurality of archival medium groups, each archival medium group includes at least two archival storage media, and the verification data blocks corresponding to the archival data blocks stored in the at least two archival storage media are stored in the same storage medium; The recycling value is also related to the total amount of garbage data blocks in the archiving medium group to which the archiving storage medium belongs.

24. The device according to claim 23, characterized in that The data management module is used to determine a first archival storage medium and a fifth archival storage medium to be recycled based on the sorting of recycling values ​​of the plurality of archival storage media, wherein the first archival storage medium and the fifth archival storage medium belong to the same archival medium group; The garbage collection module and the data encoding module are further configured to recycle the garbage data blocks in the fifth archive storage medium during the process of reclaiming the garbage data blocks in the first archive storage medium.

25. The device according to any one of claims 14 to 24, characterized in that The capacity of the first cache area is smaller than the capacity of the first archival storage medium, and the amount of data of the target data block set in the first archival storage medium is smaller than or equal to the capacity of the first cache area; The data management module is further configured to, after the garbage collection module writes the target data block set located in the first cache area into the second archive storage medium in the first storage area, write the next data block set located after the target data block set in the first archive storage medium as a new target data block set into the first cache area, until the garbage collection module completes the recovery of garbage data blocks in the first archive storage medium.

26. The device according to any one of claims 14 to 25, characterized in that The data management module is used to call the garbage collection module to write the target data block set in the first archival storage medium into the first cache area if the data volume of the data to be archived is greater than the data volume of the garbage data blocks in the first archival storage medium.

27. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 13.

28. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 13.

29. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 13.

Images