Apparatus and method for improved sequential access data redundancy storage

By storing predefined data in sequential access data storage and independently storing parity information, the problem of low redundant storage efficiency in large-scale disk systems is solved, and efficient data redundant storage and flexible RAID-level support are achieved.

CN118922822APending Publication Date: 2024-11-08HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202280094003.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-06-14
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

When the prior art realizes effective data redundant storage in large-scale disk systems, a large number of tape drives are required to support RAIDs with small overhead, and zigzag encoding is not practical in more than 10 disk drive systems, making it difficult to efficiently process sequential access to data. redundant storage.

Method used

By storing predefined data in sequential access data memory (SADS) and independently storing parity information, ensuring that parity information is fully calculated before storage, reducing overhead, and storing it in Direct Access Data Memory (DADS) to Supports efficient direct access.

Benefits of technology

Improves efficiency of storing sequential access to data, reduces overhead, and supports flexible RAID-level implementations for multi-tape drive systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118922822A_ABST
    Figure CN118922822A_ABST
Patent Text Reader

Abstract

The present disclosure relates to the field of storage, data redundancy processing, RAID and tape drives. The invention relates to a device (100) for redundant processing of sequential access data, in which the device (100) is configured to store a predefined number of data (101) in a sequential access data store (SADS) (102), in particular to a device (100) for redundant processing of sequential access data, in which the device (100) is configured to store a predefined number of data (101) in a sequential access data store (SADS) (102). The apparatus is further configured to determine parity information (103) based on the predefined number of data (101), and to store the parity information (103) in the SADS (102) when the predefined number of data (101) is stored in the SADS (102).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the fields of storage, data redundancy processing, redundant array of independent disks (RAID) and tape drives, and provides a device for improving redundant storage of data in a sequential access manner. In addition, the present disclosure provides a corresponding method and computer program. Background Art

[0002] In conventional storage systems, erasure codes are used for redundant data storage. Regeneration codes are another type of erasure code that supports faster reconstruction. Erasure codes are codes that support data recovery in the event of partial data loss. For example, redundant array of independent disks (RAID)5 (the concept is in Figure 7 ) uses an extra disk that includes parity data from all other devices and supports recovery of lost data in the event of a disk failure. For example, RAID 6 is a scheme that supports rebuilding data in the event of two drive failures. RAID 5 and RAID 6 require reading all data from the remaining disks in the event of a drive failure.

[0003] Other encodings, such as zigzag encoding, allow significantly less data to be read in the event of a drive failure, and can also repair more than one disk failure. Unfortunately, zigzag encoding scales exponentially, so it is not practical for systems with more than 10 disk drives.

[0004] As a general example, Figure 7 A RAID 5 system with two data disks and one parity disk is shown in Figure 1. The disks are sliced, and each slice retains parity. The disks containing the parity are different for each slice - for the first slice, the data is located on the first two disks A1, A2, and the parity A p = (A1 XOR A2) is stored on the third disk. For the second slice, parity is stored on the first disk and data is stored on the other two disks. In the event of a failure of Disk 2, each block in the disk can be recovered using (Disk0 XOR Disk1). In a tape system RAID (called redundant array of independent tapes (RAIT)), data is usually written to multiple tapes that appear as a single tape. For example, Figure 8It shows how subsequent blocks of file A are written to three tape drives in a round robin manner. Since file A is written to tape drives that only support sequential access, file A can also be referred to as "sequential access data". As shown in the figure, file A is striped to three physical tape drives, but these tape drives are visualized to appear as a single drive.

[0005] Tape libraries that support RAID (ie, RAIT) write data directly to tape with parity, which means that the number of tape drives required determines the available RAID for the system.

[0006] The disadvantage is that many tape drives are needed to implement a RAID with a large parity number and low overhead.

[0007] Therefore, there is a need for an improved redundancy mechanism for storing sequentially accessed data. Summary of the invention

[0008] In view of the above problems, an object of the embodiments of the present disclosure is to provide an improved redundancy mechanism for storing sequentially accessed data.

[0009] This object and other objects can be achieved by embodiments of the present disclosure as described in the attached independent claims. Advantageous implementations of embodiments of the present disclosure are further defined in the dependent claims.

[0010] A first aspect of the present disclosure provides a device for sequential access data redundancy processing, wherein the device is used to: store a predefined amount of data in a sequential access data store (SADS); determine parity information based on the predefined amount of data; and store the parity information in the SADS when the predefined amount of data is stored in the SADS.

[0011] This ensures that parity information is stored in SADS when a predefined amount of data is already stored in SADS. That is, since parity information that changes during storage of a predefined amount of data can be stored separately from the predefined amount of data, the efficiency of storing sequential access data is improved and overhead is reduced.

[0012] Specifically, the device is used to obtain the predefined amount of data before storing the predefined amount of data in the SADS. Specifically, the predefined amount of data is obtained from an external device.

[0013] Specifically, the SADS may be external to the device for sequential access data redundancy processing. Specifically, the SADS may be a part of the device for sequential access data redundancy processing.

[0014] Specifically, the sequential access data includes data to be stored in the SADS.

[0015] In an implementation manner of the first aspect, the device is further configured to: store the parity information in the SADS when the predefined amount of data is completely stored in the SADS.

[0016] This ensures that the parity information can be fully calculated before being stored in the SADS, thereby reducing overhead (eg, for temporarily storing the parity information in the SADS) and further improving efficiency.

[0017] In another implementation of the first aspect, the device is further configured to store the parity information in a direct access data store (DADS).

[0018] This ensures that the parity information can be stored in a manner that supports direct access to the parity information, which is very efficient during the calculation of the parity information because the parity information does not need to be stored in a sequential access manner.

[0019] Specifically, the DADS may be external to the apparatus for sequential access data redundancy processing. Specifically, the DADS may be a part of the apparatus for sequential access data redundancy processing.

[0020] In another implementation of the first aspect, the device is further configured to: store the parity information in the DADS before the parity information is stored in the SADS.

[0021] This is advantageous because no parity information needs to be stored to the SADS during storage of the predefined amount of data.

[0022] In another implementation of the first aspect, the device is further configured to: update the parity information for each element of the predefined amount of data being stored in the SADS.

[0023] This ensures that the parity information can be updated, for example, when it is stored in the SADS, so that the parity information only needs to be written to the SADS once.

[0024] Specifically, the element includes at least one of the following: a file; a data block.

[0025] In another implementation of the first aspect, compared with the SADS, the DADS is a storage of a higher storage level.

[0026] This is advantageous because higher storage tiers support direct access to data, whereas lower storage tiers only support sequential access to data.

[0027] In another implementation of the first aspect, the DADS includes at least one of the following: primary storage and secondary storage.

[0028] This ensures that storage types can be used that support efficient direct access to data.

[0029] In another implementation of the first aspect, the primary storage includes at least one of the following: a main memory, a random access memory.

[0030] This ensures that a memory type that supports efficient use of direct access to data can be used.

[0031] In another implementation of the first aspect, the secondary storage includes at least one of the following: a mass storage device, a hard disk drive, a solid state drive, a flash drive, a RAM disk, and a non-volatile memory.

[0032] This ensures that a memory or disk type can be used that supports direct access to the data for efficient use.

[0033] In another implementation of the first aspect, the SADS includes three levels of storage.

[0034] This ensures that a storage type that supports efficient use of sequential access to the data can be used.

[0035] In another implementation of the first aspect, the tertiary storage includes a tape drive.

[0036] This ensures that drive types can be used that support efficient use of sequential access to data.

[0037] In another implementation of the first aspect, the SADS includes X storage entities, where X is an integer greater than 1, and each of the X storage entities can store the same amount of data. It is also possible that one or more of the X storage entities can store different amounts of data.

[0038] This ensures that X storage (eg, tape drives) can be used to implement the SADS.

[0039] Specifically, the storage entity is a virtual or physical storage drive. Specifically, the storage medium in the storage entity can be replaced during the storage of a predefined amount of data in the SADS. For example, if the storage entity is a tape drive, the tape in the tape drive can be replaced when it is full.

[0040] In another implementation of the first aspect, the DADS includes storage space for Y storage entities, wherein each storage entity of the Y storage entities can store the same amount of data as a storage entity of the X storage entities.

[0041] This ensures that DADS and SADS can flexibly implement the required RAID level.

[0042] Specifically, the ratio of X to Y corresponds to a RAID level of a predefined amount of data when the predefined amount of data is stored by the device.

[0043] Specifically, the term RAID refers to Redundant Array of Inexpensive Disks.

[0044] A second aspect of the present disclosure provides a method for sequential access data redundancy processing, the method comprising the following steps: a device stores a predefined amount of data in a sequential access datastore (SADS); the device determines parity information based on the predefined amount of data; when the predefined amount of data is stored in the SADS, the device stores the parity information in the SADS.

[0045] In an implementation manner of the second aspect, the method further includes: when the predefined amount of data is completely stored in the SADS, the device stores the parity information in the SADS.

[0046] In another implementation of the second aspect, the method further includes: the device storing the parity information in a direct access data store (DADS).

[0047] In another implementation of the second aspect, the method further includes: the device storing the parity information in the DADS before the parity information is stored in the SADS.

[0048] In another implementation of the second aspect, the method further includes: the device updating the parity information for each element of the predefined amount of data being stored in the SADS.

[0049] In another implementation of the second aspect, compared with the SADS, the DADS is a storage of a higher storage level.

[0050] In another implementation of the second aspect, the DADS includes at least one of the following: primary storage and secondary storage.

[0051] In another implementation of the second aspect, the primary storage includes at least one of the following: a main memory, a random access memory.

[0052] In another implementation of the second aspect, the secondary storage includes at least one of the following: a mass storage device, a hard disk drive, a solid state drive, a flash drive, a RAM disk, and a non-volatile memory.

[0053] In another implementation of the second aspect, the SADS includes three levels of storage.

[0054] In another implementation of the second aspect, the tertiary storage includes a tape drive.

[0055] In another implementation of the second aspect, the SADS includes X storage entities, where X is an integer greater than 1, and each of the X storage entities can store the same amount of data. It is also possible that one or more of the X storage entities can store different amounts of data.

[0056] In another implementation of the second aspect, the DADS includes storage space for Y storage entities, wherein each storage entity of the Y storage entities can store the same amount of data as a storage entity of the X storage entities.

[0057] The second aspect and its implementations include the same advantages as the first aspect and its corresponding implementations.

[0058] A third aspect of the present disclosure provides a computer program comprising instructions. When the program is executed by a computer, the instructions cause the computer to perform the method according to the second aspect or any implementation manner thereof.

[0059] The third aspect includes the same advantages as the first aspect and its corresponding implementations.

[0060] It should be noted that all devices, elements, units and devices described in this application can be implemented in software or hardware elements or any combination thereof. All steps performed by various entities described in this application and the functions described to be performed by various entities are intended to indicate that the corresponding entities are used to perform the corresponding steps and functions. Although in the description of the following specific embodiments, the specific functions or steps performed by the external entity are not reflected in the description of the specific detailed elements of the entity performing the specific steps or functions, it should be clear to the technician that these methods and functions can be implemented by corresponding hardware or software elements or any combination thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In conjunction with the accompanying drawings, the following description of specific embodiments illustrates various aspects and implementations of the present disclosure. In the drawings:

[0062] Figure 1 A schematic diagram of a device provided by an embodiment of the present disclosure is shown;

[0063] Figure 2 A schematic diagram showing a device provided by an embodiment of the present disclosure in more detail;

[0064] Figure 3 A schematic diagram of a storage archiving system is shown;

[0065] Figure 4 A schematic diagram showing an operation scenario provided by the present disclosure is shown;

[0066] Figure 5 Another schematic diagram showing an operation scenario provided by the present disclosure;

[0067] Figure 6 A schematic diagram of a method provided by an embodiment of the present disclosure is shown;

[0068] Figure 7 A schematic diagram of a conventional RAID system is shown;

[0069] Figure 8 A schematic diagram of a conventional RAIT system is shown. DETAILED DESCRIPTION

[0070] Figure 1A schematic diagram of a device 100 is shown. The device 100 is used for improved redundant storage of sequential access data. To this end, the device 100 is used to store a predefined amount of data 101 in a sequential access data store (SADS) 102. The device 100 can be used to obtain a predefined amount of data 101 from an external device to store the predefined amount of data 101 in SADS 102. For example, the predefined amount of data 101 can be provided to the device from a computing device via a network connection or a system bus, etc. The device 100 is also used to determine parity information 103 based on the predefined amount of data 101. In order to store the predefined amount of data 101 in a redundant manner, the device 100 obtains parity information 103 and stores the parity information 103. More precisely, when the predefined amount of data 101 is stored in SADS 102, the device 100 stores the parity information 103 in SADS 102.

[0071] Thus, parity information 103 may be stored separately from a predefined amount of data 101 (eg, files, data objects, or blocks), which improves efficiency and reduces overhead.

[0072] Alternatively, the device 100 may first store the predefined amount of data 101 in its entirety in the SADS 102, and then store the parity information 103 in the SADS 102. That is, the parity information 103 is stored in the SADS 102 after its final state is determined and no additional changes are expected to be made to the parity information 103 (the parity information may change with each item of the predefined amount of data 101 stored in the SADS 102).

[0073] Now combine Figure 2 The apparatus 100 is described in more detail. Figure 2 The device 100 includes Figure 1 All functions and features of the device 100 are shown.

[0074] like Figure 2 As shown, the device 100 may optionally store the parity information 103 in a direct access data store (DADS) 201. In the DADS 201, changes may be easily applied to the parity information 103 based on the predefined amount of data 101 while the predefined amount of data 101 is stored in the SADS 102.

[0075] Alternatively, the device 100 may store the parity information 103 in the DADS 201 before storing the parity information 103 in the SADS 102. Therefore, the parity information 103 is stored in the SADS after being stored in the DADS 201, where the parity information 103 may be easily modified by directly accessing it.

[0076] Figure 2 It is schematically shown that the parity information 103 can be updated optionally for each element 202 of (and based on) the predefined amount of data 101 being stored in the SADS 102. That is, if the predefined amount of data 101 has not been completely stored in the SADS 102, the parity information 103 can be changed based on the element 202.

[0077] Alternatively, DADS 201 may be a higher storage tier than SADS 102. Storage tiering is a method of storing data based on how the data is accessed on different storage media (so-called storage tiers).

[0078] Alternatively, if Figure 2 As shown, DADS201 includes at least one of the following: primary storage 203 and secondary storage 204. Primary storage 203 may include at least one of the following: main memory, random access memory. Secondary storage 204 may include at least one of the following: mass storage device, hard disk drive, solid state hard disk, flash drive, RAM disk, non-volatile memory.

[0079] Alternatively, if Figure 2 As shown, SADS 102 may include tertiary storage 205. Alternatively, tertiary storage 205 may include a tape drive.

[0080] As now will refer to Figure 3 As described, the device 100 may optionally also be regarded as a storage system (or as part of a storage system), which may expose multiple protocols such as a file system and an object storage, and may store data in a tape-level long-term archive (corresponding to SADS102 and in Figure 3 In multiple levels (denoted by 102).

[0081] Device 100 can support any RAID type, regardless of the number of tapes. Device 100 can maintain RAID of tapes on a higher level (i.e., DADS201, including file system / object interface, SSD layer or HDD layer). In other words, parity information 103 is stored in a higher level (i.e., DADS201) before being stored in SADS102. Therefore, device 100 may need space for N tapes in a higher level, where N is the number of parity devices. Therefore, for example, if the tape needs double protection and each tape has 10TB of data, the total space required on the higher HDD / SSD layer will be 20TB.

[0082] The data to be written to the tape (i.e., a predefined amount of data 101) can be either tiered to the HDD / SSD (which will be written to the tape later) or written directly to the storage. The data will be written directly to the tape, and the parity of the data will be retained on a higher tier.

[0083] Since the parity algorithm is linear, the device 100 can create a RAID with a larger storage capacity than a single tape by writing to multiple tapes (i.e., SADS 102) simultaneously and updating the parity in DADS 201. Typically, data is written to the tapes sequentially, and since parity is linear, it can be assumed that tapes that have not yet been written have zero data and therefore do not affect the current parity.

[0084] The following Figure 4 and Figure 5 It shows how to use three tape drives to build a RAID 6 (four data units + two parity units). This can be achieved because at the beginning, the object is only written to the first three tape drives (which form SADS102), and the parity is kept on the HDD / SSD (i.e., DADS201).

[0085] like Figure 4 and Figure 5 As shown, SADS102 includes X storage entities, where X is an integer greater than 1, and each of the X storage entities can store the same amount of data. In the figure, X is equal to 3, and the three storage entities correspond to "Tape 1", "Tape 2" and "Tape 3".

[0086] Once the three tape cartridges in the tape drive are full, the tape cartridges in the three tape drives are replaced and three new tape cartridges are put in. As new data arrives, the data is written to cartridge 4, and as the new data is written, parity is calculated from the parity saved on the higher level. The portion of the parity that was written to the tape can then be erased from the higher level.

[0087] In the figure, DADS 201 is formed by "higher levels" (i.e., "Parity 1" and "Parity 2") and includes storage space for Y storage entities, wherein each of the Y storage entities can store the same amount of data as a storage entity in the X storage entities. Figure 4 and Figure 5 , Y is equal to 2.

[0088] It should be noted that although this example is RAID 6 (4+2) with 3 tapes, any protection level X+Y can be achieved as long as there is space for parity information in the higher levels.

[0089] Data can be written to multiple tape drives in parallel, i.e. file 1 is written to tape 1, file 2 is written to tape 2, and file 3 is again written to tape 1. Once all tapes are written to a certain location, the parity for that portion can be written to the parity tape.

[0090] Recovery of a failed tape can be performed in a similar manner, by rebuilding the data of the failed tape in a higher level, which supports the use of any number of available tapes for reconstruction.

[0091] Figure 6 A schematic diagram of a method 600 is shown. The method 600 is used for sequential access data redundancy processing and comprises a first step: the device 100 stores (601) a predefined amount of data 101 in a sequential access data store (SADS) 102. The method also comprises a second step: the device 100 determines (602) parity information 103 based on the predefined amount of data 101. The method comprises a third step: when the predefined amount of data 101 is stored in the SADS 102, the device 100 stores (603) the parity information 103 in the SADS 102.

[0092] The disclosure has been described in conjunction with various embodiments as examples and implementations. However, from a study of the drawings, the disclosure and the independent claims, other variations will be understood and implemented by those skilled in the art in practicing the claimed disclosure. In the claims and in the specification, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" does not exclude a plurality. A single element or other unit may fulfil the functions of several entities or items described in the claims. The listing of certain measures in mutually different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Claims

1. A device (100) for sequential access data redundancy processing, wherein: The device (100) is used for: storing a predefined amount of data (101) in a sequential access data store SADS (102); Determining parity information (103) based on the predefined amount of data (101); When the predefined amount of data (101) is stored in the SADS (102), the parity information (103) is stored in the SADS (102).

2. The device (100) according to claim 1, wherein The device (100) is further configured to store the parity information (103) in the SADS (102) when the predefined amount of data (101) is completely stored in the SADS (102).

3. The device (100) according to claim 1 or 2, wherein: The device (100) is further configured to store the parity information (103) in a direct access data storage DADS (201).

4. The device (100) according to any one of the preceding claims, wherein: The device (100) is further configured to store the parity information (103) in the DADS (201) before the parity information (103) is stored in the SADS (102).

5. The device (100) according to any one of the preceding claims, wherein: The device (100) is further configured to update the parity information (103) for each element (202) of the predefined amount of data (101) being stored in the SADS (102).

6. The device (100) according to any one of the preceding claims, wherein: Compared with the SADS (102), the DADS (201) is a higher storage level storage.

7. The device (100) according to any one of the preceding claims, wherein: The DADS (201) includes at least one of the following: primary storage (203) and secondary storage (204).

8. The device (100) according to claim 7, wherein The primary storage (203) includes at least one of the following: a main memory, a random access memory.

9. The device (100) according to claim 7 or 8, wherein: The secondary storage (204) includes at least one of the following: a mass storage device, a hard disk drive, a solid state drive, a flash drive, a RAM disk, a non-volatile memory.

10. The device (100) according to any one of the preceding claims, wherein The SADS (102) includes three levels of storage (205).

11. The device (100) according to claim 10, wherein The tertiary storage (205) includes a tape drive.

12. The device (100) according to any one of the preceding claims, wherein: The SADS (102) includes X storage entities, where X is an integer greater than 1, and each of the X storage entities can store the same amount of data.

13. The device (100) according to any one of claims 3 to 12, wherein: The DADS (201) includes storage space for Y storage entities, wherein each storage entity of the Y storage entities can store the same amount of data as a storage entity of the X storage entities.

14. A method (600) for sequential access data redundancy processing, wherein: The method (600) comprises the following steps: The device (100) stores (601) a predefined amount of data (101) in a sequential access data store SADS (102); The device (100) determines (602) parity information (103) based on the predefined amount of data (101); When the predefined amount of data (101) is stored in the SADS (102), the device (100) stores (603) the parity information (103) in the SADS (102).

15. A computer program comprising instructions, wherein: When the instructions are executed by a computer, the computer is caused to perform the method (600) according to claim 14.