Data storage method and system and storage medium

By grouping data blocks and local verification blocks in a multi-AZ architecture, and storing global verification blocks and local verification blocks in the same AZ, the problem of large bandwidth repair across AZ is solved, and efficient data is achieved Repair and resource utilization.

CN119088284AInactive Publication Date: 2024-12-06UNIV OF SCI & TECH OF CHINA +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310665631.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-06-05
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing erasure coding technology has caused the bandwidth of cross-AZ repairs to be too large under the multi-AZ architecture, and it is impossible to effectively reduce the amount of cross-AZ data transmission required for single-block repairs, resulting in waste of resources and repair time overhead.

Method used

By grouping data blocks and local check blocks, storing them in multiple AZs, and storing the global check blocks and their corresponding local check blocks in the same AZ, repairing within the AZ is achieved, thereby reducing cross-AZ transmissions quantity.

Benefits of technology

It effectively reduces the amount of data transmission across AZ required for single-block repair, significantly reduces the repair bandwidth across AZ under multiple AZ architectures, and improves repair efficiency and performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088284A_ABST
    Figure CN119088284A_ABST
Patent Text Reader

Abstract

The invention discloses a data storage method and system and a storage medium. The method comprises the steps that L-1 first data sets and a second data set are determined based on K data blocks corresponding to data to be processed; wherein the first data set comprises K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data set comprises G global check blocks and second local check blocks corresponding to the G global check blocks; k and L are integers greater than 1, and G is an integer greater than 0; and storing the L-1 first data sets to at least L-1 first available areas based on the G, and storing the second data sets to the second available areas.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data storage method and system, and a storage medium. Background Art

[0002] Erasure code technology can minimize the system storage overhead while ensuring data reliability. Therefore, erasure code technology is widely used in the field of data storage technology.

[0003] In common data storage methods, in order to ensure fault tolerance at the AZ level, data blocks and check blocks are often stored in different AZs. When a storage block fails, since the surviving blocks are scattered in different AZs, the repair process will introduce a large amount of cross-AZ data transmission, resulting in a serious waste of bandwidth resources.

[0004] How to ensure single-AZ fault tolerance while minimizing the cross-AZ repair bandwidth in a multi-AZ architecture has become a key issue in erasure code research. Summary of the invention

[0005] The embodiments of the present application provide a data storage method and system, and a storage medium, which can effectively reduce the amount of cross-AZ data transmission required for single-block repair and greatly reduce the repair bandwidth across AZs in a multi-AZ architecture.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a data storage method, the method comprising:

[0008] Determine L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0;

[0009] Based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in a second available area.

[0010] In a second aspect, an embodiment of the present application provides a data storage system, the data storage system comprising: a determination unit, a storage unit,

[0011] The determining unit is used to determine L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0;

[0012] The storage unit is used to store the L-1 first data groups in at least L-1 first available areas respectively based on G, and store the second data group in a second available area.

[0013] In a third aspect, an embodiment of the present application provides a data storage system, which includes a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the method described in the first aspect above is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the method described in the first aspect above is implemented.

[0015] An embodiment of the present application provides a data storage method and system, and a storage medium, wherein the data storage system determines L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in the second available area. It can be seen that in the embodiments of the present application, when storing data in multiple AZs, L-1 first data groups including data blocks and local check blocks corresponding to the data blocks can be selected to be placed in at least L-1 AZs respectively, and at the same time, the second data group including global check blocks and local check blocks corresponding to the global check blocks can be placed in the same AZ. On the one hand, intra-AZ repair can be achieved by placing them in the least number of AZs as possible, thereby reducing the amount of data transmitted across AZs; on the other hand, placing the global check blocks and the corresponding local check blocks in one AZ can achieve intra-AZ repair of the global check blocks while ensuring single AZ fault tolerance, further reducing cross-AZ traffic. In other words, the data storage method proposed in the embodiments of the present application can effectively reduce the amount of cross-AZ data transmission required for single-block repair, and greatly reduce the repair bandwidth across AZs under a multi-AZ architecture. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the implementation process of the data storage method Figure 1 ;

[0017] Figure 2 Schematic diagram of erasure coding Figure 1 ;

[0018] Figure 3 Schematic diagram of erasure coding Figure 2 ;

[0019] Figure 4 Schematic diagram of the implementation process of the data storage method Figure 2 ;

[0020] Figure 5 Schematic diagram for repairing failed blocks;

[0021] Figure 6 Schematic diagram of the implementation process of the data storage method Figure 3 ;

[0022] Figure 7 Schematic diagram of the data storage implementation scheme Figure 1 ;

[0023] Figure 8 Schematic diagram of the data storage implementation scheme Figure 2 ;

[0024] Fig. 9 A schematic diagram of data repair processing;

[0025] Fig.10 Schematic diagram of the structure of the data storage system proposed in the embodiment of the present application Figure 1 ;

[0026] Fig.11 Schematic diagram of the structure of the data storage system proposed in the embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for ease of description, only the parts related to the related applications are shown in the drawings.

[0028] An Availability Zone (AZ) is a physical area in the same region with independent power and network. The network latency between instances in the same AZ is smaller.

[0029] In order to improve the reliability and availability of data, multi-AZ architecture is widely used in enterprise data centers. Multi-AZ architecture supports data disaster recovery by increasing redundancy. Common redundancy strategies are multiple copies and erasure coding (EC). Reed-Solomon codes (RS codes) are the most widely used erasure codes. RS (k, m) divides the file into k data blocks and calculates m check blocks. The data blocks and the added redundant check blocks are organized into stripes, allowing up to m data blocks / check blocks to be damaged. Erasure coding technology can provide fault tolerance comparable to multi-copy technology with lower storage overhead. Therefore, multi-AZ architectures usually deploy erasure codes for data backup.

[0030] The data blocks and check blocks in the same stripe will be stored in different AZs to ensure fault tolerance at the AZ level. When a storage block fails, the related surviving blocks need to be read for data repair. Since the surviving blocks are scattered in different AZs, the repair process will introduce a large amount of cross-AZ data transmission, resulting in a serious waste of bandwidth resources. How to reduce the cross-AZ repair bandwidth in a multi-AZ architecture is a key issue in erasure code research.

[0031] Related technologies propose the construction of Locally Repairable Codes (LRC) and a flat data placement strategy, for example, the construction of Locally Repairable Codes is called Azure-LRC. Azure-LRC (k, l, g) divides k data blocks into l groups, calculates a local check block in each group, and then uses all k data blocks to calculate g global check blocks. Compared with Reed-Solomon coding, Azure-LRC reduces the number of blocks that need to be read in the process of repairing a single data block by adding local check blocks, thereby improving the repair performance. For the placement of Azure-LRC-encoded strips in the system, a flat data placement strategy is proposed: each block in the stripe is placed in an error domain to achieve maximum fault tolerance. The error domain can be a rack, a cluster, or an availability zone (AZ). In the scenario of multi-AZ storage, in order to ensure the fault tolerance requirements of a single AZ, each block in the stripe needs to be placed in a separate AZ according to the flat placement strategy.

[0032] Considering the problem of uneven available traffic in the hierarchical data center network structure, the related technology proposes a placement strategy for Azure-LRC. Considering the stripes of Azure-LRC encoding, this strategy places each group in as few AZs as possible. At the same time, in order to meet the single AZ fault tolerance limit, the number of blocks from the same stripe in each AZ does not exceed the upper limit of the encoding fault tolerance.

[0033] Furthermore, the related art analyzes the trade-off between repair performance and expansion performance in Azure-LRC tiered storage systems, and designs a placement strategy that can achieve the best balance between repair and expansion performance.

[0034] Furthermore, the related art analyzes the fault tolerance of an LRC encoding called Optimal-LRC, and provides a placement strategy that minimizes the single-block repair cost of Optimal-LRC in a tiered storage system.

[0035] However, the flat placement strategy proposed by Azure-LRC's related technologies places each block in a stripe in a separate AZ to ensure maximum fault tolerance. This flat placement strategy does not take into account the problem that the available communication bandwidth between AZs is much smaller than the available bandwidth within an AZ, which will generate a large amount of cross-AZ traffic during the single-block repair process, resulting in a large repair time overhead.

[0036] Although other related technologies take into account the characteristic that the available bandwidth between AZs is much smaller than the available bandwidth within AZs, there are still certain defects. For example, on the one hand, for the Azure-LRC encoded stripes, the cross-AZ traffic generated during the repair of data blocks and local check blocks is reduced by centrally placing the data blocks and local check blocks in the group. However, this placement strategy does not optimize the repair of global check blocks. Therefore, according to this placement strategy, when repairing global check blocks, a large amount of data still needs to be transmitted across AZs, resulting in a large repair overhead. On the other hand, a placement strategy for optimizing the repair of data blocks and global check blocks is proposed for Azure-LRC. Compared with the placement strategy of centrally placing data blocks and local check blocks in the group, this placement strategy optimizes the repair of global check blocks, but because Azure-LRC encoding does not generate local check blocks for global check blocks. Therefore, the repair of global check blocks still requires reading a large amount of data, generating a large amount of cross-AZ data transmission. On the other hand, for the placement strategy that minimizes the single-block repair cost of Optimal-LRC, due to the relatively complex structure of Optimal-LRC, the research remains at the theoretical level and is still a certain distance away from industrial application.

[0037] It can be seen that the common data storage methods and data repair processes often generate a large amount of cross-AZ data transmission, which in turn generates a large repair overhead. How to ensure single-AZ fault tolerance while minimizing the cross-AZ repair bandwidth in a multi-AZ architecture has become a key issue in erasure coding research.

[0038] In order to solve the above problems, in an embodiment of the present application, the data storage system determines L-1 first data groups and one second data group based on the K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in the second available area. It can be seen that in the embodiments of the present application, when storing data in multiple AZs, L-1 first data groups including data blocks and local check blocks corresponding to the data blocks can be selected to be placed in at least L-1 AZs respectively, and at the same time, the second data group including global check blocks and local check blocks corresponding to the global check blocks can be placed in the same AZ. On the one hand, intra-AZ repair can be achieved by placing them in the least number of AZs as possible, thereby reducing the amount of data transmitted across AZs; on the other hand, placing the global check blocks and the corresponding local check blocks in one AZ can achieve intra-AZ repair of the global check blocks while ensuring single AZ fault tolerance, further reducing cross-AZ traffic. In other words, the data storage method proposed in the embodiments of the present application can effectively reduce the amount of cross-AZ data transmission required for single-block repair while ensuring single AZ fault tolerance, greatly reducing the repair bandwidth across AZs under a multi-AZ architecture.

[0039] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0040] An embodiment of the present application provides a data storage method, which can be applied to a data storage system, wherein the data storage system can be deployed in a multi-AZ environment, each AZ includes one or more data nodes. Accordingly, each AZ corresponds to at least one coding group (data group), and at least one coding group is stored in one or more data nodes in the AZ.

[0041] Furthermore, in the embodiments of the present application, Figure 1 Schematic diagram of the implementation process of the data storage method Figure 1 ,like Figure 1 As shown, the method for performing data storage by the data storage system may include the following steps:

[0042] Step 101, determine L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0.

[0043] In an embodiment of the present application, the data storage system may first determine L-1 first data groups and one second data group based on the K data blocks corresponding to the data to be processed. That is, the data storage system may obtain L different data groups based on the K data blocks corresponding to the data to be processed. Wherein K and L are integers greater than 1.

[0044] It should be noted that, in an embodiment of the present application, the data storage system may be deployed in a multi-AZ environment, the data storage system may include a number of data nodes, and each AZ includes one or more data nodes of the data storage system.

[0045] For example, in some embodiments, it is assumed that the data storage system includes 7 data nodes, and the data storage system is arranged in an environment with 3 AZs, wherein AZ0 includes 2 data nodes, AZ1 includes 1 data node, and AZ2 includes 3 data nodes.

[0046] Furthermore, in an embodiment of the present application, the data storage system may divide the data to be processed, thereby determining corresponding K data blocks.

[0047] It is understandable that in the embodiment of the present application, the data storage system may first divide the data to be processed into multiple parts, and define each of the data as a data block. The number of data blocks K is not limited in the embodiment of the present application.

[0048] It should be noted that, in the embodiment of the present application, the data amounts of different data blocks in the K data blocks may be equal or unequal, which is not specifically limited in the present application.

[0049] It is understandable that in the embodiments of the present application, the data to be processed can be understood as data that needs to be stored, and the specific source is not limited in the embodiments of the present application. For example, when a terminal device generates data that needs to be stored and transmits it to a server, the server stores and processes the received data as data to be processed. For another example, the data generated by each terminal device is first stored in a storage space. When the amount of stored data reaches a preset amount of data, the data stored in the storage space is stored and processed as data to be processed; correspondingly, the storage space can delete the stored and processed data, or not delete it.

[0050] Exemplarily, in some embodiments, the data to be processed may be a data request, and the data request may be further divided into K sub-requests, each sub-request being a data block.

[0051] Exemplarily, in some embodiments, the data to be processed may also be transaction data to be stored, and the transaction data may be processed by sharding to obtain K data blocks.

[0052] Exemplarily, in some embodiments, the data to be processed may also be user registration data, and the user registration data may be processed by slicing to obtain K data blocks.

[0053] Furthermore, in an embodiment of the present application, the first data group determined based on the K data blocks corresponding to the data to be processed may include K / (L-1) data blocks, and the first data group may also include a first local check block corresponding to the K / (L-1) data blocks.

[0054] That is to say, in the embodiment of the present application, the K data blocks corresponding to the data to be processed can be divided into L-1 data groups, wherein each data group includes K / (L-1) data blocks. At the same time, a local check block corresponding to the K / (L-1) data blocks in the data group can be determined in each data group. It can be seen that each first data group can include K / (L-1)+1 blocks.

[0055] It can be understood that in an embodiment of the present application, when L-1 first data groups are determined based on the K data blocks corresponding to the data to be processed, for each K / (L-1) data block among the K data blocks, it is possible to first encode according to the K / (L-1) encoding parameters corresponding to the K / (L-1) data blocks to determine the first local check block; and then generate the first data group based on the K / (L-1) data blocks and the first local check block.

[0056] It should be noted that in an embodiment of the present application, when encoding according to K / (L-1) encoding parameters corresponding to K / (L-1) data blocks, multiple encoding methods can be used to determine the corresponding first local check block, and the present application does not make specific limitations.

[0057] Exemplarily, in some embodiments, the method for calculating the first local check block can be an erasure code encoding algorithm. That is, the K / (L-1) encoding parameters corresponding to each K / (L-1) data block can be used to encode through the erasure code encoding algorithm to obtain the corresponding local check block. Among them, the erasure code (Erasure Code) is a coding fault-tolerant technology. Its basic principle is to shard the stored data, and generate k+m copies of data through a certain verification calculation method for the k copies of original data obtained by sharding, and can restore the original data through any k copies of the k+m copies. In this way, even if part of the data is lost, the distributed system can still restore the original data.

[0058] It should be noted that, in the embodiment of the present application, among the L-1 first data groups obtained by division, each first data group includes at least K / (L-1) data blocks, K / (L-1) encoding parameters corresponding to the K / (L-1) data blocks, and a first local check block. The K / (L-1) encoding parameters can be used to generate a local check block and a global check block in combination with the corresponding K / (L-1) data blocks.

[0059] Further, in an embodiment of the present application, the second data group determined based on the K data blocks corresponding to the data to be processed may include G global check blocks, and the second data group may also include second local check blocks corresponding to the G global check blocks, where G is an integer greater than 0.

[0060] That is to say, in the embodiment of the present application, one data group can be selected to store the global check block corresponding to K data blocks. At the same time, a local check corresponding to the global check block can be calculated based on the global check block. It can be seen that the second data group can include G+1 blocks.

[0061] It can be understood that in an embodiment of the present application, when determining a second data group based on K data blocks corresponding to the data to be processed, it is possible to first encode according to K encoding parameters corresponding to the K data blocks to determine G global check blocks; then determine the second local check blocks based on the G global check blocks; finally, the second data group can be generated based on the G global check blocks and the second local check blocks.

[0062] It is understandable that in the embodiment of the present application, the data storage system may first perform encoding based on the K encoding parameters corresponding to the K data blocks, thereby determining any number of global check blocks, for example, G global check blocks may be determined. Then, the data storage system may also perform calculations on the corresponding local check blocks based on the G global check blocks, thereby determining the corresponding second local check blocks. Finally, the data storage system may choose to form a second data group with the G global check blocks and the corresponding second local check blocks.

[0063] It should be noted that in the embodiments of the present application, when encoding is performed according to the K encoding parameters corresponding to the K data blocks, a variety of encoding methods can be used to determine the corresponding G global check blocks, and the present application does not make specific limitations.

[0064] Exemplarily, in some embodiments, the method of calculating the global check block may be an erasure coding algorithm, that is, K coding parameters corresponding to K data blocks may be used to encode through the erasure coding algorithm to obtain the corresponding global check block.

[0065] It should be noted that, in an embodiment of the present application, a second data group obtained by division includes at least G global check blocks, K coding parameters corresponding to the K data blocks, and a second local check block. The K coding parameters can be used to generate a local check block and a global check block in combination with the corresponding K data blocks.

[0066] It is understandable that in the embodiments of the present application, the specific encoding method of the global check block is not limited in this embodiment, and can be specifically set according to the actual application scenario. In one example, the global check block can be determined according to the sum of the products of K data blocks and corresponding K encoding parameters.

[0067] Exemplarily, in some embodiments, for the generation of a global check block, taking the generation of a global check block as an example, encoding can be performed according to K data blocks and K encoding parameters to generate a global check block. In the case where at least two global check blocks need to be generated, the K data blocks may include at least two parameter groups, the parameter groups include K encoding parameters corresponding to the K data blocks respectively, and the encoding parameters in different parameter groups are different, so that at least two global check blocks can be generated by encoding according to the K data blocks and at least two different parameter groups respectively.

[0068] It can be understood that the common Azure-LRC (k, l, g) means that k data blocks are divided into l groups, each group has k / l data blocks, a local check block is generated using the data blocks in the group, and a global check block is generated using all the data blocks. Taking the local check coding method LRC erasure code as an example, the core idea of ​​LRC erasure code is to divide the check block (parity block) into a global check block (global parity) and a local check block (local reconstruction parity), which can be calculated in groups during fault recovery. LRC coding is generally expressed as LRC (k, l, g), where k represents the total number of data blocks, z represents the number of local check blocks, and g represents the number of global check blocks.

[0069] Exemplarily, in some embodiments, Figure 2 Schematic diagram of erasure coding Figure 1 ,like Figure 2 As shown, taking LRC (12, 3, 6) erasure code as an example, it means there are 12 data blocks D 0 -D 11 , 3 local check blocks L 0 -L 2 , and 6 global check blocks G 0 -G 5 ; Among them, the global check block G 0 -G 5 Can be composed of data block D 0 -D 11 Encode and generate, and use data block D 0 -D 3 and the global check block G 0 -G 1 XOR generates a local check block L 0 , using data block D 4 -D 7 and the global check block G 2 -G 3 XOR generates a local check block L 1 , using data block D 8 -D 11 and the global check block G 4 -G 5 XOR generates a local check block L 2 .

[0070] For example, in some embodiments, Figure 2 As shown, data block D 0 -D 3 , global check block G 0 -G 1 , local check block L 0 The first coding group, data block D 4 -D 7 , global check block G 2 -G 3 , local check block L 1 The second coding group, data block D 8 -D 11 , global check block G 4 -G 5 , local check block L 2 Form the third coding group.

[0071] It should be noted that compared with common data storage methods, the data storage method proposed in the embodiment of the present application can be understood as the Azure-LRC+1 method based on Azure-LRC, which can divide all global check blocks into a group, and then generate an additional local check block.

[0072] It can be understood that in the embodiments of the present application, based on the data storage method of Azure-LRC+1, a local check block is generated for all global check blocks, that is, the second local check block corresponding to the global check block. Therefore, when the global check block is damaged, you can choose to use the corresponding second local check block for repair.

[0073] Exemplarily, in some embodiments, Figure 3 Schematic diagram of erasure coding Figure 2 ,like Figure 3 As shown, taking LRC (12, 3, 6) erasure code as an example, it means there are 12 data blocks D 0 -D 11 , 3 local check blocks L 0 -L 2 , and 6 global check blocks G 0 -G 5 ; Among them, the global check block G 0 -G 5 Can be composed of data block D 0 -D 11 Encode and generate, and calculate data block D 0 -D 5 The corresponding local check block L 0 , calculate the data block D 6 -D 11 The corresponding local check block L 1 , and the global check block G 0 -G 5 The corresponding local check block L 2 .

[0074] For example, in some embodiments, Figure 3 As shown, data block D 0 -D 5 and local check block L 0 (first local check block) constitutes the first coding group, that is, the first data group, data block D 6 -D 11 and local check block L 1 (first local check block) constitutes the second coding group, namely the first data group, and the global check block G 0 -G 5 and local check block L 2(The second local check block) constitutes the third coding group, namely the second data group.

[0075] It can be seen that in the embodiments of the present application, compared with the data storage method of the scene, the data storage method proposed in the present application chooses to generate a corresponding local check block based on the global check block, and divides the global check block and the local check block corresponding to the global check block into the same data group, so that when the global check block is damaged, it can be repaired using the corresponding second local check block.

[0076] Step 102: based on G, store L-1 first data groups in at least L-1 first available areas respectively, and store the second data group in the second available area.

[0077] In an embodiment of the present application, after determining L-1 first data groups and one second data group based on the K data blocks corresponding to the data to be processed, the L-1 first data groups can be further stored in at least L-1 first available areas based on G, and the second data group can be stored in the second available area.

[0078] It should be noted that in an embodiment of the present application, after completing the division and encoding processing of the K data blocks corresponding to the data to be processed, when storing the determined L-1 first data groups and one second data group, the L-1 first data groups can be selected to be stored in at least L-1 first available areas respectively, and the second data group can also be stored in the second available area.

[0079] It should be noted that, in the embodiment of the present application, the data storage system may choose to store the second data group in an available area, namely, the second available area.

[0080] That is, in an embodiment of the present application, the data storage system may choose to store the G global check blocks and the corresponding second local check blocks in the same available area.

[0081] It can be understood that in the embodiments of the present application, since the corresponding second local check block can be used to repair the global check block when it is damaged, the data storage system chooses to place the global check block and the corresponding local check block in one availability zone (AZ), which can achieve in-AZ repair of the global check block while ensuring single AZ fault tolerance, that is, the repair of the global check block can be completed within the AZ, thereby reducing cross-AZ traffic.

[0082] Further, in an embodiment of the present application, M=K / (L-1) is set, where M is an integer greater than 0. Accordingly, when L-1 first data groups are stored in at least L-1 first available areas respectively based on G, it is possible to select to store the L-1 first data groups in at least L-1 first available areas respectively based on M and G.

[0083] It should be noted that in the embodiment of the present application, M can be understood as the number of data blocks in the first data group, that is, M is the data block included in each first data group, and accordingly, M=K / (L-1). G is the number of global check blocks in the second data group.

[0084] That is to say, in the embodiment of the present application, storing the first data group according to M and G can be understood as storing the first data group according to the number of data blocks in the first data group and the number of global check blocks in the second data group.

[0085] Further, in an embodiment of the present application, when L-1 first data groups are stored in at least L-1 first available areas respectively according to M and G, for any first data group, when G is greater than or equal to M, the first data group can be selected to be stored in one first available area.

[0086] That is to say, in an embodiment of the present application, for a first data group, if the number of data blocks K / (L-1) in the first data group is not greater than the number of global check blocks G in the second data group, that is, when G is greater than or equal to M, then the K / (L-1) data blocks in the first data group and the corresponding first local check blocks can be selected to be stored in the same available area, that is, stored in the first available area.

[0087] Furthermore, in an embodiment of the present application, when L-1 first data groups are stored in at least L-1 first available areas respectively according to M and G, for any first data group, when G is less than M, it is possible to choose to first divide the first data group to determine a plurality of sub-data groups corresponding to the first data group; and then store the plurality of sub-data groups in a plurality of first available areas respectively.

[0088] That is to say, in an embodiment of the present application, for a first data group, if the number of data blocks K / (L-1) in the first data group is greater than the number of global check blocks G in the second data group, that is, when G is less than M, the K / (L-1) data blocks in the first data group and the corresponding first local check blocks cannot be directly stored in the same available area, but all the blocks in the first data group need to be divided and processed first to obtain corresponding multiple sub-data groups, and then different sub-data groups are stored in different first available areas respectively.

[0089] That is to say, in the embodiments of the present application, based on the data storage method of Azure-LRC+1, at most G+1 blocks from the same group can fail simultaneously, where G is the number of global parity blocks. Suppose the number of data blocks in one of the data groups is M, and the number of the first local parity blocks is 1, that is, the number of blocks in this data group is M+1. Then, it is possible to determine whether to store the M+1 blocks in this group in the same available area by comparing G+1 with M+1. Correspondingly, it is possible to select whether to store the M+1 blocks in this group in the same available area by comparing G with M.

[0090] It can be seen from this that in the embodiments of the present application, when storing the first data group, for any first data group, in the case where the number of blocks in the first data group is greater than or equal to the number of blocks in the second data group, the first data group is stored in a first available area.

[0091] Correspondingly, in the embodiments of the present application, when storing the first data group, for any first data group, in the case where the number of blocks in the first data group is less than the number of blocks in the second data group, the first data group is stored in multiple first available areas.

[0092] Exemplarily, in some embodiments, for the stripe encoded by Azure-LRC+1(k, l, g), at most g+1 blocks from the same group can fail simultaneously. Correspondingly, suppose the number of data blocks in one of the data groups is b+1. If g+1≥b+1, that is, g≥b, then the blocks in this data group can be placed in the same AZ, otherwise, the blocks in this data group are placed in different AZs.

[0093] Furthermore, in the embodiments of the present application, if the number of data blocks in the first data group is greater than or equal to the number of global parity blocks in the second data group, or rather, if the number of blocks in the first data group is greater than or equal to the number of blocks in the second data group, then when performing a partitioning process on the first data group to determine multiple sub-data groups corresponding to the first data group, it is possible to select to partition the K / (L-1) data blocks and the first local parity blocks in the first data group according to the number of global parity blocks G, so as to obtain corresponding multiple sub-data groups.

[0094] Exemplarily, in some embodiments, for the stripe encoded by Azure-LRC+1(k, l, g), at most g+1 blocks from the same group can fail simultaneously. Correspondingly, suppose the number of data blocks in one of the data groups is b+1. If g < b, then this data group can be divided into small sets of size g+1, and each small set is placed in a separate AZ.

[0095] Exemplarily, in some embodiments, for the second data group where the global parity block is located, the number of global parity blocks is g, plus one second local parity block, and the total number of blocks in the group is g + 1. Therefore, the second data group where the global parity block is located can be placed in the same AZ, and even if all the nodes in this AZ fail, data loss will not occur.

[0096] Exemplarily, in some embodiments, for the stripe encoded by Azure-LRC+1(K, L, G), each first data group includes M = K / (L - 1) data blocks and a corresponding first local parity block, and the second data group includes G global parity blocks and a corresponding second local parity block. When storing the first data group, if G ≥ M, then the first data group where the M data blocks are located can be stored in the first available zone, that is, both the data blocks and the first local parity block included in the first data group are placed in the same AZ; if G < M, then the first data group where the M data blocks are located can be first divided, divided into sub-data groups of size G + 1, and then each sub-data group is stored in the first available zone, that is, the blocks included in different sub-data groups are placed in different AZs respectively. When storing the second data group, it can be directly stored in the second available zone, that is, the global parity blocks and their corresponding second local parity blocks included in the second data group are placed in the same AZ.

[0097] Further, in the embodiments of the present application, Figure 4 Schematic diagram of the implementation process of the data storage method Figure 2 , as Figure 4 shown, after completing the storage of the first data group and the second data group, the method for the data storage system to store data may include the following steps:

[0098] Step 103, perform repair processing on the failed blocks in the first data group and / or the second data group according to a preset repair strategy; wherein, the preset repair strategy is used to perform repair processing according to some candidate blocks, and the candidate blocks are other blocks except the failed blocks.

[0099] After storing the L - 1 first data groups in at least L - 1 first available zones based on G and storing the second data group in the second available zone, for the failed blocks in the first data group and / or the second data group, it can be selected to perform repair processing on the failed block according to the preset repair strategy.

[0100] It should be noted that, in the embodiments of the present application, the preset repair strategy can be used to perform repair processing according to some candidate blocks, where the candidate blocks are other blocks except the failed blocks, that is, based on the preset repair strategy, a part of the surviving candidate blocks except the failed block can be selected to repair the failed block.

[0101] It is understandable that in the embodiments of the present application, the preset repair strategy can be understood as a repair method of partial decoding. That is, if a data block or a check block is damaged, a partial decoding method can be used to perform repair processing.

[0102] Furthermore, in an embodiment of the present application, after a damaged data block and / or check block is determined, that is, after a failed block is determined, a candidate block in the same data group as the failed block can be selected as a first candidate block; at the same time, a candidate block in a different data group from the failed block can also be determined as a second candidate block.

[0103] Exemplarily, in some embodiments, it is assumed that the first data group and the second data group include 12 data blocks D 0 -D 11 , 3 local check blocks L 0 -L 2 , and 6 global check blocks G 0 -G 5 Among them, a first data group includes data block D 0 -D 5 and local check block L 0 (first local check block), another first data group includes data block D 6 -D 11 and local check block L 1 (first local check block), the second data group includes a global check block G 0 -G 5 and local check block L 2 If data block D is determined 11 If the block is a failure block, then the 11 In the same data set, D 6 -D 10 and L 1 Determine as the first candidate block, and set D 0 -D 5 , L 0 , G 0 -G 5 and L 2 Determined as the second candidate block.

[0104] Furthermore, in an embodiment of the present application, when repairing the failed blocks in the first data group and / or the second data group according to a preset repair strategy, it is possible to select to generate a third candidate block based on part or all of the candidate blocks in the second candidate block; and then the failed block can be repaired based on the third candidate block and the first candidate block.

[0105] It should be noted that in an embodiment of the present application, the third candidate block can be a block calculated based on multiple candidate blocks, that is, a third candidate block can be generated using multiple second candidate blocks, so that in the process of repairing a failed block, multiple second candidate blocks transmitted across AZs can be replaced with a third candidate block transmitted across AZs, thereby reducing the data traffic across AZs.

[0106] It can be understood that in the embodiments of the present application, the number of third candidate blocks can be 1 or greater than 1, and the present application does not make a specific limitation. However, in order to effectively reduce the data traffic across AZs, it is necessary to ensure that the number of second candidate blocks used to generate the third candidate blocks is greater than the number of third candidate blocks.

[0107] Exemplarily, in some embodiments, Figure 5 A schematic diagram of repairing a failed block, such as Figure 5 As shown, in a data storage system consisting of two AZs, AZ 1 and A-Z 2 The storage nodes in AZ communicate through switches. The stripe consists of five storage blocks: A, B, C, D, and E. A and B are distributed in AZ. 1 ; C, D, E are distributed in AZ 2 Among them, A, B, C, D, E can be data blocks and / or check blocks. Assume that block A fails, that is, A is a failed block, and the remaining four blocks B, C, D, and E need to be repaired, that is, A = B + C + D + E. One way to achieve this is to move C, D, and E from AZ 2 Cross-AZ transfer to AZ 1 , A.Z. 1 Collect the surviving blocks B and AZ in AZ 2 C, D, E are used to reconstruct the failed block A. It can be seen that the common repair method transmits three blocks C, D, and E across AZ. In contrast, the partial decoding repair strategy proposed in the embodiment of the present application is adopted, B is regarded as the first candidate block, and C, D, and E are regarded as the second candidate blocks. Before transmitting multiple second candidate blocks C, D, and E across AZ, a partial block F = C + D + E can be calculated, where F is regarded as the third candidate block. Then, in the subsequent repair process, only the partial block F is transmitted across AZ, that is, only the third candidate block F is transmitted. AZ 1 Using the AZ 2 Part of blocks F and AZ 1 The surviving block B is used to repair A, that is, A = B + F. In this way, the number of blocks transmitted across AZs during the repair process is reduced from 3 to 1, reducing the data traffic across AZs.

[0108] Furthermore, in an embodiment of the present application, the erasure code algorithm can tolerate a maximum amount of damaged data that is the same as the amount of data in the check block, that is, at most a number of data blocks equal to the number of check blocks can be damaged. In the event of such damage, data recovery can be performed according to the decoding method corresponding to the erasure code algorithm.

[0109] It can be understood that in the embodiments of the present application, verification data is calculated for each piece of data (a data block in the first data group or a global verification block in the second data group) to obtain verification data for each piece of data (a first local verification block or a second local verification block). According to the verification data obtained by this calculation method, when there is data damage in the global verification block in the first data group and the corresponding first local verification block, or when there is data damage in the data block in the second data group and the corresponding second local verification block, if the amount of damaged data is less than the amount of verification data, data recovery can be performed through the remaining undamaged data to calculate and obtain the damaged data. For example, a data group has N data blocks, and Q verification blocks are calculated based on the N data blocks. When less than or equal to Q data blocks are damaged among the N+Q data blocks, the damaged data blocks can be restored based on the remaining undamaged data blocks. Both N and Q are positive integers.

[0110] It should be noted that, in the embodiments of the present application, the specific repair method for the failed block can be determined according to the generation method when generating the verification data, that is, if the damaged data is verification data (first local verification block or second local verification block), the verification data (first local verification block) can be regenerated according to the data block in the first data group, or the verification data (second local verification block) can be regenerated according to the global verification block in the second data group; if the damaged data includes part or all of the data group, and is less than or equal to the data amount of the verification data, the data group and the undamaged data in the verification data can be decoded accordingly according to the encoding method of the verification data generated by the data group to achieve data recovery. If the data group calculates the verification data by the erasure code encoding method, the undamaged data in the data set with the same data amount as the sub-storage data can be decoded according to the decoding method of the erasure code to recover the sub-storage data.

[0111] In summary, the data storage method proposed in steps 101 to 103 adopts an erasure code data placement mechanism for implementing intra-AZ repair in multi-AZ storage. On the one hand, different placement strategies are designed according to stripes encoded with different parameters. Accordingly, by designing a placement strategy, the cross-AZ data transmission in the single block repair process can be minimized, thereby improving the repair efficiency. For parameters that meet the conditions, a single failed block can be repaired within the AZ without generating cross-AZ repair traffic; because in a multi-AZ storage system, the bottleneck of repair performance lies in the tight bandwidth across AZs, therefore, by reducing the cross-AZ repair traffic, the repair speed can be greatly improved. At the same time, for situations where cross-AZ repair may be required, the partial decoding method can be selected to repair a single failed block, further reducing the cross-AZ repair traffic.

[0112] It can be understood that the data storage method proposed in the embodiment of the present application can place blocks of the same group in the Azure-LRC+1 stripe in the minimum number of AZs under the constraint of tolerating single AZ failures, targeting the data placement problem of erasure codes in multi-AZ storage. Under certain parameters (such as Azure-LRC+1 (12, 3, 6)), blocks of the same group can be placed in the same AZ to achieve intra-AZ repair. Compared with the common flat placement strategy and random placement strategy, the placement strategy proposed in the embodiment of the present application greatly reduces the number of AZs spanned by each Azure-LRC+1 group while taking into account the tolerance of AZ failures, and therefore can effectively reduce the amount of cross-AZ network transmission required for single-block repair.

[0113] It should be noted that the data storage method proposed in the embodiment of the present application, in order to solve the single block repair problem of erasure codes in multi-AZ storage, chooses to combine local repair within AZ + global repair across AZ to reconstruct the failed block. When reconstructing a failed block requires data reading across multiple AZs, firstly, local repair is performed within each AZ to construct a partial block, and then the partial blocks are transmitted across AZs and the partial blocks are combined to reconstruct the failed block. Compared with the method of directly reading the original data block across AZs, the repair strategy proposed in the embodiment of the present application can greatly reduce the cross-AZ network transmission amount of single-block repair through local repair within AZ + global repair across AZ technology, thereby improving the repair efficiency.

[0114] It is understandable that when the data storage method proposed in the embodiment of the present application is implemented in an actual system, the system needs to perceive the data layout and coordinate various parts to complete the repair task. In the actual system, the coordinator takes on this function.

[0115] The Coordinator needs to have the ability to perceive the network topology. When the Coordinator node starts, it reads the AZ configuration file and generates network topology information, and saves the network topology information in local memory throughout the life cycle of the Coordinator node. The proxy node needs to perceive the network topology of the AZ when storing data, reading data, deleting data, repairing data, and so on. Before performing specific operations, the Coordinator node sends the corresponding network topology information to the Proxy node, and the Proxy node executes the task after obtaining the network topology information. However, the Proxy node will not hold the network topology information for a long time, and will discard it after use, thereby ensuring that the proxy node will not use the old network topology and maintain global consistency.

[0116] When a single failed block needs to be repaired, the Coordinator coordinates each Proxy to complete the repair work. If the failed data block and the surviving block that needs to be read are distributed in the same AZ, the Coordinator sends a repair command to the AZ where the failed data block is located, and the Proxy in the AZ completes the specific repair steps. If the failed data block and the surviving block are distributed in different AZs, the Coordinator generates a repair plan, sends a repair command to the AZ where the failed block is located, and sends auxiliary repair commands such as partial decoding to the AZ where the surviving block is located. The Proxy in the AZ where the surviving block is located receives the auxiliary repair command, reads the surviving block, calculates the "partial block", and passes the "partial block" to the AZ where the failed block is located. The Proxy in the AZ where the failed block is located collects the surviving blocks and "partial blocks", reconstructs the failed block and places it in a new node in the AZ. After the repair is completed, the Coordinator updates the metadata.

[0117] It can be seen that the data storage method proposed in the embodiment of the present application can achieve the goal of minimizing cross-AZ data transmission during the repair process and reduce the impact of the repair process on the repair performance of the multi-AZ storage system. At the same time, the data storage method has good versatility and is applicable to all encoding parameters.

[0118] An embodiment of the present application provides a data storage method, in which a data storage system determines L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in the second available area. It can be seen that in the embodiments of the present application, when storing data in multiple AZs, L-1 first data groups including data blocks and local check blocks corresponding to the data blocks can be selected to be placed in at least L-1 AZs respectively, and at the same time, the second data group including global check blocks and local check blocks corresponding to the global check blocks can be placed in the same AZ. On the one hand, intra-AZ repair can be achieved by placing them in the least number of AZs as possible, thereby reducing the amount of data transmitted across AZs; on the other hand, placing the global check blocks and the corresponding local check blocks in one AZ can achieve intra-AZ repair of the global check blocks while ensuring single AZ fault tolerance, further reducing cross-AZ traffic. In other words, the data storage method proposed in the embodiments of the present application can effectively reduce the amount of cross-AZ data transmission required for single-block repair, and greatly reduce the repair bandwidth across AZs under a multi-AZ architecture.

[0119] Based on the above embodiments, the embodiments of the present application provide a data storage method, which can be applied to a data storage system, wherein the data storage system can be arranged in a multi-AZ environment, and each AZ includes one or more data nodes. Accordingly, each AZ corresponds to at least one coding group (data group), and at least one coding group is stored in one or more data nodes in the AZ.

[0120] It should be noted that the data storage method proposed in the embodiment of the present application can be understood as an erasure code data placement mechanism that implements intra-AZ repair in multi-AZ storage.

[0121] Furthermore, in the embodiments of the present application, Figure 6 Schematic diagram of the implementation process of the data storage method Figure 3 ,like Figure 6 As shown, the method for performing data storage by the data storage system may include the following steps:

[0122] Step 201: Encode the strip using Azure-LRC+1.

[0123] In an embodiment of the present application, the data storage system may first determine L-1 first data groups and one second data group based on the K data blocks corresponding to the data to be processed. That is, the data storage system may obtain L different data groups based on the K data blocks corresponding to the data to be processed. Wherein K and L are integers greater than 1.

[0124] Furthermore, in an embodiment of the present application, the first data group determined based on the K data blocks corresponding to the data to be processed may include K / (L-1) data blocks, and the first data group may also include a first local check block corresponding to the K / (L-1) data blocks.

[0125] Further, in an embodiment of the present application, the second data group determined based on the K data blocks corresponding to the data to be processed may include G global check blocks, and the second data group may also include second local check blocks corresponding to the G global check blocks, where G is an integer greater than 0.

[0126] The common Azure-LRC (k, l, g) means dividing k data blocks into l groups, each group has k / l data blocks, using the data blocks in the group to generate a local parity block, and using all the data blocks to generate a global parity block. Taking the local parity coding method LRC erasure code as an example, the core idea of ​​LRC erasure code is to divide the parity block into a global parity block and a local reconstruction parity block, which can be calculated in groups during fault recovery. LRC coding is generally expressed as LRC (k, l, g), where k represents the total number of data blocks, z represents the number of local parity blocks, and g represents the number of global parity blocks.

[0127] Compared with common data storage methods, the data storage method proposed in the embodiment of the present application can be understood as the Azure-LRC+1 method based on Azure-LRC, which can divide all global check blocks into a group, and then generate an additional local check block.

[0128] Based on Azure-LRC, Azure-LRC+1 can also divide all global check blocks into a data group, and then generate an additional local check block, namely the second local check block, to reduce the repair traffic overhead when a single global check block is damaged.

[0129] In multi-AZ storage, the repair of Azure-LRC global check blocks still requires the transmission of a large amount of data across AZs; while Azure-LRC+1 can repair global check blocks using local check blocks, so that the repair of global check blocks can be completed within the AZ. The repair of a single global check block in Azure-LRC requires reading k surviving blocks, but in order to meet the single AZ fault tolerance limit, all blocks required for the repair of the global check block cannot be placed in the same AZ. Therefore, the repair of the global check block inevitably requires the transmission of a large amount of data across AZs.

[0130] Azure-LRC+1 adds a local check block to the global check block. Therefore, when placing stripes, all global check blocks and corresponding local check blocks can be placed in one AZ, ensuring single-AZ fault tolerance while achieving intra-AZ repair of global check blocks.

[0131] In order to reduce the single-block repair cost of data blocks and check blocks at the same time, the data storage method proposed in the embodiment of the present application chooses to use Azure-LRC+1 encoding stripes. In Azure-LRC+1(k, l, g), k represents the number of data blocks, l represents the number of local check blocks, and g represents the number of global check blocks. Azure-LRC+1(k, l, g) divides k data blocks into l-1 first data groups, each first data group has k / (l-1) data blocks, and calculates a local check block in each first data group, that is, each first data group also includes a first local check block.

[0132] Different from Azure-LRC, Azure-LRC+1 also places all g global check blocks in one group and adds a local check block, namely the second local check block. All global check blocks and the second local check block together constitute the second data group.

[0133] Step 202: Generate a data placement strategy for implementing intra-AZ repair.

[0134] In an embodiment of the present application, after determining L-1 first data groups and one second data group based on the K data blocks corresponding to the data to be processed, the L-1 first data groups can be further stored in at least L-1 first available areas based on G, and the second data group can be stored in the second available area.

[0135] To ensure that single-AZ failures are repairable and the cross-AZ traffic generated by repairing a single block is minimized, the blocks in a group should be placed in one AZ as much as possible. When all the blocks in a group are placed in the same AZ, the cross-AZ traffic for single-block repair is minimized and is 0. According to the analysis of the fault tolerance of Azure-LRC, for the stripes encoded by Azure-LRC+1(k, l, g), up to g+1 blocks from the same group can fail simultaneously. Let the number of blocks in a certain first data group be b+1. If g+1≥b+1, that is, g≥b, the blocks in the first data group can be placed in the same AZ. If g<b, the first data group is divided into small sets of size g+1, and each small set is placed in a separate AZ.

[0136] For the second data group where the global parity block is located, the number of global parity blocks is g, plus one second local parity block, and the total number of blocks in the group is g+1; then the second data group where the global parity block is located is placed in the same AZ, and even if all the nodes in this AZ fail, data loss will not occur.

[0137] For the stripes encoded by Azure-LRC+1(k, l, g), each group has b = k / (l - 1) data blocks and one local parity block, and the specific placement strategy can be expressed as follows:

[0138] When g≥b:

[0139] 1. Place each group where the data blocks are located (including the data blocks and local parity blocks in this group) in a separate AZ.

[0140] 2. Place the global parity block and its local parity block in a separate AZ.

[0141] When g<b:

[0142] 1. For a group with b data blocks and 1 local parity block, divide it into small sets of size g+1, and place the blocks in each small set in a separate AZ.

[0143] 2. Place the global parity block and its local parity block in a separate AZ.

[0144] That is to say, in the embodiments of the present application, when the Azure-LRC+1(k, l, g) encoding parameters satisfy g≥b, that is, when the number of global parity blocks is more than the number of data blocks in each group, the data blocks and local parity blocks in a group can be placed in one AZ to achieve the goal of repairing a single failed block within the AZ and minimizing the cross-AZ repair traffic. When g<b, the placement strategy generated by this algorithm can ensure that the cross-AZ repair traffic generated during the repair of a single failed block is minimized.

[0145] Step 203: Repair the invalid data using a partial decoding method.

[0146] After L-1 first data groups are stored in at least L-1 first available areas respectively based on G, and the second data group is stored in the second available area, if a data block or a check block is damaged, for the failed block in the first data group and / or the second data group, you can choose to repair the failed block according to the preset repair strategy.

[0147] The preset repair strategy can be understood as a partial decoding repair method. That is, if a data block or a check block is damaged, a partial decoding method can be used to perform repair processing.

[0148] When repairing a single failed block, the partial decoding method is used to repair the failed data, thereby reducing the cross-AZ repair traffic.

[0149] Further, in an embodiment of the present application, after determining a damaged data block and / or a check block, that is, determining a failed block, a candidate block in the same data group as the failed block can be selected as a first candidate block; and a candidate block in a different data group from the failed block can also be determined as a second candidate block. When repairing the failed blocks in the first data group and / or the second data group according to a preset repair strategy, a third candidate block can be generated based on part or all of the candidate blocks in the second candidate block; and then the failed block can be repaired based on the third candidate block and the first candidate block.

[0150] It should be noted that in an embodiment of the present application, the third candidate block can be a block calculated based on multiple candidate blocks, that is, a third candidate block can be generated using multiple second candidate blocks, so that in the process of repairing a failed block, multiple second candidate blocks transmitted across AZs can be replaced with a third candidate block transmitted across AZs, thereby reducing the data traffic across AZs.

[0151] For example, in some embodiments, Figure 5 As shown, in a data storage system consisting of two AZs, AZ 1 and A-Z 2 The storage nodes in AZ communicate through switches. The stripe consists of five storage blocks: A, B, C, D, and E. A and B are distributed in AZ. 1 ; C, D, E are distributed in AZ 2Among them, A, B, C, D, E can be data blocks and / or check blocks. Assume that block A fails, that is, A is a failed block, and the remaining four blocks B, C, D, and E need to be repaired, that is, A = B + C + D + E. One way to achieve this is to move C, D, and E from AZ 2 Cross-AZ transfer to AZ 1 , A.Z. 1 Collect the surviving blocks B and AZ in AZ 2 C, D, E are used to reconstruct the failed block A. It can be seen that the common repair method transmits three blocks C, D, and E across AZ. In contrast, the partial decoding repair strategy proposed in the embodiment of the present application is adopted, B is regarded as the first candidate block, and C, D, and E are regarded as the second candidate blocks. Before transmitting multiple second candidate blocks C, D, and E across AZ, a partial block F = C + D + E can be calculated, where F is regarded as the third candidate block. Then, in the subsequent repair process, only the partial block F is transmitted across AZ, that is, only the third candidate block F is transmitted. AZ 1 Using the AZ 2 Part of blocks F and AZ 1 The surviving block B is used to repair A, that is, A = B + F. In this way, the number of blocks transmitted across AZs during the repair process is reduced from 3 to 1, reducing the data traffic across AZs.

[0152] That is to say, for stripes encoded by Azure-LRC+1, the repair of all individual blocks can be completed within the group. When the repair process starts, the group where the failed block is located is located, and then all surviving blocks in the group are read and repaired through the decoding matrix; if cross-AZ data transmission is involved, the preset repair strategy (partial decoding) is used for repair processing. Specifically, before transmission, the block to be transmitted is calculated as a partial block, and cross-AZ data transmission is reduced by transmitting partial blocks. When the reconstruction is completed, a new node is selected in the AZ to store the reconstructed block.

[0153] When the encoding parameter satisfies g≥b, using the placement strategy described in step 202, all single failed block repairs can be completed within the AZ, minimizing the repair traffic across AZs.

[0154] In summary, in an embodiment of the present application, an erasure code data placement mechanism is adopted to implement intra-AZ repair in multi-AZ storage. On the one hand, different placement strategies are designed according to stripes encoded with different parameters. Accordingly, by designing a placement strategy, the cross-AZ data transmission in the single block repair process can be minimized, thereby improving the repair efficiency. For parameters that meet the conditions, a single failed block can be repaired within the AZ without generating cross-AZ repair traffic; because in a multi-AZ storage system, the bottleneck of repair performance lies in the tight bandwidth across AZs, therefore, by reducing the cross-AZ repair traffic, the repair speed can be greatly improved. At the same time, for situations where cross-AZ repair may be required, the partial decoding method can be selected to repair a single failed block, further reducing the cross-AZ repair traffic.

[0155] It can be understood that the data storage method proposed in the embodiment of the present application can place blocks of the same group in the Azure-LRC+1 stripe in the minimum number of AZs under the constraint of tolerating single AZ failures, targeting the data placement problem of erasure codes in multi-AZ storage. Under certain parameters (such as Azure-LRC+1 (12, 3, 6)), blocks of the same group can be placed in the same AZ to achieve intra-AZ repair. Compared with the common flat placement strategy and random placement strategy, the placement strategy proposed in the embodiment of the present application greatly reduces the number of AZs spanned by each Azure-LRC+1 group while taking into account the tolerance of AZ failures, and therefore can effectively reduce the amount of cross-AZ network transmission required for single-block repair.

[0156] It should be noted that the data storage method proposed in the embodiment of the present application, in order to solve the single block repair problem of erasure codes in multi-AZ storage, chooses to combine local repair within AZ + global repair across AZ to reconstruct the failed block. When reconstructing a failed block requires data reading across multiple AZs, firstly, local repair is performed within each AZ to construct a partial block, and then the partial blocks are transmitted across AZs and the partial blocks are combined to reconstruct the failed block. Compared with the method of directly reading the original data block across AZs, the repair strategy proposed in the embodiment of the present application can greatly reduce the cross-AZ network transmission amount of single-block repair through local repair within AZ + global repair across AZ technology, thereby improving the repair efficiency.

[0157] It can be seen that the data storage method proposed in the embodiment of the present application can achieve the goal of minimizing cross-AZ data transmission during the repair process and reduce the impact of the repair process on the repair performance of the multi-AZ storage system. At the same time, the data storage method has good versatility and is applicable to all encoding parameters.

[0158] An embodiment of the present application provides a data storage method, in which a data storage system determines L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in the second available area. It can be seen that in the embodiments of the present application, when storing data in multiple AZs, L-1 first data groups including data blocks and local check blocks corresponding to the data blocks can be selected to be placed in at least L-1 AZs respectively, and at the same time, the second data group including global check blocks and local check blocks corresponding to the global check blocks can be placed in the same AZ. On the one hand, intra-AZ repair can be achieved by placing them in the least number of AZs as possible, thereby reducing the amount of data transmitted across AZs; on the other hand, placing the global check blocks and the corresponding local check blocks in one AZ can achieve intra-AZ repair of the global check blocks while ensuring single AZ fault tolerance, further reducing cross-AZ traffic. In other words, the data storage method proposed in the embodiments of the present application can effectively reduce the amount of cross-AZ data transmission required for single-block repair, and greatly reduce the repair bandwidth across AZs under a multi-AZ architecture.

[0159] Based on the above embodiments, the embodiments of the present application provide a data storage method, which can be applied to a data storage system, wherein the data storage system can be arranged in a multi-AZ environment, and each AZ includes one or more data nodes. Accordingly, each AZ corresponds to at least one set of coding groups (data groups), and the at least one set of coding groups is stored in one or more data nodes in the AZ.

[0160] It should be noted that the data storage method proposed in the embodiment of the present application can be understood as an erasure code data placement mechanism that implements intra-AZ repair in multi-AZ storage. The data storage method proposed in the embodiment of the present application is described below using Azure-LRC+1 (12, 3, 6) as an example.

[0161] Based on the erasure code data placement mechanism for implementing intra-AZ repair in multi-AZ storage proposed in the embodiment of the present application, Azure-LRC+1 (12, 3, 6) encoded stripes can be deployed in multi-AZ storage scenarios according to specific placement strategies and possible failed blocks can be repaired. Among them, there are 12 data blocks, 3 local check blocks, and 6 global check blocks in the Azure-LRC+1 (12, 3, 6) encoded stripes, and the data blocks and global check blocks are divided into 3 groups. Since the available bandwidth within an AZ in a multi-AZ storage system is much smaller than the available bandwidth between AZs, it is necessary to reasonably arrange the placement strategy and repair mechanism of the stripes to achieve intra-AZ repair of a single failed block and improve the repair performance.

[0162] Exemplarily, in some embodiments, the strips may be encoded using Azure-LRC+1 first. Figure 7 Schematic diagram of the data storage implementation scheme Figure 1 ,like Figure 7 As shown, the data block can be divided into a first data group, and the first data group also includes a local check block corresponding to the data block; the global check block corresponding to the data block can also be divided into a second data group, and the second data group also includes a local check block corresponding to the global check block. Figure 8 Schematic diagram of the data storage implementation scheme Figure 2 ,like Figure 8 As shown, there are 12 data blocks D 0 -D 11 , 3 local check blocks L 0 -L 2 , 6 global check blocks G 0 -G 5 , where D 0 -D 5 and L 0 As a group, L 0 By D 0 -D 5 Calculated; D 6 -D 11 and L 1 As a group, L 1 By D 6 -D 11 Calculated; G 0 -G 5 and L 2 As a group, L 2 By G 0 -G 5 Calculated.

[0163] For example, in some embodiments, the LRC placement strategy that can achieve data repair within the AZ can be used to store different data groups in different AZs. In the Azure-LRC+1 (12, 3, 6) encoded strip, b = 6, g = 6, then the condition g ≥ b is satisfied, and the calculation is Then one group should be placed in each AZ, and the three groups in the Azure-LRC+1 (12, 3, 6) encoded stripe should be placed in three AZs respectively, such as AZ0, AZ1, and AZ2.

[0164] Exemplarily, in some embodiments, for a failed block in the first data group and / or the second data group, the failed block may be repaired according to a preset repair strategy, that is, the failed data may be repaired using a partial decoding method. Fig. 9 This is a schematic diagram of data repair processing, such as Fig. 9 As shown, for a group of Azure-LRC+1 (12, 3, 6) coded stripes, such as the first data group, all blocks of the group are placed in the same AZ. 0 When the proxy starts the repair process, it collects D 0 All surviving blocks in the group are decoded and reconstructed 0 =α 1 D 1 +α 2 D 2 +α 3 D 3 +α 4 D 4 +α 5 D 5 +α 6 L 0 . And randomly select a new node in the original AZ to store the repaired D 0 .

[0165] In summary, in an embodiment of the present application, an erasure code data placement mechanism is adopted to implement intra-AZ repair in multi-AZ storage. On the one hand, different placement strategies are designed according to stripes encoded with different parameters. Accordingly, by designing a placement strategy, the cross-AZ data transmission in the single block repair process can be minimized, thereby improving the repair efficiency. For parameters that meet the conditions, a single failed block can be repaired within the AZ without generating cross-AZ repair traffic; because in a multi-AZ storage system, the bottleneck of repair performance lies in the tight bandwidth across AZs, therefore, by reducing the cross-AZ repair traffic, the repair speed can be greatly improved. At the same time, for situations where cross-AZ repair may be required, the partial decoding method can be selected to repair a single failed block, further reducing the cross-AZ repair traffic.

[0166] It can be understood that the data storage method proposed in the embodiment of the present application can place blocks of the same group in the Azure-LRC+1 stripe in the minimum number of AZs under the constraint of tolerating single AZ failures, targeting the data placement problem of erasure codes in multi-AZ storage. Under certain parameters (such as Azure-LRC+1 (12, 3, 6)), blocks of the same group can be placed in the same AZ to achieve intra-AZ repair. Compared with the common flat placement strategy and random placement strategy, the placement strategy proposed in the embodiment of the present application greatly reduces the number of AZs spanned by each Azure-LRC+1 group while taking into account the tolerance of AZ failures, and therefore can effectively reduce the amount of cross-AZ network transmission required for single-block repair.

[0167] It should be noted that the data storage method proposed in the embodiment of the present application, in order to solve the single block repair problem of erasure codes in multi-AZ storage, chooses to combine local repair within AZ + global repair across AZ to reconstruct the failed block. When reconstructing a failed block requires data reading across multiple AZs, firstly, local repair is performed within each AZ to construct a partial block, and then the partial blocks are transmitted across AZs and the partial blocks are combined to reconstruct the failed block. Compared with the method of directly reading the original data block across AZs, the repair strategy proposed in the embodiment of the present application can greatly reduce the cross-AZ network transmission amount of single-block repair through local repair within AZ + global repair across AZ technology, thereby improving the repair efficiency.

[0168] It can be seen that the data storage method proposed in the embodiment of the present application can achieve the goal of minimizing cross-AZ data transmission during the repair process and reduce the impact of the repair process on the repair performance of the multi-AZ storage system. At the same time, the data storage method has good versatility and is applicable to all encoding parameters.

[0169] An embodiment of the present application provides a data storage method, in which a data storage system determines L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in the second available area. It can be seen that in the embodiments of the present application, when storing data in multiple AZs, L-1 first data groups including data blocks and local check blocks corresponding to the data blocks can be selected to be placed in at least L-1 AZs respectively, and at the same time, the second data group including global check blocks and local check blocks corresponding to the global check blocks can be placed in the same AZ. On the one hand, intra-AZ repair can be achieved by placing them in the least number of AZs as possible, thereby reducing the amount of data transmitted across AZs; on the other hand, placing the global check blocks and the corresponding local check blocks in one AZ can achieve intra-AZ repair of the global check blocks while ensuring single AZ fault tolerance, further reducing cross-AZ traffic. In other words, the data storage method proposed in the embodiments of the present application can effectively reduce the amount of cross-AZ data transmission required for single-block repair, and greatly reduce the repair bandwidth across AZs under a multi-AZ architecture.

[0170] Based on the above embodiment, in another embodiment of the present application, Fig.10 Schematic diagram of the structure of the data storage system proposed in the embodiment of the present application Figure 1 ,like Fig.10 As shown, the data storage system 10 proposed in the embodiment of the present application may include a determination unit 111, a storage unit 112,

[0171] The determining unit 111 is used to determine L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0;

[0172] The storage unit 112 is configured to store the L-1 first data groups in at least L-1 first available areas respectively based on G, and store the second data group in a second available area.

[0173] In the embodiments of the present application, further, Fig.11 Schematic diagram of the structure of the data storage system proposed in the embodiment of the present application Figure 2,like Fig.11 As shown, the data storage system 10 proposed in the embodiment of the present application may also include a processor 121, a memory 122 storing executable instructions of the processor 121, and further, the data storage system 10 may also include a communication interface 123, and a bus 124 for connecting the processor 121, the memory 122 and the communication interface 123.

[0174] In the embodiment of the present application, the processor 121 can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the function of the processor can also be other, and the embodiment of the present application is not specifically limited. The data storage system 10 can also include a memory 122, which can be connected to the processor 121, wherein the memory 122 is used to store executable program code, the program code includes computer operation instructions, and the memory 122 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.

[0175] In the embodiment of the present application, the bus 124 is used to connect the communication interface 123, the processor 121 and the memory 122, and the mutual communication between these devices.

[0176] In the embodiment of the present application, the memory 122 is used to store instructions and data.

[0177] Further, in an embodiment of the present application, the above-mentioned processor 121 is used to determine L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in the second available area.

[0178] In practical applications, the memory 122 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or a combination of the above types of memories, and provide instructions and data to the processor 121.

[0179] In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.

[0180] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., which can store program code.

[0181] An embodiment of the present application provides a data storage system, which determines L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in the second available area. It can be seen that in the embodiments of the present application, when storing data in multiple AZs, L-1 first data groups including data blocks and local check blocks corresponding to the data blocks can be selected to be placed in at least L-1 AZs respectively, and at the same time, the second data group including global check blocks and local check blocks corresponding to the global check blocks can be placed in the same AZ. On the one hand, intra-AZ repair can be achieved by placing them in the least number of AZs as possible, thereby reducing the amount of data transmitted across AZs; on the other hand, placing the global check blocks and the corresponding local check blocks in one AZ can achieve intra-AZ repair of the global check blocks while ensuring single AZ fault tolerance, further reducing cross-AZ traffic. In other words, the data storage method proposed in the embodiments of the present application can effectively reduce the amount of cross-AZ data transmission required for single-block repair, and greatly reduce the repair bandwidth across AZs under a multi-AZ architecture.

[0182] An embodiment of the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the data storage method described above is implemented.

[0183] Specifically, the program instructions corresponding to a data storage method in this embodiment can be stored in a storage medium such as a CD, a hard disk, a USB flash drive, etc. When the program instructions corresponding to a data storage method in the storage medium are read or executed by an electronic device, the following steps are included:

[0184] Determine L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0;

[0185] Based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in a second available area.

[0186] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0187] The present application is described with reference to implementation flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0188] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the implementation flow diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process described in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0190] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A data storage method, characterized in that: The method comprises: Determine L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; Based on G, the L-1 first data groups are respectively stored in at least L-1 first available areas, and the second data group is stored in a second available area.

2. The method according to claim 1, characterized in that: The step of determining L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed includes: For each K / (L-1) data block of the K data blocks, encoding is performed according to K / (L-1) encoding parameters corresponding to the K / (L-1) data blocks to determine the first local check block; The first data group is generated based on the K / (L-1) data blocks and the first local check block.

3. The method according to claim 1, characterized in that The step of determining L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed includes: Encoding is performed according to K encoding parameters corresponding to the K data blocks to determine the G global check blocks; Determine the second local check block based on the G global check blocks; The second data group is generated based on the G global check blocks and the second local check block.

4. The method according to any one of claims 1 to 3, characterized in that: The storing the L-1 first data groups respectively to at least L-1 first available areas based on G includes: According to M and G, the L-1 first data groups are stored in at least L-1 first available areas respectively; wherein M=K / (L-1), and M is an integer greater than 0.

5. The method according to claim 4, characterized in that According to M and G, storing the L-1 first data groups respectively in at least L-1 first available areas comprises: For any first data group, when G is greater than or equal to M, the first data group is stored in one of the first available areas.

6. The method according to claim 4, characterized in that According to M and G, storing the L-1 first data groups respectively in at least L-1 first available areas comprises: For any of the first data groups, when G is smaller than M, the first data group is divided to determine a plurality of sub-data groups corresponding to the first data group; The plurality of sub-data groups are stored in a plurality of the first available areas respectively.

7. The method according to claim 6, characterized in that The dividing the first data group to determine a plurality of sub-data groups corresponding to the first data group includes: The K / (L-1) data blocks and the first local check block in the first data group are divided according to G to obtain the multiple sub-data groups.

8. The method according to claim 5 or 6, characterized in that: The method further comprises: For any one of the first data groups, if the number of blocks in the first data group is greater than or equal to the number of blocks in the second data group, storing the first data group in one of the first available areas; For any one of the first data groups, when the number of blocks in the first data group is less than the number of blocks in the second data group, the first data group is stored in a plurality of the first available areas.

9. The method according to claim 1, characterized in that: The method further comprises: Repair the failed blocks in the first data group and / or the second data group according to a preset repair strategy; wherein the preset repair strategy is used to perform repair processing based on some candidate blocks, and the candidate blocks are blocks other than the failed blocks.

10. The method according to claim 9, characterized in that The method further comprises: Determine the candidate block in the same data group as the failed block as a first candidate block; The candidate block in a different data group from the failed block is determined as a second candidate block.

11. The method according to claim 10, characterized in that The repairing the failed blocks in the first data group and / or the second data group according to the preset repair strategy includes: Generate a third candidate block according to part or all of the candidate blocks in the second candidate blocks; The failed block is repaired according to the third candidate block and the first candidate block.

12. A data storage system, characterized in that: The data storage system comprises: a determination unit, a storage unit, The determining unit is used to determine L-1 first data groups and one second data group based on K data blocks corresponding to the data to be processed; wherein the first data group includes K / (L-1) data blocks and first local check blocks corresponding to the K / (L-1) data blocks; the second data group includes G global check blocks and second local check blocks corresponding to the G global check blocks; K and L are integers greater than 1, and G is an integer greater than 0; The storage unit is used to store the L-1 first data groups in at least L-1 first available areas respectively based on G, and store the second data group in a second available area.

13. A data storage system, characterized in that: The data storage system includes a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Citation Information

Patent Citations

  • Data placement method for optimizing local recoverable code repair performance

    CN115617551A

  • Managing a storage array

    US20140040702A1