Method and apparatus for balancing redundancy conversion of cross-rack aware erasure code storage system

Abstract

The application discloses a kind of cross-rack perception's erasure code storage system equalization redundancy conversion method and device, to solve the problem of large cross-rack transmission flow overhead and rack load imbalance in the redundancy conversion of erasure code storage system, method includes: formulating inter-rack balanced data layout and intra-rack aggregated check block layout, and maintaining the layout in the redundancy conversion process, to reduce the cross-rack transmission flow caused by data block relocation and check block update;Design data block selection and parity check block incremental update strategy to further reduce the cross-rack transmission flow of check block update;Proposed a heuristic algorithm, select strip group and carry out redundancy conversion in parallel, to balance the flow load of each rack.The application balances the load between racks while suppressing the cross-rack transmission flow caused by redundancy conversion, and can shorten the time to complete redundancy conversion.

Description

Technical Field

[0001] This invention relates to the field of storage technology, and in particular to a method and apparatus for balanced redundancy conversion in a rack-aware erasure coding storage system. Background Technology

[0002] With the increasing prevalence of failures in large-scale storage systems, most existing storage systems employ backups and erasure coding to maintain additional data redundancy in advance, ensuring data reliability is protected against unexpected failures. Backups copy data N times and store them across N different nodes, resulting in a storage overhead N times that of the original data. Erasure coding, on the other hand, introduces lightweight computation. It takes a fixed-size data block as input and calculates a small amount of redundant data of the same size (called parity blocks). Specifically, erasure coding is configured with two parameters, k and m. During encoding, k data blocks are used to generate m parity blocks. These (k+m) blocks form a "strip" and are stored across (k+m) different nodes (ensuring each node contains only one block from this strip). Furthermore, any k blocks in the strip can be decoded to obtain the remaining m blocks. Erasure coding thus ensures data recovery in the event of a failure. Compared to storing multiple identical copies of a backup, erasure coding is more space-efficient and can maintain the same fault tolerance as backups with less data redundancy. Currently, erasure coding is widely used in various storage systems, primarily for persistent storage of cold data and mitigating delays caused by frequent data access.

[0003] In most cases, storage systems typically employ a range of different redundancy levels (i.e., storage overhead, calculated as...). Erasure coding at different redundancy levels is used to adapt access performance to changing access characteristics and reliability requirements. Therefore, the conversion between different redundancy levels of erasure coding (called redundancy conversion) is crucial for modern storage systems, but this process can generate significant conversion traffic (i.e., data conversion over the network). Typically, redundancy conversion represents the operation of converting (k,m) erasure coding to (k',m) erasure coding. Existing methods consider redundancy conversion operations that make k' > k. Specifically, redundancy conversion requires disassembling some older stripes of (k,m) coding (called stripe decomposition) and allocating data blocks from them to other (k,m) coded stripes (called stripe stretching) for recoding under the new (k',m) coding. Assume {D1,D2,...,D...} k} and {P1,P2,...,P m} represents a certain strip containing k data blocks and m parity blocks. Each parity block is a linear combination of the k data blocks, which can be expressed by the formula under the Galois field [1]. To calculate, where α i,j (1≤i≤k, 1≤j≤m) is derived from data block D. i Calculate the check block P j The coding coefficients used at that time. For decomposing stripes, the old linear relationship needs to be decoupled, that is, its parity block is invalidated and its data block is added to the stretched strip as a new data block; for stretched stripes, these new data blocks need to be received and its own parity block is updated, using P. j 'Indicates the updated check block P j Then, in the formula, it can be calculated as follows: Where 1≤j≤m, △P j This is called a check increment block.

[0004] The traffic overhead of redundancy conversion consists of two parts: data block migration and parity block update. During the conversion process, some data blocks need to be migrated to other nodes to ensure that the (k'+m) blocks of the new stripe encoded by (k',m) are still stored in (k'+m) different nodes. The resulting traffic is the data block migration traffic. At the same time, the stretched stripe is converted from (k,m) encoding to (k',m) encoding. Since new data blocks have been added, the m parity blocks of each stretched stripe need to be recalculated to ensure that the fault tolerance capability is still effective after the conversion. This part of the traffic is the parity block update traffic. Existing work mainly accelerates the redundancy conversion process by reducing traffic. For example, SRS[2] eliminates data block migration traffic by designing a data layout; ERS[3] designs a special data layout like SRS, which not only eliminates data block migration traffic but also further reduces parity block update traffic; StripeMerge[4] achieves the conversion from k to 2k by directly merging two stripes. The existing methods have the following shortcomings: (i) The selection of parameters for redundant conversion lacks flexibility (SRS and ERS require parameters to be given in advance to configure the layout, and StripeMerge can only perform conversions from k to multiples thereof); (ii) Although the single conversion traffic is optimized, they are still prone to generating a large amount of cross-rack transmission traffic when performing continuous conversions (i.e., performing multiple conversions) (SRS and ERS require readjustment of the layout before each conversion); (iii) The problem of load imbalance between racks during the conversion process is not considered, thus prolonging the conversion time.

[0005] [1]Plank J S,Simmerman S,Schuman C D.Jerasure:A library in C / C++facilitating erasure coding for storage applications-Version 1.2[J].University of Tennessee,Tech.Rep.CS-08-627,2008,23.

[0006] [2]Taranov K,Alonso G,Hoefler T.Fast and strongly-consistent per-itemresilience in key-value stores[C] / / Proceedings of the Thirteenth EuroSysConference.2018:1-14.

[0007] [3]Wu S,Shen Z,Lee P P C.Enabling I / O-efficient redundancytransitioning in erasure-coded KV stores via elastic Reed-Solomon codes[C] / / 2020International Symposium on Reliable Distributed Systems(SRDS).IEEE,2020:246-255.

[0008] [4]Yao Q,Hu Y,Cheng L,et al.Stripemerge:Efficient wide-stripegeneration for large-scale erasure-coded storage[C] / / 2021IEEE 41stInternational Conference on Distributed Computing Systems(ICDCS).IEEE,2021:483-493. Summary of the Invention

[0009] The main objective of this invention is to overcome the problems of high cross-rack transmission traffic overhead and unbalanced load between racks in the storage redundancy conversion of erasure coding storage systems. It proposes a cross-rack-aware method and apparatus for balanced redundancy conversion in erasure coding storage systems, which can balance the load between racks while suppressing cross-rack transmission traffic caused by redundancy conversion, and can shorten the time to complete the redundancy conversion.

[0010] The present invention adopts the following technical solution:

[0011] Firstly, a method for balanced redundancy conversion in a rack-aware erasure coding storage system includes:

[0012] The block layout formulation steps involve aggregating the verification blocks of all stripes within the redundancy conversion group into the same verification rack, and evenly distributing the data blocks of each stripe within the redundancy conversion group across various data racks; the stripes include stretched stripes and decomposed stripes.

[0013] The verification block update step uses a decoupled approach of directly reading data blocks and decomposing strip verification blocks to update the verification blocks of the stretched strip;

[0014] The load balancing process involves selecting redundant conversion groups from different racks to form execution groups, using a heuristic algorithm to iteratively replace execution groups, and selecting the execution group with the smallest upload load balancing ratio. The redundant conversion groups within the execution group are then executed in parallel.

[0015] Preferably, the block layout formulation step specifically includes:

[0016] Step 1.1: For a redundant conversion group consisting of k stretch strips and k'-k decomposed strips, aggregate the verification blocks of all strips in the group into the same inspection frame;

[0017] Step 1.2: Distribute the data blocks of each stripe in the redundancy conversion group evenly across the data racks; the data blocks are also evenly distributed across the data racks; the data racks are racks other than the verification racks.

[0018] Step 1.3: After the redundancy operation is initiated, priority is assigned to each data rack according to the number of data blocks of each stripe in the data rack, and a priority queue is established. The k'-k data blocks of the k'-k decomposed strips are allocated to the first k'-k stretch strips in the order of the data racks in the priority queue. The remaining stretch strips are called the remaining stretch strips.

[0019] Step 1.4: Construct a network flow graph for the remaining data blocks and remaining stretch strips. In the network flow graph, each stretch strip, decomposition strip, and data rack is represented by a vertex. Vertices of decomposition strips are connected to directed edges pointing to vertices of data racks, and each vertex of a decomposition strip is connected to... The edge capacity of each rack vertex is determined by the number of data blocks in the corresponding rack for the decomposed stripe, indicating the number of data blocks in the decomposed stripe within that rack. The number of data blocks that can be provided in each data rack; the rack vertices are connected to directed edges pointing to the vertices of the stretch strips, and the edge capacity is determined by the number of data blocks that the stretch strip can receive in the corresponding rack; a source point is established, and the source point is connected to directed edges pointing to the vertices of each decomposed strip, and the edge capacity depends on the number of remaining data blocks in that decomposed strip; a sink point is established, and each vertex of the stretch strip is connected to a directed edge pointing to the sink point, and the edge capacity depends on the number of data blocks that the stretch strip needs to receive;

[0020] Step 1.5: Run the Dinic algorithm to find the maximum flow, and allocate the remaining data blocks and remaining stretched strips in the decomposition strips according to the found maximum flow.

[0021] Preferably, data racks containing fewer data blocks have higher priority.

[0022] Preferably, for the remaining stretch strips, data blocks are allocated according to a priority queue until a completely balanced state is reached, that is, the number of data blocks in each data rack is equal.

[0023] Preferably, the number of directed edges includes k'-k edges.

[0024] Preferably, the verification block update step specifically includes:

[0025] Step 2.1: Send the k'-k data blocks allocated to the remaining stretch strips to the first verification node of the corresponding decomposed strip on the verification rack;

[0026] Step 2.2: Calculate the required verification increment blocks for the remaining stretch strip based on the first verification node of the decomposed strip; the verification increment blocks include m blocks, and send these m verification increment blocks to the verification nodes of the remaining stretch strip;

[0027] Step 2.3: The first verification node of the decomposed strip forwards the read data block to the remaining verification nodes. The verification node decouples the received data block from its local verification block and calculates a total of m verification increment blocks. These m verification increment blocks are then distributed to the verification node of the previous stretched strip.

[0028] Step 2.4: For each stretched strip, the verification node uses the received verification increment block to update the verification block stored in itself.

[0029] Preferably, the load balancing step specifically includes:

[0030] Step 3.1: Group all redundant conversion groups according to the rack where the verification block is located, and group the verification blocks located in different racks. A combination of redundant conversion groups is called an execution group; among them... This represents the total number of racks.

[0031] Step 3.2: Traverse the redundant conversion groups that have not been selected, and select the redundant conversion groups that have different verification racks from the current execution group and add them to the execution group;

[0032] Step 3.3: Record the load of the execution group each time a new redundancy conversion group is added, and add upload load to the execution group based on the number of data blocks uploaded to each data rack of the redundancy conversion group;

[0033] Step 3.4: Calculate the load balancing ratio of the execution group, which is the ratio of the maximum upload load of the execution group to the average upload load of the execution group;

[0034] Step 3.5: Select the execution group with the smallest load balancing ratio, and all redundant conversion groups in the execution group perform redundant conversion in parallel;

[0035] Step 3.6: Repeat steps 3.2 to 3.5 above until all redundant conversion groups have been selected.

[0036] Secondly, a cross-rack-aware erasure coding storage system redundancy switching device includes:

[0037] The block layout specification module is used to aggregate the verification blocks of all stripes within the redundancy conversion group into the same verification rack, and to evenly distribute the data blocks of each stripe within the redundancy conversion group across various data racks; the stripes include stretched stripes and decomposed stripes.

[0038] The verification block update module is used to update the verification block of the stretched strip by directly reading the data block and decoupling the strip verification block.

[0039] The load balancing module is used to select different redundant conversion groups of the verification rack to form an execution group. It uses a heuristic algorithm to iteratively replace the execution groups and select the execution group with the smallest upload load balancing ratio. The redundant conversion groups in the execution group are then executed in parallel.

[0040] Thirdly, a computer device includes a program or instructions that, when the program or instructions are executed, execute the cross-rack-aware erasure coding storage system balanced redundancy conversion method.

[0041] Fourthly, a storage medium includes a program or instructions, wherein when the program or instructions are executed, the cross-rack-aware erasure coding storage system balanced redundancy conversion method is performed.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] (1) By formulating a balanced data layout between racks and an aggregated parity block layout within racks, and maintaining this layout during the redundancy conversion process, the cross-rack transmission traffic caused by data block relocation and parity block updates is reduced; by designing a data block selection and parity block incremental update strategy, the cross-rack transmission traffic of parity block updates is further reduced; by proposing a heuristic algorithm, stripe groups are carefully selected and parallelized for redundancy conversion to balance the traffic load of each rack.

[0044] (2) The redundant conversion parameters of the present invention can be flexibly selected and can maintain good performance in continuous conversion. It is a more universal design, while balancing the load between racks and making better use of I / O parallelism.

[0045] (3) The parameters of the redundant conversion of the present invention are not limited and can be used for any given k to k' conversion, and there is no need to give parameters in advance;

[0046] (4) The present invention can still maintain good and stable performance in continuous redundant conversion operations, which existing SRS and ERS cannot guarantee;

[0047] (5) This invention takes into account the load balancing problem between racks in redundancy conversion and proposes a heuristic algorithm to carefully select redundancy conversion groups and execute redundancy conversion in parallel in order to balance the load of each rack. Attached Figure Description

[0048] Figure 1 A flowchart of a cross-rack-aware erasure coding storage system redundancy conversion method provided in an embodiment of the present invention;

[0049] Figure 2 The schematic diagram of the block layout design strategy provided in the embodiments of the present invention is shown in which (a) represents a strip layout schematic diagram before redundancy conversion; (b) represents a data block allocation schematic diagram; (c) represents a network flow graph establishment and maximum flow finding schematic diagram; and (d) represents a strip layout schematic diagram after redundancy conversion.

[0050] Figure 3 This is an example diagram of the optimized check block update algorithm provided in an embodiment of the present invention;

[0051] Figure 4 Example diagram of the heuristic algorithm for selecting stretch strips provided in the embodiments of the present invention;

[0052] Figure 5 This is a structural block diagram of a cross-rack-aware erasure coding storage system redundancy conversion device provided in an embodiment of the present invention;

[0053] Figure 6 This is a schematic diagram of the prototype system structure provided in an embodiment of the present invention;

[0054] Figure 7 This is a diagram showing the experimental results of data block relocation and check block update traffic in a large-scale simulation experiment according to an embodiment of the present invention;

[0055] Figure 8 The figure shows the experimental results of continuous flow conversion in a large-scale simulation experiment provided in the embodiments of the present invention;

[0056] Figure 9 The following are experimental results diagrams for different numbers of racks in a large-scale simulation experiment provided in the embodiments of the present invention; wherein, (a) represents the redundancy switching traffic result diagram when the number of racks is 10; (b) represents the redundancy switching traffic result diagram when the number of racks is 20; and (c) represents the redundancy switching traffic result diagram when the number of racks is 30.

[0057] Figure 10 The following are experimental results diagrams for different m values ​​in a large-scale simulation experiment provided in the embodiments of the present invention; wherein, (a) represents the redundancy conversion flow result diagram when the verification increment block is 3; (b) represents the redundancy conversion flow result diagram when the verification increment block is 4; and (c) represents the redundancy conversion flow result diagram when the verification increment block is 5.

[0058] Figure 11 This is a result diagram of a load balancing experiment in a large-scale simulation provided in an embodiment of the present invention;

[0059] Figure 12 The experimental results of continuous conversion time and computation time in the Amazon cloud environment experiment provided for the embodiments of the present invention are shown in the figure; wherein, (a) represents the experimental results of continuous redundant conversion time; and (b) represents the experimental results of total computation time.

[0060] Figure 13 The following are experimental results for different block sizes in an Amazon cloud environment experiment provided for embodiments of the present invention; wherein, (a) represents the experimental results for conversion time when the block size is 16MB; and (b) represents the experimental results for conversion time when the block size is 64MB.

[0061] Figure 14 This is a graph showing the experimental results of different network bandwidths in an Amazon cloud environment experiment provided in this embodiment of the invention; wherein,

[0062] (a) shows the experimental results of the conversion time when the network bandwidth is 1GB / s; (b) shows the experimental results of the conversion time when the network bandwidth is 3GB / s. Detailed Implementation

[0063] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0064] See Figure 1 As shown in this embodiment, a method for balanced redundancy conversion in a cross-rack-aware erasure coding storage system includes:

[0065] Step 1, Block Layout Formulation Step: Aggregate the verification blocks of all stripes within the redundancy conversion group into the same verification rack, and evenly distribute the data blocks of each stripe within the redundancy conversion group into various data racks; the stripes include stretched stripes and decomposed stripes;

[0066] Step 2, the check block update step, uses a decoupling method of directly reading data blocks and decomposing strip check blocks to update the check blocks of the stretched strip;

[0067] Step 3, load balancing step: Select redundant conversion groups with different verification racks to form execution groups, use heuristic algorithms to iteratively replace execution groups, select the execution group with the smallest upload load balancing ratio, and execute the redundant conversion groups in the execution group in parallel.

[0068] Specifically, step 1 is implemented as follows.

[0069] Step 1.1: Assume there are k stretched strips and k′-k decomposed strips. The combination of the k stretched strips and their corresponding k′-k decomposed strips is called a redundancy conversion group. For each redundancy conversion group, the check blocks of all strips within the group are aggregated in the same rack. The rack containing the check blocks is called the check rack, and the racks other than the check rack are called data racks.

[0070] Step 1.2 involves uniformly distributing the data blocks of each stripe in the redundancy conversion group across the data racks. Uniform distribution means that the difference in the number of data blocks in any two data racks for a stripe is less than or equal to 1. Simultaneously, the data blocks for the entire redundancy conversion group also satisfy the uniform distribution across all data racks.

[0071] Step 1.3: After the redundancy operation is initiated, priorities are assigned to each data rack based on the number of data blocks in each stripe (data racks with fewer data blocks have higher priorities), and a priority queue is established. The k'-k data blocks of the k'-k decomposed stripes are then allocated to the first k'-k stretched stripes according to the rack order in the priority queue. The remaining stretched stripes are called the residual stretched stripes. For the residual stretched stripes, data blocks are allocated according to the priority queue until a completely balanced state is reached (i.e., the number of data blocks in each data rack is equal).

[0072] Step 1.4: Construct a network flow graph for the remaining data blocks and remaining stretch strips. In the network flow graph, each stretch strip, decomposition strip, and data rack is represented by a vertex. The vertices of the decomposition strips (referred to as decomposition vertices) are connected to directed edges pointing to the vertices of the data racks (referred to as rack vertices). Each decomposition vertex should be connected to... Each rack vertex (because the data block of each decomposition strip is stored in) In a data rack, the edge capacity is determined by the number of data blocks in the corresponding rack for the decomposed stripe, indicating the number of data blocks in the rack for that decomposed stripe. The number of data blocks that can be provided in each data rack. Each rack vertex is connected to a directed edge pointing to a vertex of the stretch strip (referred to as a stretch vertex). The edge capacity is determined by the number of data blocks the stretch strip can receive in the corresponding rack, indicating that the stretch strip can obtain data blocks from that data rack and maintain a balanced distribution of the strip across all data racks. A source vertex is established, connected to each decomposition vertex by directed edges (i.e., k'-k edges). The edge capacity depends on the number of remaining data blocks in that decomposition strip. A sink vertex is established, with each stretch vertex connected to a directed edge pointing to the sink. The edge capacity depends on the number of data blocks the stretch strip needs to receive. At this point, the network flow graph is complete.

[0073] Step 1.5: Run the Dinic algorithm to find the maximum flow. The Dinic algorithm is an existing algorithm for finding the maximum flow. The remaining data blocks and remaining stretched strips in the decomposition strip are then allocated based on the found maximum flow.

[0074] See Figure 2 As shown, a block layout strategy using RS(5,3) encoding is demonstrated in rack {R1, R2, R3, R4}. This strategy decomposes data blocks in stripe {S6, S7, S8} to stretch stripe {S1, S2, S3, S4, S5}, completing the redundancy conversion from k=5 to k'=8. Figure 2 (a) The check blocks of all stripes in the redundancy conversion group are aggregated and placed in check rack R1. The data blocks are evenly distributed in data racks {R2, R3, R4}. It can be seen that the number of data blocks in each data rack is balanced. Figure 2In (b), k'-k=3 data blocks from the decomposed strips {S6, S7, S8} are allocated to the stretch strips {S1, S2, S3} according to the rack order in the priority queue. For the remaining stretch strips {S4, S5}, the strips are stretched to a fully balanced state according to the priority queue order. Figure 2 (c) A network flow graph is constructed according to the rules described in step 1 to allocate the remaining data blocks. Decompose vertices {S7, S8} (since there are no remaining data blocks in decomposed strip S8, there are no directed edges connecting to rack vertices) and connect them to rack vertices {R2, R3, R4}, each with a capacity of 1. Rack vertices are connected to stretched vertices {S4, S5}, each with a capacity of 1. Source vertices are connected to decomposed vertices, each with a capacity of 2. Stretched vertices are connected to sink vertices, each with a capacity of 2. The red edges between rack vertices and stretched vertices represent part of the maximum flow of the network flow graph, indicating which rack data blocks are allocated to the corresponding stretched strips. Figure 2 (d) shows the stretched strip layout after the redundancy conversion. It can be seen that the stretched layout still maintains the balance between racks, and does not generate additional cross-rack data migration overhead.

[0075] The specific implementation of step 2 is as follows.

[0076] Step 2.1: Send the k'-k data blocks allocated to the remaining stretch strips to the first verification node of the corresponding decomposed strip on the verification rack;

[0077] Step 2.2, direct reading of data blocks. The first verification node of the decomposed strip calculates the verification increment blocks required for the verification blocks of the remaining stretched strip, that is, it needs to calculate m verification increment blocks, and sends these m verification increment blocks to the verification nodes of the remaining stretched strip;

[0078] Step 2.3, Decoupling of strip verification blocks. The first verification node of the decomposed strip forwards the data block read in step 2.1 to the remaining verification nodes. The verification nodes decouple the received data block from their local verification blocks, that is, calculate m verification increment blocks and distribute these m verification increment blocks to the verification nodes of the previous stretched strip;

[0079] Step 2.4: For each stretched strip, the verification node uses the received verification increment block to update the verification block stored in itself.

[0080] Specifically, in step 2, the corresponding data blocks used for stretching the remaining stretch strips are sent to the verification rack to update the verification blocks of the remaining stretch strips and decouple the verification blocks of the decomposed strips. Then, the first k'-k stretch strips read the verification increment blocks generated after decoupling the verification blocks of the decomposed strips from the verification rack and update the verification blocks stored in their own nodes. See [link to relevant documentation] Figure 3 The example shown illustrates a set of redundant conversion groups for updating the check block, where k=4, m=2, and k'=5. First, according to the layout strategy allocated by the block layout design module, the data blocks corresponding to the remaining stretched strips (i.e., D18, D19, and D20) are read into the check rack, and the original check block of the remaining stretched strips is updated. Then, these data blocks are calculated with the check blocks P9 and P10 of the decomposed strip S5 to generate a check increment block, which is used to update the original check block of the stretched strip S1.

[0081] The specific implementation of step 3 is as follows.

[0082] Step 3.1: Group all redundant conversion groups according to the rack where the verification block is located, and group the verification blocks located in different racks. A combination of redundant conversion groups (i.e., the total number of racks in the system) is called an execution group;

[0083] Step 3.2: Traverse the redundant conversion groups that have not been selected, and try to select redundant conversion groups with different verification racks from the current execution group to add to the execution group;

[0084] Step 3.3: Record the load of the execution group each time a new redundancy conversion group is added. Add upload load to the execution group based on the number of data blocks uploaded to each data rack of the redundancy conversion group (only upload load is recorded here, because in Step 3.1 the execution group is composed of redundancy conversion groups with verification blocks located on different racks, and the download load of the execution group is the same on each rack).

[0085] Step 3.4: Calculate the load balancing ratio of the execution group, which is the ratio of the maximum upload load of the execution group to the average upload load of the execution group;

[0086] Step 3.5: Select the execution group with the smallest load balancing ratio, and all redundant conversion groups in the execution group perform redundant conversion in parallel;

[0087] Step 3.6: Repeat the above steps until all redundant conversion groups have been selected.

[0088] Specifically, in step 3, by selecting different redundant conversion groups from the verification racks to form execution groups, the download load of each rack is balanced. Then, a heuristic algorithm is used to iteratively replace the combination of execution groups to select the execution group with the smallest upload load balancing ratio. Finally, the redundant conversion groups in the execution group are executed in parallel, and the above steps are repeated to select execution groups until all redundant conversion groups in the system have been executed. See also Figure 4The diagram illustrates an example of a heuristic algorithm selecting redundant conversion groups for parallel execution. The data blocks and check blocks in the diagram represent the data blocks that need to be sent for redundant conversion and the check blocks that need to be updated. These blocks are distributed across three racks. Therefore, to balance the download bandwidth across racks, three redundant conversion groups are selected for parallel execution of redundant conversion each time. First, the first group G1 is selected from the redundant conversion group located on the first rack. At this point, racks R2 and R3 need to upload 7 and 5 data blocks respectively. Next, a group is selected from the redundant conversion groups G3 and G4 located on the second rack. G3 is selected first, as racks R1 and R3 both need to upload 6 blocks, resulting in a load balancing ratio of 11 / 8. Then, G4 is selected, with a load balancing ratio of 10 / 8, so G4 is added to the parallel execution group. Finally, redundant conversion group G6 is selected from the redundant conversion group located on the third rack, resulting in the lowest system upload load balancing ratio. The selection process is now complete.

[0089] See Figure 5 As shown, this embodiment also discloses a cross-rack-aware erasure coding storage system redundancy leveling and conversion device, comprising:

[0090] The block layout specification module 501 is used to aggregate the verification blocks of all stripes in the redundancy conversion group into the same verification rack, and to evenly distribute the data blocks of each stripe in the redundancy conversion group into each data rack; the stripe includes stretched stripe and decomposed stripe.

[0091] The verification block update module 502 is used to update the verification block of the stretched strip by directly reading the data block and decoupling the strip verification block;

[0092] The load balancing module 503 is used to select different redundant conversion groups of the verification rack to form an execution group. It uses a heuristic algorithm to iteratively replace the execution group and select the execution group with the smallest upload load balancing ratio. The redundant conversion groups in the execution group are then executed in parallel.

[0093] For the specific implementation of each module in a cross-rack-aware erasure coding storage system redundancy conversion device, please refer to a cross-rack-aware erasure coding storage system redundancy conversion method, which will not be repeated in this embodiment.

[0094] Further, see Figure 6As shown, this embodiment discloses a system architecture prototype. The system prototype includes a centralized controller (equipped with the aforementioned cross-rack-aware erasure coding storage system balanced redundancy conversion method / device) and multiple agents. The controller runs in a metadata server and is responsible for generating conversion decisions based on data distribution and conversion parameters (i.e., k, m, k'), and guiding redundancy conversion by accessing metadata information (e.g., the location of data blocks and check blocks for each stripe). Conversion decisions can be represented by a customized data structure specifying the unique ID of the block to be transferred and its destination node, as well as the agent of the corresponding node executing the conversion decision. Agent components on each storage node are used to listen for conversion decisions and perform redundancy conversion. When a conversion request is reported to the metadata server, the coordinator first generates a conversion decision and then distributes it to the corresponding agents of the participating nodes (…). Figure 6 Steps in Upon receiving the transition decision, each agent can parse the decision to understand its tasks, including which blocks should be relocated and which blocks need to be sent for parity block updates. Figure 6 Steps in The agent will notify the coordinator of the completion of its task by returning an ACK command to the coordinator. Figure 6 Steps in Once all ACKs from the participating nodes have been collected, the coordinator can know that the transition operation is complete.

[0095] The following performance test was conducted using the redundancy conversion method / redundancy conversion device of the present invention.

[0096] The performance of this invention was evaluated through large-scale simulations and cloud environment experiments. This invention was compared with two other state-of-the-art redundancy conversion methods: (i) SRS, which establishes a stripe layout based on pre-fixed coding parameters, eliminating data migration in the first conversion operation; and (ii) ERS, which also establishes a stripe layout based on pre-fixed coding parameters, eliminating data migration in the first conversion operation, but further reduces the check update traffic by pre-expanding the coding matrix. In the experimental results figures, (k,m,k') represents the redundancy conversion of erasure code (k,m) to erasure code (k',m).

[0097] A. Large-scale simulation experiment

[0098] First, simulations were conducted to reveal the performance of this invention when deployed on a large-scale storage system. Experimental setup: Network transmissions and storage operations were removed, and redundancy conversion traffic was measured under a common erasure coding configuration. Specifically, the experiment first deployed erasure coding (k,m), continuously increasing the value of k while keeping the value of m constant. Unless otherwise specified, the following default configuration was selected: the number of stripes was set to 100,000, distributed across 100 nodes, and the block size was set to 64MB (used in Hadoop HDFS).

[0099] A.1 Data relocation and check block update traffic experiment:

[0100] This experiment was conducted to demonstrate that the majority of traffic during continuous redundancy transitions originates from checksum block updates, while traffic due to data block relocation approaches zero. The system was configured with 10 racks, and the value of k was increased from 6 to 26 to perform eight transition operations. Traffic from data block relocation and checksum block updates was measured separately. See [link to relevant documentation] Figure 7 As shown in the two bar charts (which represent the results of data block relocation and check block update, respectively), in most cases, the traffic caused by data block relocation is almost negligible compared to the traffic caused by check block update.

[0101] A.2 Continuous Flow Conversion Experiment:

[0102] Perform 14 transformation operations by increasing the value of k from 6 to 96, and measure the transformation flow for the three methods. See also Figure 8 As shown in the three bar charts (representing the results of this invention, ERS, and SRS respectively), the method of this invention significantly reduces redundant conversion traffic, while SRS and ERS amplify the conversion traffic in continuous conversion operations. Overall, this invention reduces redundant conversion traffic by an average of 93.8% and 96.5% compared to ERS and SRS, respectively.

[0103] A.3 Experiments with different rack numbers:

[0104] Increase the number of racks in the system from 10 to 30 and measure the conversion flow rate for the three methods. See [link / reference] Figure 9 As shown in the three bar charts (representing the results of this invention, ERS, and SRS respectively), the conversion throughput of the method of this invention remains almost unchanged under different rack counts. The fundamental reason is that this invention maintains a relatively balanced block layout during continuous conversions, while ERS and SRS require additional overhead for layout adjustments. Compared to ERS and SRS, this invention reduces redundant conversion throughput by an average of 88.9% and 95.4%, respectively.

[0105] A.4 Experiments with different m values:

[0106] Measure the conversion flow rate at different values ​​of m. See [reference needed]. Figure 10 The experimental results shown in the three bar charts (representing the results of this invention, ERS, and SRS, respectively) indicate that the conversion traffic of the method described in this invention remains almost unchanged. This is because the invention aggregates the parity blocks of redundant conversion groups within the same rack; therefore, changes in the number of parity blocks do not alter the cross-rack transmission traffic during updates. As m increases, ERS and SRS require transmitting more data blocks for layout adjustments and reading more data blocks for parity updates. Compared to ERS and SRS, this invention reduces redundant conversion traffic by an average of 88.7% and 95.7%, respectively.

[0107] A.5 Load Balancing Experiment:

[0108] The load balancing ratio of the three methods was evaluated experimentally. See [link / reference] Figure 11 As shown in the three bar charts (representing the results of this invention, ERS, and SRS, respectively), the present invention effectively balances inter-rack load by employing a download bandwidth balancing layout to ensure balanced download bandwidth across racks. Simultaneously, careful selection of transition groups further balances the upload bandwidth within each rack. In all consecutive conversion operations, the average load balancing ratio of ERS and SRS was 2.2, while the method of this invention reduces the load balancing ratio to 1.1, which is closer to the optimal value (optimal value is 1).

[0109] B. Alibaba Cloud Environment Experiment

[0110] The prototype of this invention was deployed in an Alibaba Cloud environment, and its performance was evaluated to reveal its capabilities in a real-world cloud data center. The experiment used 19 virtual machine instances (ecs.g7.large), each equipped with 2 vCPUs (2.7GHz 3rd generation Intel Xeon Scalable processors) and 8GB of memory. The operating system was Ubuntu 18.04, and network bandwidth between any two instances was measured to be approximately 10Gb / s using iperf.

[0111] Of the 19 instances, one instance was deployed with the controller from the prototype to guide redundancy transition, while the remaining 18 instances ran the agents from the prototype, one agent per instance, with three agents forming a rack (i.e., six racks in total). Initially, 300 stripes encoded with erasure coding (6,3) were deployed across the 18 instances, with the value of k progressively increased to 15. The block size was set to a default of 64MB. The Linux tool tc was used to control the network bandwidth between racks. Each experiment was repeated five times, and the average redundancy transition time for each stretched strip was calculated. Error bars were also plotted to show the maximum and minimum values ​​(some of which may be too small) across all experiments.

[0112] B.1 Continuous Conversion Time Experiment:

[0113] See Figure 12 (a) (The three bar charts represent the results of the present invention, ERS, and SRS, respectively) show the experimental results of continuous redundancy conversion time, which shows that the method of the present invention can always achieve the minimum conversion time, on average 72.3% and 89.7% less than ERS and SRS, respectively.

[0114] B.2 Time Calculation Experiment:

[0115] Experiments were conducted to measure the total computation time required to generate redundant conversion schemes using the method of this invention under different rack and stripe numbers. The number of racks was increased from 10 to 40, and the number of stripes was increased from 1000 to 100000, considering (6,3,8) conversions. See [link to documentation]. Figure 12 (b) The results show that the running time of the conversion scheme generated by the present invention is always small and can be ignored compared with the transmission time.

[0116] B.3 Experiments with different block sizes:

[0117] Finally, the conversion time was measured by changing the block size from 16MB to 64MB. See also Figure 13 The experimental results shown (the three bar charts represent the results of this invention, ERS, and SRS, respectively) indicate that the conversion time of all three methods increases with increasing block size, because a larger block size amplifies the conversion throughput and storage I / O. Overall, compared to ERS and SRS, the average conversion time of this invention is reduced by 65.6% and 89.1%, respectively.

[0118] B.4 Experiments with different cross-rack transmission bandwidths:

[0119] Experiments were conducted to study the performance of the method of this invention under different cross-rack network transmission bandwidths, utilizing tc to increase the bandwidth from 1Gb / s to 3Gb / s. See also Figure 14 The experimental results shown (the three bar charts represent the results of this invention, ERS, and SRS, respectively) demonstrate that the method of this invention has the shortest conversion time under different bandwidth conditions. Furthermore, this invention exhibits better performance when cross-rack bandwidth is relatively scarce because it reduces cross-rack conversion communication and improves load balancing. Overall, compared to ERS and SRS, this invention reduces conversion time by an average of 56.7% and 87.0%, respectively.

[0120] This invention first analyzes the data layout scheme in redundancy conversion and proposes a striped layout to accelerate conversion, which helps reduce cross-rack traffic in continuous redundancy conversion. Then, it uses old parity blocks when recalculating new parity blocks and explores load balancing to accelerate conversion. Numerical simulations and experimental results in an Alibaba Cloud environment demonstrate that the conversion performance of this invention is excellent.

[0121] This invention also provides a computer device, including a program or instructions, which, when executed, are used to perform a cross-rack-aware erasure coding storage system balanced redundancy conversion method and any optional method provided in this invention.

[0122] This invention provides a storage medium including a program or instructions, which, when executed, are used to perform a cross-rack-aware erasure coding storage system balanced redundancy conversion method and any optional method provided in this invention.

[0123] Finally, it should be noted that those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0124] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for balanced redundancy conversion in a cross-rack-aware erasure coding storage system, characterized in that, include: The block layout formulation steps involve aggregating the verification blocks of all stripes within the redundancy conversion group into the same verification rack, and evenly distributing the data blocks of each stripe within the redundancy conversion group across various data racks; the stripes include stretched stripes and decomposed stripes. The verification block update step uses a decoupled approach of directly reading data blocks and decomposing strip verification blocks to update the verification blocks of the stretched strip; The load balancing process involves selecting redundant conversion groups from different verification racks to form execution groups, using a heuristic algorithm to iteratively replace execution groups, selecting the execution group with the smallest upload load balancing ratio, and then executing the redundant conversion groups in the execution group in parallel. The block layout formulation steps specifically include: Step 1.1, for including One stretch strip and The redundant conversion group of each decomposed strip aggregates the verification blocks of all strips in the group into the same inspection rack; Step 1.2: Distribute the data blocks of each stripe in the redundancy conversion group evenly across the data racks; the data blocks are also evenly distributed across the data racks; the data racks are racks other than the verification racks. Step 1.3: After the redundancy operation is initiated, assign priorities to each data rack based on the number of data blocks for each stripe in the data rack, and establish a priority queue to sequentially allocate data blocks to each data rack. Each decomposition band Data blocks are allocated to the first data rack in the priority queue according to the data rack order. One stretch strip, the remaining stretch strip is called the residual stretch strip; Step 1.4: Construct a network flow graph for the remaining data blocks and remaining stretch strips. In the network flow graph, each stretch strip, decomposition strip, and data rack is represented by a vertex. Vertices of decomposition strips are connected to directed edges pointing to vertices of data racks, and each vertex of a decomposition strip is connected to... The edge capacity of each rack vertex is determined by the number of data blocks in the corresponding rack for the decomposed stripe, indicating the number of data blocks in the decomposed stripe within that rack. The number of data blocks that can be provided in each data rack; the rack vertices are connected to directed edges pointing to the vertices of the stretch strips, and the edge capacity is determined by the number of data blocks that the stretch strip can receive in the corresponding rack; a source point is established, and the source point is connected to directed edges pointing to the vertices of each decomposed strip, and the edge capacity depends on the number of remaining data blocks in that decomposed strip; a sink point is established, and each vertex of the stretch strip is connected to a directed edge pointing to the sink point, and the edge capacity depends on the number of data blocks that the stretch strip needs to receive; Step 1.5: Run the Dinic algorithm to find the maximum flow, and allocate the remaining data blocks and remaining stretched strips in the decomposition strip based on the found maximum flow; The load balancing steps specifically include: Step 3.1: Group all redundant conversion groups according to the rack where the verification block is located, and group the verification blocks located in different racks. A combination of redundant conversion groups is called an execution group; among them... This represents the total number of racks. Step 3.2: Traverse the redundant conversion groups that have not been selected, and select the redundant conversion groups that have different verification racks from the current execution group and add them to the execution group; Step 3.3: Record the load of the execution group each time a new redundancy conversion group is added, and add upload load to the execution group based on the number of data blocks uploaded to each data rack of the redundancy conversion group; Step 3.4: Calculate the load balancing ratio of the execution group, which is the ratio of the maximum upload load of the execution group to the average upload load of the execution group; Step 3.5: Select the execution group with the smallest load balancing ratio, and all redundant conversion groups in the execution group perform redundant conversion in parallel; Step 3.6: Repeat steps 3.2 to 3.5 above until all redundant conversion groups have been selected.

2. The method for balanced redundancy conversion in a cross-rack-aware erasure coding storage system according to claim 1, characterized in that, Data racks containing fewer data blocks have higher priority.

3. The method for balanced redundancy conversion in a cross-rack-aware erasure coding storage system according to claim 1, characterized in that, For the remaining stretch strips, data blocks are allocated according to the priority queue until a fully balanced state is reached, meaning that the number of data blocks in each data rack is equal.

4. The method for balanced redundancy conversion in a cross-rack-aware erasure coding storage system according to claim 1, characterized in that, The number of directed edges includes strip.

5. The method for balanced redundancy conversion in a cross-rack-aware erasure coding storage system according to claim 1, characterized in that, The verification block update step specifically includes: Step 2.1, allocate the remaining tension strips Each data block is sent to the first verification node of the corresponding decomposed strip on the verification rack; Step 2.2: Calculate the required verification increment blocks for the remaining tensile strip based on the first verification node of the decomposed strip; the verification increment blocks include... One, will this Each verification increment block is sent to the verification nodes of the remaining stretched strips; Step 2.3: The first check node in the stripe decomposition forwards the read data block to the remaining check nodes. Each check node decouples the received data block from its local check block, calculating a total of [number missing]. A check increment block, which will... Each verification increment block is sent to the verification node of the previous stretch strip; Step 2.4: For each stretched strip, the verification node uses the received verification increment block to update the verification block stored in itself.

6. A cross-rack-aware erasure coding storage system redundancy leveling and conversion device, characterized in that, For implementing the method as described in any one of claims 1 to 5, comprising: The block layout specification module is used to aggregate the verification blocks of all stripes within the redundancy conversion group into the same verification rack, and to evenly distribute the data blocks of each stripe within the redundancy conversion group across various data racks; the stripes include stretched stripes and decomposed stripes. The verification block update module is used to update the verification block of the stretched strip by directly reading the data block and decoupling the strip verification block. The load balancing module is used to select different redundant conversion groups of the verification rack to form an execution group. It uses a heuristic algorithm to iteratively replace the execution groups and select the execution group with the smallest upload load balancing ratio. The redundant conversion groups in the execution group are then executed in parallel.

7. A computer device, characterized in that, Includes a program or instructions, which, when executed, cause the method described in any one of claims 1 to 5 to be performed.

8. A storage medium, characterized in that, Includes a program or instructions, which, when executed, cause the method described in any one of claims 1 to 5 to be performed.

Images