Method and system for improving wide-strip erasure code fault repair performance in distributed storage system

Abstract

The invention relates to the technical field of distributed storage systems, and discloses a method for improving the fault repair performance of a wide-band erasure code in a distributed storage system, which comprises the following steps of: initializing the system and identifying a fault, and acquiring fault block information and a system topology and bandwidth state; performing local group division based on the position of the fault block, and constructing a candidate node set for each local group; selecting an optimal relay node for each local group needing to be repaired; performing dynamic role allocation and adjustment on the repaired links in each local group; and executing a relay node-based hybrid repair task in each local group. According to the method for improving the fault repair performance of the wide-band erasure code in the distributed storage system, the relay node is introduced to shunt the pressure of the target node, parallel repair, dynamic optimization and repair of a link and global cooperative transmission optimization are realized by utilizing locality, the average repair time of a single block and a fault can be remarkably shortened, and the fault repair efficiency of the distributed storage system is improved. And the repair performance and reliability of the erasure code system are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of distributed storage system technology, and in particular to a method for significantly reducing repair time and improving system repair performance and reliability in a distributed storage system employing wide stripe erasure coding when data failure occurs, through relay node sensing, dynamic link adjustment and local group parallel repair mechanism. Background Technology

[0002] Cloud storage systems typically employ erasure coding technology, encoding data into striped data blocks and parity blocks as a low-cost redundancy method to achieve data reliability. Compared to traditional erasure coding, wide-striped erasure coding, proposed in recent years, achieves even lower storage redundancy (as low as 1.03 times) by significantly increasing the stripe width (e.g., k=64 to 150), and has become the mainstream technology to replace multi-replica strategies. However, while reducing storage redundancy, wide-striped coding also significantly increases the probability of data failure. Simulation studies show that when the stripe width k=64, due to batch node failures (such as rack power outages), the system faces pressure to repair individual blocks and failures. Therefore, how to efficiently repair these failures has become the core challenge in ensuring the data reliability of wide-striped storage systems.

[0003] Existing repair methods are mainly divided into two categories: centralized repair (CR) and independent repair (IR). CR downloads all surviving blocks to a new node for decoding, but the bandwidth of the central node can easily become a system bottleneck. IR runs multiple single-block repair pipelines in parallel, but surviving nodes need to repeatedly upload data, leading to bandwidth resource competition. Although existing hybrid repair strategies (such as HMBR) combine CR and IR, alleviating repair pressure to some extent, they still have significant drawbacks in wide-strip scenarios: First, the repair process heavily relies on the target node, making its bandwidth prone to becoming a system bottleneck; second, as the stripe width k increases, the efficiency of the CR task weakens, hindering the full realization of the overall advantages of hybrid repair.

[0004] Therefore, given the importance of wide-strip erasure coding repair and the challenges faced by existing methods, how to provide a method that can effectively reduce repair time and improve system repair performance has become an urgent technical problem to be solved. Summary of the Invention

[0005] In view of this, the present invention mainly addresses how to minimize the repair time and maximize the system repair performance and reliability during the wide stripe erasure coding fault repair process by introducing relay node awareness, dynamic link adjustment and local group parallel repair mechanisms.

[0006] This invention is implemented as follows: a method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system, the method comprising:

[0007] Step S1: System initialization and fault identification, obtaining fault block information and system topology and bandwidth status;

[0008] Step S2: Divide the local groups based on the location of the fault block, and construct a candidate node set for each local group;

[0009] Step S3: Select the optimal relay node for each local group that needs repair;

[0010] Step S4: Dynamically assign and adjust roles for the repair links within each local group;

[0011] Step S5: Execute a relay node-based hybrid repair task within each local group;

[0012] Step S6: Coordinate the local groups with the global check block to perform global collaborative data transmission;

[0013] Step S7: Calculate and output the final repair time to complete the repair process.

[0014] Furthermore, the system initialization and fault identification in step S1 includes: after the system detects a data block fault, it identifies the stripe and specific location of the faulty block, and at the same time collects the uplink and downlink bandwidth data of all relevant nodes in the system, as well as network topology information.

[0015] Furthermore, the local group division based on the fault block location in step S2 includes: grouping fault blocks belonging to the same local group together according to the joint locality (CL code) structure of erasure codes to form independent repair subtask units.

[0016] Furthermore, step S3, which involves selecting the optimal relay node for each local group requiring repair, includes:

[0017] S31: For the current local group, exclude the target node (i.e. the node that stores the block to be repaired) from all the live nodes to form a candidate node set;

[0018] S32: Traverse the set of candidate nodes and compare the download bandwidth of each node;

[0019] S33: Select the node with the largest download bandwidth as the repair relay node for this local group.

[0020] Furthermore, the dynamic role allocation and adjustment of the repair links within each local group in step S4 includes:

[0021] S41: Identify the node with the lowest download bandwidth within the local group (excluding relay nodes) and mark it as a "upload-only" tail node;

[0022] S42: Assign a specific transmission role to each node within the local group: relay node, tail node (upload only), or standard node (participates in uplink and downlink transmission normally).

[0023] S43: Based on role assignment, reconstruct and repair the data flow pipeline to ensure that low-bandwidth nodes do not become the bottleneck of IR tasks.

[0024] Furthermore, step S5, which involves performing a relay node-based hybrid repair task within each local group, includes:

[0025] S51: Divide each data block to be repaired into a CR sub-block of size p×B and an IR sub-block of size (1-p)×B;

[0026] S52: CR task: All surviving nodes send their respective CR sub-blocks to the relay node for centralized decoding;

[0027] S53: IR task: Surviving nodes transmit and decode IR sub-blocks in a pipeline manner through the adjusted pipeline;

[0028] S54: CR and IR tasks are executed in parallel within a local group.

[0029] Furthermore, the step S6 of coordinating the local groups and the global verification block to perform global collaborative data transmission includes:

[0030] S61: The relay nodes of each local group will transmit the locally repaired data blocks or intermediate results to the target node through the pipeline;

[0031] S62: The global verification node participates in the repair process and collaborates with the relay node to transmit the global verification information required for repair to the target node at the optimal ratio μ.

[0032] S63: The target node receives data from the relay node and the global verification node, and completes the final reconstruction of all faulty blocks.

[0033] Furthermore, the calculation and output of the final repair time in step S7 includes: the final time of the entire repair process is the maximum value of the repair time of all local groups and the global collaborative transmission time, i.e., TRAHR = max( max(Tlocali), Tglobal ).

[0034] To address the technical challenges of low repair efficiency, target node bandwidth bottlenecks, and uneven network resource utilization in existing wide-strip erasure coding fault repair technologies, this invention proposes an innovative method to improve repair performance. Compared with existing technologies, the technical solution of this invention achieves breakthrough progress in repair efficiency and resource utilization through systematic architecture optimization and intelligent scheduling mechanisms.

[0035] 1. This invention innovatively introduces a relay node traffic offloading mechanism, fundamentally solving the bandwidth bottleneck problem of target nodes in wideband repair. The performance of existing hybrid repair strategies (such as HMBR) is severely limited by the download bandwidth of the target node. This invention intelligently selects the surviving node with the highest download bandwidth in the system as a relay node, successfully offloading and diverting the centralized decoding (CR) task data traffic originally borne by the target node. This not only directly avoids the target node becoming a system bottleneck but also fully utilizes the high-bandwidth node resources that are not effectively used in the system, so that the execution efficiency of the CR task is no longer limited by the performance limit of a single node.

[0036] 2. This invention deeply explores and utilizes the joint locality property of erasure codes to achieve refined parallel processing of repair tasks. Traditional global repair methods are inefficient in complex scenarios with concurrent faults across wide stripes. Based on the inherent local group structure of CL codes, this invention innovatively decomposes the global repair task into multiple independent, local group-based repair subtasks. These subtasks can be executed in parallel, thus transforming a large-scale repair problem into multiple small-scale, concurrently processable sub-problems. This significantly shortens the overall system recovery time from faults, with particularly pronounced advantages in large-scale concurrent fault scenarios.

[0037] 3. A dynamic link role allocation mechanism is proposed, effectively eliminating performance bottlenecks in the repair pipeline. In Independent Repair (IR) tasks, the slowest node on the link determines the overall pipeline transmission speed. This invention intelligently identifies and locates low-bandwidth nodes in local groups by dynamically sensing real-time node bandwidth and adjusts their role to "upload only," preventing them from participating in downstream data downloads. This innovative role adjustment restructures the data flow pipeline, completely avoiding the tailing effect caused by low-bandwidth nodes on IR tasks, thereby improving the overall data throughput and transmission efficiency of the pipeline.

[0038] 4. A global collaborative transmission optimization mechanism was designed to achieve optimal allocation of system-level repair resources. For repair scenarios involving global verification blocks, this invention no longer simply aggregates data to the target node, but constructs a highly efficient collaborative transmission network involving relay nodes and global verification nodes. Through theoretical modeling and optimization, the optimal data allocation ratio μ between relay nodes and global verification nodes was determined, balancing the data transmission time from both to the target node, minimizing transmission latency in the global phase, and achieving efficient collaboration of overall system repair resources.

[0039] Through large-scale simulation experiments and prototype system testing, the method described in this invention (RAHR) was compared with the current state-of-the-art HMBR method. The experimental results fully demonstrate the inventive technical effectiveness of this invention: in typical single-block fault repair scenarios, the average repair time is reduced by 29%; in more challenging concurrent fault repair scenarios, repair performance is improved by 37%. These data not only verify the effectiveness of each technical module of this invention but also demonstrate the synergistic amplification effect brought about by their combination. This invention systematically solves the efficiency bottleneck problem faced by wide-striped erasure coding in dealing with faults, providing key technical support for building next-generation high-performance, highly reliable distributed storage systems.

[0040] The expected benefits and commercial value of the technical solution after its transformation are as follows: This technical solution can be directly applied to various cloud storage platforms, distributed file systems, and big data storage infrastructures, demonstrating significant commercial transformation potential. By improving the repair performance of wide-strip erasure coding, it can bring the following core values ​​to enterprises:

[0041] Reduce operating costs: Repair time is reduced by approximately 30%, directly reducing the time window for the system to operate in a degraded state, thus reducing the risk of data loss and the resulting business losses;

[0042] Improve resource utilization: By using relay nodes and dynamic link scheduling, bandwidth resource idleness and competition are avoided, and network resource utilization is improved by more than 20%.

[0043] Enhanced system reliability: Especially suitable for rack-level and node-level multi-fault concurrent scenarios, and can meet the stringent requirements of high-reliability industries such as finance and healthcare for storage system availability;

[0044] Drive storage architecture upgrades: Provide key technical support for wide stripe deployments with lower redundancy (e.g., 1.03 times) in hyperscale data centers, helping to further reduce the total cost of ownership of storage.

[0045] The technical solution of this invention fills a technological gap in the industry both domestically and internationally: Current research in the field of wide-strip erasure coding repair largely focuses on single-block repair optimization or static strategy adjustments within a hybrid repair framework. There is a lack of a systematic solution that combines relay node selection, local group parallel repair, dynamic link role allocation, and reinforcement learning adaptive adjustment. This invention proposes for the first time a "hybrid repair architecture centered on relay nodes," forming a closed-loop optimization system from local repair to global collaboration. This fills the technological gap in system-level methods for efficient wide-strip fault repair and is at the forefront of the industry.

[0046] The technical solution of this invention solves a long-standing but unresolved technical problem: in wide-strip erasure coding applications, "how to significantly improve fault repair efficiency without increasing redundancy" has been a common challenge faced by both industry and academia. Existing methods, such as HMBR, although attempting to combine CR and IR, still cannot overcome core obstacles such as the target node bandwidth bottleneck and the low-bandwidth node tailing effect. This invention, through relay splitting and dynamic role reconfiguration, fundamentally solves the above problems, achieving a leapfrog improvement in repair efficiency and accomplishing the goal that previous technical approaches failed to achieve.

[0047] The technical solution of this invention overcomes a long-standing technical bias: There is a prevalent bias in the industry that improving erasure coding repair efficiency must be achieved by increasing storage redundancy or sacrificing some data locality. This invention breaks this established perception, demonstrating that by introducing intelligent relays and dynamic link optimization at the scheduling level, a significant improvement in repair performance can be achieved without changing the coding structure or increasing redundancy. Furthermore, traditional views hold that AI methods such as reinforcement learning are too costly to deploy in low-level systems like storage repair. This invention, through lightweight models and key parameter optimization, achieves adaptive repair in complex environments, providing a feasible engineering practice path for AI-enabled storage systems. Attached Figure Description

[0048] Figure 1 A flowchart illustrating the overall process of a method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system, provided by an embodiment of the present invention.

[0049] Figure 2 A flowchart of a relay node selection algorithm provided for an embodiment of the present invention;

[0050] Figure 3 A flowchart of the dynamic link adjustment algorithm provided for embodiments of the present invention;

[0051] Figure 4 A schematic diagram illustrating the execution of a hybrid repair task within a single local group, provided for an embodiment of the present invention;

[0052] Figure 5 A schematic diagram illustrating multi-local group parallel repair and global collaborative transmission provided for embodiments of the present invention;

[0053] Figure 6 is a schematic diagram of data allocation ratio optimization in global collaborative transmission provided by an embodiment of the present invention;

[0054] Figure 7 A bar chart comparing the repair times of RAHR and HMBR under different numbers of faulty blocks;

[0055] Figure 8Heatmap showing the percentage performance improvement of RAHR compared to HMBR;

[0056] Figure 9 A three-dimensional surface plot showing the impact of z / r ratio and number of faulty blocks on RAHR performance improvement. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0058] Figure 1 This is a flowchart illustrating the overall process of a method for improving the fault repair performance of wide-striped erasure coding in a distributed storage system, according to an embodiment of the present invention. Figure 1 As shown, the method provided by the embodiments of the present invention includes:

[0059] Step S1: System Initialization and Fault Identification. The system continuously monitors the status of data blocks. When a faulty block is detected, it immediately identifies the stripe and local group to which it belongs, and collects the real-time uplink bandwidth Ui and downlink bandwidth Di information of all relevant nodes in the current system topology.

[0060] Step S2: Local Grouping. Based on the predefined structure of the CL code, faulty blocks are categorized according to their local groups. Each local group containing faulty blocks will be processed as an independent repair unit.

[0061] Step S3: Relay Node Selection. The relay node selection algorithm is executed independently for each local group requiring repair (corresponding to...). Figure 2 ).

[0062] Step S4: Dynamic Link Adjustment. Within each local group, the dynamic link adjustment algorithm is executed (corresponding to...). Figure 3 Assign transmission roles to nodes and optimize the IR task pipeline.

[0063] Step S5: Local Group Hybrid Repair. Each local group executes its internal hybrid repair task in parallel (CR and IR in parallel). The relay node undertakes data aggregation and preliminary calculation tasks in this stage (corresponding to...). Figure 4 ).

[0064] Step S6: Global Cooperative Transmission. Relay nodes in each local group transmit the repair results to the target node, while global verification nodes participate in data transmission at the optimal ratio μ, jointly completing the final repair (corresponding to...). Figure 5 (Figure 6).

[0065] Step S7: Repair Completed and Performance Output. The system calculates and records the total repair time (T RAHR), completing the reconstruction of all faulty blocks.

[0066] This invention addresses the problems of high bandwidth consumption, high repair latency, and uneven link utilization in the repair of wide-striped erasure codes (such as CL codes) in distributed storage systems under node failure scenarios. It proposes a group-based collaborative hybrid repair method for heterogeneous bandwidth environments. The core idea is to fully exploit the inherent bandwidth potential of the system and reduce the overall repair time by combining local group parallel repair, relay collaboration, and dynamic link scheduling while maintaining the reliability of erasure codes.

[0067] During system operation, a continuous monitoring mechanism is first used to achieve real-time awareness of the data block status. Once a data block failure is detected, the system can quickly locate the stripe to which the block belongs and its local group position within the stripe, and simultaneously obtain the uplink and downlink bandwidth status of each participating node in the current network topology. This information provides the basis for adaptive optimization of subsequent repair strategies.

[0068] Based on the hierarchical and locally recoverable structure of CL codes, the system divides fault blocks into local groups, making each local group a relatively independent repair unit. This not only reduces the data range involved in a single repair but also naturally supports parallel repair between multiple local groups, reducing repair latency at the structural level.

[0069] For each local group requiring repair, the system selects relay nodes based on node bandwidth conditions, prioritizing nodes with high overall transmission capabilities as data relays and initial calculation nodes to avoid concentrating repair traffic on bandwidth bottleneck nodes. Furthermore, a dynamic link adjustment mechanism flexibly allocates transmission roles to each node, rationally splitting IR (relay repair) and CR (direct repair) tasks, enabling a pipelined execution of the repair process and maximizing bandwidth utilization.

[0070] During the local repair phase, each local group executes a hybrid repair strategy in parallel, with relay nodes undertaking data aggregation and some computational tasks, effectively reducing the instantaneous reception pressure on the target nodes. Subsequently, in the global coordination phase, relay nodes from each local group and the global verification node jointly complete the remaining data transmission and verification recovery according to the optimal participation ratio, achieving a balance between local repair and global consistency.

[0071] Finally, the system calculates and outputs the total time for this repair, and completes the reconstruction of all fault blocks, thereby significantly improving repair efficiency and overall system performance in wide stripe erasure coding scenarios.

[0072] Figure 2A flowchart of a relay node selection algorithm provided according to an embodiment of the present invention. In a preferred embodiment of the present invention, such as Figure 2 As shown, the relay node selection algorithm specifically includes:

[0073] S31: Enter the list of nodes in the current local group and the target node;

[0074] S32: Initialize an empty set of candidate nodes;

[0075] S33: Traverse all nodes within the local group. If a node is not the target node, add it to the candidate set.

[0076] S34: Temporarily designate the first node in the candidate set as the relay node;

[0077] S35: Traverse all nodes in the candidate set and compare their download bandwidth Di;

[0078] S36: Select the node with the largest download bandwidth as the final relay node;

[0079] S37: Output relay node.

[0080] The core idea of ​​this algorithm is that the CR task requires relay nodes to receive data from multiple source nodes, and their download bandwidth is most likely to become a bottleneck. Therefore, maximizing the download bandwidth of relay nodes is the most direct way to optimize the performance of the CR task.

[0081] The relay node selection algorithm provided by this invention addresses the high sensitivity of the CR (Cooperative Repair) task to network bandwidth during wide-strip erasure coding local repair process. It optimizes the selection of relay nodes from the perspective of node download bandwidth, aiming to avoid the formation of an overall system performance bottleneck due to insufficient relay node bandwidth during the repair process.

[0082] When a node fails in a distributed storage system, each local group containing the failed block needs to independently complete the repair operation. In CR repair mode, the relay node needs to receive the data fragments required for repair from multiple surviving nodes simultaneously. Therefore, this node exhibits a clear "multiple-input, single-output" data flow characteristic during the repair phase, with its download link often bearing the largest instantaneous load. If the relay node's download bandwidth is insufficient, even if other nodes have high upload capabilities, the overall repair process will still be forced to wait, thus prolonging the repair time.

[0083] Based on the above understanding, in the relay node selection phase, this invention first uses the current local group as the scope of action, considering all nodes within the group that can participate in repair as candidate objects, and explicitly excludes the target node itself to avoid conflicts between repair and receiving roles. By constructing a candidate node set, the relay node selection process is localized and lightweight, reducing computational complexity and facilitating parallel execution.

[0084] Subsequently, the algorithm uses the download bandwidth of nodes in the candidate set as the core evaluation metric, and compares them according to a simple and effective maximum value criterion, that is, selecting the node with the largest download bandwidth among all candidate nodes as the final relay node. The advantage of this strategy is that it does not rely on complex global network modeling or prediction, but only uses real-time available bandwidth status information to ensure that the selected relay node has the strongest data receiving capability in the current local group repair phase, thereby supporting multi-source concurrent transmission.

[0085] By maximizing the download bandwidth of relay nodes, data from upstream nodes can be aggregated to relay nodes more quickly, reducing queuing time and improving the parallelism and pipeline efficiency of CR tasks. At the same time, this selection strategy is naturally compatible with subsequent dynamic link adjustments and hybrid repair mechanisms, ensuring that relay nodes neither become bottlenecks in local repair nor cause unnecessary congestion to other links in the system.

[0086] In summary, this relay node selection algorithm achieves precise optimization of the key bottleneck in CR repair through the principle of simplicity and deterministic bandwidth maximization, providing important basic support for shortening the repair latency of wide-strip erasure codes.

[0087] Figure 3 A flowchart of a dynamic link adjustment algorithm provided according to an embodiment of the present invention. In a preferred embodiment of the present invention, such as Figure 3 As shown, the dynamic link adjustment algorithm specifically includes:

[0088] S41: Enter the list of nodes in the current local group and the selected relay nodes;

[0089] S42: Initialize the tail node as the first node in the local group (excluding the relay node itself).

[0090] S43: Traverse all nodes in the local group (excluding relay nodes), find the node with the minimum download bandwidth, and mark it as the tail node;

[0091] S44: Begin assigning roles to each node;

[0092] S45-S47: If the node is a tail node, set its role to "TAILUPLOADONLY" (upload only).

[0093] S48-S410: If the node is a relay node, set its role to "RELAY" (relay);

[0094] S411-S413: Set the role of all other nodes to "STANDARD".

[0095] S414: Output the list of nodes after role assignment.

[0096] This algorithm ensures that the node with the lowest download bandwidth no longer participates in data downloading in the IR task, but is only responsible for uploading its own data, thus preventing that node from becoming a bottleneck in the IR pipeline.

[0097] The dynamic link adjustment algorithm proposed in this invention aims to solve the problem of pipeline imbalance caused by heterogeneous node bandwidth during the local repair process of wide-strip erasure coding. By adaptively dividing the communication roles of nodes within a local group, it avoids weak-bandwidth nodes from slowing down the overall repair process, thereby improving repair efficiency and system stability.

[0098] After the local group determines the relay node, the repair process will inevitably involve multiple data flow patterns: on the one hand, some nodes need to upload repair data to the relay node; on the other hand, some nodes need to download and forward data from other nodes or the relay node, forming an IR pipeline structure. During this process, if any node with weak download capabilities is incorrectly assigned to a role requiring heavy downloads, it will cause link congestion, ultimately affecting the repair performance of the entire local group.

[0099] To address the aforementioned issues, this algorithm first takes the current local group as the unit, inputting the list of nodes within that group and the information of the identified relay nodes. During the initialization phase, relay nodes themselves are excluded to avoid being misclassified as tail nodes. Subsequently, the algorithm iterates through all non-relay nodes within the local group, comparing the download bandwidth of each node in real time, identifying the node with the lowest download bandwidth, and marking it as the tail node. This tail node is the least suitable for handling download tasks under the current network conditions and is the node most likely to become a performance bottleneck in the IR pipeline.

[0100] After identifying tail nodes, the algorithm enters the role assignment phase. Different types of nodes are assigned specific and mutually exclusive communication roles. Nodes identified as tail nodes are set to "upload-only," meaning they only send their own data blocks to other nodes or relay nodes and no longer participate in any operations requiring data download and forwarding. This strategy effectively reduces the burden on weak download nodes in the IR task, preventing them from blocking the pipeline due to limited download speed.

[0101] For identified relay nodes, they are uniformly assigned the "relay" role, allowing them to focus on receiving data concurrently from multiple nodes, performing necessary data aggregation or computation operations, and providing efficient support for subsequent CR and global repair phases. Other nodes, besides tail nodes and relay nodes, are set to the "standard" role, which can participate in downloading and also undertake some forwarding tasks, thereby enhancing the overall flexibility and parallelism of the IR pipeline.

[0102] Through the above methods, this dynamic link adjustment algorithm achieves a refined role division based on node download capabilities, transforming the IR repair process from a coarse-grained mode of "all nodes participating equally" to an adaptive mode of "cooperative division of labor based on capabilities." This mechanism effectively avoids the negative impact of weak-bandwidth nodes on repair performance without increasing additional communication overhead, significantly improving the stability, pipeline efficiency, and overall repair speed of local repair phases, providing crucial technical support for the efficient fault tolerance of wide-strip erasure coding systems.

[0103] Figure 4 This explains the core idea of ​​the RAHR repair mechanism of this invention. In this example, rack1 is selected as the relay node due to its higher download bandwidth. By introducing a relay node, rack4 is no longer the bottleneck for the CR task repair. Simultaneously, by dynamically adjusting rack3, which has the lowest download bandwidth, to be the "TAILUPLOADONLY" transmission link, rack3 can be prevented from becoming the bottleneck for the IR task. The repair process is as follows: the hybrid repair task of this invention is executed between rack1, rack2, and rack3. Finally, rack1 transmits the repaired data to the target rack rack4 via a pipeline. The repair time is then calculated.

[0104] The calculation formula is: , where (i, j) are adjacent node pairs in the pipeline. This refers to the actual bandwidth between the two (one-to-one mode).

[0105] The calculation formula is: Where p is the CR sub-block ratio and B is the block size. For the survival rack Actual bandwidth to the relay node (many-to-single mode).

[0106] Through calculation The optimal CR sub-block ratio p can be obtained, and the time for the mixed repair stage can be determined. .

[0107] Simultaneously, the pipeline transfer time between rack1 and rack4 is calculated. .

[0108] Therefore, the total repair time in this single-block failure scenario The maximum value between the mixed repair time and the pipeline transfer time, i.e. This example demonstrates how relay nodes can reduce the load on target nodes and how dynamic link adjustments can optimize recovery time.

[0109] Figure 5 This invention demonstrates the mechanism for handling concurrent data block failures in the repair process. In this scenario, each local group independently selects a rack with the highest download bandwidth as its relay node, allowing the mixed repair process between different local groups to be executed in parallel (step S5). During the repair process of each local group, its relay nodes (Relay 1, Relay 2) coordinate to pipeline the data to the target rack. The Global Parity block also participates in this process, collaborating with the relay nodes to transmit data to the target node.

[0110] The total repair time of the system is determined by the repair time of the slowest local group, i.e. ,in The time for repairing g local groups is given. This mechanism effectively improves fault repair efficiency through parallel repair of local groups and relay coordination.

[0111] Figure 6 is a schematic diagram of the data allocation ratio optimization in global collaborative transmission provided by the embodiment of the present invention (step S6). In the data block fault repair scenario, the data of each local group is finally reconstructed by the relay node in conjunction with the global verification node. Figure 6(a) quantitatively shows the data block size required to be transmitted by each relay node and the global verification node, providing a basis for the optimized allocation of transmission load.

[0112] Based on this transmission load model, Figure 6(b) further illustrates the core mechanism (step S62) of minimizing global transmission time through dynamic data allocation. This is achieved by dynamically dividing the data block to be forwarded by each relay node into two parts: a portion (μ) of the data is directly sent to the target node, while the remaining portion (1-μ) is sent to the global verification node. By establishing and solving the transmission time model, the download time of the target node is determined. Download time with global verification node When the allocation ratios are equal, the corresponding allocation ratio μ0 is the optimal solution for the system. This strategy ensures the total system repair time. The theoretical minimum value is reached, thus efficiently completing the final data reconstruction in step S63.

[0113] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described above.

[0114] This terminal device can be deployed on the management node of a distributed storage system to coordinate and execute the entire repair process and achieve corresponding beneficial results.

[0115] This invention verifies the technical effectiveness of the RAHR (Relay-Aware Hybrid Repair) method in a real distributed storage environment through extensive simulation experiments and prototype system testing. The experiments use the CL (Combined Locality) code proposed by ECWide as the basic coding structure, and the test environment simulates the hierarchical network topology of a typical cloud storage system. To comprehensively evaluate the performance advantages of this invention, various scenarios ranging from single-block failures to concurrent failures were set up in the experiments, and a systematic comparison was made with the current best HMBR (Hybrid Multi-block Repair) method. The following is a summary of the findings. Figures 7 to 9 Evidence of the technical effects of the present invention in different dimensions is presented separately.

[0116] 1. Figure 7 Bar chart comparing RAHR and HMBR repair times under different numbers of faulty blocks.

[0117] Figure 7 A direct comparison was made between the average repair times of the RAHR and HMBR methods under five CL code configurations with varying numbers of faulty blocks f (f=1 to 4). The bar chart shows that in all test scenarios, the repair time of RAHR was significantly lower than that of HMBR. For example, in the single-block fault (f=1) scenario, RAHR reduced the repair time by an average of 29% compared to HMBR, and the reduction reached 32.8% in the ultra-wide stripe configuration CL(136,128,27,34). As the number of faulty blocks increased (f=2,3,4), the repair time of RAHR increased more gradually, while the repair time of HMBR increased sharply. Especially at f=3, the repair time of RAHR in the CL(73,64,11,19) configuration was only 0.317 seconds, while that of HMBR was 0.546 seconds, representing a 41.9% improvement in repair efficiency. The data in the figure strongly demonstrates that the present invention significantly reduces the repair time for single-block and concurrent faults through relay node diversion, local group parallel repair, and global collaborative transmission mechanisms.

[0118] 2. Figure 8 Heatmap of RAHR's percentage performance improvement over HMBR

[0119] Figure 8The performance improvement percentage of RAHR compared to HMBR is presented in the form of a heatmap under different CL code configurations and the number of faulty blocks. The figures clearly show that RAHR achieves performance improvement across all test configurations in single-block to four-block fault scenarios, with an improvement rate consistently around 31%. For example, in the CL(56,48,11,14) configuration and f=3 scenario, the improvement rate reaches 32.0%; in the CL(136,128,27,34) configuration and f=1 scenario, the improvement rate is 32.9%. The color gradient and numerical distribution of the heatmap indicate that the performance improvement effect of RAHR remains stable and significant under different coding structures and fault scales, verifying the broad adaptability and robustness of the method of this invention to various wide-strip scenarios.

[0120] 3. Figure 9 3D surface plot showing the impact of z / r ratio and number of faulty blocks on RAHR performance improvement

[0121] Figure 9 This study reveals how the CL code structure parameters (z / r ratio) and the number of faulty blocks (f) jointly influence the performance improvement of RAHR compared to HMBR. Three-dimensional surface plots show that when the z / r ratio is low (e.g., 0.6-0.8), RAHR mainly relies on dynamic link adjustment mechanisms, with an improvement rate of approximately 28%-34%. As the z / r ratio increases to a moderate range (0.8-1.2), the performance improvement jumps significantly, exceeding 40% at f=3. In wide-strip scenarios with high z / r ratios (>1.2), the performance improvement stabilizes in the 40%-45% range, indicating that RAHR's local group parallel repair and relay node scheduling mechanisms have a more significant advantage in environments with high cross-rack complexity. The surface plot results demonstrate that the proposed method can adaptively utilize the coding locality structure, achieving significant performance optimization under various storage topologies, and is particularly suitable for current mainstream wide-strip distributed storage systems.

[0122] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0123] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system, characterized in that, The method includes: Step S1: System initialization and fault identification, obtaining fault block information and system topology and bandwidth status; Step S2: Divide the local groups based on the location of the fault block, and construct a candidate node set for each local group; Step S3: Select the optimal relay node for each local group that needs repair; Step S4: Dynamically assign and adjust roles for the repair links within each local group; Step S5: Execute a relay node-based hybrid repair task within each local group; Step S6: Coordinate the local groups with the global check block to perform global collaborative data transmission; Step S7: Calculate and output the final repair time to complete the repair process.

2. The method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system according to claim 1, characterized in that, The system initialization and fault identification in step S1 include: After the system detects a data block fault, it identifies the stripe and specific location of the faulty block, and at the same time collects the uplink bandwidth Ui and downlink bandwidth Di data of all relevant nodes in the system, as well as network topology information.

3. The method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system according to claim 1, characterized in that, The local group division based on the fault block location in step S2 includes: Based on the joint locality CL code structure of erasure codes, fault blocks belonging to the same local group are grouped together to form independent repair subtask units.

4. The method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system according to claim 1, characterized in that, The step S3, which involves selecting the optimal relay node for each local group requiring repair, includes: S31: For the current local group, exclude the target node from all surviving nodes to form a candidate node set; S32: Traverse the set of candidate nodes and compare the download bandwidth of each node; S33: Select the node with the largest download bandwidth as the repair relay node for this local group.

5. The method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system according to claim 1, characterized in that, The dynamic role allocation and adjustment of the repair links within each local group in step S4 includes: S41: After excluding relay nodes within the local group, identify the node with the lowest download bandwidth and mark it as a "upload only" tail node; S42: Assign a transmission role to each node within the local group: relay node, tail node, or standard node; S43: Reconstruct and repair the data flow pipeline based on role assignment.

6. The method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system according to claim 1, characterized in that, The step S5, which involves performing a relay node-based hybrid repair task within each local group, includes: S51: Divide each data block to be repaired into a CR sub-block of size p×B and an IR sub-block of size (1-p)×B; S52: CR task: All surviving nodes send their respective CR sub-blocks to the relay node for centralized decoding; S53: IR task: Surviving nodes transmit and decode IR sub-blocks in a pipeline manner through the adjusted pipeline; S54: CR and IR tasks are executed in parallel within a local group.

7. The method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system according to claim 1, characterized in that, The step S6, which involves coordinating the local groups and the global verification block to perform global collaborative data transmission, includes: S61: The relay nodes of each local group will transmit the locally repaired data blocks or intermediate results to the target node through the pipeline; S62: The global verification node participates in the repair process and collaborates with the relay node to transmit the global verification information required for repair to the target node at the optimal ratio μ. S63: The target node receives data from the relay node and the global verification node, and completes the final reconstruction of all faulty blocks.

8. The method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system according to claim 1, characterized in that, The calculation and output of the final repair time in step S7 includes: The final time of the entire repair process is the maximum value of the repair time of all local groups and the global collaborative transmission time, i.e., TRAHR = max(max(Tlocal group i), Tglobal).

9. The method for improving the fault repair performance of wide stripe erasure coding in a distributed storage system according to claim 1, characterized in that, This method introduces a lightweight reinforcement learning model to dynamically adjust the CR sub-block ratio parameter p and the relay node selection strategy based on the real-time network status.

10. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 9.

Images