A method for adaptive repair of RS code storage nodes based on subspace chain

Abstract

The application discloses a RS code storage node self-adaptive repairing method based on a subspace chain and belongs to the technical field of information and communication. The method selects a subspace on a finite field extension, constructs a coset based on mutually different base field elements to form an RS code value point set, and constructs a nested subspace chain to determine a unified response base. When a certain node is invalid, each participating node outputs a response symbol on the base field according to a predetermined order to form a streaming response sequence which does not depend on a pre-fixed complete help node set. A replacement node selects a repairing mode according to the number of effective help groups, the response condition of a host group and the number of response symbols, and accurately recovers the invalid coding symbol through linear combination and post-processing.

Description

Technical Field

[0001] This invention belongs to the fields of information and communication technology, coding theory, and distributed storage technology, and specifically relates to an adaptive repair method for distributed storage nodes. Background Technology

[0002] With the rapid development of technologies such as cloud storage, data centers, and edge computing, massive amounts of data are typically distributed across multiple nodes in encoded form to improve system reliability, fault tolerance, and storage efficiency. RS codes, due to their maximum distance separability, are widely used in distributed storage systems. When a storage node fails, data from the remaining surviving nodes is used to recover the encoded symbols from the failed node.

[0003] Among existing RS code node repair methods, the most direct approach is to contact surviving nodes of at least the code dimension and download their complete storage content to recover the failed symbols. While this method is simple to implement and highly flexible, it incurs significant download overhead. To reduce repair bandwidth, existing research has proposed low-bandwidth repair methods based on trace functions and subspace polynomials, enabling replacement nodes to complete accurate repairs by downloading only a portion of the sub-symbol information.

[0004] However, most existing low-bandwidth repair solutions are typically designed for a pre-defined set of helper nodes, and the data returned by these helper nodes often depends on the complete set of helper nodes. In real-world distributed storage systems, when node availability changes dynamically, some nodes respond slowly, or some helper groups fail to return complete responses, replacement nodes struggle to flexibly switch repair modes based on the currently received valid responses, thus impacting repair efficiency and dynamic adaptability. Therefore, it is necessary to propose an adaptive node repair method that does not rely on a pre-fixed set of complete helper nodes. This method would allow replacement nodes to adaptively select the repair mode based on the actual number of valid helper groups and response items received, achieving a flexible trade-off between repair bandwidth, single-node download volume, and waiting time. Summary of the Invention

[0005] The purpose of this invention is to address the problem that existing low-bandwidth repair methods for RS codes typically rely on a pre-fixed set of helper nodes and struggle to adapt to dynamic changes in node availability. This invention proposes an adaptive repair method for distributed storage nodes based on RS codes and nested subspace chains. This method constructs a set of RS code value points consisting of multiple cosets, divides this set into multiple disjoint groups, and combines nested subspace chains and a unified streaming response mechanism. This allows replacement nodes to adaptively select a repair mode based on the actual number of valid helper groups and response symbols received, even when the set of helper nodes is not pre-fixed, thereby restoring the encoded symbols of failed nodes.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] An adaptive repair method for distributed storage nodes includes:

[0008] In finite field extension Select one - Subspace Based on several distinct base field elements, multiple cosets are constructed to obtain the set of values ​​for the RS code. This set of values ​​is then divided into multiple disjoint groups, where each group of values ​​represents a base field element in the subspace. Translation;

[0009] In the extended field of the finite field Construct multiple arrays arranged in ascending order of dimension. - Subspaces are formed into nested subspace chains, and a unified response basis is determined based on the nesting relationship between the image spaces of the corresponding subspace polynomials, so that different recovery levels adopt a unified response order;

[0010] The data to be stored is encoded using RS codes and then stored in the corresponding storage nodes in the corresponding groups. Each storage node corresponds to a value point in the set of value points and stores the corresponding encoding symbol.

[0011] When a storage node fails, the group to which the failed node belongs is defined as the host group, and the replacement node initiates a repair request to the remaining surviving nodes. During the repair process, each participating node outputs response symbols on multiple base fields in a predetermined order based on the encoded symbols, corresponding value points, and the unified response base, forming a unified streaming response sequence, wherein the streaming response does not depend on a pre-fixed complete set of helper nodes.

[0012] The replacement node selects a target repair mode that meets the current conditions from a number of pre-set repair modes based on the number of valid help groups received, the response status of the host group, and the number of response items returned by each participating node. The definition of a valid help group is: when all nodes in a certain group, except for the host group, participate in the response and return data to the replacement node during a node repair process, the group is defined as a valid help group.

[0013] The replacement node uses the subspace polynomial, pre-calculated coefficients, trace function relationship and basis expansion relationship corresponding to the selected target repair mode to perform linear combination and post-processing on the received streaming response to restore the coded symbol corresponding to the failed node.

[0014] Furthermore, the subspace Extending a finite field On dimension - subspace, where It is a positive integer and satisfies as well as .

[0015] Furthermore, the subspace and They intersect only at the zero element, and for any... All have .

[0016] Furthermore, the set of value points consists of several cosets, the first... The set of values ​​for each group is represented as:

[0017]

[0018] in for The elements in.

[0019] Furthermore, the unified streaming response sequence consists of response symbols on multiple base domains, and the response space required for different repair levels is spanned by the first few terms of the unified response base.

[0020] Furthermore, the replacement node adaptively determines the target repair mode based on the number of effective help groups, the host group response status, and the number of response items returned by each participating node.

[0021] Furthermore, the set of value points is designed to ensure that the relevant dual code multipliers satisfy the base field processable condition, so that after the actual effective help group is determined, the replacement node can perform scalar adjustment and post-processing on the received response to restore the coded symbol corresponding to the failed node.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1) This invention constructs a grouped set of RS code value points and combines it with nested subspace chains, enabling multiple repair modes to be implemented uniformly within the same streaming response framework. Replacement nodes do not need to pre-fix a complete set of helper nodes; they can select the appropriate repair mode based on real-time reception, thereby improving adaptability in dynamic distributed storage environments.

[0024] 2) In this invention, each participating node returns response symbols on multiple base domains, rather than complete encoded symbols, which can effectively reduce the download overhead during the repair process of failed nodes. At the same time, this invention allows for flexible trade-offs between the number of participating nodes, the download volume per node, and the total repair bandwidth as the number of effective help groups changes.

[0025] 3) This invention adopts a unified streaming response mechanism. Once the replacement node meets the repair conditions, it can stop receiving and perform recovery. Therefore, in cases where the node response speed is inconsistent or the availability of some nodes changes dynamically, it is beneficial to shorten the average waiting time and improve the repair efficiency.

[0026] 4) This invention is applicable to the accurate repair of failed nodes in RS-encoded distributed storage systems. It can achieve adaptive repair even when the set of helper nodes is not fixed in advance, and has good practicality and scalability. Attached Figure Description

[0027] Figure 1 is a flowchart of the overall process of the adaptive repair method for distributed storage nodes based on RS code and nested subspace chains according to the present invention.

[0028] Figure 2 This is a schematic diagram of the RS code value point set and node structure in this invention;

[0029] Figure 3 is a schematic diagram of adaptive repair when the present invention is applied to a distributed storage node repair scenario. Detailed Implementation

[0030] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0031] The core idea of ​​this invention is as follows: First, a set of RS code value points with a grouping structure is constructed on a finite field extension field; second, multiple subspaces arranged in ascending order of dimension are constructed to form a nested subspace chain, and a unified response basis is determined accordingly; subsequently, when a storage node fails, each participating node outputs response symbols sequentially in a streaming manner based on its local storage symbol, corresponding value points, and unified response basis; finally, the replacement node adaptively selects the appropriate repair mode according to the actual number of valid help groups and response items received, and completes the recovery of the coded symbols of the failed node.

[0032] like Figure 1 As shown, the present invention includes the following steps:

[0033] S1: Construct the set of values ​​for the RS code and group the set of values. Select a subspace that satisfies predetermined conditions on the extended field of a finite field, and construct multiple cosets based on several distinct base field elements, thereby obtaining the set of values ​​for the RS code; further, divide the set of values ​​into multiple disjoint groups, where each group of values ​​can be represented as a translation of a base field element onto the subspace.

[0034] S2: Construct a nested subspace chain and determine a unified response basis. Construct multiple subspaces arranged in ascending order of dimension on the extended field of the finite field to form a nested subspace chain; determine a unified response basis based on the nesting relationship between the image spaces of the corresponding subspace polynomials of each subspace, so that different repair levels can adopt a unified response order.

[0035] S3: Perform RS encoding storage and scenario deployment. After encoding the data to be stored using RS codes, store it in the storage nodes of the corresponding groups, so that each storage node corresponds to a value point and stores the corresponding encoding symbol.

[0036] S4: Participating nodes generate a unified streaming response. When a storage node fails, the replacement node initiates a repair request to the remaining surviving nodes. Each participating node, based on the encoded symbols stored locally, the corresponding value points, and the unified response base, outputs response symbols on multiple base fields in a predetermined order, forming a unified streaming response sequence. The streaming response is generated independently by each participating node and does not depend on a pre-fixed complete set of helper nodes.

[0037] S5: The control terminal adaptively selects the repair mode. The replacement node selects the target repair mode that meets the current conditions from multiple pre-set repair modes based on the number of valid help groups received, the host group response status, and the number of response items returned by each participating node. Different repair modes correspond to different numbers of valid help groups and response items.

[0038] S6: Post-process the received response and complete the repair of the failed node. The replacement node uses the subspace polynomial, pre-calculated coefficients, trace function relationship and basis expansion relationship corresponding to the selected repair mode to perform linear combination and post-processing on the received streaming response, and finally restore the coded symbol corresponding to the failed node.

[0039] In this invention, the Extending a finite field one above - Subspace, the dimension of which can be selected according to system parameters, where To expand the degree. To ensure the algebraic processing during subsequent grouping point construction, subspace polynomial construction, and recovery process, the subspace... It needs to satisfy predetermined algebraic properties. In one implementation, a positive integer can be chosen. ,in and And construct a when the predetermined conditions are met. dimension - Subspace So that it is connected to the base domain They intersect only at the zero element, and for any... All have Based on this type of subspace, multiple cosets obtained by shifting the elements of the base field can be constructed as the set of block values ​​for the RS code.

[0040] In one specific construction, the subspace Can be taken as In the form of, Let be a non-zero element in the extended field, and satisfy . as well as .

[0041] Example:

[0042] This embodiment is used for single-node repair in distributed storage. For example... Figure 2 As shown, in this embodiment, consider a... A distributed storage system consisting of storage nodes. All storage nodes are divided into... There are 1 group, each containing 1 group. 1 node, satisfying Given a limited field and an extended field as... Select the one that satisfies the above on this extended domain. - Subspace And select several distinct ones elements on Construct value point grouping

[0043] ,

[0044] Therefore, the complete set of all possible values ​​for the RS code can be represented as:

[0045] ,

[0046] In this set, each group of value points is pairwise distinct. After constructing the RS code using this set of value points, the... The first in the group The encoded symbols stored in each node are ,in Let be the encoding polynomial corresponding to the data to be stored, for any... , This grouping and value-taking construction ensures that each group of nodes naturally corresponds to a coset group, providing a unified algebraic basis for subsequent multi-helper group degree repair.

[0047] make , Furthermore, nested subspace chains are constructed on the extended field of the aforementioned finite field.

[0048] ,

[0049] Each subspace is - Subspaces, and the image spaces of the corresponding subspace polynomials satisfy a nesting relationship. From this nesting relationship, a unified response basis can be selected.

[0050] ;

[0051] This allows the response space required at different recovery levels to be spanned by the first few terms of the unified response basis. Thus, the participating points in different recovery modes can generate streaming responses in the same order, differing only in the actual number of response terms used, thereby unifying multiple recovery levels into a single streaming response framework.

[0052] In this embodiment, as Figure 3 As shown, if node Failure, i.e., encoding symbol Lost, at this time it is called the first The group is the host group. The replacement node initiates a unified streaming response request to the remaining nodes.

[0053] During the recovery process, a group is considered a valid helper group if all nodes in it (excluding the eraser node) participate in the response and return data to the replacement node. If only some nodes in a group return a response, or if some nodes return partial data but ultimately do not participate in the recovery as a complete group, then that group is not considered a valid helper group. For a given successful repair, the number of valid helper groups is called the valid helper group degree. The concept of valid helper groups characterizes the number of helper resources that can participate in recovery as complete groups in a real-world dynamic environment, allowing the replacement node to select the appropriate recovery mode.

[0054] Each node Encoded symbols based on local storage The value point corresponding to this node And the unified response base mentioned above, in turn, returns responses in the following forms:

[0055] ,

[0056] in Indicates from arrive trace mapping, The value point is the one corresponding to the failed node. The above response depends only on the encoded symbol stored locally on the node and the value point corresponding to that node, and not on the complete set of helper nodes. Therefore, even if the helper set is not fixed in advance before repair, each node can still generate a response independently.

[0057] During the repair process, the replacement node adaptively selects the appropriate recovery mode based on the actual number of valid help groups received and the number of response items returned by each participating node. For a given recovery level, when the replacement node has received a sufficient number of preceding response symbols from all surviving nodes in the host group and several valid help groups, it can invoke the recovery rule corresponding to that recovery level to perform linear combination and post-processing of the received responses, thereby recovering the failed symbols. Therefore, when the availability of help nodes changes dynamically, replacement nodes can switch repair modes in real time based on the responses received, achieving a trade-off between the number of help groups, the download volume per node, and the total repair bandwidth.

Claims

1. An adaptive repair method for RS code storage nodes based on subspace chains, characterized in that, include: In finite field extension Select one - Subspace Based on several distinct base field elements, multiple cosets are constructed to obtain the set of values ​​for the RS code. This set of values ​​is then divided into multiple disjoint groups, where each group of values ​​represents a base field element in the subspace. Translation; In the extended domain of the finite field Construct multiple arrays arranged in ascending order of dimension. - Subspaces are formed into nested subspace chains, and a unified response basis is determined based on the nesting relationship between the image spaces of the corresponding subspace polynomials, so that different recovery levels adopt a unified response order; The data to be stored is encoded using RS codes and then stored in the corresponding storage nodes in the corresponding groups. Each storage node corresponds to a value point in the set of value points and stores the corresponding encoding symbol. When a storage node fails, the group to which the failed node belongs is defined as the host group, and the replacement node sends a repair request to the remaining surviving nodes. During the repair process, each participating node calculates and outputs the response symbols on the base domain in a predetermined order based on the locally stored encoded symbols, corresponding value points, and the unified response base, forming a unified streaming response sequence. The streaming response sequence does not depend on a pre-fixed complete set of helper nodes. The response symbols on the base domain are obtained by the participating nodes performing a trace function operation from the finite field to the base domain on the product of the locally stored encoded symbols and the pre-calculated coefficients. The pre-calculated coefficients are pre-determined based on the unified response base, the corresponding value points, and the subspace polynomials corresponding to each preset repair mode. The replacement node selects a target repair mode that meets the current conditions from a number of pre-set repair modes based on the number of valid help groups received, the response status of the host group, and the number of response items returned by each participating node. The definition of a valid help group is: when all nodes in a certain group, except for the host group, participate in the response and return data to the replacement node during a node repair process, the group is defined as a valid help group. The replacement node uses the subspace polynomial, pre-calculated coefficients, trace function relationship and basis expansion relationship corresponding to the selected target repair mode to perform linear combination and post-processing on the received streaming response to restore the coded symbol corresponding to the failed node.

2. The adaptive repair method for RS code storage nodes based on subspace chains according to claim 1, characterized in that, The subspace Extending a finite field On dimension - subspace, where It is a positive integer and satisfies as well as .

3. The adaptive repair method for RS code storage nodes based on subspace chains according to claim 2, characterized in that, The subspace and They intersect only at the zero element, and for any... All have .

4. The adaptive repair method for RS code storage nodes based on subspace chains according to claim 1, characterized in that, The set of value points consists of several cosets, the first... The set of values ​​for each group is represented as:

5. Among them for The elements in.

6. The adaptive repair method for RS code storage nodes based on subspace chains according to claim 1, characterized in that, The unified streaming response sequence consists of response symbols on multiple base domains, and the response space required for different repair levels is spanned by the first few terms of the unified response base.

7. The adaptive repair method for RS code storage nodes based on subspace chains according to claim 1, characterized in that, The replacement node adaptively determines the target repair mode based on the number of effective help groups, the host group response status, and the number of response items returned by each participating node.

8. The adaptive repair method for RS code storage nodes based on subspace chains according to claim 1, characterized in that, The set of value points is further designed to ensure that the relevant dual code multipliers satisfy the base field processing condition, so that after the actual effective help group is determined, the replacement node can perform scalar adjustment and post-processing on the received response symbols on the base field to restore the coded symbols corresponding to the failed node.

Images