A log-based high-availability erasure code update method

By using a log-based highly available erasure coding update method, the update can be completed as long as only a portion of the verification nodes respond successfully. Combined with refactoring to ensure the update success, this method solves the problems of high latency and low availability caused by long-tail delays and failures in erasure coding updates, and achieves a fast and reliable update process.

CN118971895BActive Publication Date: 2025-11-18UNIV OF SCI & TECH OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410964648.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2025-11-18
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

Existing erasure coding update technology requires all check nodes to respond successfully when updating the check block, resulting in long-tail latency and low update availability, especially when some check nodes respond too slowly or fail, making it impossible to complete the update quickly.

Method used

A log-based high-availability erasure coding update method is adopted, which can complete the update by only requiring a portion of the check nodes to respond successfully. Combined with a new check block incremental calculation method, a refactored write fallback is used to ensure the update success, including writing the update information to the log, merging the log, and calculating the check block incrementally.

Benefits of technology

It reduces the latency of erasure coding updates, improves update speed and availability, ensures the fault tolerance of stripes, and avoids update failures caused by long-tail latency or check node failures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118971895B_ABST
    Figure CN118971895B_ABST
Patent Text Reader

Abstract

The application relates to the computer technical field and discloses a log-based high-availability erasure code updating method, which is based on log technology and can complete updating only by responding successfully to part of check nodes during Reed-Solomon code updating, and the method comprises the following steps: when a single data block in a strip is updated, it is judged whether information about simultaneous updating of multiple data blocks exists in a log; if yes, the updating information in the log is combined, and then the process of responding to the check nodes is executed; if not, the process of responding to the check nodes is directly executed; when multiple data blocks in a strip are updated, the calculated check block increment is sent to the check nodes; if a set number of check nodes respond successfully within a fixed time, the new data block is saved into the log of the corresponding data node, and the strip updating frequency is increased; the application can update successfully only by responding successfully to part of check nodes during erasure code updating, thereby reducing updating delay, accelerating updating speed and improving updating availability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a log-based high-availability erasure coding update method. Background Technology

[0002] Erasure coding offers data fault tolerance with low storage overhead and is widely used in storage systems. The classic Reed-Solomon Codes (RS) divide a file into k data blocks, multiplying the encoding matrix by the k data blocks to calculate m parity blocks, which can tolerate any m errors. However, erasure coding requires frequent updates to the parity blocks during data block updates, generating significant network traffic and I / O operations, resulting in substantial update overhead. Conventional update techniques include two types: parity-based write and read-to-write. Parity-based write requires reading and transmitting all old data blocks during updates, resulting in huge network traffic and I / O overhead; read-to-write requires frequent reading and writing of old parity blocks, also leading to very high I / O overhead. Existing logging techniques, such as Parity Logging and Full Logging, optimize read-to-write by appending update information to the log, merging the update information in the log after multiple updates to update the parity blocks, thus reducing the I / O overhead of frequently reading old parity blocks. The specific process is as follows:

[0003] After receiving a new data block, a data node reads the relevant old data block, calculates the data block increment using the new and old data blocks, and then calculates the parity block increment. Both Parity Logging and Full Logging require sending the parity block increment to the parity nodes. Upon receiving the parity block increment, the parity nodes save it in their logs and then return a success message. After all parity nodes have responded successfully, Parity Logging will overwrite the old data block with the new data block on the corresponding data node, while Full Logging will save the new data block in the log of the corresponding data node while retaining the old data block.

[0004] Both Parity Logging and Full Logging require all validator nodes to respond successfully before the update process can be completed. Therefore, there are some problems: when some validator nodes respond too slowly or fail, erasure coding updates need to wait for all validator nodes to respond successfully, which will be affected by long-tail latency and result in excessively long update times; or the update cannot be completed due to the failure of some validator nodes, resulting in excessively long update times and low availability. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a log-based highly available erasure coding update method. For RS coding, based on log technology and a novel check block increment calculation method, the update can be completed with only a subset of check nodes responding successfully. Even when some check nodes respond slowly or malfunction, the erasure coding update can still be completed quickly and effectively, accelerating the update speed and improving update availability.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A log-based high-availability erasure coding update method, based on log technology, allows for updates to be completed only when a portion of the check nodes respond successfully during Reed-Solomon coding. The method includes the following steps:

[0008] Step 1: Write update information to the log: Set the expected update time t each time a data block is updated. ′ Minimum fault tolerance m during the update process ′ ;

[0009] When a single data block in a stripe is updated, it is determined whether there is information about multiple data blocks being updated simultaneously in the data node log and the check node log. If so, the update information in the data node log and the check node log is merged, and then process A is executed; otherwise, process A is executed directly. Process A involves using Parity Logging technology to send the calculated check block increment to the check node and save it in the corresponding check node's log. If at time t... ′ It contains m ′ If a verification node responds successfully, the new data block is used to overwrite the old data block, and the stripe update count is incremented by 1.

[0010] When multiple data blocks in a stripe are updated, Full Logging technology is used to send the calculated checksum block increment to the check nodes and save it in the log of the corresponding check node. If at time t ′ It contains m ′ If a verification node responds successfully, the new data block is saved to the log of the corresponding data node, and the stripe update count is incremented by 1.

[0011] Step 2, determine whether to merge logs: repeat Step 1 and determine whether the number of stripe updates has reached the threshold τ. If yes, merge the update information in the data node log and the check node log and end the update process; otherwise, end the update process directly.

[0012] Furthermore, in step one, when a single data block in the strip is updated, or when multiple data blocks in the strip are updated, if time t ′ The number of validator nodes that successfully responded was less than m. ′If there are no new check blocks, then use the refactoring write to write all the new check blocks to the nodes that were not previously data nodes or check nodes, as the new check nodes, and use the new data blocks to overwrite the old data blocks, reset the stripe update count to 0, clear the update information about the stripe in the logs of the data nodes, and end the update process.

[0013] Furthermore, in step one, for the case of updating a single data block in a stripe, the calculation method for the check block increment of the check node that failed to respond is as follows:

[0014] Use m ′ The increments of the check blocks not saved by the check nodes that successfully responded are calculated from the increments of the check blocks saved by the other check nodes:

[0015] Suppose that during the r-th update, a single data block in the stripe is... Updated to The m stored in the log of the verification node ′ The increment of each check block is The increment of the unsaved check block is The increment of an unsaved check block can be calculated using the increment of any already saved check block. The calculation method for calculating the increment of unsaved check blocks is as follows:

[0016]

[0017] in, express generate The coding coefficients at that time.

[0018] Furthermore, in step one, for the case of multiple data blocks being updated in a stripe, the calculation method for the check block increment of the check node that failed to respond is as follows:

[0019] The increments of new data blocks and a portion of the parity block increments are stored in the data node log and the parity node log, respectively. The increments of unsaved parity blocks are calculated using the new data blocks, old data blocks, and the already saved parity block increments on the data nodes.

[0020] Data nodes store the old data blocks before each update and the new data blocks after the update, and can calculate the data block increment. Let the increment of the u data blocks in the r-th update be... The verification node log saves m ′ The increment of each check block is If the data node is not faulty when calculating the increment of the unsaved check block, then u data block increments are calculated from the data node, and the increment of the unsaved check block is then calculated; if the data node fails, and the node failure does not exceed m... ′If there are at least *u* remaining data block increments and check block increments, then the total number of remaining increments is no less than *u*. We then select *u* of these increments. Calculate the increments of all data blocks using u increments. Right now

[0021]

[0022] Then, the increment of the unsaved check block is calculated, i.e.

[0023]

[0024] The calculation method for the unsaved check block increment is as follows:

[0025]

[0026] express generate The coding coefficients at that time.

[0027] Furthermore, in steps one and two, merging the update information in the data node log and the verification node log specifically includes:

[0028] For each update of the Reed-Solomon coding, the increment of the unsaved check block is calculated using the new data block, the old data block, and the increment of the already saved check block on the data node. Then, the old check block is read out and merged with all the relevant check block increments to obtain the new check block.

[0029] Then, on each verification node, the new verification block is used to overwrite the old verification block. If the latest data block exists in the log of the data node, the latest data block is used to overwrite the old data block.

[0030] Finally, reset the stripe update count to 0 and clear the relevant update information in the log.

[0031] Compared with the prior art, the beneficial technical effects of the present invention are:

[0032] Compared to Parity Logging and Full Logging, this invention only requires a portion of the check nodes to respond successfully during erasure coding updates, thus reducing update latency, accelerating update speed, and improving update availability.

[0033] The new check block increment calculation method ensures that the stripe can accommodate any m during the update process. ′ This ensures the stripe's fault tolerance. Furthermore, using a refactored write fallback ensures successful updates even when too many check nodes respond slowly or fail, further enhancing the stripe's fault tolerance. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the single data block update log writing process according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram illustrating the write log result of a single data block update according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram illustrating the result of multiple updates to the write log of a single data block in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the incremental result of updating and completing the verification block for a single data block according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of the single data block update merge log result according to an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram illustrating the update write log results of two data blocks according to an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram illustrating the result of multiple updates to the write log for two data blocks according to an embodiment of the present invention;

[0041] Figure 8 This is a schematic diagram illustrating the incremental results of updating and completing the verification block for two data blocks according to an embodiment of the present invention;

[0042] Figure 9 This is a schematic diagram illustrating the single data block update check block increment calculation method according to an embodiment of the present invention;

[0043] Figure 10 This is a schematic diagram illustrating the incremental calculation method for updating the check block between two data blocks according to an embodiment of the present invention;

[0044] Figure 11 This is a flowchart of the update mechanism of the present invention. Detailed Implementation

[0045] A specific embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0046] The log-based high-availability erasure coding update method of this invention includes a high-availability erasure coding update mechanism and a new method for calculating check block increments, specifically including the following steps:

[0047] Step 1: Write the update information to the log:

[0048] Expected update time t for each input update ′ Minimum fault tolerance m during the update process ′ m ′ The default setting is half the number of check blocks, i.e. The update process can accommodate any number of m. ′ wrong.

[0049] When a single data block in a stripe is updated, if the log does not contain information about multiple data blocks being updated simultaneously, Parity Logging is used to send the calculated parity block increment to the parity node and save it in its log. This is done as long as time t... ′ It contains m ′ If a verification node responds successfully, the new data block is used to overwrite the old data block, and the stripe update count is incremented by 1. If multiple data blocks are updated simultaneously in the log when a single data block is updated, step three is executed first to merge the existing update information in the log before executing the current step.

[0050] When multiple data blocks in a stripe are updated, Full Logging is used to send the calculated check block increment to the check node, as long as it is within time t. ′ It contains m ′ If a verification node responds successfully, the new data block is saved to the log of the corresponding data node, and the stripe update count is incremented by 1.

[0051] Regardless of whether a single data block or multiple data blocks are updated, if time t ′ The number of validator nodes that successfully responded was less than m. ′ If there are too many check nodes in the stripe, it is considered that too many check nodes cannot respond in time, failing to meet the required fault tolerance. A refactored write is used to write all new check blocks to other nodes, and the new data blocks overwrite the old data blocks. The stripe update count is reset to 0, and the update information for this stripe is cleared from the data node logs. The update is successful, and the update process ends.

[0052] Unlike Parity Logging and Full Logging, this invention only requires m ′ An update is successful as long as a single verification node responds, effectively solving the problems of excessively long update times due to long-tail latency or low update availability due to verification node failure. Unlike direct refactoring, this invention only performs refactoring when there are too few successful verification nodes, avoiding the significant overhead of direct refactoring.

[0053] Step 2, determine whether to merge logs:

[0054] After the verification node successfully responds and stores the new data block on the data node in step one, the corresponding update information is saved in the log, and it is determined whether the number of stripe updates has reached the threshold τ. If the number of stripe updates has not reached the threshold τ, the update is successful and the update process ends; if the number of stripe updates has reached the threshold τ, step three is executed to merge the update information in the log. After merging the log, the update is successful and the update process ends.

[0055] Step 3: Merge the update information in the logs:

[0056] Use the update information in the log to update the data blocks and checksum blocks to the latest state. Step 1, the log writing process, only requires m seconds per update. ′ Each check node receives and saves the check block increment, but some check nodes may not receive the required check block increment. At this stage, the check block increment calculation method proposed in this invention is used to calculate the unsaved check block increments on other check nodes using the update information already saved in the log, and then merges them with the old check blocks.

[0057] First, the existing method for calculating the incremental check block is formalized. Then, the method for calculating the incremental check block used for single data block updates and multiple data block updates, as well as the log merging process, are given.

[0058] 3.1 Existing method for calculating check block increments: Suppose the first update will increment u data blocks in the stripe. Updated to The updated value of the i-th check block is P. i (1) Using the existing formula for calculating the parity block increment, the increment ΔP of the i-th parity block in this update is... i (1) for:

[0059]

[0060] in express Generate P i The coding coefficients at that time, for This indicates the j-th node of the first update. t The increment of each data block, and the calculation of all check block increments, can be represented in matrix form, i.e.:

[0061]

[0062] 3.2 Calculation of incremental update of checksum block for single data block:

[0063] For a single data block update, m ′ The increments of the check blocks are saved on each successfully responding check node. The increments of the remaining unsaved check blocks are calculated using these saved increments. The specific calculation method is as follows:

[0064] Let the r-th update be Updated to The m stored in the log of the verification node ′ The increment of each check block is Unsaved check block increment This can be calculated using an already saved check block increment. Assuming we use... The calculation is performed as follows:

[0065]

[0066] For each update of a single data block, the increment of the unsaved check block can be calculated using an already saved check block increment.

[0067] 3.3, Incremental calculation of multiple update check blocks:

[0068] For updates to multiple data blocks, the increments of the new data blocks and a portion of the checksum blocks are stored in the log. The increments of the remaining unsaved checksum blocks can be calculated using the increments of the old and new data blocks on the data node and the already stored checksum block increments. The specific calculation method is as follows:

[0069] Data nodes store the old data blocks before each update and the new data blocks after the update, allowing the calculation of data block increments. Let the increment of the u data blocks in the r-th update be... The verification node log saves m ′ The increment of each check block is If the data node is not faulty when calculating the increments of the remaining parity blocks, the increments of these u data blocks can be calculated from the data node, and then the increments of the remaining parity blocks can be calculated; if the data node fails, and the node failure does not exceed m... ′ If there are 1, then the total number of remaining data block increments and check block increments is no less than u. Let u be the number of increments we choose. Using these u increments, we can first calculate the increments of all data blocks. Right now:

[0070]

[0071] Then, the increment of the unsaved check block is calculated, that is:

[0072]

[0073] The calculation method for the unsaved check block increment is as follows:

[0074]

[0075] Using the above method, the increment of unsaved check blocks can be calculated each time multiple data blocks are updated.

[0076] 3.4, Merging Logs:

[0077] For each update, the increment of the unsaved checksum block is calculated using the above method. Then, the old checksum block is read out and merged with the increments of all related checksum blocks to obtain a new checksum block. For example, starting from the first update until the number of updates reaches the threshold τ for log merging, the i-th checksum block is... Next, on each check node, the new check block overwrites the old check block. If a new data block exists in the log of a data node, the new data block is used to overwrite the old data block. Finally, the stripe update count is reset to 0, relevant information in the log is cleared, the update is successful, and the update process ends.

[0078] This invention takes RS(10,4) erasure coding as an example and provides... Figures 1 to 10 Examples in [the text].

[0079] Example 1:

[0080] This embodiment describes the update of a single data block in a stripe based on a high-availability erasure coding update mechanism.

[0081] Step 1: Write the update information to the log:

[0082] Figure 1 The log writing process is given, showing a single data block in an RS(10,4) stripe. Updated to The logs of the check nodes (node11 to node14) do not store information about multiple data blocks being updated simultaneously. The check block increment is calculated using read-modify-write and sent to the check nodes. Upon receiving the check block increment, the check nodes save it in their logs and return a success message. In t ′ It contains m ′ = Both check nodes (node11 and node12) responded successfully, and the new data block is used. Overwrite old data blocks Figure 2 The log writing results are given; data node1 only stores the new data blocks. The logs of check nodes node11 and node12 store the check block increments. Before the update, the stripe update count was 0; after this update, the stripe update count is incremented by 1, becoming 1. Then... After two updates, it became The load on each node varies at different times, and different nodes will verify successful responses. Figure 3 The results of multiple update write logs are given. Only the latest data block is saved on node1. The logs from node11 to node14 store the increment of the checksum block received for each update. Additionally, each subsequent update increments the stripe update count by 1.

[0083] Step 2, determine whether to merge logs:

[0084] Assuming a stripe update threshold of 3, the first two updates involved 1 and 2 stripe updates respectively, failing to reach the threshold, resulting in successful updates and the end of the update process. The third update... Updated to The stripe update count reached 3, which met the threshold, and the log merging process was initiated.

[0085] Step 3: Merge the update information in the logs:

[0086] Figure 3 The logs on the checkpoint nodes do not save the checkpoint block increments required for each update, resulting in gaps. These gaps need to be filled to ensure the correctness of the checkpoint blocks. Based on the proposed method for calculating checkpoint block increments for single data block updates, all required checkpoint block increments are calculated using the checkpoint block increments saved for each update. Figure 4 The information saved in the log after completing the checksum block increment is provided. Each node from node11 to node14 saves all the necessary checksum block increments. Merging the checksum block with its corresponding increment on each checksum node successfully updates the checksum block. Data nodes do not have the latest data block in their logs and require no further action. Figure 5 The merged result is given, all check blocks are in the latest state, then the stripe update count is reset to 0, the update information in the log is cleared, the update is successful, and the update process ends.

[0087] Example 2:

[0088] This embodiment describes the updating of two data blocks in a stripe based on a high-availability erasure coding update mechanism.

[0089] Step 1: Write the update information to the log:

[0090] Two data blocks in the RS(10,4) stripe Updated to The parity block increment is calculated using read-modify-write and sent to the parity nodes (node11 to node14). Upon receiving the parity block increment, the parity nodes save it in their logs and return a success message. In t ′ It contains m ′ =If both verification nodes (node11 and node12) respond successfully, then Each data node's log is saved separately. Figure 6 The log writing results are shown; the logs of data nodes node1 and node2 respectively saved the new data blocks. The logs of check nodes node11 and node12 store the check block increments. Before the update, the stripe update count was 0; after this update, the stripe update count is incremented by 1, becoming 1. Then... After two updates, it became The verification node for each successful log write response is different. Figure 7 The results of multiple update write logs are presented. The logs on node1 and node2 store the new data blocks from each update, while the logs on nodes11 to 14 store the received checksum block increments. Additionally, each subsequent update increments the stripe update count by 1.

[0091] Step 2, determine whether to merge logs:

[0092] Set the stripe update threshold to 3. In the first two updates, the stripe update counts are 1 and 2 respectively, which do not reach the threshold, so the updates are successful and the update process ends. In the third update, the stripe update count reaches 3, thus reaching the threshold, and the log merging process is initiated.

[0093] Step 3: Merge the update information in the logs:

[0094] Figure 7 The logs on the intermediate check nodes do not store the check block increments required for each update, resulting in gaps that need to be filled to ensure the correctness of the check blocks. Based on the proposed multi-block update check block increment calculation method, all required check block increments are calculated using the information stored in each update log. Figure 8 The information saved in the log after completing the checksum block increment is given. Each node from node11 to node14 saves all the necessary checksum block increments. On each checksum node, the checksum block is merged with the corresponding checksum block increment, and the checksum block is updated. If the latest data block exists in the logs of data nodes node1 and node2, the latest data block is used to overwrite the write data block. Then, the stripe update count is reset to 0, the update information saved in the log is cleared, the update is successful, and the update process ends. The final state of the stripe is... Figure 5 Similarly, all data blocks and check blocks are now up-to-date.

[0095] Example 3:

[0096] This embodiment is based on embodiment 1 and explains the specific calculation method for the increment of the check block that is not saved when updating a single data block.

[0097] Figure 2 Given Updated to As a result of writing logs, nodes 13 and 14 did not save the required checksum block increments. Utilize already saved logs during log merging. Calculated. Figure 9 The calculation method is given above. By substituting the indices of the existing blocks into the given formula, the corresponding coding coefficient α can be obtained. 31,α 41 ,α 11 By performing matrix operations, the required check block increment can be obtained.

[0098] Example 4:

[0099] This embodiment is based on embodiment 2 and illustrates the specific calculation method for the increment of multiple updated but unsaved check blocks.

[0100] Figure 6 Given Updated to The log write results show that nodes 13 and 14 did not save the required checksum block increments. The calculation is performed using information from the logs during log merging. This is done using the logs on node1. It can be calculated use calculate Figure 10 The calculation method is given. Based on the calculation formula given above, the index of the existing block is substituted and matrix operation is performed to obtain the required check block increment.

[0101] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0102] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A log-based highly available erasure coding update method, characterized in that, Based on log technology, during Reed-Solomon coding updates, only a portion of the check nodes need to respond successfully to complete the update. Specifically, this includes the following steps: Step 1: Write update information to the log: Set the expected update time each time a data block is updated. Minimum fault tolerance during the update process ; When a single data block in a stripe is updated, it is determined whether there is information about multiple data blocks being updated simultaneously in the data node log or the check node log. If so, the update information in the data node log and the check node log is merged, and then process A is executed; otherwise, process A is executed directly. Process A involves using Parity Logging technology to send the calculated check block increment to the check node and save it in the corresponding check node's log. Inside If a verification node responds successfully, the new data block is used to overwrite the old data block, and the stripe update count is incremented by 1. When multiple data blocks in a stripe are updated, Full Logging technology is used to send the calculated checksum block increment to the check nodes and save it in the logs of the corresponding check nodes. If the update occurs within a certain time frame... Inside If a verification node responds successfully, the new data block is saved to the log of the corresponding data node, and the stripe update count is incremented by 1. Step 2, determine whether to merge logs: repeat step 1 and determine whether the number of stripe updates has reached the threshold. If so, the update information in the data node log and the verification node log will be merged, and the update process will end; otherwise, the update process will end directly.

2. The log-based high-availability erasure coding update method according to claim 1, characterized in that, In step one, when a single data block in the strip is updated, or when multiple data blocks in the strip are updated, if time... Insufficient number of verification nodes that successfully responded internally If there are no new check blocks, then use the refactoring write to write all the new check blocks to the nodes that were not previously data nodes or check nodes, as the new check nodes, and use the new data blocks to overwrite the old data blocks, reset the stripe update count to 0, clear the update information about the stripe in the logs of the data nodes, and end the update process.

3. The log-based high-availability erasure coding update method according to claim 1, characterized in that, In step one, for the case of updating a single data block in a stripe, the calculation method for the check block increment of the check node that failed to respond is as follows: use The increments of the check blocks not saved by the check nodes that successfully responded are calculated from the increments of the check blocks saved by the other check nodes: Let the first The next update will include individual data blocks within the stripe. Updated to The logs of the verification nodes are saved The increment of each check block is The increment of unsaved check blocks is The increment of an unsaved check block can be calculated using the increment of any already saved check block. The calculation method for calculating the increment of unsaved check blocks is as follows: ; in, express generate The coding coefficients at that time.

4. The log-based high-availability erasure coding update method according to claim 1, characterized in that, In step one, for the case of multiple data blocks being updated in a stripe, the calculation method for the check block increment of the check node that failed to respond is as follows: The increments of new data blocks and a portion of the parity block increments are stored in the data node log and the parity node log, respectively. The increments of unsaved parity blocks are calculated using the new data blocks, old data blocks, and the already saved parity block increments on the data nodes. Data nodes store the old data blocks before each update and the new data blocks after the update, and can calculate the data block increment. Let the first data node be... Next update The increment of each data block is Verify the node logs saved The increment of each check block is ; If the data node did not fail when calculating the increment of the unsaved check block, then the increment is calculated from the data node. The increment of each data block is used to calculate the increment of the unsaved check block; If a data node fails, and the failure rate does not exceed [a certain threshold], then [the failure rate will not exceed [a certain threshold], and the failure rate will not exceed [a certain threshold], then [the failure rate will not exceed [a certain threshold]]. If there are 1, then the total number of remaining obtainable data block increments and check block increments is not less than 1. One, take one of them. Increment ,use Calculate the increment of all data blocks using the increment. ,Right now ; Then, the increment of the unsaved check block is calculated, i.e. ; The calculation method for the unsaved check block increment is as follows: ; express generate The coding coefficients at that time.

5. The log-based high-availability erasure coding update method according to claim 1, characterized in that, In steps one and two, merging the update information in the data node log and the verification node log specifically includes: For each update of the Reed-Solomon coding, the increment of the unsaved check block is calculated using the new data block, the old data block, and the increment of the already saved check block on the data node. Then, the old check block is read out and merged with all the relevant check block increments to obtain the new check block. Then, on each verification node, the new verification block is used to overwrite the old verification block. If the latest data block exists in the log of the data node, the latest data block is used to overwrite the old data block. Finally, reset the stripe update count to 0 and clear the relevant update information in the log.

Citation Information

Patent Citations

  • Differential log type erasure code updating method for single storage pool

    CN107766170A

  • Erasure code updating method and system based on copy data logs

    CN110262922A