A log replication decoupling consensus method and storage medium using erasure coding

Abstract

This invention provides a log replication decoupling consensus method and storage medium using erasure coding. The log replication decoupling consensus method includes: a first step: the client sends a request to the cluster; a second step: the cluster begins processing the client's request after receiving it; and a third step: after committing the log, the cluster replies to the client, indicating that the client's request has been completed. The beneficial effects of this invention are: 1. The log replication decoupling consensus method of this invention can effectively reduce the load on the leader node, solving the system performance problem caused by the leader's performance bottleneck; 2. The log replication decoupling consensus method of this invention can handle a large amount of non-transactional data in IoT scenarios; 3. The log replication decoupling consensus method of this invention ensures that even when the number of healthy nodes decreases, it can still effectively reduce the system's network and storage resource consumption.

Description

Technical Field

[0001] This invention relates to the field of blockchain, and more particularly to a log replication decoupling consensus method and storage medium using erasure coding. Background Technology

[0002] Blockchain is a revolutionary technology, essentially a traceable and ever-growing distributed ledger database. It achieves decentralization and information traceability through the comprehensive use of cryptography, consensus algorithms, and other technologies. These characteristics of blockchain bring new solutions to trust and security in IoT devices. Currently, most blockchain technologies used in IoT scenarios are consortium blockchains. The consensus mechanism is a crucial part of a consortium blockchain, affecting its performance, scalability, and security. Since nodes in a consortium blockchain are authenticated, most consortium blockchains employ the Raft algorithm for higher efficiency. The Raft algorithm coordinates log replication in the distributed system by electing a stable leader node, ensuring that the log state of all nodes remains consistent. The leader node is responsible for handling client requests, converting them into log entries, and replicating them to other nodes. When a majority of nodes have saved a log entry, the entry is committed and applied to the state machine by the leader. At any time, if the leader fails or cannot communicate with a majority of nodes, the system automatically elects a new leader. This election and log replication process is cyclical until the system reaches a consistent state. The Raft algorithm is designed for ease of understanding and implementation while guaranteeing strong consistency and fault tolerance.

[0003] The consensus mechanism of a consortium blockchain determines the overall performance of the system. In geographically distributed IoT scenarios, there is high network latency between nodes, and IoT devices generate a high volume of requests. The Raft algorithm relies on a single leader node to manage log replication and maintain consistency. This design limits the system's performance to the processing power of the leader node. In geographically distributed IoT scenarios, the leader node needs to handle a large number of requests from IoT devices. However, due to high network latency and packet loss rates, a large amount of IoT data is retransmitted when the leader performs log replication, resulting in high traffic between nodes and a decline in system performance. To address the problems of the Raft algorithm, the academic community has proposed many solutions, such as modifying the leader election mechanism to allow higher-performing nodes to assume the leader role, and introducing erasure coding to save network and storage resources. Existing work has made many improvements to the Raft algorithm, mainly focusing on improving the leader's performance. However, this approach cannot fundamentally solve the leader performance bottleneck. A large amount of non-transactional data exists in IoT scenarios, and the academic community has not yet proposed optimization schemes specifically for this characteristic of IoT scenarios. Summary of the Invention

[0004] To address the problems in the prior art, this invention provides a log replication decoupling consensus method using erasure coding.

[0005] This invention provides a method comprising the following steps:

[0006] First step: The client sends the request to the cluster;

[0007] The second step: After receiving a client request, the cluster begins processing it. Upon receiving data from the client, the leader node generates a corresponding Term and a normal log index for the request, assembles it into a normal log entry, adds it to its local normal log, and then initiates the log replication process. It uses dynamic erasure coding to divide the data into several encoded data fragments, replicates the encoded entries to the follower nodes, and commits the entry after receiving responses from a set number of follower nodes. Upon receiving non-transactional data from the client, the follower nodes generate a corresponding ServerID and a decoupling log index for the request, assemble it into a decoupling log entry, and add it to their local normal log. The decoupling log is stored locally, and then the decoupling log replication process is initiated. The dynamic erasure coding protocol is used to divide the data into several encoded data fragments. The decoupling entries containing complete data are replicated to other follower nodes, and the decoupling entries containing encoded data fragments are replicated to the leader node. After receiving a set number of replies, the follower node responsible for replicating this decoupling log marks the corresponding decoupling entry as weakly committed and reports it to the leader node. After receiving the report from the follower node, the leader node generates a marked entry for the corresponding decoupling entry and then replicates this marked entry to other follower nodes. After receiving a set number of replies, the leader node marks the decoupling entry corresponding to the marked entry as strongly committed.

[0008] The third step: After the cluster completes the log submission, it replies to the client to indicate that the client's request has been completed.

[0009] As a further improvement of the present invention, in the second step, when a decoupled entry is marked as a weak commit state, the follower node updates its weak commit index and transmits this information to the leader node through the leader node's heartbeat mechanism. After receiving the weak commit index from the follower node, the leader node merges all decoupled entries that have entered the weak commit state but have not yet been merged into the normal log.

[0010] As a further improvement of the present invention, in the second step, the marker entry generated by the leader node does not contain specific command data, but references the instruction data in the corresponding decoupling entry through a pointer. When the leader node copies the marker entry to the follower node, it checks whether the corresponding decoupling entry exists in the local log according to the pointer. If the corresponding decoupling entry is missing in the local decoupling log, the follower node will request the corresponding decoupling entry from the leader node.

[0011] As a further improvement of the present invention, in the second step, the leader node encodes normal entries and the follower node encodes decoupling entries using the same k,m erasure coding strategy, that is, the data is divided into k data blocks and m redundant blocks are generated. The leader node records the specific coding strategy used in its normal entries, and the follower node records the specific coding strategy used in its decoupling entries. The leader node assigns a Term and a normal log index to the encoded data block and combines them with the coding strategy to form a normal entry. The follower node assigns a ServerID and a decoupling log index to the encoded data block and combines them with the coding strategy to form a decoupling entry.

[0012] As a further improvement of the present invention, in the second step, the leader node broadcasts the encoded normal entry to all follower nodes. After successfully receiving the entry, the follower nodes send an acknowledgment to the leader node. Once the leader node receives acknowledgment replies from F+k-1 follower nodes, the normal entry is committed. The follower nodes send the encoded decoupling entry to other follower nodes and send the complete decoupling entry to the leader node. When a follower node receives acknowledgment replies from the leader node and F+k-2 other follower nodes, the corresponding decoupling entry is marked as weakly committed. At this time, the leader node can create the corresponding marked entry and push the decoupling entry into a strong committed state.

[0013] As a further improvement of the present invention, in the second step, it is assumed that the erasure coding parameter currently used is k. old ,m old The leader node detects the number of healthy nodes in the system through a heartbeat mechanism. When the number of healthy nodes in the system is less than F+k, the leader node will terminate the process. old When the coding strategy change process is initiated, the leader node first increments its term, then sets the value of the next array to the largest index value among the committed entries, and adjusts the coding strategy to meet the new system health status. The new erasure coding parameters must ensure F+k. new It equals the number of healthy nodes in the system, and k new +m new This equals the total number of nodes in the system. The coding strategies of follower nodes must be synchronized with those of the leader node. After changing its coding strategy, the leader node must notify the follower nodes. Upon receiving the new coding strategy, the follower nodes will send a confirmation reply to the leader node. When the leader node receives F+k... new After receiving a confirmation reply, the coding strategy change is confirmed to be successful.

[0014] As a further improvement of the present invention, in the second step, when the original leader node encounters a crash or network partition, the cluster will start a new leader election process. The newly elected leader node contains two types of entries in its log: committed entries and uncommitted entries. These entries all contain encoded data fragments. In order to ensure the consistency of the log, the new leader node needs to use the follower nodes to recover these data fragments in order to reconstruct the complete dataset.

[0015] As a further improvement of the present invention, in the second step, for submitted normal entries, the new leader node obtains at least k encoded data fragments from F follower nodes to complete data recovery. For submitted marked entries, the new leader node only needs to request the required data from the corresponding follower node. If this acquisition process fails, the new leader node requests data fragments from other follower nodes. Similarly, the new leader node can obtain at least k encoded data fragments from F follower nodes to recover the decoupled entry corresponding to the marked entry. For entries that have not yet been submitted, the new leader node will perform data recovery. If an entry cannot be recovered, the new leader node takes deletion measures to remove the entry and its subsequent entries.

[0016] As a further improvement of the present invention, in this log replication decoupled consensus method, the requests sent by the client include transactional data requests and non-transactional data requests. The leader node can handle both transactional and non-transactional data requests, while the follower nodes only handle non-transactional data requests. The transactional data requests include activity records, plans, bills, payment records, and invoices. The non-transactional data requests include GPS data, consumer feedback, audio data, image data, and machine-generated log data.

[0017] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the log replication decoupling consensus method described in the present invention when invoked by a processor.

[0018] The beneficial effects of this invention are: 1. The log replication decoupling consensus method of this invention can effectively reduce the load on the leader node and solve the system performance problem caused by the leader's performance bottleneck; 2. The log replication decoupling consensus method of this invention can handle a large amount of non-transactional data in IoT scenarios; 3. The log replication decoupling consensus method of this invention ensures that even when the number of healthy nodes decreases, the network and storage resource consumption of the system can still be effectively reduced; 4. The log replication decoupling consensus method of this invention maintains the consistency and reliability of the distributed system by ensuring the consistency of system logs. Attached Figure Description

[0019] Figure 1 This is a flowchart of the log replication decoupling consensus method of the present invention;

[0020] Figure 2 This is the overall architecture diagram of the log replication decoupling consensus method of the present invention;

[0021] Figure 3 This is a flowchart of the decoupled log replication process of the present invention;

[0022] Figure 4 This is the weak commit state diagram of the consensus method for replication decoupling of this invention;

[0023] Figure 5 This is a diagram showing the generation and copying status of the marked entries in this invention; Detailed Implementation

[0024] like Figure 1-2 As shown, this invention discloses a log replication decoupling consensus method using erasure coding, comprising the following steps:

[0025] The first step: The client sends a request to the cluster; the requests sent by the client are divided into two types: transactional data requests (activity records, plans, bills, payment records, invoices, etc.) and non-transactional data requests (GPS data, consumer feedback, audio data, image data, machine-generated log data, etc.).

[0026] The second step: After receiving a client request, the cluster begins processing it. The leader node can handle both transactional and non-transactional data, while follower nodes can only handle non-transactional data. Upon receiving data from the client, the leader node generates a corresponding Term (the term of the cluster node, an incrementing variable) and a normal log index for the request. It assembles the data into a normal log entry and adds it to its local normal log. Then, it initiates the log replication process, using dynamic erasure coding to divide the data into several encoded data fragments. The encoded entries are copied to the follower nodes. After receiving responses from a set number of follower nodes, the leader node commits the entry. Upon receiving non-transactional data from the client, the follower nodes generate a corresponding ServerID (a node ID, uniquely identifying a node) and a decoupling log index for the request. They assemble the decoupling log into a normal log entry. The log entries are added to the local decoupling log, and then the decoupling log replication process is started. The data is divided into several coded data fragments using the dynamic erasure coding protocol. The decoupling entries containing complete data are copied to other follower nodes, and the decoupling entries containing coded data fragments are copied to the leader node. After receiving a set number of replies, the follower node responsible for replicating this decoupling log marks the corresponding decoupling entry as weakly committed and reports it to the leader node. After receiving the report from the follower node, the leader node generates a marked entry for the corresponding decoupling entry and then copies this marked entry to other follower nodes. After receiving a set number of replies, the leader node marks the decoupling entry corresponding to the marked entry as strongly committed.

[0027] The third step: After the cluster completes the log submission, it replies to the client to indicate that the client's request has been completed.

[0028] The log replication decoupling consensus method of this invention mainly consists of two parts: decoupling logs and a dynamic erasure coding protocol. The core mechanism of the log replication decoupling consensus mechanism, through the introduction of decoupling logs and an adaptive fragmentation coding protocol, significantly improves the efficiency and reliability of the Raft consensus algorithm. The design concept of decoupling logs is to distribute log replication tasks to various follower nodes without weakening the authority of the leader node, thereby effectively reducing the load on the leader node and solving the system performance problem caused by the leader's performance bottleneck. Furthermore, the adaptive fragmentation coding protocol proposes an innovative solution to the problem that other Raft optimization schemes, when applying erasure coding to reduce network and storage resource consumption, suffer from protocol performance degradation to full replication as the number of healthy nodes decreases. This ensures that even with a decrease in the number of healthy nodes, the log replication decoupling consensus mechanism can still effectively reduce the system's network and storage resource consumption. This invention will be elaborated in detail from three aspects:

[0029] 1. How are decoupled logs generated and replicated to all nodes in the cluster?

[0030] 2. How does the leader node merge entries from the decoupled log into the normal log and perform the commit operation?

[0031] 3. What is the specific operating mechanism of the adaptive fragmentation coding strategy protocol?

[0032] I. Decoupling Log Replication

[0033] When discussing the decoupled log replication mechanism in the consensus mechanism of log replication decoupling, it is important to note that each follower node maintains an independent decoupled log. The corresponding follower node is responsible for replicating decoupled log entries among nodes, and the log commit and state machine application are achieved through the leader node's merge operation. For distinction, this invention refers to the log in the Raft protocol as the normal log, its entries as normal entries, and the entries in the decoupled log as decoupled entries.

[0034] Within this framework, when each follower node receives a non-transactional data request from a client, it first creates a corresponding decoupling entry and appends it to its local decoupling log. Then, the follower node initiates the decoupling log replication process.

[0035] like Figure 3As shown, the decoupling log is structured as a series of consecutive decoupling entries. Each entry contains the ServerID of the follower node that generated the entry, the index of the decoupling entry, and the corresponding non-transactional state machine command data. When a decoupling entry is replicated by a majority of nodes and then replicated to the leader node, it enters a weak commit state. It's important to note that, unlike Raft's commit mechanism, the weak commit state only indicates that the decoupling entry is ready to be merged into the normal log by the leader node; it does not guarantee its ultimate safety. The weak commit state is a necessary stage for a decoupling entry to reach the strong commit state.

[0036] Follower nodes maintain a variable called the weak commit index to track which entries in their local decoupling logs have reached the weak commit state. Unlike the commit index in the Raft protocol, the weak commit index is not monotonically increasing; it may roll back in the event of a leader node failure. Figure 4 As shown, at time T1, follower node S0 successfully replicates decoupled entry D to leader node S1 and another follower node S2, and marks the entry as weakly committed. However, at time T2, when leader node S1 fails and S3 is elected as the new leader, decoupled entry D fails to meet the conditions for replication to the new leader, and therefore its weakly committed state is revoked. This mechanism only ensures that decoupled entries entering the weakly committed state can be merged into the normal log by the leader node, but it cannot guarantee the security of the log.

[0037] Within the framework of the Raft algorithm, the leader node is responsible for broadcasting log entries to other nodes in the cluster to ensure the consistency of the system logs, thereby maintaining the consistency and reliability of the distributed system.

[0038] In the log replication decoupled consensus mechanism, decoupled entries in the decoupled log are eventually transformed into marked entries in the normal log maintained by the leader node. These marked entries are generated and replicated by the leader node. Notably, these marked entries do not carry actual data content, thus consuming almost no additional network resources during replication. This means that decoupled entries in the decoupled log will eventually be integrated into the normal log, and the leader node is responsible for replication and management, thereby ensuring that the authority of the leader node in the log replication decoupled Raft framework is not weakened.

[0039] II. Tagging Entries and Two-Phase Submission

[0040] When a decoupling entry is marked as weakly committed, follower nodes update their weakly committed index and transmit this information to the leader via the leader node's heartbeat mechanism. Upon receiving the weakly committed index from the follower nodes, the leader node merges all decoupling entries that have entered a weakly committed state but have not yet been merged into the normal log. The leader node does not directly copy the decoupling entry's data content into the normal log. Instead, it creates a marker entry. This marker entry does not contain actual command data but contains a pointer to the corresponding decoupling entry, identifying itself as a marker entry pointing to the decoupling entry. Therefore, there are two types of entries in the normal log maintained by the leader node: one is a normal entry generated by the leader node after receiving a client request, which contains command data; the other is a marker entry generated based on the decoupling log, which does not contain command data but references the command data in the corresponding decoupling entry through a pointer. When the marker entry corresponding to a decoupling entry is committed by the leader node, the decoupling entry enters a strongly committed state, ensuring eventual consistency of its data.

[0041] During log replication, the leader node copies the marked entries to the follower nodes. Since the marked log does not contain data, after receiving a marked entry, the follower node needs to check if a corresponding decoupling entry exists locally based on the pointer within it. If the local decoupling log lacks the corresponding decoupling entry, the follower node will request the corresponding decoupling entry from the leader node. The following example illustrates this situation: Figure 5 As shown, at time T1, the leader node S0 receives non-transactional data A from a client and generates a normal entry, which is then propagated to other follower nodes via log replication. Simultaneously, follower node S2, after receiving non-transactional data from a client, generates the corresponding decoupling entry and attempts to replicate it to other nodes. However, due to network issues, the decoupling entry generated by S2 fails to be replicated to node S4. At this point, decoupling entries B and C are marked as weakly committed. At time T2, the leader node generates corresponding marked entries for decoupling entries B and C and propagates these marked entries to other nodes via log replication, marking them as strongly committed. Since follower node S4 does not contain the decoupling entries corresponding to the marked entries replicated by the leader node, it requests the required decoupling entries from the leader node. At time T3, based on the feedback from follower node S4, the leader node S0 sends the missing decoupling entries B and C to follower node S4.

[0042] III. Adaptive Coding Fragmentation Protocol

[0043] In the Raft framework for decoupled log replication, the adaptive coding fragmentation protocol plays a crucial role, designed to optimize network and storage resource consumption during log replication. This protocol consists of two main parts: entry coding and the dynamic adjustment of the coding strategy during log replication. First, the coding phase involves the leader node coding normal entries and the follower nodes coding decoupled entries. Both use the same (k, m) erasure coding strategy, dividing the data into k data blocks and generating m redundant blocks. Since the coding strategy may change over time, the specific coding strategy used must be recorded in the entry. The leader node is responsible for assigning Term and Index to the coded data blocks and combining them with the coding strategy to form normal entries. Correspondingly, follower nodes assign ServerID and Index to the coded data blocks and record the coding strategy, thus forming decoupled entries.

[0044] Assuming a cluster with 2F+1 nodes and using an (k, m) erasure coding strategy, the log replication phase can be discussed from two aspects: the replication of normal entries by the leader node and the replication of decoupled entries by the follower nodes. The leader node broadcasts the encoded normal entries to all follower nodes, and each follower node sends an acknowledgment to the leader node upon successful reception. Once the leader node receives acknowledgments from F+k-1 follower nodes, the normal entry is committed. Similarly, follower nodes send encoded decoupled entries to other nodes and send complete decoupled entries to the leader node. When a follower node receives acknowledgments from the leader node and F+k-2 other follower nodes, the corresponding decoupled entry is marked as weakly committed. At this point, the leader node can create the corresponding marked entry and push the decoupled entry into a strong committed state.

[0045] When the number of healthy nodes in the system is insufficient to meet the requirement of F+k, the leader node will be unable to commit logs. At this point, the system enters the coding strategy adjustment phase. In the adaptive coding strategy adjustment section, it is assumed that the current erasure coding parameter is k. old m old The leader node detects the number of healthy nodes in the system through a heartbeat mechanism. When the number of healthy nodes in the system is less than F+k, the leader node will terminate the process. old When this happens, the coding strategy change process is initiated. The leader node first increments its term, then sets the value of the `next` array to the largest index value among committed entries, and adjusts the coding strategy to meet the new system health status. Specifically, the new erasure coding parameters must ensure F+k new It equals the number of healthy nodes in the system, and k new +m newThis equals the total number of nodes in the system. Follower nodes' coding strategies must be synchronized with the leader node. After changing its coding strategy, the leader node must notify the follower nodes. Upon receiving the new coding strategy, the follower nodes will send a confirmation reply to the leader node. When the leader node receives F+k... new After receiving a confirmation reply, the encoding strategy change is confirmed to be successful.

[0046] Preparation phase of the new leader node

[0047] When the original leader node fails or experiences a network partition, the cluster will initiate a new leader election process. The newly elected leader node contains two types of entries in its log: committed entries and uncommitted entries. These entries all contain encoded data fragments. To ensure log consistency, the new leader node needs to use follower nodes to recover these data fragments and reconstruct the complete dataset.

[0048] To fulfill its function, a new leader node must undergo a phase called leader preparation before processing client requests, aimed at recovering complete log information. For committed entries, there are two types: normal entries containing data and marked entries containing no data. For normal entries, the new leader node can ensure that it obtains at least k encoded data fragments from F follower nodes to complete data recovery. For marked entries, since they point to a specific decoupled entry, and the follower node that generated the decoupled entry holds the complete data, the leader node only needs to request the required data from the corresponding follower node. If this process fails, the leader node can request data fragments from other follower nodes. Similarly, the leader node has the ability to obtain at least k encoded data fragments from F follower nodes to recover the decoupled entry corresponding to the marked entry.

[0049] For entries that have not yet been committed, the new leader node will do its best to recover the data. However, if an entry cannot be recovered, the leader node will delete it and all subsequent entries to prevent data consistency from being affected. This strategy ensures that the system maintains data integrity and consistency even in the event of a leader change, while also guaranteeing the robustness and reliability of the distributed system.

[0050] The present invention also discloses a computer-readable storage medium storing a computer program configured to implement the steps of the log replication decoupling consensus method described in the present invention when invoked by a processor.

[0051] The beneficial effects of this invention are: 1. The log replication decoupling consensus method of this invention can effectively reduce the load on the leader node and solve the system performance problem caused by the leader's performance bottleneck; 2. The log replication decoupling consensus method of this invention can handle a large amount of non-transactional data in IoT scenarios; 3. The log replication decoupling consensus method of this invention ensures that even when the number of healthy nodes decreases, the network and storage resource consumption of the system can still be effectively reduced; 4. The log replication decoupling consensus method of this invention maintains the consistency and reliability of the distributed system by ensuring the consistency of system logs.

[0052] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A log replication decoupling consensus method using erasure coding, characterized in that, Includes the following steps: First step: The client sends the request to the cluster; The second step: After receiving the client's request, the cluster begins to process the client's request; After receiving data from a client, the leader node generates a corresponding Term and a normal log index for the request, assembles it into a normal log entry, adds it to its local normal log, and initiates the log replication process. It uses dynamic erasure coding to divide the data into several encoded data fragments, replicates the encoded entries to follower nodes, and commits the entry after receiving responses from a set number of follower nodes. Similarly, after receiving non-transactional data from a client, follower nodes generate a corresponding ServerID and a decoupling log index for the request, assemble it into a decoupling log entry, add it to their local decoupling log, and initiate the process. The decoupled log replication process also uses a dynamic erasure coding protocol to divide the data into several encoded data fragments. Decoupled entries containing complete data are copied to other follower nodes, while decoupled entries containing encoded data fragments are copied to the leader node. After receiving a set number of replies, the follower node responsible for replicating this decoupled log marks the corresponding decoupled entry as weakly committed and reports it to the leader node. After receiving the report from the follower node, the leader node generates a marked entry for the corresponding decoupled entry and then copies this marked entry to other follower nodes. After receiving a set number of replies, the leader node marks the decoupled entry corresponding to the marked entry as strongly committed. The third step: After the cluster completes the log submission, it replies to the client to indicate that the client's request has been completed; In the second step, the leader node encodes normal entries and the follower nodes encode decoupling entries using the same k, m erasure coding strategy, that is, the data is divided into k data blocks and m redundant blocks are generated. The leader node records the specific coding strategy used in its normal entries, and the follower nodes record the specific coding strategy used in their decoupling entries. The leader node assigns a Term and a normal log index to the encoded data block and combines them with the coding strategy to form a normal entry. The follower nodes assign a ServerID and a decoupling log index to the encoded data block and combine them with the coding strategy to form a decoupling entry.

2. The log replication decoupling consensus method according to claim 1, characterized in that, In the second step, when a decoupled entry is marked as weak commit, the follower node updates its weak commit index and transmits this information to the leader node through the leader node's heartbeat mechanism. After receiving the weak commit index from the follower node, the leader node merges all decoupled entries that have entered the weak commit state but have not yet been merged into the normal log.

3. The log replication decoupling consensus method according to claim 1, characterized in that, In the second step, the marker entries generated by the leader node do not contain specific command data, but instead reference the instruction data in the corresponding decoupling entries through pointers. When the leader node copies the marker entries to the follower nodes, it checks whether the corresponding decoupling entries exist in the local log based on the pointers. If the corresponding decoupling entries are missing in the local decoupling log, the follower nodes will request the corresponding decoupling entries from the leader node.

4. The log replication decoupling consensus method according to claim 1, characterized in that, In the second step, the leader node broadcasts the encoded normal entry to all follower nodes. After successfully receiving the entry, the follower nodes send an acknowledgment to the leader node. Once the leader node receives the acknowledgment replies from F+k-1 follower nodes, the normal entry is committed. Follower nodes send the encoded decoupling entry to other follower nodes and send the complete decoupling entry to the leader node. When a follower node receives confirmation responses from the leader node and F+k-2 other follower nodes, the corresponding decoupling entry is marked as weakly committed. The leader node then creates the corresponding marked entry and pushes the decoupling entry into a strong committed state.

5. The log replication decoupling consensus method according to claim 4, characterized in that, In the second step, assume that the erasure coding parameter currently used is k. old m old The leader node detects the number of healthy nodes in the system through a heartbeat mechanism. When the number of healthy nodes in the system is less than F+k, the leader node will terminate the process. old When this happens, the coding strategy change process is initiated. The leader node increments its term, sets the value of the next array to the largest index value among the committed entries, and adjusts the coding strategy to meet the new system health status. The new erasure coding parameters must ensure F+k. new equals the number of healthy nodes in the system, k new +m new This equals the total number of nodes in the system. The coding strategies of follower nodes must be synchronized with those of the leader node. After changing its coding strategy, the leader node must notify the follower nodes. Upon receiving the new coding strategy, the follower nodes will send a confirmation reply to the leader node. When the leader node receives F+k... new After receiving a confirmation reply, the coding strategy change is confirmed to be successful.

6. The log replication decoupling consensus method according to claim 1, characterized in that, In the second step, when the original leader node encounters a crash or network partition, the cluster will start a new leader election process. The newly elected leader node contains two types of entries in its log: committed entries and uncommitted entries. These entries contain encoded data fragments. To ensure log consistency, the new leader node needs to use follower nodes to recover these data fragments in order to reconstruct the complete dataset.

7. The log replication decoupling consensus method according to claim 6, characterized in that, In the second step, for committed normal entries, the new leader node obtains at least k encoded data fragments from F follower nodes to complete data recovery. For committed marked entries, the new leader node only needs to request the required data from the corresponding follower node. If this acquisition process fails, the new leader node requests data fragments from other follower nodes. The new leader node can obtain at least k encoded data fragments from F follower nodes to recover the decoupled entry corresponding to the marked entry. For entries that have not yet been committed, the new leader node will perform data recovery. If an entry cannot be recovered, the new leader node takes deletion measures to remove the entry and its subsequent entries.

8. The log replication decoupling consensus method according to claim 2, characterized in that: In this log replication decoupled consensus method, the requests sent by the client include transactional data requests and non-transactional data requests. The leader node can handle both transactional and non-transactional data requests, while the follower nodes only handle non-transactional data requests. The transactional data requests include activity records, plans, bills, payment records, and invoices. The non-transactional data requests include GPS data, consumer feedback, audio data, image data, and machine-generated log data.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to implement the steps of the log replication decoupling consensus method according to any one of claims 1-8 when invoked by a processor.

Images