Accounting method based on distributed account book
By deploying the Raft consensus algorithm in the distributed ledger system to detect and process the accounting fork caused by network partitions, the problem of accounting data fork in the distributed ledger system in the cloud environment is solved, and the consistency of the ledger data and reliability.
Patent Information
- Application Number
- CN202411552191.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-01
- Publication Date
- 2025-02-11
AI Technical Summary
In the cloud environment, in a distributed ledger system based on consensus algorithms, the fork of the accounting data caused by network partitioning problems seriously threatens data consistency and integrity.
By deploying the Raft consensus algorithm in the distributed ledger system, the master node detects the network partition in the log replication stage, starts the fork restore mechanism, compares the log offsets between nodes and submits indexes, determines the fork point, and passes the Raft consensus algorithm Log replication mechanism to keep the logs of forked nodes consistent with other nodes.
Effectively detect and restore the accounting fork problems caused by network partitioning in distributed ledger systems in cloud environments to ensure the consistency and reliability of ledger data.
Smart Images

Figure CN119415599A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information technology, and in particular to a bookkeeping method based on a distributed ledger. Background Art
[0002] In the cloud environment, there is a key technical problem in the distributed ledger technology based on the consensus algorithm: when the primary node receives incomplete log blocks due to network partition problems during the log replication phase, the secondary node will receive incomplete log blocks, which will cause the fork of accounting data. In this case, the data consistency and integrity of the distributed system are seriously threatened. Specifically, the primary node is responsible for replicating its log entries to all secondary nodes to maintain the consistency of the system state, but when a network partition occurs, the connection between the primary node and some secondary nodes may be temporarily interrupted. This interruption may cause some secondary nodes to fail to receive the latest log entries in time, or receive incomplete log entries. Since the distributed ledger relies on the consistency of logs between all nodes to ensure the correctness of transactions and the stability of the system, incomplete log replication will cause the ledger state of some nodes to be out of sync with other nodes, forming the so-called "accounting fork". This fork not only affects the execution efficiency of transactions, but may also cause data security issues, such as duplicate transactions or wrong transaction records. Therefore, how to effectively solve the accounting fork problem caused by network partitions is a major challenge facing distributed ledger technology in the cloud environment. Summary of the invention
[0003] The present invention provides a bookkeeping method based on a distributed ledger, which mainly includes:
[0004] A distributed ledger system based on the Raft consensus algorithm is deployed in a cloud environment. Several nodes of the distributed ledger system work together. In the log replication phase, the master node copies new transaction log entries to other nodes and waits for the nodes to confirm before submitting the logs.
[0005] When the master node is partitioned during the log replication phase, the network partition is detected and heartbeat messages are sent regularly between nodes within a preset time. If a node does not receive a heartbeat from the leader or other nodes within the preset time, it is determined that the node is disconnected and the disconnected node starts the fork recovery mechanism.
[0006] Obtain the index values of the submitted log entries of each node and compare them to obtain the log synchronization progress. The log offset is obtained by calculating the byte offset of each entry in the log file. The system determines the log synchronization progress of each node by comparing the log offset and submission index between nodes. If a node's log offset or submission index is found to be significantly different from that of other nodes, it indicates that the node has a bookkeeping fork.
[0007] For a node with a bookkeeping fork, the system first obtains the node's latest log block, calculates the Merkle tree Hash value of the log block, and compares the Hash value with the Hash value of the corresponding log block of other nodes to determine the location of the fork point;
[0008] According to the location of the fork point, the system obtains the complete log block after the fork point from other nodes and sends it to the node where the fork occurs. The system uses the log replication mechanism of the Raft consensus algorithm to keep the log of the forked node consistent with other nodes;
[0009] After the fork-restore mechanism is triggered, the system performs incremental synchronization of log blocks after the fork point, sends logs to different followers through batch transmission and multi-threaded concurrency, writes received log batches in parallel through followers, and determines whether to commit the current log based on the previous log entries received;
[0010] During the fork recovery process, the system continuously monitors the network connection and health status of all nodes. If any new anomalies are detected during the recovery process, the process is temporarily stopped. When the anomaly is resolved, the fork recovery is restarted.
[0011] After the fork is recovered, the secondary node re-receives the log entry or re-receives the log entry, thereby restoring the financial accounting records in the cloud environment and synchronizing the ledger state of the node that caused the network partition problem with other nodes.
[0012] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:
[0013] The present invention discloses a bookkeeping method based on a distributed ledger. When a network partition occurs in the log replication phase of the master node, the present invention detects the network partition by regularly sending heartbeat messages between nodes. If the node does not receive a heartbeat from a leader or other nodes within a preset time, it is determined that the node has been disconnected, and the disconnected node starts a fork recovery mechanism. The present invention determines the log synchronization progress of each node by comparing the log offset and the commit index between nodes, and determines the node where the bookkeeping fork occurs. For the node where the bookkeeping fork occurs, the present invention calculates the fork point position, obtains the complete log block after the fork point from other nodes, and makes the log of the forked node consistent with other nodes through the log replication mechanism of the Raft consensus algorithm. During the fork recovery process, the present invention continuously monitors the network connection and health status of all nodes, and pauses and restarts the fork recovery if there is an abnormality. Through the above technical scheme, the present invention can effectively detect and recover the bookkeeping fork problem caused by network partition in the distributed ledger system under the cloud environment, and ensure the consistency and reliability of the ledger data. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1The present invention is a flowchart of a bookkeeping method based on a distributed ledger.
[0015] Figure 2 A schematic diagram of a bookkeeping method based on a distributed ledger of the present invention.
[0016] Figure 3 This is another schematic diagram of a distributed ledger-based accounting method of the present invention. DETAILED DESCRIPTION
[0017] The following will describe the technical solutions in the embodiments of the present invention in detail in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention.
[0018] like Figure 1-3 In this embodiment, a bookkeeping method based on a distributed ledger may specifically include:
[0019] S101. Deploy a distributed ledger system based on the Raft consensus algorithm in a cloud environment. Several nodes of the distributed ledger system work together. In the log replication phase, the master node copies new transaction log entries to other nodes and waits for the nodes to confirm before submitting the log.
[0020] The address list of running nodes is obtained from the registry, and a heartbeat detection package is sent to the address list of running nodes to score and sort the online nodes according to the response time of the heartbeat detection package; the integrity checksum of the latest block data is calculated for the online nodes, and the node with the highest response time score and the largest data integrity checksum is selected as the master node; after receiving the transaction data, the master node generates a block structure, and the block structure includes a block header record and a block body record, and the block header record includes a timestamp, a previous block hash value, and a Merkle tree root value, and the block body record includes a transaction list; wherein the timestamp is generated when the master node receives new transaction data, identifies the exact time when the block is created, and is used to verify the sequentiality of the transaction, and the previous block hash value is the information of the previous block that the master node refers to when generating a new block. The hash value of the previous block is calculated by a hash algorithm to ensure the security and immutability of the blockchain. The Merkle tree root value is obtained by hashing the transaction data in the block to improve the integrity and validity of the data. The transaction list is a transaction list formed by storing verified valid transactions in sequence in the block body; the master node sends a detection packet to each slave node to obtain a round-trip delay, and the slave nodes are divided into node groups according to the round-trip delay. The node groups include a fast group, a medium-speed group, and a slow group; for the node groups, the master node sends log data according to a preset batch value, and the slave node calculates the Merkle tree root value after receiving the log data and compares it with the master node data. If the comparison is consistent, a confirmation message is returned, and the master node submits the log after receiving all confirmation messages.
[0021] Specifically, the address list of running nodes is obtained from the node registry, and a heartbeat detection packet is sent for each node address. The heartbeat interval is set to 100 milliseconds. If there is no response for more than 300 milliseconds, the node is considered offline. The online nodes are scored and sorted based on the node response time, and the node with the highest score and the largest data integrity checksum is selected as the master node. The data integrity checksum is obtained by calculating the latest 1000 blocks. When the master node receives the transaction data, a block structure is generated for the transaction data. The block header records the current timestamp, the hash value of the previous block, and the Merkle tree root value. The block body records the transaction list. Each transaction item in the transaction list contains the sender address, the receiver address, the transaction amount, and the transaction sequence number. A unique identifier is assigned to each transaction data in a sequential ascending manner. According to the round-trip delay obtained by the master node sending a detection packet to each slave node, the node groups are divided according to the network delay time. Those less than 50 milliseconds are classified into the fast group, 50 to 150 milliseconds are classified into the medium-speed group, and those greater than 150 milliseconds are classified into the slow group. The synchronization interval for the fast group is set to 200 milliseconds, the medium-speed group is set to 500 milliseconds, and the slow group is set to 1000 milliseconds. For each slave node group, the master node sends log data according to the preset batch size. The starting batch value is 100 transaction records. After each successful replication, the batch value increases by 50 until it reaches the upper limit of 500. After receiving the log data, the slave node calculates the Merkle tree root value and compares it with the master node data. If the comparison results are consistent, a confirmation message is returned to the master node. The master node submits the log after receiving the confirmation message from all slave nodes. The server ID, network address, and port number of each node are recorded in the node registry. The node ID, timestamp, load index, memory usage, and processor occupancy are included in the heartbeat packet data structure. When the master node receives the heartbeat response packet from the slave node, the response time is recorded. The heartbeat packet size is controlled within 2KB. The response time accounts for 0.4, the load index accounts for 0.3, the memory usage accounts for 0.2, and the processor occupancy accounts for 0.1 in the scoring weight. The SHA-256 algorithm is used to calculate the hash checksum for the latest 1,000 blocks. The block structure design adopts a combination of block header and block body. The block header has a fixed size of 128 bytes, including a 64-bit timestamp, a 256-bit previous block hash value, and a 256-bit Merkle tree root value. The transaction list in the block body is stored in a binary tree. The sender address in the transaction item structure is 32 bytes, the receiver address is 32 bytes, the transaction amount is 8 bytes, and the transaction sequence number is 8 bytes. The master node sends a detection packet to the slave node through the UDP protocol. The detection packet size is 64 bytes. Five detection packets are sent each time and the average value is taken as the network delay. The detection packet sending interval is 20 milliseconds. The master node maintains a delay routing table to record the real-time delay value of each node.During the log replication process, a sliding window mechanism is used to control the batch size. The starting value of the window is 100 transaction records. When the replication success rate exceeds 95%, the window increases by 50 records, and when it is lower than 85%, the window decreases by 50 records. The upper limit of the window is 500 records and the lower limit is 50 records. The slave node uses the Merkle tree verification method to verify data integrity, and the verification data block size is 4KB.
[0022] S102. When a network partition occurs in the log replication phase of the master node, the network partition is detected. Heartbeat messages are sent regularly between nodes within a preset time. If a node does not receive a heartbeat from the leader or other nodes within the preset time, it is determined that the node is disconnected, and the disconnected node starts a fork recovery mechanism.
[0023] A slave node address list is obtained from the node connection table, and a heartbeat message is sent to each slave node according to the slave node address list, wherein the heartbeat message includes a node identifier, a timestamp, a block height, and a load status; the number of disconnected nodes is counted according to the heartbeat message response, and if the number of disconnected nodes exceeds a preset proportion of the total number of nodes, it is determined that a network partition has occurred; a hash value is calculated for the online node identifier using a hash ring algorithm, and the nodes are divided into groups according to the hash value, and the voting weight is calculated according to the node online time, load status, and response speed to obtain the regional master node; a quadtree structure is used to record the forked chain data of the network partition, and the tree node of the quadtree structure includes a timestamp, a data hash, and a child node pointer, and a fork record table is established to maintain the block number mapping relationship; the data consistency score of each network partition is calculated according to block integrity, transaction consistency, timestamp continuity, and block link correctness, and the blockchain data is obtained from the network partition with the highest consistency score, and the transaction records during the fork are reordered and merged.
[0024] Specifically, the address list of all slave nodes is obtained from the node connection table, and the heartbeat message interval is set to 500 milliseconds in the node communication queue. The heartbeat message structure contains a 64-bit node ID, a 32-bit timestamp, a 32-bit block height, and a 16-bit load status. The message size is limited to 1KB. When the node does not receive a heartbeat response within 2000 milliseconds, it is recorded that the node enters a disconnected state. By collecting the results of 5 consecutive heartbeat detections, it is determined that a network partition occurs when the number of disconnected nodes exceeds 50% of the total number of nodes. The routing table is queried according to the address information of the disconnected node to obtain the network topology structure. The routing table records the node ID, network address, group to which it belongs, and online status. The hash ring algorithm is used to calculate the hash value of the online node ID. The hash value space is divided into 8 segments as the basis for node grouping. When the number of nodes in each group exceeds 3, voting is started. The voting weight is calculated according to the node online time, load status, and response speed. The node with more than 80% of the total number of nodes in the group becomes the regional master node. The forked chain data is recorded for the network partition where the disconnected node is located. The quadtree structure is used to store the forked data. The tree node contains a 64-bit timestamp, a 256-bit data hash, and 4 child node pointers. An index node is established every 100 blocks to form a forked chain structure. The mapping relationship between the forked start block number, the end block number, and the original chain block number is maintained in the fork record table, and subsequent transaction records are appended by incremental writing. After the network is restored to connectivity, the data consistency score of each partition is calculated based on the Bayesian network, with block integrity accounting for 40%, transaction consistency accounting for 30%, timestamp continuity accounting for 20%, and block link correctness accounting for 10%. The complete blockchain data is obtained from the partition with the highest score, and 4MB blocks are used for transmission. Bloom filters are used to mark duplicate transactions. The transaction records during the fork period are reordered according to the block timestamp, and the transaction records are merged by recursive comparison. The node connection table is implemented according to the bidirectional linked list storage structure. Each node record contains a 128-bit node identifier, a 32-bit network address, a 16-bit port number, an 8-bit node status, and an 8-bit load level. The 64-bit node identifier in the heartbeat message is generated by a mixed hash of the timestamp and the network address. The load status is divided into 16 levels from 0 to 15, 0 means idle and 15 means fully loaded. The block height is stored in 4 bytes and supports up to 4 billion blocks. The routing table adopts a red-black tree storage structure, and the mapping relationship between node identifiers and network addresses is stored in the form of key-value pairs. The node grouping adopts a consistent hash ring. The hash value space 0-2^32 is divided into 8 equal parts. The hash value range of each group occupies 2^29. The node identifier is calculated by SHA-256 to obtain the hash value to determine the grouping. The fork chain index table adopts a B+ tree structure. The leaf node stores block data, and the non-leaf node stores index information. Each index node contains the start timestamp, end timestamp, block height range, and data location pointer. The quadtree node size is 4KB, and each node can store up to 64 transaction records.In the data consistency score, the integrity of the block is calculated by verifying the block header hash chain, the transaction consistency is judged by comparing the transaction Merkle tree root value, the timestamp continuity requires that the difference between the timestamps of adjacent blocks is within 60 seconds, and the correctness of the block link is judged by verifying the hash value of the previous block. The block transmission adopts sliding window control, the window size is 4MB, and the Bloom filter parameters are set as follows: the number of hash functions is 4, the bit array size is 65536 bits, and the misjudgment rate is controlled within 0.01.
[0025] S103. Obtain the index values of the submitted log entries of each node and compare them to obtain the log synchronization progress. The log offset is obtained by calculating the byte offset of each entry in the log file. The system determines the log synchronization progress of each node by comparing the log offset and submission index between nodes. If it is found that the log offset or submission index of a node is significantly different from that of other nodes, it indicates that an accounting fork has occurred in the node.
[0026] Obtain log entry information to establish an index table, where the index node in the index table includes a sequence number, offset, length and content hash value, and the index node is maintained through a red-black tree structure; collect node log status according to the index node information to establish a state table, where the state table records the node identifier, log sequence number, file size and submission position pointer; construct a log progress matrix for the node information recorded in the state table, where the log progress matrix obtains the log synchronization rate between nodes by calculating the difference between the log sequence numbers of the row node and the column node and dividing it by the collection time interval; process the log progress difference sequence using a Kalman filter according to the log synchronization rate, and determine whether the node is in a forked state by calculating the standard deviation of the node log progress; establish a log file snapshot for the forked state node, use a Bloom filter to record the transaction hash set, and verify the consistency of the log content by comparing the transaction hash. If the verification is inconsistent, use a sliding window method to transmit the log data to the node.
[0027] Specifically, the node log file structure is obtained from the log record table, and an index table is established for log entries based on the hash index. The index node contains a 32-bit sequence number, a 64-bit offset, a 32-bit length, and a 256-bit content hash value. The index node is maintained through a red-black tree structure, and each index node is updated at intervals of 1 second. The log file is stored in blocks of a fixed size of 4MB, and the offset value of each entry is recorded in an incremental manner. The node status table is established by collecting node log status information at intervals of 1 second. The status table records the node ID, the latest log sequence number, the log file size, and the submission position pointer. The log progress matrix is constructed according to the number of nodes. The number of rows and columns in the matrix is equal to the number of nodes. The value of the matrix element is equal to the difference in the log sequence number between the row node and the column node divided by the collection time interval, and the log synchronization rate between nodes is obtained. The log progress difference sequence is preprocessed based on the Kalman filter. The fluctuation range is obtained by calculating the standard deviation of the node log progress. The potential fork state is marked for nodes whose standard deviation exceeds the preset threshold of 500 per second. The binary search method is used to locate the inconsistent position. The hash value of the 256KB data block is compared each time, and the precise fork position is located up to 20 times. According to the fork position, the log file snapshot is traced back 200 blocks forward, and the transaction hash set is recorded by using 4 hash functions and a Bloom filter with a bit array size of 8 times the number of transactions. The consistency of the log content is verified by comparing the transaction hash. The sliding window method is used to transmit log data for inconsistent nodes. The window size is 1MB. The sender maintains the confirmed sequence number, and the receiver records the received sequence number. The retransmission timeout is set to 500 milliseconds. The log record table uses a sequential append write method to record transaction data. A single log entry is organized in a fixed format: the header stores metadata in 32 bytes, including an 8-byte timestamp, a 4-byte sequence number, a 4-byte transaction number, and a 16-byte checksum. The data part is variable length to store transaction content. Each transaction contains a fixed 32-byte sender address, a 32-byte receiver address, an 8-byte amount, and a 4-byte sequence number. The index table uses a red-black tree to store index nodes. The nodes are sorted by sequence number. The balance factor of the tree is kept within plus or minus 1, and the tree height does not exceed 32 layers. The node status table records the status history of the last 100 seconds. The rolling update of the status record is realized through a ring buffer. The rows and columns in the log progress matrix represent different nodes. The matrix update uses a double buffering method to avoid read-write conflicts. The Kalman filter parameter settings in data preprocessing are: process noise covariance 0.01, measurement noise covariance 0.1, and initial estimate covariance 1.0. The fork location uses a binary search, starting from the middle of the log file, and narrowing the search range by half each time. The hash value of the comparison data block is calculated using SHA-256. Log transmission uses a sliding window, which contains a maximum of 64 data blocks. Each data block is 16KB in size. The receiver uses a bitmap to mark the data block reception status. The bitmap is stored in a byte-aligned manner. The transmission bandwidth can be dynamically adjusted from 1MB / s to 10MB / s.
[0028] S104. For a node where accounting bifurcation occurs, the system first obtains the latest log block of the node and calculates the Merkle tree Hash value of the log block; compares the Hash value with the Hash value of the corresponding log block of other nodes to determine the location of the bifurcation point.
[0029] Obtain log block data in the node log storage area, wherein the log block data includes block header information and transaction records, wherein the block header information includes a pre-order hash value, a timestamp, a block height, and a transaction quantity, and the transaction record includes a sender address, a receiver address, an amount, a timestamp, and a signature; wherein the pre-order hash value is the hash value of the previous block calculated when a new block is generated, which is used to ensure the immutability of the blockchain, the timestamp records the time when the block is generated, and is used to verify the generation order of the block, the block height is used to identify the order and hierarchical relationship of the block, the transaction quantity is used to verify the integrity and validity of the block, the sender address and the receiver address are the key identifiers of the transaction, which are encrypted addresses, the amount is the amount of funds clearly recorded in the transaction, and the signature is generated by the sender using its private key to encrypt the transaction data; according to the transaction records in the log block A quad Merkle tree is constructed, wherein the leaf nodes of the quad Merkle tree store the hash value of a single transaction record, and the non-leaf nodes of the quad Merkle tree store the concatenation result of the hash values of the child nodes, and the root hash value of the tree is obtained by recursive calculation as the log block feature value; random forest is used to group the nodes of the entire network, and the random forest constructs a feature vector based on the node communication delay, synchronization rate, online time, and load status, and the communication delay between nodes is measured by a detection packet; if the communication delay between nodes is less than a preset threshold and the log synchronization rate is greater than a preset rate threshold, the nodes are divided into the same group, and the log block feature values of the nodes in the group are obtained; the log block feature values of the forked nodes are compared block by block with the nodes in the group, and the inconsistent positions are located by binary search, and the verified transactions are recorded through a Bloom filter, and the last block with the same feature value is recorded as the fork point.
[0030] Specifically, the most recently submitted log block data is read from the node log storage area. The log block contains 64 bytes of block header information and 128,000 bytes of transaction records. The block header information contains 32 bytes of pre-order hash value, 16 bytes of timestamp, 8 bytes of block height, and 8 bytes of transaction number. A single transaction record is fixed to 128 bytes, including 32 bytes of sender address, 32 bytes of receiver address, 8 bytes of amount, 8 bytes of timestamp, and 48 bytes of signature. The log block contains up to 1,000 transactions. A quad Merkle tree is generated based on the transaction records in the log block, with a maximum depth of 6 layers and an upper limit of 4,096 leaf nodes. Leaf nodes store the hash value of a single transaction record, and non-leaf nodes store the concatenation of the hash values of 4 child nodes. Each node occupies 32 bytes of storage space. The root hash value is calculated recursively as the log block feature value, and the root hash value is added to the block header. The random forest algorithm is used to group the nodes of the entire network. The forest contains 10 decision trees with a maximum depth of 8 layers. The feature vector is constructed based on the node communication delay, synchronization rate, online time, and load status. The communication delay between nodes is measured by the detection packet. The nodes with network delay less than 50 milliseconds and log synchronization rate greater than 1000 per second are divided into the same group, and the log block feature values of all nodes in the group are obtained. The log block feature values of the forked node are compared block by block with the nodes in its group. The hash values of 256 blocks are compared each time. The inconsistent positions are located by binary search starting from the latest block. Four hash functions and a Bloom filter with a bit array size of 8 times the number of transactions are used to record verified transactions. During the comparison process, the last block with the same feature value is recorded as the fork point. Log block storage adopts continuous memory allocation. The total size of a single log block is 128064 bytes, of which the block header of 64 bytes is located at the beginning of the memory, followed by the transaction record data of 128000 bytes. The transaction record is stored in 128-byte alignment. The sender and receiver addresses use the elliptic curve public key format, and the signature data uses the DER encoding format. The quad Merkle tree is built from the bottom leaf node. Every 4 leaf nodes are calculated by SHA-256 to obtain the parent node hash value. The process is repeated until the root node is generated. Each layer of the tree is stored in an array, and the array size decreases according to the power of 4. Each decision tree in the random forest is built based on different feature subsets. The feature vector contains the average and standard deviation of the round-trip delay of the last 10 detection packets, the average synchronization rate and fluctuation range in the past 1 hour, the continuous online time of the node, the CPU usage and the memory occupancy rate. In the process of node grouping, the sliding time window is used to count the feature values. The window size is 1 hour and the sliding step is 1 minute. The fork point is located using a binary search with logarithmic time complexity. The 256 blocks compared in each round are transmitted in blocks of 8MB in size. The Bloom filter uses MurmurHash3 to calculate the 4 index positions of the transaction hash, and the bit array is stored in byte alignment.
[0031] S105. According to the location of the fork point, the system obtains the complete log block after the fork point from other nodes and sends it to the node where the fork occurs. The system uses the log replication mechanism of the Raft consensus algorithm to keep the log of the forked node consistent with that of other nodes.
[0032] The node identification, load rate and response time are obtained from the node status table, and the data source nodes that meet the preset threshold are screened out according to the load rate and response time; a Bloom filter is used to compare the hash value of the log block for the data source node, and the Bloom filter obtains the master data provider with the highest data integrity according to the preset bit array and hash function; the log data is transmitted in slices through the master data provider, and the log slices obtain the transmission rate according to the preset slice size and sliding window parameters; a temporary buffer of a circular queue structure is established according to the transmission rate, and the circular queue structure stores the transactions to be written through write pointers and read pointers; a thread pool is used to asynchronously write the transactions to be written, and if the thread pool receives confirmation messages from more than half of the nodes, the submission index is updated, and the confirmation message is sent after the log entry is successfully written.
[0033] Specifically, normal nodes with a load rate of less than 50% and a network delay of less than 100 milliseconds are selected from the node status table as data source nodes. The node status table is updated every 500 milliseconds to record the node identification, load rate, response time, and synchronization status. The log block hash values of each source node are compared through the Bloom filter. The Bloom filter sets 4 hash functions, the bit array size is 16 times the number of blocks, and the misjudgment rate is controlled within 0.01. The source node with the highest data integrity is selected as the primary data provider to obtain the complete log block sequence starting from the fork point. The log data is sharded based on the processing performance of the forked node. The size of a single shard is set to 4MB. The shard contains block link information and transaction content. The sliding window is used to control data transmission. The window size is fixed to 4 shards. The transmission rate ranges from 500KB / s to 10MB / s. The transmission rate is dynamically adjusted by the processing delay of the receiver. The receiver reports the processing delay data every 100 milliseconds. The time window for forked nodes to append logs is calculated according to the preset consensus cycle. The consensus cycle is set to 1000 milliseconds. The new transaction writing operation of the forked node is suspended during the window period. A temporary buffer of a circular queue structure is established to store transactions to be written. The queue capacity is 2048 transactions, and the interval between the buffer write pointer and the read pointer does not exceed 1024 transactions. Log appending and consensus messages are processed simultaneously through multiplexing. The thread pool is used to write the appended log blocks asynchronously. The number of threads is set to twice the number of processor cores. The write operation notifies the consensus module through the callback function. After the log entry is successfully written, a confirmation message is sent to the master node. After the master node receives confirmation from more than half of the nodes, the log entry is submitted and the submission index is updated. The node status table uses a double buffer mechanism to achieve concurrent reading and writing. Each buffer contains 1024 node status record slots. The status record occupies 64 bytes, including 16-byte node identification, 8-byte timestamp, 8-byte load rate, 8-byte response time, 8-byte synchronization status, and 16-byte reserved fields. The Bloom filter bit array is stored in 64-bit alignment, and MurmurHash3 is used to generate 4 hash values. The fragment transmission uses TCP long connection. A single fragment carries a 4KB data header. The data header includes the fragment sequence number, fragment size, and checksum. The receiver maintains a sliding window buffer with a buffer size of 16MB. The bandwidth delay product is calculated by the receiving window size and the network round-trip delay to dynamically adjust the transmission rate. The ring buffer is implemented using an array. Each array element stores a single transaction data. The write pointer and the read pointer are updated through atomic operations. The write operation is blocked when the pointer interval exceeds the threshold. The asynchronous log writing uses a multi-level cache, with a memory cache of 128MB. The write thread corresponds to an independent write queue with a queue depth of 256. The write completion is notified to the consensus thread through a semaphore. The consensus thread uses edge triggering to handle the completion event. The thread pool task queue depth is 1024, and the thread idle timeout is 5 seconds.
[0034] S106. After triggering the fork restore mechanism, the system performs incremental synchronization on the log blocks after the fork point, sends logs to different followers through batch transmission and multi-threaded concurrency, writes the received log batches in parallel through the followers, and determines whether to commit the current log based on the previous log entries received.
[0035] The log data to be synchronized is obtained from the bifurcation point, and the nodes are grouped by hash ring according to the log data to be synchronized, and the node slot distribution data is obtained by the hash ring grouping; the node processing capacity parameter is determined according to the node slot distribution data, and the adaptive transmission window size is set according to the node processing capacity parameter, and the adaptive transmission window is used to control the log transmission rate; a token generation threshold is set for the log transmission rate, and if the number of tokens reaches the token generation threshold, the log block is transmitted to the writing thread through the sending thread, and the writing thread obtains the log block data; the writing speed is controlled by the watermark method according to the log block data obtained by the writing thread, and if the write buffer usage exceeds the watermark threshold, the transmission rate is reduced, and the log block data is written to the disk through page alignment; a Bloom filter index is established for the log block data written to the disk, and the continuous complete score of the log batch is calculated through the Bloom filter index, and if the continuous complete score is higher than the preset threshold, the log batch data is submitted.
[0036] Specifically, the logs to be synchronized are read from the fork point, and the follower nodes are grouped through the hash ring structure. The hash ring space is divided into 2^16 slots, and each node is responsible for 4096 consecutive slots. The overlapping rate of adjacent node slots is maintained at 20%. The transmission parameters are set based on the node processing performance. The single batch size is set to 2MB. The adaptive window is used to control the transmission rate. The window starting value is 8 batches, and the transmission rate is dynamically adjusted in the range of 200KB / s to 20MB / s. A fixed-size thread pool is created according to the number of processor cores. The thread pool contains 16 sending threads and 32 writing threads. The task submission rate is limited by the token bucket. The token generation rate is 1000 per second and the token bucket capacity is 5000. The sending thread is responsible for log block transmission, and the writing thread performs log persistence. Data is transferred between the sending thread and the writing thread through a shared memory queue, and the queue depth is set to 64. The high and low watermarks are used to control the log writing speed. The buffer usage is sampled every 100 milliseconds. The write buffer size is set to 64MB. When the buffer usage exceeds 80%, the transmission rate is halved. When the usage is less than 20%, the transmission rate increases by 200KB / s. The write buffer is aligned according to 4KB pages, and the buffer data is persisted to the disk. The submitted log entries are recorded through the Bloom filter. The Bloom filter sets 5 hash functions. The bit array size is 10 times the number of log entries. The continuity score and integrity score are calculated for each log batch. The continuity accounts for 60% to reflect the completeness of the previous log, and the integrity accounts for 40% to reflect the correctness of the batch data. The batch log is submitted when the weighted average of the two scores exceeds 0.95. The hash ring uses the consistent hashing algorithm to achieve load balancing. The hash value is obtained by calculating the node identifier through SHA-256 and mapped to the range of 0 to 2^16. The space is divided into 65536 slots. Each node is allocated 4096 consecutive slots by default. The node weight dynamically adjusts the number of allocated slots according to the processing performance. Adjacent nodes achieve fault tolerance by jointly maintaining overlapping slots. The data transmission adopts a sliding window mechanism. The window size is calculated according to the bandwidth-delay product. The initial window has 8 batches, each batch is 2MB. The sender maintains a 32KB send buffer and a 64KB retransmission buffer. The thread pool task queue adopts a multi-level feedback queue. The priority is divided into 4 levels. The queue length of each level is 256. The high priority queue time slice is 50 milliseconds. The subsequent queue time slices are doubled in sequence. The token bucket current limiter generates tokens at an interval of 1 millisecond. The write buffer adopts a double buffer design, with a single buffer of 64MB. When the buffer is full, it triggers asynchronous flushing. The write data is aligned with 4KB pages to improve the disk write efficiency. The buffer usage is sampled every 100 milliseconds to adjust the transmission rate. The Bloom filter bitmap is stored in byte alignment, and 5 different prime numbers are used as hash seeds to generate index positions. When verifying log batches, the continuity score is calculated based on the integrity of the previous log, and the integrity score is obtained by verifying the transaction signature and block link.
[0037] S107. During the fork recovery process, the system continuously monitors the network connection and health status of all nodes. If any new anomaly is detected during the recovery process, the process is temporarily stopped. When the anomaly is resolved, the fork recovery is restarted.
[0038] A heartbeat detection packet sent by a node status detection module is obtained, wherein the heartbeat detection packet includes a node identifier and resource usage data, and the resource usage data is obtained by sampling according to the processor occupancy rate, memory usage rate, and disk read / write rate; a random forest algorithm is used to evaluate the node health status according to the processor occupancy rate, memory usage rate, and disk read / write rate in the heartbeat detection packet, and if the processor occupancy rate exceeds a first preset percentage, the memory usage rate exceeds a second preset percentage, and the disk read / write rate is lower than a preset disk read / write rate, it is marked as an abnormal state; a fork recovery pause operation is performed on the abnormal state node to obtain the current log entry position and the unfinished transmission data, and the fork recovery pause operation is sent to all network nodes by broadcasting; a breakpoint resume table is established using a B-tree structure, and the breakpoint resume table records the node identifier and timestamp combination key value and the log position, data cache, and transmission status value; a health score of the abnormal state node is calculated by exponential moving average, and when the health score reaches a preset threshold, the recovery progress data is read from the breakpoint resume table, and the fork recovery process is re-executed based on the data cache.
[0039] Specifically, the node status detection module obtains the operation data of all nodes in the network, sends a heartbeat detection packet every 1000 milliseconds, and the size of the heartbeat packet is controlled at 256 bytes, including node identification, timestamp, processor occupancy, memory usage, disk read and write rate, and records 5 groups of resource usage data at a sampling frequency of 200 milliseconds to take the average value, and records the offline status of nodes that have not responded for more than 3000 milliseconds. The node health status is evaluated based on the random forest algorithm. The random forest contains 8 decision trees. The input features include response delay, resource occupancy, and log synchronization speed. The feature values are normalized to the range of 0-1, with a response delay weight of 0.4, a resource occupancy weight of 0.3, and a synchronization speed weight of 0.3. When the processor occupancy exceeds 85%, the memory usage exceeds 90%, and the disk write speed is less than 1MB per second, the abnormal mark is triggered. According to the abnormal mark, the fork recovery is suspended, the current restored log entry position and the unfinished transmission data are recorded, and the suspension status is broadcast to all network nodes. The breakpoint resume table uses a B-tree structure to store the recovery progress. The key value is a combination of node identifier and timestamp. The value includes log position, data cache, and transmission status. The receiving node stops writing logs after receiving the pause broadcast. The node in the abnormal state is isolated, and the node health score is calculated by exponential moving average. The decay factor is set to 0.8, the sampling period is 5 seconds, and the node rejoins the network after the score exceeds 80 points for 60 consecutive cycles. The recovery progress is read from the breakpoint resume table, and the fork recovery process is re-executed based on the cached data block. The node status detection adopts a hierarchical sampling architecture. The underlying resource monitoring collects data every 200 milliseconds. The latest 25 groups of sampling values are stored in the ring buffer. The buffer size is 64KB. Each sampling point contains a 16-byte timestamp, an 8-byte processor occupancy rate, an 8-byte memory usage, a 16-byte disk read and write rate, and a 16-byte network transceiver rate. The heartbeat detection data packet is encoded in TLV format, with a type field of 1 byte, a length field of 2 bytes, and a data field of variable length. The detection packet is sent via the UDP protocol, and the number of timeout retransmissions is 3. The random forest training data set contains 10,000 sets of labeled samples. The maximum depth of each decision tree is 8 layers. The feature normalization adopts the maximum and minimum method. The response delay normalization benchmark value is set to 1000 milliseconds, the resource occupancy benchmark value is set to 100%, and the synchronization speed benchmark value is set to 10,000 per second. The breakpoint resume table uses a B-tree index, the tree order is 64, the leaf node size is 4KB, the node split threshold is 85%, the merge threshold is 40%, and the key value encoding adopts a compact binary format. The health score uses a sliding time window with a window size of 300 seconds. The exponential moving average is calculated every 5 seconds. If the score is less than 60 points, it is marked as abnormal. If it exceeds 80 points and the volatility is less than 5%, it is judged to be restored. The observation period after the node joins the network is set to 180 seconds.
[0040] S108. After the fork is restored, the secondary node re-receives the log entry or re-receives the log entry, thereby restoring the financial accounting records in the cloud environment and synchronizing the ledger state of the node that caused the network partition problem with other nodes.
[0041] The ledger difference data is obtained according to the bifurcation time recorded in the node status table, and the marking result of the synchronized record is obtained through the Bloom filter, and the Bloom filter sets a hash function and a bit array for recording the mark; a transaction feature vector is constructed based on the ledger difference data, and the transaction feature vector is classified and sorted by a support vector machine to obtain a packet data packet, and the transaction feature vector includes transaction time, amount, account hash, and correlation; the packet data packet is transmitted to the target node through a multiplexed channel, and the number of multiplexed channels is determined by the number of processor cores; a hierarchical verification structure is used to verify the packet data packet, and a packet verification value is obtained by hash chain calculation, and a Merkle tree is used to verify the packet verification value; whether the packet verification is passed is determined according to the packet verification value, and if the verification is passed, a read-write lock is used to control the concurrent sharding to write the packet to the target node, and the read-write lock controls the double buffer for data caching.
[0042] Specifically, the ledger difference data is obtained according to the fork time point recorded in the node status table, and the synchronized accounting records are marked through the Bloom filter. The filter sets 6 hash functions, the bit array size is 12 times the number of records, and the error rate is controlled within 0.01. The incremental method is used to transmit 1,000 records per batch. The transmission schedule is stored in a skip table structure. The keywords include the start and end positions of the data block, and the values record the transmission status and timestamp. Based on the support vector machine, the ledger records are classified and sorted, and a four-dimensional feature vector containing transaction time, amount, account hash, and correlation is constructed. The feature value is normalized to the range of 0-1, and the transaction records are grouped and packaged. The size of a single data packet is limited to 2MB, and it is transmitted to the target node concurrently through multiplexing. The number of transmission channels is twice the number of CPU cores, and the bandwidth of each channel is limited to 50MB per second. A hierarchical verification structure is used to verify the synchronized data. The hash chain of every 1,000 records is calculated by the SHA-256 algorithm. The transaction record is verified by a Merkle tree with a maximum depth of 8 layers. The ECDSA algorithm is used for digital signature verification. Data packets that fail verification are retransmitted. Retransmission is triggered when the verification of a single data packet times out by 500 milliseconds or the verification value does not match. The transmission channel is reestablished when the number of retransmissions exceeds 3 times. The benchmark ledger status is obtained from the master node, and a read-write lock is used to control concurrent shard writing. The number of write threads is set to the number of CPU cores, the size of a single shard is set to 10,000 records, the size of the double buffer is 256MB each, and the buffer switching cycle is 1 second. When the buffer usage exceeds 75%, data reception is suspended, and transmission is resumed when it is less than 25%. During the data synchronization process, the Bloom filter generates a hash function using 6 different prime numbers as seeds. The bit array is stored in a byte-aligned manner. Every 1 million records occupies 12MB of memory space, and the filter can process 100,000 queries per second. The jump table structure adopts a 32-layer design. Each layer generates an index node with a probability of 1 / 4. The node stores an 8-byte starting position, an 8-byte ending position, a 4-byte status code, and an 8-byte timestamp. In the support vector machine feature extraction, the transaction time is recorded using a relative timestamp, the amount is normalized by logarithmic transformation, the account hash uses the last 16 bytes to calculate the similarity, and the correlation is calculated based on the node degree in the transaction graph. Multiplexed transmission uses an asynchronous channel, with an 8MB send buffer and a 16MB receive buffer. The channel is automatically closed if it is idle for more than 5 seconds. In the layered verification, the hash chain is stored in 4KB alignment, the Merkle tree is built from the bottom up, the leaf node stores a single transaction hash, and the parent node is calculated by cascading the child node hash value. The ECDSA signature uses the secp256k1 curve, and the signature data is appended to the end of the transaction record. The double buffer write adopts a ping-pong buffer design. The background buffer is full and triggers switching. The write thread accesses the buffer synchronously through a spin lock. The read-write lock sets a 100 microsecond spin timeout, and falls into sleep waiting after the timeout.
[0043] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and supplements without departing from the principle of the present invention. These improvements and supplements should also be regarded as the scope of protection of the present invention.
Claims
1. A bookkeeping method based on a distributed ledger, characterized in that: The method comprises: A distributed ledger system based on the Raft consensus algorithm is deployed in a cloud environment. Several nodes of the distributed ledger system work together. In the log replication phase, the master node copies new transaction log entries to other nodes and waits for the nodes to confirm before submitting the logs. When the master node is partitioned during the log replication phase, the network partition is detected and heartbeat messages are sent regularly between nodes within a preset time. If a node does not receive a heartbeat from the leader or other nodes within the preset time, it is determined that the node is disconnected and the disconnected node starts the fork recovery mechanism. Obtain the index values of the submitted log entries of each node and compare them to obtain the log synchronization progress. The log offset is obtained by calculating the byte offset of each entry in the log file. The system determines the log synchronization progress of each node by comparing the log offset and submission index between nodes. If a node's log offset or submission index is found to be significantly different from that of other nodes, it indicates that the node has a bookkeeping fork. For a node with a bookkeeping fork, the system first obtains the node's latest log block, calculates the Merkle tree Hash value of the log block, and compares the Hash value with the Hash value of the corresponding log block of other nodes to determine the location of the fork point; According to the location of the fork point, the system obtains the complete log block after the fork point from other nodes and sends it to the node where the fork occurs. The system uses the log replication mechanism of the Raft consensus algorithm to keep the log of the forked node consistent with other nodes; After the fork-restore mechanism is triggered, the system performs incremental synchronization of log blocks after the fork point, sends logs to different followers through batch transmission and multi-threaded concurrency, writes received log batches in parallel through followers, and determines whether to commit the current log based on the previous log entries received; During the fork recovery process, the system continuously monitors the network connection and health status of all nodes. If any new anomalies are detected during the recovery process, the process is temporarily stopped. When the anomaly is resolved, the fork recovery is restarted. After the fork is recovered, the secondary node re-receives the log entry or re-receives the log entry, thereby restoring the financial accounting records in the cloud environment and synchronizing the ledger state of the node that caused the network partition problem with other nodes.
2. The method according to claim 1, characterized in that The distributed ledger system based on the Raft consensus algorithm is deployed in a cloud environment. Several nodes of the distributed ledger system work together. In the log replication phase, the master node copies the new transaction log entry to other nodes and waits for the node to confirm before submitting the log, including: Obtain a running node address list from a registry, send a heartbeat detection packet to the running node address list, and score and sort the online nodes according to the response time of the heartbeat detection packet; Calculate the integrity checksum of the most recent block data for the online nodes, and select the node with the highest response time score and the largest data integrity checksum as the master node; After receiving the transaction data, the master node generates a block structure, wherein the block structure includes a block header record and a block body record, wherein the block header record includes a timestamp, a hash value of a previous block, and a Merkle tree root value, and the block body record includes a transaction list; The timestamp is generated when the master node receives new transaction data, identifies the exact time when the block is created, and is used to verify the order of transactions. The hash value of the previous block is calculated by the hash algorithm with reference to the information of the previous block when the master node generates a new block, which is used to ensure the security and immutability of the blockchain. The Merkle tree root value is obtained by hashing the transaction data in the block, which is used to improve the integrity and validity of the data. The transaction list is a transaction list formed by storing the verified valid transactions in the block body in order. The master node sends a detection packet to each slave node to obtain a round trip delay, and the slave nodes are divided into node groups according to the round trip delay, wherein the node groups include a fast group, a medium speed group and a slow speed group; For the node group, the master node sends log data according to the preset batch value. After receiving the log data, the slave node calculates the Merkle tree root value and compares it with the master node data. If the comparison is consistent, a confirmation message is returned. After receiving all confirmation messages, the master node submits the log.
3. The method according to claim 1, characterized in that When a network partition occurs in the log replication phase of the master node, the network partition is detected, and heartbeat messages are sent regularly between nodes within a preset time. If a node does not receive a heartbeat from the leader or other nodes within the preset time, it is determined that the node is disconnected, and the disconnected node starts a fork recovery mechanism, including: Obtain a slave node address list from the node connection table, and send a heartbeat message to each slave node according to the slave node address list, wherein the heartbeat message includes a node identifier, a timestamp, a block height, and a load status; Counting the number of disconnected nodes according to the heartbeat message response, and if the number of disconnected nodes exceeds a preset proportion of the total number of nodes, determining that a network partition occurs; A hash ring algorithm is used to calculate a hash value for the online node identifier, and the nodes are divided into groups according to the hash value. The voting weight is calculated according to the node online time, load status and response speed to obtain the regional master node. A quadtree structure is used to record the forked chain data of the network partition, wherein the tree node of the quadtree structure includes a timestamp, a data hash, and a child node pointer, and a fork record table is established to maintain the block number mapping relationship; The data consistency score of each network partition is calculated based on block integrity, transaction consistency, timestamp continuity and block link correctness, blockchain data is obtained from the network partition with the highest consistency score, and the transaction records during the fork are reordered and merged.
4. The method according to claim 1, characterized in that: The index values of the submitted log entries of each node are obtained and compared to obtain the log synchronization progress. The log offset is obtained by calculating the byte offset of each entry in the log file. The system determines the log synchronization progress of each node by comparing the log offset and the submitted index between nodes. If it is found that the log offset or submitted index of a node is significantly different from that of other nodes, it indicates that the node has an accounting fork, including: Obtain log entry information to establish an index table, where the index nodes in the index table include a sequence number, an offset, a length, and a content hash value, and the index nodes are maintained by a red-black tree structure; Collect node log status according to index node information to establish a status table, the status table records node identification, log sequence number and file size and submission position pointer; Constructing a log progress matrix for the node information recorded in the state table, wherein the log progress matrix obtains the log synchronization rate between nodes by calculating the difference between the log sequence numbers of the row nodes and the column nodes and dividing the difference by the collection time interval; A Kalman filter is used to process the log progress difference sequence according to the log synchronization rate, and a node is judged whether it is in a bifurcation state by calculating the node log progress standard deviation; A log file snapshot is created for the forked state node, and a Bloom filter is used to record the transaction hash set. The consistency of the log content is verified by comparing the transaction hash. If the verification is inconsistent, the log data is transmitted to the node using a sliding window method.
5. The method according to claim 1, characterized in that: For the node with accounting bifurcation, the system first obtains the latest log block of the node and calculates the Merkle tree Hash value of the log block; compares the Hash value with the Hash value of the corresponding log block of other nodes to determine the location of the bifurcation point, including: Obtain log block data in the node log storage area, wherein the log block data includes block header information and transaction records, wherein the block header information includes a preceding hash value, a timestamp, a block height, and a transaction quantity, and the transaction record includes a sender address, a receiver address, an amount, a timestamp, and a signature; Among them, the pre-order hash value is the hash value of the previous block calculated when generating a new block, which is used to ensure the immutability of the blockchain. The timestamp records the time when the block was generated and is used to verify the generation order of the blocks. The block height is used to identify the order and hierarchical relationship of the blocks. The transaction quantity is used to verify the integrity and validity of the block. The sender address and the receiver address are the key identifiers of the transaction and are encrypted addresses. The amount is the amount of funds clearly recorded in the transaction. The signature is generated by the sender using its private key to encrypt the transaction data. A quad Merkle tree is constructed according to the transaction records in the log block, wherein the leaf nodes of the quad Merkle tree store the hash value of a single transaction record, and the non-leaf nodes of the quad Merkle tree store the concatenation result of the hash values of the child nodes, and the root hash value of the tree is obtained by recursive calculation as the log block feature value; The nodes in the whole network are grouped by using random forest. The random forest constructs feature vectors based on node communication delay, synchronization rate, online time and load status, and measures the communication delay between nodes through detection packets. If the inter-node communication delay is less than a preset threshold and the log synchronization rate is greater than a preset rate threshold, the nodes are divided into the same group, and the log block feature values of the nodes in the group are obtained; The log block feature values of the forked node are compared block by block with those of the nodes in the group, and the inconsistent positions are located using binary search. The verified transactions are recorded through the Bloom filter, and the last block with the same feature value is recorded as the fork point.
6. The method according to claim 1, characterized in that According to the location of the fork point, the system obtains the complete log block after the fork point from other nodes and sends it to the node where the fork occurs. The system uses the log replication mechanism of the Raft consensus algorithm to make the log of the forked node consistent with that of other nodes, including: Obtaining the node identification, load rate and response time from the node status table, and filtering out data source nodes that meet the preset threshold according to the load rate and response time; A Bloom filter is used to compare the hash values of log blocks for the data source node, and the Bloom filter obtains the primary data provider with the highest data integrity according to a preset bit array and a hash function; The master data provider transmits the log data in slices, and the log slices obtain a transmission rate according to a preset slice size and a sliding window parameter; Establishing a temporary buffer of a ring queue structure according to the transmission rate, wherein the ring queue structure stores transactions to be written through a write pointer and a read pointer; A thread pool is used to asynchronously write the transaction to be written. If the thread pool receives confirmation messages from more than half of the nodes, the submission index is updated. The confirmation message is sent after the log entry is successfully written.
7. The method according to claim 1, characterized in that After the fork-restore mechanism is triggered, the system performs incremental synchronization on the log blocks after the fork point, sends logs to different followers through batch transmission and multi-threaded concurrency, writes received log batches in parallel through followers, and determines whether to commit the current log based on the previous log entries received, including: Obtain the log data to be synchronized from the bifurcation point, perform hash ring grouping on the nodes according to the log data to be synchronized, and obtain node slot distribution data through the hash ring grouping; Determine a node processing capacity parameter according to the node slot distribution data, and set an adaptive transmission window size according to the node processing capacity parameter, wherein the adaptive transmission window is used to control the log transmission rate; A token generation threshold is set for the log transmission rate. If the number of tokens reaches the token generation threshold, the log block is transmitted to the writing thread through the sending thread, and the writing thread obtains the log block data. The write speed is controlled by a watermark method according to the log block data obtained by the write thread. If the write buffer usage exceeds the watermark threshold, the transmission rate is reduced and the log block data is written to the disk by page alignment. A Bloom filter index is established for the log block data written to the disk, and a continuous complete score of the log batch is calculated using the Bloom filter index. If the continuous complete score is higher than a preset threshold, the log batch data is submitted.
8. The method according to claim 1, characterized in that: During the fork recovery process, the system continuously monitors the network connection and health status of all nodes. If any new anomalies are detected during the recovery process, the process is temporarily stopped. When the anomaly is resolved, the fork recovery is restarted, including: Obtain a heartbeat detection packet sent by a node status detection module, wherein the heartbeat detection packet includes a node identifier and resource usage data, and the resource usage data is obtained by sampling a processor occupancy rate, a memory usage rate, and a disk read / write rate; According to the processor occupancy rate, memory usage rate and disk read / write rate in the heartbeat detection packet, a random forest algorithm is used to evaluate the node health status. If the processor occupancy rate exceeds a first preset percentage, the memory usage rate exceeds a second preset percentage, and the disk read / write rate is lower than a preset disk read / write rate, it is marked as an abnormal state; Perform a fork recovery pause operation on the abnormal state node to obtain the current log entry position and unfinished transmission data, and the fork recovery pause operation is sent to all network nodes by broadcasting; A breakpoint-resume table is established using a B-tree structure, wherein the breakpoint-resume table records a node identifier and a timestamp combination key value, a log position, a data cache, and a transmission status value; The health score of the abnormal state node is calculated by exponential moving average. When the health score reaches a preset threshold, the recovery progress data is read from the breakpoint resume table, and the fork recovery process is re-executed based on the data cache.
9. The method according to claim 1, characterized in that: After the fork is restored, the secondary node re-receives the log entry or re-receives the log entry, thereby restoring the financial accounting records in the cloud environment and synchronizing the ledger state of the node that caused the network partition problem with other nodes, including: The ledger difference data is obtained according to the fork time recorded in the node status table, and the marking result of the synchronized record is obtained through the Bloom filter, wherein the Bloom filter sets a hash function and a bit array for recording the marking; Constructing a transaction feature vector based on the account book difference data, and classifying and sorting the transaction feature vector by a support vector machine to obtain a grouped data packet, wherein the transaction feature vector includes transaction time, amount, account hash and correlation; Transmitting the packet data packet to the target node via multiplexing channels, wherein the number of the multiplexing channels is determined by the number of processor cores; A hierarchical verification structure is used to verify the packet data, a packet verification value is obtained by hash chain calculation, and a Merkle tree is used to verify the packet verification value; Whether the data packet is verified is determined according to the data packet check value. If the verification is passed, a read-write lock is used to control concurrent fragmentation to write the data packet to the target node. The read-write lock controls the double buffer for data caching.
Citation Information
Cited By
Node detection method and device, electronic equipment and storage medium
CN120416088A
Method for correspondingly managing IPV4 and IPV6 addresses of same user of network dual stacks
CN120729831A
Method for managing ipv4 and ipv6 addresses corresponding to a same user in a network dual stack
CN120729831B
Block chain-based bulk commodity transaction system
CN121481660A