A distributed consistency method, apparatus, device, storage medium and product

Abstract

The application discloses a distributed consistency method, device, equipment, storage medium and product. The method comprises the following steps: in the process that a to-be-joined node joins a high-availability cluster, a master node and the to-be-joined node exchange remote direct memory access remote addresses of themselves; a two-phase commit protocol is initiated, and the master node encapsulates and writes data sent by a client into the remote direct memory access remote address of a newly-joined node; the newly-joined node checks, through a coroutine receiving a remote direct memory access request, whether a new distributed consistency sending message exists on a local address, and writes a processed reply message into the remote direct memory access remote address of the master node after the new distributed consistency sending message is found; and the master node checks, through a coroutine replying to a remote direct memory access request, whether a new distributed consistency reply message exists on a local address, and processes the new distributed consistency reply message after the new distributed consistency reply message is found. The method can effectively reduce consistency delay.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of data consistency technology, and in particular to a distributed consistency method, apparatus, device, storage medium and product. Background Technology

[0002] The Paxos consensus algorithm is the theoretical foundation of distributed consensus algorithms. It is used to achieve data consistency among a majority of replicas in a distributed environment. The Paxos consensus algorithm mainly relies on the message passing model and uses a specific communication protocol to achieve consistency among distributed nodes.

[0003] The MGR Paxos algorithm is an optimization of the Paxos consensus algorithm. It mainly improves performance by reducing the prepare stage and log write latency by designating a master node.

[0004] However, both the Paxos consensus algorithm and the MGR Paxos algorithm use core Internet communication protocols, such as the Transmission Control Protocol (TCP). Since distributed consensus algorithms based on the TCP / IP communication protocol rely on the network layer of the operating system kernel for their communication process, this can lead to TCP protocol stack latency. Summary of the Invention

[0005] This invention provides a distributed consensus method, apparatus, device, storage medium, and product to solve the problem that existing distributed consensus algorithms suffer from a sharp increase in consistency latency due to the communication process relying on the network layer of the operating system kernel.

[0006] According to one aspect of the present invention, a distributed consensus method is provided, comprising:

[0007] During the process of a node joining a high-availability cluster, the master node and the node to be joined exchange their remote addresses for remote direct memory access. These remote addresses are used to form a security mechanism for remote direct memory access.

[0008] A two-phase commit protocol is initiated, in which the master node encapsulates the data sent by the client and writes it into the remote address of the newly joined node's remote direct content access. The node to be joined becomes the newly joined node after joining the high-availability cluster.

[0009] The newly added node, through a coroutine that receives remote direct memory access requests, polls and checks whether there are new distributed consistency messages on its local address, and then writes the processed reply message to the remote address of the master node's remote direct memory access.

[0010] The master node processes the new distributed consistency response message after polling and detecting the existence of a new distributed consistency response message on the local address through the coroutine that responds to remote direct memory access requests.

[0011] According to another aspect of the present invention, a distributed consensus device is provided, comprising:

[0012] The switching module 110 is used to exchange the remote addresses of remote direct memory access between the master node and the node to be joined during the process of the node to be joined joining the high availability cluster. The remote addresses are used to form a security mechanism for remote direct memory access.

[0013] The first write module 120 is used to initiate a two-phase commit protocol. The master node encapsulates the data sent by the client and writes it to the remote address of the remote direct content access of the newly joined node. The node to be joined becomes the newly joined node after joining the high availability cluster.

[0014] The second writing module 130 is used by the newly added node to poll and check the local address for a new distributed consistency sending message by a coroutine that receives a remote direct memory access request, and then write the processed reply message to the remote address of the master node's remote direct memory access.

[0015] The processing module 140 is used by the master node to process the new distributed consistency reply message after polling and detecting the existence of a new distributed consistency reply message on the local address through the coroutine that replies to the remote direct memory access request.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: at least one processor;

[0017] and a memory communicatively connected to the at least one processor;

[0018] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the distributed consensus method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the distributed consensus method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the distributed consensus method described in any embodiment of the present invention.

[0021] The technical solution of this invention uses one-sided read and write operations of remote direct memory access, which solves the problem of a sharp increase in consistency latency and achieves the beneficial effect of effectively reducing consistency latency.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a distributed consensus method provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart illustrating a distributed consensus method provided in Embodiment 2 of the present invention;

[0026] Figure 3 This is a schematic diagram of the structure of a distributed consensus device provided in Embodiment 3 of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the distributed consensus method of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention. It should be understood that the various steps described in the method embodiments of the present invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0029] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] It should be noted that the terms "a" and "a plurality of" used in this invention are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a distributed consistency method provided in Embodiment 1 of the present invention. This method is applicable to situations where data consistency is achieved among a majority of replicas in a distributed environment. This method can be executed by a distributed consistency device, which can be implemented by software and / or hardware and is generally integrated on an electronic device. In this embodiment, the electronic device includes, but is not limited to, a computer device.

[0035] like Figure 1 As shown, a distributed consensus method provided in Embodiment 1 of the present invention includes the following steps:

[0036] S110. During the process of a node joining a high-availability cluster, the master node and the node to be joined exchange their remote addresses for remote direct memory access. These remote addresses are used to form a security mechanism for remote direct memory access.

[0037] Remote Direct Memory Access (RDMA) is a network technology that allows a computer to directly access the memory of another computer without the intervention of the operating system kernel and central processing unit.

[0038] The remote address consists of raddr and rkey. raddr and rkey work together to form the security mechanism for RDMA remote memory access. raddr represents the memory address of the remote machine, used to directly read and write the other party's memory without CPU intervention. rkey is the security access credential in RDMA technology, equivalent to the "password" for remote memory, used to authorize other hosts to operate on locally registered memory areas through RDMA.

[0039] In this embodiment, after the high-availability cluster selects a master node, it can send a cluster formation command to the master node and a join command to a new node to be joined. The node to be joined still uses the core communication protocol of the Internet to send a join request to the master node, and at this time, the RDMA initialization operation is performed. After the master node hears the join request, it executes the distributed consensus protocol to enable each node in the high-availability cluster to reach a consensus protocol. The master node sends its own RDMA remote address to the node to be joined, and the node to be joined passes its own RDMA remote address to the master node. The master node sends a snapshot of the state information of the high-availability cluster to the node to be joined, so that the node to be joined can determine the node pair based on the snapshot, and realize the exchange of RDMA remote addresses between the two nodes in each node pair.

[0040] In this embodiment, each time the master node hears a join request, it creates a coroutine to receive core communication protocol requests and a coroutine to receive direct memory access requests. The coroutine to receive remote direct memory access requests is used to receive remote direct memory access requests from the consensus protocol.

[0041] Each time a new node, i.e. a node to be added, wants to join the high-availability cluster, it sends a join request to the master node. When the master node hears a join request, in addition to creating a coroutine acceptor_learner_task to receive core communication protocol requests, it can also create an additional coroutine rdma_learner_task to receive direct memory access requests. acceptor_learner_task is used to handle requests sent using core Internet communication protocols, such as TCP / IP requests; rdma_learner_task is used to receive RDMA requests for the consensus protocol.

[0042] In this embodiment, each time the master node hears a join request, it creates a coroutine to reply to the core communication protocol request and a coroutine to reply to the direct memory access request. The coroutine to reply to the direct memory access request is used to handle the remote direct memory access request of the consistency protocol.

[0043] Each time a new node (i.e., a node to be added) wants to join the high-availability cluster, it sends a join request to the master node. When the master node hears a join request, in addition to creating a coroutine `reply_handler_task` to reply to core communication protocol requests, it can also create an additional coroutine `reply_rdma_handler_task` to reply to direct memory access requests. `reply_handler_task` is used to handle requests sent using core Internet communication protocols, such as TCP / IP requests; `reply_rdma_handler_task` is used to handle RDMA requests sent using consistency protocols.

[0044] S120. Initiate a two-phase commit protocol. The master node encapsulates the data sent by the client and writes it to the remote address of the newly joined node's remote direct content access.

[0045] The node to be added is considered as the newly added node after it joins the high-availability cluster.

[0046] Among them, the Two-Phase Commit Protocol (2PC) is the core protocol for ensuring the atomicity of transactions in distributed systems. It is used to ensure that transactions across multiple nodes are either all committed or all rolled back.

[0047] In this embodiment, the client sends an SQL request to the database read / write instance node, i.e., the master node. The proposer_task goroutine retrieves the data sent by the client from the pipeline and initiates a two-phase commit protocol. This process iterates through each node in the high-availability cluster and wakes up the corresponding sender_task goroutine of the node.

[0048] In this embodiment, the sender_task goroutine of the master node checks whether the RDMA flag is set. If it is, the data sent by the client is directly encapsulated and written to the RDMA remote address of the newly joined node in IBV_WR_RDMA_WRITE mode. The system then checks whether the completion queue has been successfully written. If the writing is unsuccessful, the Internet core communication protocol is still used for communication.

[0049] S130. The newly added node, through the coroutine that receives the remote direct memory access request, polls and checks whether there is a new distributed consistency sending message on the local address, and then writes the processed reply message to the remote address of the master node's remote direct memory access.

[0050] Among them, the coroutine of the newly added node that receives remote direct memory access requests periodically polls the local address configured to see if there are any new Paxos messages. If so, it reads the message, enters the processing flow, and returns a processed reply message. The reply message is written to the remote address of the remote direct memory access of the master node to be added in the form of IBV_WR_RDMA_WRITE.

[0051] S140. The master node, through the coroutine that replies to the remote direct memory access request, polls and detects the existence of a new distributed consistency reply message on the local address, and then processes the new distributed consistency reply message.

[0052] The master node uses the coroutine reply_rdma_handler_task, which responds to remote direct memory access requests, to poll and check if there are any new Paxos messages on the configured local address. If not, it means that no new data has arrived in this poll. If so, it reads a new Paxos message of a specified length, enters the processing flow, and clears this memory segment.

[0053] Furthermore, each node in the high-availability cluster maintains a data structure, which is used to maintain the remote address of the node's remote direct memory access for writing messages in a preset writing manner; wherein, the node that receives the message obtains the message length from the received remote direct memory access remote address and dynamically adjusts the receive buffer offset according to the message length.

[0054] In this high-availability cluster, each node maintains a data structure that stores the remote address of the remote direct memory access (RDM) written to the node in the IBV_WR_RDMA_WRITE mode. The node that receives the message can obtain the message length from the content parsed from the remote address of the RDM, and thus the physical address of the next message can be increased by the response offset.

[0055] This completes one round of message processing based on RDMA. It should be noted that after using the RDMA ibv_post_send function, the requesting end receives the acknowledgment message from the network card, generates a completion queue entry, places it in the completion queue, and the task is completed.

[0056] The distributed consistency method provided in Embodiment 1 of this invention firstly involves the master node and the node to be joined exchanging their remote direct memory access (RDA) addresses during the process of a node joining a high-availability cluster. These remote addresses form the security mechanism for RDA. Then, a two-phase commit protocol is initiated. The master node encapsulates the data sent by the client and writes it to the RDA address of the newly joined node. Next, the newly joined node, through a coroutine that receives RDA requests, polls and checks its local address for new distributed consistency messages, then writes the processed reply message to the master node's RDA address. Finally, the master node, through a coroutine that replies to RDA requests, polls and checks its local address for new distributed consistency reply messages, and processes these new messages. In this method, each node processes requests through coroutines that receive and reply to RDA requests, without requiring intervention from the operating system kernel and CPU. By bypassing the traditional TCP / IP protocol stack, latency is significantly reduced and throughput is increased.

[0057] Example 2

[0058] Figure 2 This is a flowchart illustrating a distributed consensus method according to Embodiment 2 of the present invention. Embodiment 2 is an optimization based on the above embodiments. For details not covered in this embodiment, please refer to Embodiment 1.

[0059] like Figure 2 As shown in Embodiment 2 of the present invention, a distributed consensus method includes the following steps:

[0060] S210. The node to be joined sends a join request to the master node using the core communication protocol of the Internet and performs the initialization operation of remote direct memory access.

[0061] The high-availability cluster can send a cluster formation command to the selected master node and a join command to the node to be joined. After receiving the command, the node to be joined can send a join request of type add_node_type to the master node using the TCP / IP protocol and perform RDMA initialization operation.

[0062] S220. After the master node listens to the join request through the coroutine that receives the core communication protocol request, it executes the distributed consensus protocol on the join request. After the protocol is reached, each node in the high availability cluster executes the distributed consensus protocol.

[0063] In this high-availability cluster, each node executes the Paxos consensus protocol, which adds node information to the cluster status information. The cluster status information records the liveness status of each node.

[0064] S230. After the master node detects that the status information of the high-availability cluster has changed, it sends the remote address of the master node's remote direct memory access to the node to be joined.

[0065] Specifically, after the master node detects a change in the status information of the high-availability cluster through the liveness detection goroutine, it sends a liveness detection message to the node to be joined through a goroutine that sends the core communication protocol. The liveness detection message includes the remote address of the master node's remote direct memory access.

[0066] Furthermore, sending a liveness probe message to the node to be joined via a coroutine that sends the core communication protocol includes: establishing a communication connection between the coroutine that sends the core communication protocol and the node to be joined, creating a reliable connection protection domain, creating a reliable connection completion queue, creating a work completion array, registering remotely accessible local memory, adding the remote address of the master node's remote direct memory access to the liveness probe message, and sending the liveness probe message to the node to be joined.

[0067] S240, the node to be joined saves the remote address of the master node's remote direct memory access into its own memory, passes the remote address of the node to be joined to the master node, and requests to obtain the status information of the high availability cluster.

[0068] Specifically, after receiving the liveness probe message through the coroutine that receives the core communication protocol request, the node to be joined responds to the liveness probe message, extracts the remote address of the master node's remote direct memory access from the liveness probe message, and saves the remote address of the master node's remote direct memory access to its own memory; at the same time, the node to be joined creates a reliable connection protection domain, a reliable connection completion queue, and a work completion array, and registers remote access to local memory, adds the remote address of the master node's remote direct memory access to the response liveness probe message, and sends the liveness probe message to the master node; simultaneously, the node to be joined requests to obtain the status information of the high availability cluster.

[0069] S250, the master node saves the remote address of the node to be joined to its own memory and sends a snapshot of the status information of the high availability cluster to the node to be joined.

[0070] In this process, after the master node receives the information of all nodes in the high-availability cluster from the coroutine that responds to the core communication protocol request, it obtains the remote address of the remote direct memory access of the node to be joined from the response liveness message and saves it in memory; it sends a snapshot of the status information of the high-availability cluster to the node to be joined, and subsequent messages can be written directly to the remote address of the remote direct memory access of the node to be joined using RDMA.

[0071] S260. After the node to be added receives the snapshot through the coroutine that replies to the core communication protocol request, it executes steps S230 and S240 for each node pair to realize the exchange of remote addresses for remote direct memory access between the two nodes in each node pair.

[0072] Here, a node pair consists of the node to be added and each node in the high-availability cluster.

[0073] In this embodiment, each time the master node hears a join request, it creates a coroutine to receive core communication protocol requests and a coroutine to receive direct memory access requests. The coroutine for receiving remote direct memory access requests is used to receive remote direct memory access requests from the consensus protocol. Each time the master node hears a join request, it creates a coroutine to reply to core communication protocol requests and a coroutine to reply to direct memory access requests. The coroutine for replying to direct memory access requests is used to process remote direct memory access requests from the consensus protocol.

[0074] S270. Initiate a two-phase commit protocol, whereby the master node encapsulates the data sent by the client and writes it to the remote address of the newly joined node's remote direct content access.

[0075] The node to be added is considered as the newly added node after it joins the high-availability cluster.

[0076] S280. The newly added node, through the coroutine that receives the remote direct memory access request, polls and checks whether there is a new distributed consistency sending message on the local address, and then writes the processed reply message to the remote address of the master node's remote direct memory access.

[0077] S290. The master node, through the coroutine that replies to remote direct memory access requests, polls and detects the existence of new distributed consistency reply messages on the local address, and then processes the new distributed consistency reply messages.

[0078] The second embodiment of the present invention provides a distributed consensus method that can significantly reduce latency and improve throughput.

[0079] Based on the technical solutions of the above embodiments, this invention provides a specific implementation method.

[0080] As a specific implementation method of this embodiment, a distributed consensus method includes a cluster formation process and a protocol achievement process.

[0081] The cluster setup process includes the following steps:

[0082] 1. The high-availability cluster selects the master node node0 based on the unique identifier of the transaction.

[0083] 2. The high-availability cluster sends a cluster formation command to node0.

[0084] 3. The high-availability cluster sends a command to node1 to join the cluster.

[0085] 4. After node1 joins the high-availability cluster, it sends an add_node_type join request to node0. At this time, the TCP / IP protocol is still used to perform RDMA initialization operation.

[0086] 5. When node0 listens for a join request, the goroutine listening for the peer request is still acceptor_learner_task, which is the goroutine that receives the core communication protocol request. It executes the Paxos consensus protocol on the request. After the protocol is reached, each node in the high-availability cluster executes the consensus protocol.

[0087] 6. When the live_task of node0 detects a change in the cluster state information, it sends a live_task message to node1. The live_task message is sent by the sender_task of node0, which sends the core communication protocol. This goroutine first establishes a TCP connection with node1 and sends the live_task message. At this time, a reliable connection protection domain is also created, a reliable connection completion queue is created, a work completion array is created, remote access to local memory is registered, and (raddr, rkey) that node1 can directly access from node0 is included in the live_task message.

[0088] 7. After receiving the liveness detection message sent by node0, node1's acceptor_learner_task, which is the coroutine that receives requests from the core communication protocol, responds to the liveness detection message by retrieving node0's (raddr, rkey) from the message and storing it in node1's memory. At the same time, node1 creates a reliable connection protection domain, a reliable connection completion queue, a work completion array, registers remote access to local memory, and includes (raddr, rkey) that node0 can directly access in the response message. It also requests messages from all nodes in the high-availability cluster from node0.

[0089] 8. After receiving the message from node1 requesting to obtain all nodes in the high-availability cluster in reply_handler_task, node0 retrieves node1's (raddr, rkey) from the response liveness message and saves it in node0's memory. It then sends the snapshot information of the cluster where node0 is located to node1. Subsequent messages can be written directly to node1's raddr using RDMA.

[0090] 9. After receiving the status information of all nodes in the cluster sent by node0, node1 synchronizes the (raddr, rkey) of the peer node with the nodes in the cluster using steps 6 to 8.

[0091] 10. Whenever node0 hears a new node joining request, in addition to creating the acceptor_learner_task coroutine, it also creates an rdma_learner_task coroutine, which is used to receive direct memory access requests. This coroutine is used to receive rdma requests for the consensus protocol.

[0092] 11. Whenever node0 detects a new node joining request, in addition to creating the reply_handler_task coroutine, it also creates the reply_rdma_handler_task coroutine, which is used to handle RDMA requests.

[0093] The agreement-reaching process includes the following steps:

[0094] 1. The client sends an SQL request to the database read / write instance node0. The proposer_task goroutine retrieves the data sent by the client from the pipeline and initiates a two-phase commit protocol. This process will loop through each node in the high-availability cluster and wake up the corresponding sender_task goroutine on each node.

[0095] 2. The sender_task goroutine of node0 checks whether the RDMA flag is set. If it is, it encapsulates the data and writes it to the raddr of node1 in the IBV_WR_RDMA_WRITE mode, and checks whether the completion queue has been successfully written. If not, it still uses TCP for communication.

[0096] 3. The rdma_learner_task coroutine of node1 periodically polls the configured local address for new Paxos messages. If so, it reads the message, processes it, and returns a processed reply message. The reply message is written to the raddr of node0 in the form of IBV_WR_RDMA_WRITE.

[0097] 4. The reply_rdma_handler_task polls to check if there are any new PaxosReply messages on the configured local address. If so, it receives the message and processes it. This node can retain the offset of the message read in the last poll. The next poll will read from after that offset. First, it checks whether the message header after the offset contains a valid data length. If not, it means that no new data has arrived in this poll. If so, it reads the message of the specified length, processes it, and clears this memory segment.

[0098] 5. Each node in the cluster maintains a data structure that is used to maintain the (raddr, rkey) of messages written to the node in the IBV_WR_RDMA_WRITE mode. The node that receives the message can obtain the message length from the content parsed by RDMA, and then add the response offset to the physical address of the next message received. It should be noted that this memory is used cyclically and can be used for message flow control mechanisms.

[0099] At this point, a round of message processing based on RDMA is complete. It should be noted that after using the ibv_post_send function of RDMA, the requesting end generates a completion queue entry after receiving the ACK from the network card, places it in the completion queue, and the task is completed.

[0100] Example 3

[0101] Figure 3 This is a schematic diagram of a distributed consistency device provided in Embodiment 3 of the present invention. The device is applicable to situations where data consistency is achieved among a majority of replicas in a distributed environment. The device can be implemented by software and / or hardware and is generally integrated into an electronic device.

[0102] like Figure 3 As shown, the device includes: a switching module 110, a first writing module 120, a second writing module 130, and a processing module 140.

[0103] The switching module 110 is used to exchange the remote addresses of remote direct memory access between the master node and the node to be joined during the process of the node to be joined joining the high availability cluster. The remote addresses are used to form a security mechanism for remote direct memory access.

[0104] The first write module 120 is used to initiate a two-phase commit protocol. The master node encapsulates the data sent by the client and writes it to the remote address of the remote direct content access of the newly joined node. The node to be joined becomes the newly joined node after joining the high availability cluster.

[0105] The second writing module 130 is used by the newly added node to poll and check the local address for a new distributed consistency sending message by a coroutine that receives a remote direct memory access request, and then write the processed reply message to the remote address of the master node's remote direct memory access.

[0106] The processing module 140 is used by the master node to process the new distributed consistency reply message after polling and detecting the existence of a new distributed consistency reply message on the local address through the coroutine that replies to the remote direct memory access request.

[0107] In this embodiment, the device first uses the exchange module 110 to exchange the remote addresses of their Remote Direct Memory Access (RDA) during the process of a node joining a high-availability cluster. These remote addresses are used to form a security mechanism for RDA. Then, the first write module 120 initiates a two-phase commit protocol. The master node encapsulates the data sent by the client and writes it to the remote address of the newly joined node's RDA. The node to be joined becomes the newly joined node after joining the high-availability cluster. Next, in the second write module 130, the newly joined node, through a coroutine that receives RDA requests, polls and checks for new distributed consistency messages on its local address, and then writes the processed reply message to the remote address of the master node's RDA. Finally, in the processing module 140, the master node, through a coroutine that replies to RDA requests, polls and checks for new distributed consistency reply messages on its local address, and then processes the new distributed consistency reply messages.

[0108] This embodiment provides a distributed consistency device that can effectively reduce consistency latency.

[0109] Furthermore, the switching module 110 includes:

[0110] The first execution submodule is used by the node to be joined to send a join request to the master node using the core communication protocol of the Internet and to perform the initialization operation of remote direct memory access.

[0111] The second execution submodule is used to execute a distributed consensus protocol on the join request after the master node listens to the join request through a coroutine that receives core communication protocol requests. After the protocol is reached, each node in the high-availability cluster executes the distributed consensus protocol.

[0112] The sending submodule is used to send the remote address of the master node's remote direct memory access to the node to be joined after the master node detects that the status information of the high availability cluster has changed.

[0113] The transmission submodule is used by the node to be joined to save the remote address of the master node's remote direct memory access into its own memory, transmit the remote address of the node to be joined to the master node, and request to obtain the status information of the high availability cluster.

[0114] The storage submodule is used by the master node to save the remote address of the node to be added to its own memory and send a snapshot of the status information of the high availability cluster to the node to be added.

[0115] The interaction submodule is used to exchange remote addresses for remote direct memory access between the two nodes in each node pair after the node to be joined receives the snapshot through a coroutine that replies to the core communication protocol request. The node pair consists of the node to be joined and each node in the high availability cluster.

[0116] Based on the above optimizations, the sending submodule is specifically used for: after the master node detects a change in the status information of the high-availability cluster through the liveness detection coroutine, it sends a liveness detection message to the node to be added through a coroutine of the core communication protocol. The liveness detection message includes the remote address of the master node's remote direct memory access.

[0117] The process of sending a liveness probe message to the node to be joined via a coroutine that sends the core communication protocol includes: establishing a communication connection between the coroutine that sends the core communication protocol and the node to be joined; creating a reliable connection protection domain; creating a reliable connection completion queue; creating a work completion array; registering remote access to local memory; adding the remote address of the master node's remote direct memory access to the liveness probe message; and sending the liveness probe message to the node to be joined.

[0118] Based on the above technical solution, the transmission submodule specifically includes:

[0119] The storage unit is used to respond to the liveness detection message after the node to be added receives the liveness detection message through the coroutine that receives the core communication protocol request, extract the remote address of the master node's remote direct memory access from the liveness detection message, and save the remote address of the master node's remote direct memory access into its own memory.

[0120] The addition unit is used to create a reliable connection protection domain, a reliable connection completion queue, and a work completion array for the node to be added, and to register remotely accessible local memory, add the remote address of the master node's remote direct memory access to the response liveness probe message, and send the liveness probe message to the master node.

[0121] The acquisition unit is used for the node to be added to request the status information of the high-availability cluster.

[0122] Furthermore, the device also includes a first creation module and a second creation module;

[0123] The first creation module is used to create a coroutine for receiving core communication protocol requests and a coroutine for receiving direct memory access requests each time the master node listens for a join request. The coroutine for receiving remote direct memory access requests is used to receive remote direct memory access requests of the consistency protocol.

[0124] The second creation module is used to create a coroutine that responds to the core communication protocol request and a coroutine that responds to the direct memory access request each time the master node listens for a join request. The coroutine that responds to the direct memory access request is used to handle the remote direct memory access request of the consistency protocol.

[0125] Furthermore, each node in the high-availability cluster maintains a data structure, which is used to maintain the remote address of the node's remote direct memory access for writing messages in a preset writing manner; wherein, the node that receives the message obtains the message length from the received remote direct memory access remote address and dynamically adjusts the receive buffer offset according to the message length.

[0126] The aforementioned distributed consistency device can execute the distributed consistency method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0127] Example 4

[0128] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0129] like Figure 4As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0130] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a distributed consensus method.

[0132] In some embodiments, a distributed consensus method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of a distributed consensus method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a distributed consensus method by any other suitable means (e.g., by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] In some embodiments, a distributed consensus method may be implemented as a computer program, which is implicitly included in a computer program product. When executed by a processor, the computer program implements a distributed consensus method of the present invention. The computer program product can be understood as a software product that primarily implements its solution through a computer program. The computer program used to implement the method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program may be executed entirely on a machine, partially on a machine, partially on a remote machine as a standalone software package, or entirely on a remote machine or server.

[0135] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0138] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A distributed consensus method, characterized in that, The method includes: During the process of a node joining a high-availability cluster, the master node and the node to be joined exchange their remote addresses for remote direct memory access. These remote addresses are used to form a security mechanism for remote direct memory access. A two-phase commit protocol is initiated, in which the master node encapsulates the data sent by the client and writes it to the remote address of the newly joined node's remote direct memory access. The node to be joined becomes the newly joined node after joining the high-availability cluster. The newly added node, through a coroutine that receives remote direct memory access requests, polls and checks whether there are new distributed consistency messages on its local address, and then writes the processed reply message to the remote address of the master node's remote direct memory access. The master node processes the new distributed consistency response message after polling and detecting the existence of a new distributed consistency response message on the local address through the coroutine that responds to remote direct memory access requests. The process of adding a node to a high-availability cluster, in which the master node and the node to be added exchange remote direct memory access addresses, includes the following steps: Step 1: The node to be joined sends a join request to the master node using the core communication protocol of the Internet, and performs the initialization operation of remote direct memory access; Step 2: After the master node listens to the join request through the coroutine that receives the core communication protocol request, it executes the distributed consensus protocol on the join request. After the protocol is reached, each node in the high availability cluster executes the distributed consensus protocol. Step 3: After the master node detects that the status information of the high availability cluster has changed, it sends the remote address of the master node's remote direct memory access to the node to be joined. Step 4: The node to be joined saves the remote direct memory access address of the master node into its own memory, passes the remote direct memory access address of the node to be joined to the master node, and requests to obtain the status information of the high availability cluster. Step 5: The master node saves the remote address of the node to be joined to its own memory and sends a snapshot of the state information of the high availability cluster to the node to be joined. Step Six: After the node to be added receives the snapshot through the coroutine that replies to the core communication protocol request, it executes Step Three and Step Four for each node pair to realize the exchange of remote direct memory access remote addresses between the two nodes in each node pair. The node pair consists of the node to be added and each node in the high availability cluster.

2. The method according to claim 1, characterized in that, After the master node detects a change in the status information of the high-availability cluster, it sends the remote address of the master node's remote direct memory access to the node to be joined, including: After the master node detects a change in the status information of the high-availability cluster through the liveness detection goroutine, it sends a liveness detection message to the node to be joined through a goroutine that sends the core communication protocol. The liveness detection message includes the remote address of the master node's remote direct memory access.

3. The method according to claim 2, characterized in that, Sending a liveness probe message to the node to be joined via a coroutine that sends the core communication protocol, including: The coroutine that sends the core communication protocol establishes a communication connection with the node to be joined, creates a reliable connection protection domain, creates a reliable connection completion queue, creates a work completion array, registers remote access to local memory, adds the remote address of the master node's remote direct memory access to the liveness probe message, and sends the liveness probe message to the node to be joined.

4. The method according to claim 1, characterized in that, The node to be joined saves the remote direct memory access address of the master node into its own memory, passes the remote direct memory access address of the node to be joined to the master node, and requests the status information of the high-availability cluster, including: After receiving the liveness detection message through the coroutine that receives the core communication protocol request, the node to be joined responds to the liveness detection message, extracts the remote address of the master node’s remote direct memory access from the liveness detection message, and saves the remote address of the master node’s remote direct memory access into its own memory. Simultaneously, the node to be joined creates a reliable connection protection domain, a reliable connection completion queue, and a work completion array, registers remotely accessible local memory, adds the remote address of the node to be joined to the response liveness message, and sends the response liveness message to the master node. Meanwhile, the node to be joined requests the status information of the high-availability cluster.

5. The method according to claim 1, characterized in that, The method further includes: Each time the master node hears a join request, it creates a coroutine to receive core communication protocol requests and a coroutine to receive remote direct memory access requests. The coroutine to receive remote direct memory access requests is used to receive remote direct memory access requests from the consensus protocol. Each time the master node hears a join request, it creates a coroutine to respond to the core communication protocol request and a coroutine to respond to the direct memory access request. The coroutine that responds to the direct memory access request is used to handle the remote direct memory access request of the consistency protocol.

6. The method according to claim 1, characterized in that, Each node in the high-availability cluster maintains a data structure, which is used to maintain the remote address of the node's remote direct memory access where messages are written in a preset writing manner. The node that receives the message obtains the message length from the remote address of the received remote direct memory access and dynamically adjusts the offset of the receive buffer according to the message length.

7. A distributed consensus device, characterized in that, The device includes: The switching module is used to exchange remote addresses of remote direct memory access between the master node and the node to be joined during the process of the node to be joined joining the high availability cluster. The remote addresses are used to form a security mechanism for remote direct memory access. The first write module is used to initiate a two-phase commit protocol. The master node encapsulates the data sent by the client and writes it to the remote address of the newly joined node's remote direct memory access. The node to be joined becomes the newly joined node after joining the high-availability cluster. The second write module is used by the newly added node to poll and check the local address for a new distributed consistency message by receiving a remote direct memory access request through a coroutine, and then write the processed reply message to the remote address of the master node's remote direct memory access. The processing module is used by the master node to process the new distributed consistency response message after polling and detecting the existence of a new distributed consistency response message on the local address through the coroutine that replies to the remote direct memory access request. The switching module includes: The first execution submodule is used by the node to be joined to send a join request to the master node using the core communication protocol of the Internet and to perform the initialization operation of remote direct memory access. The second execution submodule is used to execute a distributed consensus protocol on the join request after the master node listens to the join request through a coroutine that receives core communication protocol requests. After the protocol is reached, each node in the high-availability cluster executes the distributed consensus protocol. The sending submodule is used to send the remote address of the master node's remote direct memory access to the node to be joined after the master node detects that the status information of the high availability cluster has changed. The transmission submodule is used by the node to be joined to save the remote address of the master node's remote direct memory access into its own memory, transmit the remote address of the node to be joined to the master node, and request to obtain the status information of the high availability cluster. The storage submodule is used by the master node to save the remote address of the node to be added to its own memory and send a snapshot of the status information of the high availability cluster to the node to be added. The interaction submodule is used to exchange remote addresses for remote direct memory access between the two nodes in each node pair after the node to be joined receives the snapshot through a coroutine that replies to the core communication protocol request. The node pair consists of the node to be joined and each node in the high availability cluster.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory that is communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the distributed consensus method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the distributed consensus method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the distributed consensus method according to any one of claims 1-6.

Images