A Paxos message sending method, device, equipment, medium and product

Abstract

The present invention discloses a Paxos message sending method, device, equipment, medium and product. The method is applied to a first node in a distributed system, and the first node runs a first thread and a second thread, including: obtaining a business request of the application layer from a local pipeline through the first thread, and saving the business request to a first message queue; constructing a Paxos message according to the business request in the first message queue through the second thread, and saving the Paxos message to a second message queue; initiating a proposal request to each second node according to the Paxos message in the second message queue through the first thread to execute the Paxos consensus algorithm process. This technical solution solves the throughput bottleneck problem under high concurrency pressure caused by the serial execution of coroutines in a single-threaded mode by transforming the Paxos protocol from a single-threaded multi-coroutine mode to a multi-threaded multi-coroutine mode, and optimizes the execution performance of the Paxos consensus algorithm; the system throughput can be improved by increasing the number of concurrent Paxos proposals initiated at the same time.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a Paxos message sending method, device, equipment, medium and product. Background Art

[0002] In a distributed system based on MGR (MySQL Group Replication), the Paxos consensus algorithm is often used to achieve data consistency between majority replicas. Due to the complexity of the internal state during the implementation of the Paxos protocol, the existing technology uses a single-instance single-threaded multi-coroutine approach to implement it, that is, starting a thread, abstracting the logic of each component into a coroutine within the thread, and each coroutine occupies the thread's time slice in a round-robin manner. However, the single-threaded multi-coroutine approach may affect the stability of the entire system in some extreme scenarios, such as large transactions, due to the long execution time of some coroutines, or in high-concurrency scenarios, the single thread may reach a bottleneck, thereby affecting the overall throughput of the system. Summary of the invention

[0003] The present invention provides a Paxos message sending method, device, equipment, medium and product, so as to transform the Paxos protocol from a single-threaded multi-coroutine mode to a multi-threaded multi-coroutine mode, thereby optimizing the execution performance of the Paxos consensus algorithm.

[0004] According to one aspect of the present invention, a Paxos message sending method is provided, which is applied to a first node in a distributed system, where the first node runs a first thread and a second thread; the method comprises:

[0005] Obtaining a service request of the application layer from the local pipeline through the first thread, and saving the service request to the first message queue;

[0006] Constructing a Paxos message according to the service request in the first message queue through the second thread, and saving the Paxos message to the second message queue;

[0007] A proposal request is initiated to each second node according to the Paxos message in the second message queue through the first thread to execute the Paxos consensus algorithm process; wherein the second node includes a node connected to the first node in the distributed system.

[0008] According to another aspect of the present invention, a Paxos message sending device is provided, which is applied to a first node in a distributed system, where the first node runs a first thread and a second thread; the device comprises:

[0009] A service request saving module, used for obtaining a service request of the application layer from the local pipeline through the first thread, and saving the service request to the first message queue;

[0010] A message construction module, used to construct a Paxos message according to the service request in the first message queue through a second thread, and save the Paxos message to a second message queue;

[0011] A message sending module is used to initiate a proposal request to each second node according to the Paxos message in the second message queue through the first thread to execute the Paxos consensus algorithm process; wherein the second node includes a node connected to the first node in the distributed system.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the Paxos message sending method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the Paxos message sending method described in any embodiment of the present invention when executed.

[0017] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the Paxos message sending method according to any embodiment of the present invention is implemented.

[0018] The technical solution of the embodiment of the present invention is applied to the first node in the distributed system. The first node runs a first thread and a second thread. The first thread obtains the service request of the application layer from the local pipeline and saves the service request to the first message queue; the second thread constructs a Paxos message according to the service request in the first message queue and saves the Paxos message to the second message queue; the first thread initiates a proposal request to each second node according to the Paxos message in the second message queue to execute the Paxos consensus algorithm process. This solution solves the throughput bottleneck problem under high concurrency pressure caused by the serial execution of the coroutine in the single-thread mode by transforming the Paxos protocol from a single-thread multi-coroutine mode to a multi-thread multi-coroutine mode, and optimizes the execution performance of the Paxos consensus algorithm; at the same time, due to the extraction of non-Paxos logic coroutines, the thread that originally executes the Paxos logic coroutine can load more proposal initiation coroutines, increasing the number of concurrent Paxos proposals initiated at the same time, thereby greatly increasing the system throughput.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a flowchart of a Paxos message sending method provided according to Embodiment 1 of the present invention;

[0022] Figure 2 This is a flowchart of a Paxos message sending method provided according to Embodiment 2 of the present invention;

[0023] Figure 3 is a flowchart of a Paxos message sending method provided according to Embodiment 3 of the present invention;

[0024] Figure 4 2 is a schematic diagram of the structure of a Paxos message sending device provided according to Embodiment 4 of the present invention;

[0025] Figure 5 It is a structural diagram of an electronic device that implements the Paxos message sending method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] Embodiment 1

[0029] In the prior art, the MGR Paxos majority protocol is implemented by starting multiple types of coroutines in one thread. In this process, there may be more than one coroutine of a certain type. For example, there may be multiple proposal-initiating coroutines (Multi-Paxos) to improve the overall throughput of the system. For example, each connection corresponds to a message-sending coroutine, which is responsible for message serialization and sending, and will occupy more system resources. Therefore, in high-concurrency scenarios, the serial execution of coroutines in a single-threaded manner will become a bottleneck for system throughput.

[0030] In order to solve the above problems, Figure 1 This is a flowchart of a Paxos message sending method provided in the first embodiment of the present invention. This embodiment can be applied to the case where the Paxos protocol is transformed from a single-threaded multi-coroutine mode to a multi-threaded multi-coroutine mode under the MGR architecture, thereby optimizing the execution performance of the Paxos consensus algorithm. The method can be executed by a Paxos message sending device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, a Paxos message sending method provided in the first embodiment is applied to a first node in a distributed system, and specifically includes the following steps:

[0031] S110. Obtain a service request of the application layer from the local pipeline through a first thread, and save the service request to a first message queue.

[0032] Among them, the business request may refer to a transaction update request initiated by the client to the first node. In one embodiment, the business request may be a SQL (Structured Query Language) request sent by the client. The first node may be a database node that acts as a proposer in a distributed system. It should be understood that each node in a distributed system is peer-to-peer, that is, each node will create a first thread and a second thread during the initialization process to implement the Paxos majority protocol, thereby achieving data consistency between majority replicas.

[0033] The local pipe may refer to a pipe responsible for receiving business requests from the application layer. Each node (database instance node) in the distributed system maintains a local pipe for receiving business requests.

[0034] The first thread may refer to a thread responsible for scheduling and executing non-Paxos logic-type coroutines, and the second thread may refer to a thread responsible for scheduling and executing Paxos logic-type coroutines. Among them, Paxos logic-type coroutines can be understood as a type of coroutine that is tightly coupled with the Paxos consistency logic, for example, it may include a proposal initiation coroutine (proposer_task) for initiating a Paxos proposal (proposal), a proposal execution coroutine (executor_task) for transaction authentication and playback after reaching a Paxos majority agreement, etc. Non-Paxos logic-type coroutines refer to a type of coroutine that is less related to the Paxos consistency logic, for example, it may include a message sending coroutine (sender_task) for sending Paxos messages, a pipeline data reading coroutine (local_server_task) for obtaining business requests from local pipelines, etc.

[0035] The first message queue may refer to a message queue between the first thread and the second thread for storing business requests. It can be understood that since this embodiment transforms the Paxos protocol from a single-thread multi-coroutine mode to a multi-thread (first thread + second thread) multi-coroutine mode, the coroutine between the first thread and the second thread in each node can use a message queue to synchronize messages.

[0036] In an embodiment of the present invention, after the application layer (upper layer) sends a service request (SQL request) to the first node and puts the service request into the local pipeline, the first thread can take out the service request from the local pipeline and write it into the first message queue. Subsequently, the second thread will read the service request from the first message queue and construct a corresponding Paxos message.

[0037] S120. Construct a Paxos message according to the service request in the first message queue through the second thread, and save the Paxos message to the second message queue.

[0038] The Paxos message may refer to a message constructed according to a business request for initiating a Paxos proposal, and the Paxos message may include a proposal number (a globally unique and self-increasing identifier used to identify the uniqueness of each proposal), a proposal value (a value proposed by the proposer and hoped to be agreed upon by the distributed system), and other information. The second message queue may refer to a message queue for storing Paxos messages sent by the first node to the second node.

[0039] In an embodiment of the present invention, after the service request is written to the first message queue, the corresponding coroutine in the second thread for initiating a Paxos proposal, namely the proposal initiation coroutine (proposer_task), can be awakened by the first thread, and then the service request is read from the first message queue using the proposal initiation coroutine and the corresponding Paxos message is constructed, and then the Paxos message is written to the second message queue. Subsequently, the first thread will read the Paxos message from the second message queue and send it to each second node in the distributed system. Furthermore, since the second thread may receive another service request initiated by the client and write it to the first message queue during the process of taking out the service request from the first message queue, in this case, the first message queue may have contention access. In order to avoid this conflict problem, the first message queue needs to be locked and protected before taking out the service request from the first message queue.

[0040] S130. Initiate a proposal request to each second node according to the Paxos message in the second message queue through the first thread to execute the Paxos consensus algorithm process; wherein the second node includes a node connected to the first node in the distributed system.

[0041] Among them, the second node can be a database node that plays the role of an acceptor in a distributed system, which may include a node connected to the first node in the distributed system. In one embodiment, during the initialization process of the first node, a preset node set is initialized to maintain the first node and each second node connected thereto, where the preset node set can also be understood as a replication group of the distributed system. It can be understood that the first node, as a proposer, can also serve as an acceptor (second node) of the Paxos proposal and make decisions on the proposals initiated by itself.

[0042] The Paxos consensus algorithm process can include three stages: 1) Prepare stage, Proposer sends Prepare request to Acceptors, and Acceptors make promise for the received Prepare request; 2) Accept stage, after Proposer receives the Promise promised by the majority of Acceptors, it sends Propose request to Acceptors, and Acceptors accept the received Propose request; 3) Learn stage, after Proposer receives the Accept from the majority of Acceptors, it marks the success of this Accept and reaches a majority, and sends the formed resolution to all Learners for learning.

[0043] In an embodiment of the present invention, after the Paxos message is written into the second message queue, the corresponding coroutine in the first thread for sending the Paxos message, namely the message sending coroutine (sender_task), can be awakened by the second thread, and then the Paxos message can be sent to each second node using the message sending coroutine. Subsequently, the first node and each second node will execute the corresponding Paxos consensus algorithm process based on the Paxos message.

[0044] It should be understood that in the above embodiment, the original Paxos protocol is transformed from a single-thread mode to a dual-thread (first thread + second thread) mode according to the degree of coupling with the Paxos consistency logic. However, in actual applications, according to the different granularity or rules of the coroutine division, the number of threads after the split can be greater than 2. For example, a class of coroutines responsible for the Paxos consistency logic can be divided into the same thread for scheduling, and the coroutines responsible for message serialization and sending can be divided into a thread for separate scheduling, and the coroutines responsible for data processing can be divided into a thread for separate scheduling, etc. This embodiment does not impose specific restrictions on this. In a specific embodiment, the number of corresponding divided threads can be determined according to the access to shared variables between each coroutine, that is, a class of coroutines involving the same shared variable can be divided into the same thread for scheduling. In another specific embodiment, the coroutines that occupy more system resources (such as network, CPU, etc.) in all coroutines can also be extracted from the original single thread and divided into separate threads for scheduling.

[0045] The technical solution of the embodiment of the present invention is applied to the first node in the distributed system. The first node runs a first thread and a second thread. The first thread obtains the service request of the application layer from the local pipeline and saves the service request to the first message queue; the second thread constructs a Paxos message according to the service request in the first message queue and saves the Paxos message to the second message queue; the first thread initiates a proposal request to each second node according to the Paxos message in the second message queue to execute the Paxos consensus algorithm process. This solution solves the throughput bottleneck problem under high concurrency pressure caused by the serial execution of the coroutine in the single-thread mode by transforming the Paxos protocol from a single-thread multi-coroutine mode to a multi-thread multi-coroutine mode, and optimizes the execution performance of the Paxos consensus algorithm; at the same time, due to the extraction of non-Paxos logic coroutines, the thread that originally executes the Paxos logic coroutine can load more proposal initiation coroutines, increasing the number of concurrent Paxos proposals initiated at the same time, thereby greatly increasing the system throughput.

[0046] Furthermore, based on the above-mentioned embodiments of the invention, a Paxos message sending method provided in this embodiment further includes:

[0047] During the initialization process of each node in the distributed system, the first thread and the second thread are initially created;

[0048] Initialize and create a non-Paxos logic class coroutine through the first thread, and insert the non-Paxos logic class coroutine into a first active queue corresponding to the first thread; wherein the non-Paxos logic class coroutine includes at least: a pipeline data reading coroutine and a message sending coroutine;

[0049] Initialize and create a Paxos logic class coroutine through the second thread, and insert the Paxos logic class coroutine into the second active queue corresponding to the second thread; wherein, the Paxos logic class coroutine includes at least: proposal initiation coroutine, proposal execution coroutine, heartbeat detection coroutine, message processing coroutine, memory cleaning coroutine, connection cleaning coroutine and connection processing coroutine.

[0050] Among them, the pipeline data reading coroutine (local_server_task) may refer to a coroutine used to read business requests from the local pipeline. The heartbeat detection coroutine (detector_task) may refer to a coroutine used to capture the heartbeat of a node to determine the corresponding survival status. The message processing coroutine (acceptor_learner_task) may refer to a coroutine used to process messages sent by this node or other nodes to the Paxos service of this node. The memory cleaning coroutine (sweeper_task) may refer to a coroutine used to clean up and recycle memory. The connection cleaning coroutine (tcp_reaper_task) may refer to a coroutine used to clean up TCP (Transmission Control Protocol) connections that have not been used for a long time.

[0051] The first active queue and the second active queue may refer to active queues corresponding to the first thread and the second thread respectively, and the active queues are used to store coroutines that are currently ready to execute and are waiting for CPU time slices.

[0052] In an embodiment of the present invention, during the initialization of a distributed system based on MGR, each node will initialize and create a first thread and a second thread, wherein the first thread is responsible for initializing and creating a non-Paxos logic-type coroutine, and inserting this type of coroutine into the first active queue corresponding to the first thread waiting for scheduling; the second thread is responsible for initializing and creating a Paxos logic-type coroutine, and inserting this type of coroutine into the second active queue corresponding to the second thread waiting for scheduling.

[0053] Embodiment 2

[0054] Figure 2 The flowchart of a Paxos message sending method provided in the second embodiment of the present invention is further optimized and expanded based on the above implementation, and can be combined with various optional technical solutions in the above implementation. Figure 2 As shown, a Paxos message sending method provided in the second embodiment is applied to a first node in a distributed system, and specifically includes the following steps:

[0055] S210. In response to a semaphore trigger operation on a local pipeline by the application layer, a pipeline data reading coroutine in the first thread is awakened to lock and protect a first active queue corresponding to the first thread, and the pipeline data reading coroutine is moved from a first waiting queue corresponding to the first thread to a first active queue.

[0056] The first waiting queue and the second waiting queue may refer to the waiting queues corresponding to the first thread and the second thread respectively, and the waiting queues are used to store those coroutines that cannot be executed at present and need to wait for certain conditions to be met before they can be executed.

[0057] The semaphore can be used to indicate whether new data (ie, new service request) has arrived in the local pipeline of the first node. When new data is put into the local pipeline, the pipeline data reading coroutine in the first thread can be awakened through the semaphore trigger operation.

[0058] In an embodiment of the present invention, after the application layer (upper layer) sends a business request (SQL request) to the first node and places the business request in the local pipeline, a semaphore trigger operation can be used to send a semaphore to the suspended pipeline data reading coroutine in the first thread to indicate that new data has arrived in the local pipeline, thereby waking up the pipeline data reading coroutine to move it from the first waiting queue to the first active queue, wherein the process of placing the pipeline data reading coroutine in the first active queue requires locking the first active queue to avoid competitive access to the first active queue.

[0059] S220: Lock the first active queue for protection, and retrieve the pipeline data reading coroutine from the first active queue through the first thread.

[0060] In the embodiment of the present invention, the currently executable pipeline data reading coroutine may be taken out from the first active queue by the first thread, wherein the process of taking out the pipeline data reading coroutine also requires locking and protecting the first active queue.

[0061] S230: Read the service request from the local pipeline through the pipeline data reading coroutine, and write the service request into the first message queue.

[0062] In an embodiment of the present invention, the service request can be taken out from the local pipeline by executing the pipeline data reading coroutine and placed into the first message queue, wherein the process of placing the service request also requires locking and protecting the first message queue.

[0063] S240. Waking up the proposal initiation coroutine in the second thread through the first thread to move the proposal initiation coroutine from the second waiting queue corresponding to the second thread to the second active queue.

[0064] In an embodiment of the present invention, the proposal initiation coroutine in the second thread can be awakened by the first thread, so that the proposal initiation coroutine is inserted from the second waiting queue to the second active queue to wait for execution.

[0065] S250: Lock and protect the second active queue corresponding to the second thread, and take out the proposal to initiate the coroutine from the second active queue.

[0066] In an embodiment of the present invention, the second thread can take out the executable proposal initiation coroutine from the second active queue and start scheduling execution, wherein the process of taking out the proposal initiation coroutine also requires locking and protecting the second active queue.

[0067] S260, lock the first message queue for protection, then initiate a coroutine through a proposal to read the business request from the first message queue, encapsulate the business request into a Paxos message according to the preset Paxos protocol format, and write the Paxos message into the second message queue.

[0068] Among them, the preset Paxos protocol format can be understood as a protocol format for constructing Paxos messages. Exemplarily, the preset Paxos protocol format may include fields such as a proposal number and a proposal value.

[0069] In an embodiment of the present invention, the business request to be proposed can be retrieved from the first message queue through the proposal initiation coroutine, and the business request can be encapsulated into a corresponding Paxos message according to a pre-configured preset Paxos protocol format, and the constructed Paxos message can be written into the second message queue; wherein, the process of retrieving the business request also requires locking the first message queue for protection.

[0070] S270. Move the proposal initiation coroutine from the second active queue to the second waiting queue through the second thread, and wake up the message sending coroutine in the first thread to move the message sending coroutine from the first waiting queue corresponding to the first thread to the first active queue.

[0071] In an embodiment of the present invention, after constructing the Paxos message, the proposal initiating coroutine will enter a waiting state, that is, the proposal initiating coroutine is moved from the second active queue to the second waiting queue, and at the same time, the message sending coroutine in the first thread can be awakened and moved from the first waiting queue to the first active queue.

[0072] S280. Take out a message sending coroutine from a first active queue corresponding to the first thread through the first thread.

[0073] S290, reading the Paxos message from the second message queue through the message sending coroutine, and sending the Paxos message to each second node according to the preset node set to execute the Paxos consensus algorithm process.

[0074] In an embodiment of the present invention, after the message sending coroutine starts scheduling and execution, the aforementioned constructed Paxos message will be taken out from the second message queue, and the Paxos message will be sent to the corresponding second node according to the preset node set maintained by the first node, and the corresponding Paxos consensus algorithm process will be executed based on the Paxos message.

[0075] Furthermore, based on the above-mentioned embodiments of the invention, a Paxos message sending method provided in this embodiment further includes:

[0076] After the Paxos message reaches a majority, the proposal initiation coroutine in the waiting state is awakened by the message processing coroutine in the second thread to execute the next round of proposal initiation operations through the proposal initiation coroutine.

[0077] In an embodiment of the present invention, when the Paxos message initiated in this round reaches a majority, the proposal initiation coroutine in the waiting state in the second waiting queue can be awakened by the message processing coroutine in the second thread, so that the proposal initiation coroutine can continue to read business requests from the first message queue and perform the initiation operation of the next round of proposals.

[0078] It can be understood that the technical solution of this embodiment is to describe how the first thread and the second thread construct and send Paxos messages through scheduling cooperation from the perspective of the proposer (i.e., the first node) under the new multi-threaded and multi-coroutine architecture. The subsequent Paxos consensus algorithm process of the three nodes Proposer, Acceptor, and Learner is not specifically described. The specific execution process can refer to the existing technology and will not be repeated here.

[0079] The technical solution of the embodiment of the present invention is applied to the first node in the distributed system. The first node runs the first thread and the second thread. The Paxos protocol is transformed from a single-thread multi-coroutine mode to a multi-thread multi-coroutine mode, and the pipeline data reading coroutine and the message sending coroutine are extracted to a new thread (first thread) for separate scheduling. The first thread and the second thread synchronize messages through the first message queue and the second message queue. At the same time, the protected access to the shared variables (first active queue, second active queue, first message queue) is added between each coroutine. The new architecture of multi-threaded multi-coroutine is used to solve the throughput bottleneck problem under high concurrency pressure caused by the serial execution of coroutines in a single-threaded mode, and optimize the execution performance of the Paxos consensus algorithm; at the same time, due to the extraction of non-Paxos logic coroutines, on the one hand, the pre-CPU load can be reduced, and on the other hand, the thread that originally executes the Paxos logic coroutine can load more proposal initiation coroutines, which increases the number of concurrent Paxos proposals initiated at the same time, thereby greatly increasing the system throughput.

[0080] Embodiment 3

[0081] Figure 3 This is a flowchart of a Paxos message sending method provided in Embodiment 3 of the present invention. Figure 3 As shown, a Paxos message sending method provided by Embodiment 3 of the present invention mainly includes: an initialization process and a scheduling execution process. Specifically, the initialization process includes:

[0082] Step 1: During the initialization of the MGR-based distributed system, each node is initialized to create a first thread and a second thread.

[0083] Step 2: The first thread is responsible for creating the local_server_task coroutine and putting the coroutine into the first active queue to wait for scheduling.

[0084] Step 3. After the first thread of the first node listens to the connection request of the other second node, a communication connection is established with the second node, and a corresponding sender_task coroutine is created, and the sender_task coroutine is placed in the first active queue to wait for scheduling; wherein, the sender_task coroutine can be created by the first thread or the second thread, and this embodiment does not limit this, and it is only necessary to place the created sender_task coroutine in the first active queue.

[0085] Step 4. The second thread is responsible for creating the proposer_task coroutine, the detector_task coroutine, the executor_task coroutine, the acceptor_learner_task coroutine, the sweeper_task coroutine, the tcp_reaper_task coroutine, and the incoming_connection_task coroutine, and puts this type of coroutine into the second active queue to wait for scheduling.

[0086] Step 5: The second thread is responsible for creating and initializing the linked lists, queues, state machines and other structures involved in the MGR execution process, and the first thread is only responsible for the scheduling and execution of the local_server_task coroutine and the sender_task coroutine.

[0087] Furthermore, the scheduling execution process includes:

[0088] Step 6. The first node (database read / write instance node) receives the business request (SQL request) from the application layer, puts the business request into the local pipeline, and sends a semaphore to the local_server_task coroutine suspended in the first thread to indicate that new data has arrived in the local pipeline, thereby waking up the local_server_task coroutine to put it from the first waiting queue to the first active queue; wherein, the putting process of the local_server_task coroutine requires locking the first active queue for protection.

[0089] Step 7: Access the first active queue through the first thread to retrieve the currently executable local_server_task coroutine; wherein, the retrieval process of the local_server_task coroutine requires locking the first active queue for protection.

[0090] Step 8: Take the business request from the local pipeline through the local_server_task coroutine and put it into the first message queue; the process of putting the business request into the first message queue needs to lock the first message queue for protection.

[0091] Step 9: wake up the proposer_task coroutine in the second thread through the first thread to move the proposer_task coroutine from the second waiting queue to the second active queue to wait for execution.

[0092] Step 10: Take out the executable proposer_task coroutine from the second active queue through the second thread and start scheduling execution; wherein, the process of taking out the proposer_task coroutine requires locking the second active queue for protection.

[0093] Step 11: read the business request from the first message queue through the proposer_task coroutine, encapsulate the business request into a Paxos message according to the preset Paxos protocol format, and put the Paxos message into the second message queue; wherein, the process of taking out the business request requires locking the first message queue for protection.

[0094] Step 12: Move the proposer_task coroutine from the second active queue to the second waiting queue, enter the waiting state, and wait to be awakened.

[0095] Step 13: wake up the sender_task coroutine in the first thread to move the sender_task coroutine from the first waiting queue to the first active queue.

[0096] Step 14: Read the Paxos message from the second message queue through the sender_task coroutine and send it to each second node to execute the Paxos consensus algorithm process

[0097] Step 15: After the Paxos message reaches a majority, the proposer_task coroutine in the waiting state is awakened by the acceptor_learner_task coroutine in the second thread to execute the initiation operation of the next round of proposals through the proposer_task coroutine.

[0098] In one embodiment, the executor_task coroutine in the second thread can periodically check whether the Paxos message can be pushed forward and the hole can be processed. In addition, after the Paxos message that the executor_task coroutine is waiting for reaches a majority, it will also be awakened.

[0099] Further, Figure 3 XCOM1 and XCOM2 in MGR act as communication engines, which are responsible for message delivery and consistency coordination between nodes. Specifically, the XCOM component implements the Paxos protocol to ensure that all nodes agree on the transaction order given in the global transaction sequence, thereby maintaining data consistency and cluster fault tolerance.

[0100] The technical solution of the embodiment of the present invention is implemented by transforming the Paxos protocol from a single-threaded multi-coroutine mode to a multi-threaded multi-coroutine mode, extracting the local_server_task coroutine and the sender_task coroutine to a new thread (first thread) for separate scheduling, and synchronizing messages between the first thread and the second thread through the first message queue and the second message queue, while adding protected access to shared variables (first active queue, second active queue, first message queue) between each coroutine. The new architecture of multi-threaded multi-coroutine is used to solve the throughput bottleneck problem under high concurrency pressure caused by the serial execution of coroutines in a single-threaded mode, and optimize the execution performance of the Paxos consensus algorithm; at the same time, due to the extraction of non-Paxos logic coroutines, on the one hand, the pre-CPU load can be reduced, and on the other hand, the thread that originally executes the Paxos logic coroutine can load more proposal initiation coroutines, increasing the concurrency of initiating Paxos proposals at the same time, thereby greatly increasing the system throughput. In addition, as the number of cluster nodes increases, the number of coroutines used for network communication also needs to increase accordingly. In the traditional single-threaded mode, a lot of time is spent on tasks such as network sending that are not related to the Paxos consistency logic. With the introduction of new threads, there will no longer be a limit on the number of nodes.

[0101] Embodiment 4

[0102] Figure 4 A schematic diagram of the structure of a Paxos message sending device provided in Embodiment 4 of the present invention. Figure 4As shown, the device is applied to a first node in a distributed system, and the first node runs a first thread and a second thread; the device includes:

[0103] The service request saving module 31 is used to obtain the service request of the application layer from the local pipeline through the first thread, and save the service request to the first message queue;

[0104] A message construction module 32, configured to construct a Paxos message according to the service request in the first message queue through a second thread, and save the Paxos message to a second message queue;

[0105] The message sending module 33 is used to initiate a proposal request to each second node according to the Paxos message in the second message queue through the first thread to execute the Paxos consensus algorithm process; wherein the second node includes a node connected to the first node in the distributed system.

[0106] The technical solution of the embodiment of the present invention is applied to the first node in the distributed system. The first node runs a first thread and a second thread. The first thread obtains the service request of the application layer from the local pipeline and saves the service request to the first message queue; the second thread constructs a Paxos message according to the service request in the first message queue and saves the Paxos message to the second message queue; the first thread initiates a proposal request to each second node according to the Paxos message in the second message queue to execute the Paxos consensus algorithm process. This solution solves the throughput bottleneck problem under high concurrency pressure caused by the serial execution of the coroutine in the single-thread mode by transforming the Paxos protocol from a single-thread multi-coroutine mode to a multi-thread multi-coroutine mode, and optimizes the execution performance of the Paxos consensus algorithm; at the same time, due to the extraction of non-Paxos logic coroutines, the thread that originally executes the Paxos logic coroutine can load more proposal initiation coroutines, increasing the number of concurrent Paxos proposals initiated at the same time, thereby greatly increasing the system throughput.

[0107] Further, based on the above-mentioned embodiment of the invention, the service request receiving module 31 includes:

[0108] The pipeline data reading coroutine wake-up unit is used to wake up the pipeline data reading coroutine in the first thread in response to the semaphore triggering operation of the application layer on the local pipeline, so as to lock and protect the first active queue corresponding to the first thread, and move the pipeline data reading coroutine from the first waiting queue corresponding to the first thread to the first active queue;

[0109] A pipeline data reading coroutine fetching unit is used to lock and protect the first active queue, and fetch the pipeline data reading coroutine from the first active queue through the first thread;

[0110] A business request writing unit, used to read the business request from the local pipeline through the pipeline data reading coroutine, and write the business request into the first message queue;

[0111] The proposal initiation coroutine wake-up unit is used to wake up the proposal initiation coroutine in the second thread through the first thread, so as to move the proposal initiation coroutine from the second waiting queue corresponding to the second thread to the second active queue.

[0112] Further, based on the above-mentioned embodiment of the invention, the message construction module 32 includes:

[0113] The proposal initiation coroutine extraction unit is used to lock and protect the second active queue corresponding to the second thread, and extract the proposal initiation coroutine from the second active queue;

[0114] The Paxos message encapsulation unit is used to lock and protect the first message queue, and then read the business request from the first message queue through the proposal initiation coroutine, encapsulate the business request into a Paxos message according to the preset Paxos protocol format, and write the Paxos message into the second message queue;

[0115] The message sending coroutine wake-up unit is used to move the proposal initiation coroutine from the second active queue to the second waiting queue through the second thread, and wake up the message sending coroutine in the first thread to move the message sending coroutine from the first waiting queue corresponding to the first thread to the first active queue.

[0116] Further, based on the above-mentioned embodiment of the invention, the message sending module 33 includes:

[0117] A message sending coroutine fetching unit, used for fetching a message sending coroutine from a first active queue corresponding to the first thread through a first thread;

[0118] The Paxos message sending unit is used to read the Paxos message from the second message queue through the message sending coroutine, and send the Paxos message to each second node according to the preset node set to execute the Paxos consensus algorithm process.

[0119] Further, based on the above-mentioned embodiment of the invention, the Paxos message sending device further includes:

[0120] The proposal initiation module is used to wake up the proposal initiation coroutine in the waiting state through the message processing coroutine in the second thread after the Paxos message reaches a majority, so as to execute the next round of proposal initiation operations through the proposal initiation coroutine.

[0121] Further, based on the above-mentioned embodiment of the invention, the Paxos message sending device further includes:

[0122] A thread creation module, used for initializing and creating a first thread and a second thread during the initialization process of each node in the distributed system;

[0123] A non-Paxos logic class coroutine creation module is used to create a non-Paxos logic class coroutine through the first thread initialization, and insert the non-Paxos logic class coroutine into the first active queue corresponding to the first thread; wherein the non-Paxos logic class coroutine includes at least: a pipeline data reading coroutine and a message sending coroutine;

[0124] A Paxos logic class coroutine creation module is used to initialize and create a Paxos logic class coroutine through a second thread, and insert the Paxos logic class coroutine into the second active queue corresponding to the second thread; wherein, the Paxos logic class coroutine includes at least: a proposal initiation coroutine, a proposal execution coroutine, a heartbeat detection coroutine, a message processing coroutine, a memory cleanup coroutine, a connection cleanup coroutine and a connection processing coroutine.

[0125] The Paxos message sending device provided in the embodiment of the present invention can execute the Paxos message sending method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0126] Embodiment 5

[0127] Figure 5 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0128] like Figure 5As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0129] A number of components in the electronic device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The processor 41 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 41 executes the various methods and processes described above, such as the Paxos message sending method.

[0131] In some embodiments, the Paxos message sending method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the Paxos message sending method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to execute the Paxos message sending method in any other appropriate manner (e.g., by means of firmware).

[0132] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] In some embodiments, the Paxos message sending method may be implemented as a computer program, which is invisibly included in a computer program product. When the computer program is executed by a processor, the Paxos message sending method of the present invention is implemented. The computer program product can be understood as a software product that implements its solution mainly through a computer program. The computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or completely on a remote machine or server.

[0134] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0136] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0137] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0138] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0139] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A Paxos message sending method, characterized in that: Applied to a first node in a distributed system, the first node running a first thread and a second thread; the method comprises: Obtaining a service request of the application layer from the local pipeline through the first thread, and saving the service request to the first message queue; constructing a Paxos message according to the service request in the first message queue through the second thread, and saving the Paxos message to the second message queue; Initiate a proposal request to each second node according to the Paxos message in the second message queue through the first thread to execute the Paxos consensus algorithm process; wherein the second node includes a node connected to the first node in the distributed system.

2. The method according to claim 1, characterized in that The obtaining the service request of the application layer from the local pipeline through the first thread and saving the service request to the first message queue includes: In response to the semaphore triggering operation of the application layer on the local pipeline, the pipeline data reading coroutine in the first thread is awakened to lock and protect the first active queue corresponding to the first thread, and the pipeline data reading coroutine is moved from the first waiting queue corresponding to the first thread to the first active queue; The first active queue is locked for protection, and the pipeline data reading coroutine is retrieved from the first active queue through the first thread; Reading the service request from the local pipeline through the pipeline data reading coroutine, and writing the service request into the first message queue; The proposal initiation coroutine in the second thread is awakened by the first thread to move the proposal initiation coroutine from the second waiting queue corresponding to the second thread to the second active queue.

3. The method according to claim 1, characterized in that The constructing a Paxos message according to the service request in the first message queue by the second thread, and saving the Paxos message to the second message queue, includes: The second active queue corresponding to the second thread is locked and protected, and a proposal is taken out from the second active queue to initiate a coroutine; The first message queue is locked for protection, and then the business request is read from the first message queue through the proposal initiation coroutine, the business request is encapsulated into the Paxos message according to the preset Paxos protocol format, and the Paxos message is written into the second message queue; The proposal initiation coroutine is moved from the second active queue to the second waiting queue through the second thread, and the message sending coroutine in the first thread is awakened to move the message sending coroutine from the first waiting queue corresponding to the first thread to the first active queue.

4. The method according to claim 1, characterized in that: The initiating a proposal request to each second node according to the Paxos message in the second message queue through the first thread to execute the Paxos consensus algorithm process includes: Taking out a message sending coroutine from a first active queue corresponding to the first thread through the first thread; The Paxos message is read from the second message queue through the message sending coroutine, and the Paxos message is sent to each of the second nodes according to a preset node set to execute the Paxos consensus algorithm process.

5. The method according to claim 1, characterized in that Also includes: After the Paxos message reaches a majority, the proposal initiation coroutine in the waiting state is awakened by the message processing coroutine in the second thread, so as to perform the next round of proposal initiation operation through the proposal initiation coroutine.

6. The method according to claim 1, characterized in that Also includes: During the initialization process of each of the nodes in the distributed system, initializing and creating the first thread and the second thread; Initialize and create a non-Paxos logic-type coroutine through the first thread, and insert the non-Paxos logic-type coroutine into a first active queue corresponding to the first thread; wherein the non-Paxos logic-type coroutine includes at least: a pipeline data reading coroutine and a message sending coroutine; Initialize and create a Paxos logic-class coroutine through the second thread, and insert the Paxos logic-class coroutine into the second active queue corresponding to the second thread; wherein the Paxos logic-class coroutine includes at least: a proposal initiation coroutine, a proposal execution coroutine, a heartbeat detection coroutine, a message processing coroutine, a memory cleanup coroutine, a connection cleanup coroutine and a connection processing coroutine.

7. A Paxos message sending device, characterized in that: The device is applied to a first node in a distributed system, wherein the first node runs a first thread and a second thread; the device comprises: A service request saving module, configured to obtain a service request of an application layer from a local pipeline through the first thread, and save the service request to a first message queue; a message construction module, configured to construct a Paxos message according to the service request in the first message queue through the second thread, and save the Paxos message to a second message queue; A message sending module is used to initiate a proposal request to each second node according to the Paxos message in the second message queue through the first thread to execute the Paxos consensus algorithm process; wherein the second node includes a node connected to the first node in the distributed system.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the Paxos message sending method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the Paxos message sending method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the Paxos message sending method according to any one of claims 1 to 6.

Images