Write request processing method, database system, device, storage medium and product

Abstract

One or more embodiments of the invention provide a write request processing method, a database system, equipment, a storage medium and a product. The write request processing method comprises the steps that after receiving a write request related to two partitions, a coordination node sends a preparation instruction to a database node where a main copy of each partition related to the write request is located; after the database node where the master copy is located receives the preparation instruction, the database nodes where all the slave copies corresponding to the master copy are located execute a consensus process and feed back a consensus result to the coordination node; under the condition that all the consensus results indicate that the write data corresponding to the write request can be successfully synchronized, the coordination node generates a submission instruction and sends the submission instruction to the database nodes where all the main copies are located; after the database node where the master copy is located receives the submission instruction, write operation corresponding to the stored master copy in the write request is executed, and write data corresponding to the write operation is synchronized to all the slave copies corresponding to the master copy.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of database, and in particular, to a write request processing method, a database system, an electronic device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] In related technologies, a database system generally adopts a partition mechanism to divide stored data, so as to improve overall data processing capability and system horizontal scalability. Each partition generally maintains a primary replica and at least one secondary replica. The primary replica is used to process read and write operations, and the secondary replica is used to improve system fault tolerance and availability, for example, the secondary replica can quickly take over in the event of a primary replica failure, or provide read-only services.

[0003] Such a database system is generally composed of multiple database nodes. Different database nodes cooperate through a network to complete the management and maintenance of partitions. Each database node can store at least one primary replica or secondary replica of a partition, thereby realizing distributed deployment of data in the dimension of physical resources.

[0004] However, as the data scale grows and the write request becomes more complex, the database system faces the problem of difficult data consistency guarantee when processing write requests involving cross-partitions, because the database nodes are not synchronized in time. SUMMARY

[0005] Therefore, one or more embodiments of the present specification provide a write request processing method, a database system, an electronic device, a computer readable storage medium and a computer program product.

[0006] To achieve the above-mentioned purpose, one or more embodiments of the present specification provide technical solutions as follows:

[0007] According to a first aspect of one or more embodiments of the present specification, a write request processing method is provided, applied to a database system including a coordination node, and the method comprises:

[0008] After receiving a write request involving at least two partitions, the coordination node sends a preparation instruction corresponding to the write request to a database node where a primary replica of each partition involved in the write request is located;

[0009] After receiving the preparation instruction, the database node where the primary replica is located executes a consensus process for the write request with all database nodes where secondary replicas corresponding to the primary replica are located, and feeds back a consensus result to the coordination node, the consensus result being used to indicate whether write data corresponding to the write request can be successfully synchronized to the secondary replica;

[0010] The coordination node generates a commit instruction and sends it to all the database nodes where the primary replicas are located, in a case that all the consensus results received indicate that the write data corresponding to the write request can be successfully synchronized.

[0011] The database nodes where the primary replicas are located execute the write operation in the write request corresponding to the stored primary replica after receiving the commit instruction, and synchronize the write data corresponding to the write operation to all the secondary replicas corresponding to the primary replica.

[0012] According to a second aspect of the embodiments of the present specification, a write request processing method is provided, which comprises:

[0013] After receiving a write request involving two partitions, a preparation instruction corresponding to the write request is sent to the database nodes where the primary replicas of the partitions involved in the write request are located, so that the database nodes where the primary replicas are located and the database nodes where all the secondary replicas corresponding to the primary replicas are located execute a consensus process for the write request and return consensus results; the consensus results are used to indicate whether the write data corresponding to the write request can be successfully synchronized to the secondary replicas;

[0014] In a case that all the consensus results returned by the database nodes where the primary replicas are located indicate that the write data corresponding to the write request can be successfully synchronized, a commit instruction is generated and sent to all the database nodes where the primary replicas are located, so that the database nodes where the primary replicas are located execute the write operation in the write request corresponding to the stored primary replica, and synchronize the write data corresponding to the write operation to all the secondary replicas corresponding to the primary replica.

[0015] According to a third aspect of the embodiments of the present specification, a write request processing method is provided, which comprises:

[0016] After receiving the preparation instruction corresponding to the write request involving two partitions and sent by the coordination node, the primary replica of the partition stored by itself is determined, and a consensus process for the write request is executed with the database nodes where all the secondary replicas corresponding to the primary replica are located, and a consensus result is fed back to the coordination node, the consensus result being used to indicate whether the write data corresponding to the write request can be successfully synchronized to the secondary replicas, so that the coordination node decides whether to continue to execute the write request based on the consensus result;

[0017] After receiving the commit instruction sent by the coordination node, the write operation in the write request corresponding to the stored primary replica is executed, and the write data corresponding to the write operation is synchronized to all the secondary replicas corresponding to the primary replica.

[0018] According to a fourth aspect of the embodiments of the present specification, a database system is provided, comprising a coordination node and at least two database nodes; the coordination node is a node independent of the at least two database nodes or a node selected from the at least two database nodes; the coordination node is configured to execute the method of the second aspect to coordinate processing of a write request involving at least two partitions; a database node where a primary copy of each partition involved in the write request is located is configured to execute the method of the third aspect.

[0019] According to a fifth aspect of the embodiments of the present specification, an electronic device is provided, comprising:

[0020] a processor;

[0021] a memory for storing processor-executable instructions;

[0022] wherein the processor, when executing the executable instructions, is configured to implement the method of the first aspect, the second aspect, or the third aspect.

[0023] According to a sixth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, having stored thereon a computer program, which, when executed by a processor, implements the steps of the method of the first aspect, the second aspect, or the third aspect.

[0024] According to a seventh aspect of the embodiments of the present specification, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the method of the first aspect, the second aspect, or the third aspect.

[0025] The technical solutions provided by the embodiments of the present specification can include the following beneficial effects:

[0026] In the embodiments of the present specification, through the "consistency between different partition primary copies" + "consistency between primary and secondary copies in the same partition", efficient processing and global consistency control of cross-partition write requests are achieved.

[0027] The consistency between different partition primary copies refers to, when processing a write request involving at least two data partitions, the database nodes where the primary copies of the partitions are located, under the driving of the preparation instructions and the commit instructions issued by the coordination node, collaboratively participate in the processing process of the write request. Through unified decision control, the primary copies of each partition are ensured to synchronously enter the "commit" or "rollback" state, thereby realizing consistent commit or abortion between partitions, and avoiding the problem of partial commit in distributed write operation.

[0028] The consistency between the master and the slave in the same partition refers to that, before the write request is submitted, the database node where the master is located initiates a write consensus process to the database node where the corresponding slave is located. It is confirmed in advance whether the slave has synchronization capability, so that after the write operation is completed, the write result can be timely and consistently synchronized to all slaves, ensuring the data consistency and replica reliability within each data partition.

[0029] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory and are not restrictive of the present specification. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 FIG. 1 is a schematic diagram of an architecture of a database system provided by an example embodiment.

[0031] Figure 2 FIG. 2 is a flowchart of a write request processing method provided by an example embodiment.

[0032] Figure 3 FIG. 3 is a timing diagram of a write request processing process provided by an example embodiment.

[0033] Figure 4 FIG. 4 is a flowchart of another write request processing method provided by an example embodiment.

[0034] Figure 5 FIG. 5 is a flowchart of still another write request processing method provided by an example embodiment.

[0035] Figure 6 FIG. 6 is a schematic diagram of a structure of an electronic device provided by an example embodiment. DETAILED DESCRIPTION

[0036] The example embodiments will be described in detail herein with reference to the accompanying drawings. In the following description, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following example embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Instead, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present specification, as detailed in the appended claims.

[0037] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in the present specification. In some other embodiments, the steps included in the method can be more or less than described in the present specification. In addition, a single step described in the present specification can be divided into multiple steps for description in other embodiments; and multiple steps described in the present specification can be combined into a single step for description in other embodiments.

[0038] The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the specification are information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0039] Please refer to Figure 1 The embodiment of the specification provides a database system, which divides stored data by using a partition mechanism, and each partition generally maintains a primary copy and at least one secondary copy. For example Figure 1 The stored data is divided into four partitions according to the value range in , which are partition P1∈[0000, 0999], partition P2∈[1000, 1999], partition P3∈[2000, 2999] and partition P4∈[3000, 3999], and each partition maintains one primary copy and two secondary copies.

[0040] The database system includes at least two database nodes, and each database node can store the primary copy or secondary copy of at least one partition. For example Figure 1 The database node 1 in stores the primary copy of partition P1, the secondary copy of partition P2, the primary copy of partition P3 and the secondary copy of partition P4; the database node 2 stores the secondary copy of partition P1, the primary copy of partition P2, the secondary copy of partition P3 and the secondary copy of partition P4; and the database node 3 stores the secondary copy of partition P1, the secondary copy of partition P2, the secondary copy of partition P3 and the primary copy of partition P4.

[0041] Under this structure, if a write request involves data across multiple partitions, the existing system often sends these write operations to the primary copy of the corresponding partition for execution, but if a consistency control mechanism is not used, it may lead to inconsistent data between partitions.

[0042] Assuming that a user performs an update, the data in the range of "0010~1888" needs to be updated, involving two partitions P1 and P2, and assuming that after sending an update request for partition P1 to database node 1, database node 1 successfully updates the data in partition P1; when sending an update request for partition P2 to database node 2, database node 2 fails or network packet loss occurs, resulting in unsuccessful update; in the case where the system is not configured with a global transaction coordination mechanism, the update operation is not uniformly committed or rolled back; resulting in inconsistent data update state: the data in partition P1 has been successfully updated, and the data in partition P2 remains unchanged; when the front-end user views the data, the update is found to be inconsistent, and the defect of difficult to guarantee data consistency is faced. Based on this, in order to guarantee the data consistency between multiple database nodes, especially when processing transaction requests involving multiple partitions or multiple masters and replicas, the system needs to rely on an effective synchronization mechanism to coordinate the data state and ensure a consistent view of the data.

[0043] To cope with cross-partition write requests, the database system also includes a coordination node for coordinating the processing of cross-partition write requests. The coordination node can be selected or set in multiple ways according to different system deployment architectures and running states. The following are several possible implementation ways:

[0044] In one possible implementation way, the coordination node is a node that is different from the at least two database nodes included in the database system. The coordination node is deployed as an independent service node and does not undertake any master or slave storage task of the data partition, but is only responsible for coordinating the processing of cross-partition write requests. Since the node does not participate in actual data storage and replica synchronization, it can be dedicated to logical operations such as scheduling, transaction control, and voting coordination, thereby reducing resource competition with the data read-write path and helping to improve the processing efficiency and modularization level of the system.

[0045] In another possible implementation way, the coordination node is dynamically selected from the at least two database nodes included in the database system, such as a database node whose load satisfies a first preset condition among the at least two database nodes included in the database system, wherein the first preset condition includes that the load of the coordination node is the smallest among the at least two database nodes, or the load of the coordination node is less than a preset load threshold. In this way, the coordination node can be dynamically selected based on the actual load at runtime, reducing the impact of transaction coordination on system resources, while improving the overall processing throughput and scheduling flexibility of the system, suitable for dynamically balancing the coordination load in a high-concurrency request scenario.

[0046] In yet another possible implementation, the coordination node is selected in the following manner: among the database nodes where the primary replicas of the partitions involved in the write request are located, a database node whose load satisfies a second preset condition, wherein the second preset condition comprises that the load of the coordination node is the smallest among all the database nodes where the primary replicas are located, or the load of the coordination node is smaller than a preset load threshold. The write request is a write request involving at least two partitions. In this implementation, the coordination node has higher data association and network proximity with the data nodes actually storing the primary replicas on the write path, thereby reducing cross-node communication overhead and coordination delay, and being suitable for a low-delay distributed write transaction execution scenario.

[0047] For example, the system can also form a hybrid strategy in combination with multiple selection conditions, for example, first selecting a low-load node as the coordination node from the database nodes where the primary replicas are located, and if the condition is not satisfied, falling back to selecting from the global node pool (involving all database nodes included in the database system).

[0048] For example, the system can also form a hybrid strategy in combination with multiple selection conditions, for example, first selecting a low-load node as the coordination node from the database nodes where the primary replicas are located, and if the condition is not satisfied, falling back to selecting from the global node pool (involving all database nodes included in the database system). Figure 2 Figure 3 The embodiment of the present specification provides a write request processing method, which is applied to a database system as shown in Figure 1 The embodiment of the present specification provides a write request processing method, which is applied to a database system as shown in

[0049] In S201, after receiving a write request involving at least two partitions, the coordination node sends a preparation instruction corresponding to the write request to the database nodes where the primary replicas of the partitions involved in the write request are located.

[0050] In this step, the coordination node can determine the multiple partitions involved in the write request based on the data key range or partition identifier information in the write request. Since the primary replicas of these partitions can be distributed on different database nodes, the coordination node needs to identify the database nodes where the primary replicas are located respectively, and send a preparation instruction containing write metadata (such as transaction ID, timestamp, data item digest, etc.) to each database node (301 in Figure 3 The preparation instruction is used to trigger the database nodes where the primary replicas are located to start the write-ahead verification process and enter the transaction preparation phase, but does not directly execute the write operation, so as to avoid inconsistent state before the consensus is completed.

[0051] In S202, after receiving the preparation instruction, the database nodes where the primary replicas are located perform a consensus process for the write request (302 in Figure 3 with all the database nodes where the secondary replicas corresponding to the primary replicas are located, and feed back the consensus result to the coordination node (303 in Figure 3 The consensus result is used to indicate whether the write data corresponding to the write request can be successfully synchronized to the secondary replicas.

[0052] ​In this step, to ensure that the write request has good synchronizability before execution, each primary replica needs to confirm whether the primary replica managed by the database node has the executability of the write operation. The key step is to verify the synchronization capability between the primary and secondary replicas. That is, the database node where the primary replica is located needs to establish a consistency consensus with all database nodes where the secondary replicas of the corresponding data partition are located, to avoid write failure caused by replica state lag, unavailability, etc.

[0053] For example, the consensus process can use a voting mechanism based on logical timestamps to ensure that the voting response in a distributed network environment has a clear timing judgment basis, preventing data conflicts or misjudgments caused by concurrent voting.

[0054] After the database node where the primary replica is located receives the preparation instruction, it generates a voting request about whether it can synchronize the write data corresponding to the write request, and sends the voting request to all database nodes where the secondary replicas corresponding to the primary replica are located. The voting request includes a globally unique timestamp for identifying the write request, such as combining physical time + node ID, which serves as a timing comparison reference in the voting process. Through the rule of strictly monotonically increasing timestamp, it can effectively identify and reject "historical voting requests" or "repeated submissions", preventing inconsistent replica states for the same write request among multiple replicas.

[0055] After each database node where the secondary replica is located receives the voting request, it compares the timestamp in the voting request with the timestamps of all previously responded voting requests. If the timestamp in the voting request is greater than the timestamp of any previously responded voting request (i.e., indicating that the current write transaction is the latest transaction), it returns a first voting response to the database node where the primary replica is located, indicating that it can synchronize the write data. Otherwise, it returns a second voting response to the database node where the primary replica is located, indicating that it cannot synchronize the write data. This judgment mechanism effectively avoids data disorder caused by "write backtracking" or "historical transactions overriding current state" and other problems.

[0056] The database node where the primary replica is located receives the voting responses returned by each database node where the secondary replica is located based on the voting request. The voting responses include a first voting response indicating that it can synchronize the write data and a second voting response indicating that it cannot synchronize the write data. Then, it counts the number of votes containing the first voting response. The consensus result fed back by the database node where the primary replica is located to the coordination node includes the vote count. If the vote count is not less than a preset number threshold, it indicates that the write data corresponding to the write request can be successfully synchronized to the secondary replica, and it is considered that the partition has reached the write-ahead consensus.

[0057] The voting mechanism performs synchronization capability evaluation before the write operation, ensures that the write is executed only when the majority of the replicas are ready, and avoids interruption due to unavailability of the replicas during execution. Before the write is executed, both the consistency of the master-slave replicas within each partition (intra-partition consensus) and the basic guarantee for the multi-partition write request initiated by the coordination node are ensured, thereby supporting unified submission of cross-partition transactions.

[0058] In S203, the coordination node generates a submission instruction and sends it to all database nodes where the master replicas are located, in the case that all consensus results received indicate that the write data corresponding to the write request can be successfully synchronized.

[0059] In this step, the coordination node collects the consensus results of all database nodes where the master replicas are located, and then makes a centralized decision. Only when all consensus results indicate that synchronization is possible, that is, all partitions meet the write condition, the coordination node generates a global submission instruction to ensure global consistency of the write operation. The submission instruction can carry the final transaction timestamp, version information or data confirmation digest, which is used to identify the global state of this write transaction. The coordination node sends the submission instruction to each relevant database node where the master replica is located (304-1) as the final authorization signal of the write operation. Figure 3

[0060] In S204, the database node where the master replica is located executes the write operation corresponding to the write request stored in the master replica after receiving the submission instruction, and synchronizes the write data corresponding to the write operation to all slave replicas corresponding to the master replica.

[0061] In this step, the database node where the master replica is located executes the write operation corresponding to the write request (305) after receiving the submission instruction issued by the coordination node, writes the local master replica and replicates the write data corresponding to the write operation to all slave replicas maintained by it in an asynchronous or semi-synchronous manner (306). During synchronization, data consistency replication can be achieved through pre-write logs, replication queues or timestamp-based incremental push methods. After synchronization is completed, the database node where the master replica is located can feed back the write completion status to the coordination node, thereby ensuring that the entire distributed write transaction is completed with a consistent state among multiple partitions.

[0062] The write request processing method provided in this embodiment realizes efficient processing and global consistency control of cross-partition write requests through "consistency between master replicas in different partitions" + "consistency between master-slave replicas in the same partition".

[0063] ​Consistency between different partition master replicas: refers to when processing a write request involving at least two data partitions, the database nodes where the partition master replicas are located, under the driving of the prepare instruction and the commit instruction issued by the coordination node, collaboratively participate in the processing of the write request. Through unified decision control, it ensures that the master replicas of each partition are synchronized to enter the "commit" or "rollback" state, thereby realizing consistent commit or abortion between partitions, and avoiding the problem of partial commit in distributed write operation.

[0064] Consistency between master and slave replicas in the same partition: refers to that before the write request is committed, the database node where the master replica is located initiates a write-ahead consensus process to the database node where the corresponding slave replica is located. It is confirmed in advance whether the slave replica has synchronization capability, so that after the write operation is completed, the write result can be timely and consistently synchronized to all slave replicas, ensuring the data consistency and replica reliability within each data partition.

[0065] The double-layer mechanism of "cross-partition coordination + intra-partition synchronization" not only improves the reliability and execution efficiency of cross-partition transactions, but also helps to enhance the horizontal expansion capability and overall availability of the system, thereby being more suitable for large-scale distributed database application scenarios under high concurrency and high consistency requirements.

[0066] In some embodiments, considering that the delay bottleneck of distributed transactions is usually concentrated in the final data synchronization stage, especially after the master replica writes are completed, data replication and state confirmation processes with multiple slave replicas still need to be completed, and the overall operation has a non-deterministic delay. In most scenarios, the coordination node issuing a commit instruction means that: the write request has been confirmed by the consistency of each master replica node; the involved partitions all have the execution conditions of write operation; and the decision of the entire distributed transaction has been reached.

[0067] Based on the above premise, after the coordination node sends the commit instruction to all database nodes where the master replicas are located, it does not need to wait for the feedback of the local write operation and the synchronization completion state of the master and slave replicas of each master replica database node. The coordination node can directly return a write success prompt to the client that initiated the write request, which can significantly shorten the overall time consumption of client request-response and improve user experience.

[0068] In some embodiments, if there is at least one consensus result indicating that the write data corresponding to the write request cannot be successfully synchronized to the slave replica, such as the number of votes fed back by any database node where the master replica is located being less than a preset number threshold, the coordination node determines that the current write request does not meet the write condition and there is a data consistency risk.

[0069] To avoid the data disorder problem caused by partial partition commit and partial failure of distributed write requests, the coordination node generates a rollback instruction and sends it to all database nodes where the master replicas are located Figure 3the database node where the primary copy is located terminates the write operation process corresponding to the write request after receiving the rollback instruction, for example, including data writing, buffer operation or local transaction preparation that have not been executed, and releases resources allocated for the write request, for example, locked data partition, write buffer, pre-write log space, etc., to prevent resource leakage or system blocking Figure 3 The cross-partition write operation is effectively prevented from being incorrectly committed when consistency guarantee is insufficient, avoiding partial commitment, data misplacement or logical conflict in the distributed system, and the write process that does not meet the consensus condition is terminated in advance, which helps to reduce unnecessary resource consumption and network transmission overhead, and improves the overall operation efficiency of the system.

[0070] In some embodiments, considering that in a large-scale database system, the reason for consensus failure can be a short-term network jitter, a temporary overload of the replica node, a partial replica state delay and other unstructured failures. Such problems are instantaneous and non-persistent, and do not mean that the write request itself cannot be successful. Only a little waiting and retry can achieve consensus. If the entire write request processing is directly terminated due to a single consensus failure, not only the completed preparation operation will be wasted, but also the client layer may be caused to frequently resend. Therefore, to improve the robustness of write request processing and the high availability of the system, the coordination node can introduce an automatic retry mechanism when detecting that the write request needs to be rolled back due to consensus failure.

[0071] After the coordination node sends the rollback instruction to all database nodes where the primary copy is located, it further judges whether the rollback number of the current write request has reached the preset retry number set by the system. In the case where the rollback number does not reach the preset retry number, the coordination node can regenerate the preparation instruction corresponding to the write request and send it to the database node where the primary copy is located again, so as to re-initiate the consistency coordination and processing flow of the write request. The robustness and final success rate of write request processing in a distributed unstable environment are improved.

[0072] In some embodiments, in the case where the coordination node sends the rollback instruction to all database nodes where the primary copy is located and the rollback number reaches the preset retry number, the system will regard the write request as an unrecoverable failure. The coordination node terminates the subsequent write processing flow, returns a write failure prompt information to the client sending the write request, and explicitly informs that this write request is not successfully committed and needs to be selected by the client according to the actual scene for retry, degradation processing or termination operation. This embodiment sets a reasonable retry upper limit and returns a write failure prompt information to the client in time when the limit fails, limits resource consumption, prevents performance degradation caused by invalid retry, and guarantees the timeliness and client perception ability of the system in the replica abnormal or network failure scenario.

[0073] In some embodiments, please refer to Figure 4The embodiment of the present specification also provides a write request processing method, applied to a coordinator node in a database system, and the method comprises the following steps:

[0074] In S401, after receiving a write request related to two partitions, a preparation instruction corresponding to the write request is sent to a database node where a primary copy of each partition related to the write request is located, so that the database node where the primary copy is located and a database node where all secondary copies corresponding to the primary copy execute a consensus process for the write request and return a consensus result; the consensus result is used to indicate whether the write data corresponding to the write request can be successfully synchronized to the secondary copy.

[0075] In S402, after receiving the consensus result returned by the database node where the primary copy is located, if all consensus results indicate that the write data corresponding to the write request can be successfully synchronized, a commit instruction is generated and sent to all database nodes where the primary copies are located, so that the database node where the primary copy is located executes a write operation corresponding to the stored primary copy in the write request, and synchronizes the write data corresponding to the write operation to all secondary copies corresponding to the primary copy.

[0076] In some embodiments, the method further comprises: after sending the commit instruction to all database nodes where the primary copies are located, returning a write success prompt information to a client sending the write request.

[0077] In some embodiments, the method further comprises: if there is at least one consensus result indicating that the write data corresponding to the write request cannot be successfully synchronized to the secondary copy, generating a rollback instruction and sending it to all database nodes where the primary copies are located.

[0078] In some embodiments, the method further comprises: if the rollback instruction is sent to all database nodes where the primary copies are located, and the number of rollbacks does not reach a preset retry number, the preparation instruction corresponding to the write request is re-sent to all database nodes where the primary copies are located; or if the rollback instruction is sent to all database nodes where the primary copies are located, and the number of rollbacks reaches the preset retry number, a write failure prompt information is returned to the client sending the write request.

[0079] In some embodiments, please refer to Figure 5 The embodiment of the present specification also provides a write request processing method, applied to a coordinator node in a database system, and the method comprises the following steps:

[0080] In S501, after receiving the preparation instruction sent by the coordination node and corresponding to the write request involving two partitions, the database node determines the master replica of the partition stored by itself, and performs a consensus process for the write request with all the database nodes where the slave replicas corresponding to the master replica are located, and feeds back a consensus result to the coordination node, where the consensus result is used to indicate whether the write data corresponding to the write request can be successfully synchronized to the slave replicas, so that the coordination node makes a decision based on the consensus result whether to continue to execute the write request.

[0081] In S502, after receiving the commit instruction sent by the coordination node, the database node executes the write operation corresponding to the stored master replica in the write request, and synchronizes the write data corresponding to the write operation to all the slave replicas corresponding to the master replica.

[0082] In some embodiments, the database node where all the slave replicas corresponding to the master replica are located performs a consensus process for the write request, including: sending a voting request to all the database nodes where the slave replicas corresponding to the master replica are located, about whether the write data corresponding to the write request can be synchronized; receiving a voting response returned by each database node based on the voting request, where the voting response includes a first voting response indicating that the write data can be synchronized and a second voting response indicating that the write data cannot be synchronized; counting the number of first voting responses; and wherein the consensus result includes the number of votes, and indicates that the write data corresponding to the write request can be successfully synchronized to the slave replicas when the number of votes is not less than a preset number threshold.

[0083] In some embodiments, the method further includes: after receiving the rollback instruction sent by the coordination node, terminating the write operation process corresponding to the write request, and releasing the resources allocated for the write request. Various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the features. However, due to the limited space, they are not described one by one, and therefore any combination of the various technical features in the above embodiments also belongs to the scope disclosed by the present specification.

[0084] In some embodiments, the present specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor implements the method of any one of the above embodiments by running the executable instructions.

[0085] Figure 6 FIG. 1 is a schematic structural diagram of a device provided by an example embodiment. Please refer to Figure 6At the hardware level, the device includes a processor 602, an internal bus 604, a network interface 606, a memory 608, and a non-volatile memory 610, and can also include other hardware required by functions. One or more embodiments of the present specification can be implemented in a software manner, such as reading a corresponding computer program from the non-volatile memory 610 into the memory 608 by the processor 602 and then running. Of course, in addition to the software implementation, one or more embodiments of the present specification do not exclude other implementation manners, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0086] In some embodiments, the write request processing apparatus can be applied to the device as shown in Figure 6 , to implement the technical solutions of the present specification. Wherein, the write request processing apparatus can include:

[0087] The preparation instruction sending module is configured to, after receiving the write request involving two partitions, send the preparation instruction corresponding to the write request to the database node where the primary copy of each partition involved in the write request is located, so that the database node where the primary copy is located and the database node where all the secondary copies corresponding to the primary copy execute the consensus process for the write request, and return the consensus result; the consensus result is used to indicate whether the write data corresponding to the write request can be successfully synchronized to the secondary copy.

[0088] The submission instruction sending module is configured to, after receiving the consensus result returned by the database node where the primary copy is located, in the case that all the consensus results indicate that the write data corresponding to the write request can be successfully synchronized, generate a submission instruction and send it to all the database nodes where the primary copies are located, so that the database node where the primary copy is located executes the write operation corresponding to the stored primary copy in the write request, and synchronizes the write data corresponding to the write operation to all the secondary copies corresponding to the primary copy.

[0089] In some embodiments, the write request processing apparatus can be applied to the device as shown in Figure 6 , to implement the technical solutions of the present specification. Wherein, the write request processing apparatus can include:

[0090] The consensus module is configured to, after receiving the preparation instruction sent by the coordination node and corresponding to the write request involving two partitions, determine the primary copy of the partition stored by itself, and execute the consensus process for the write request with the database node where all the secondary copies corresponding to the primary copy are located, feed back the consensus result to the coordination node, and the consensus result is used to indicate whether the write data corresponding to the write request can be successfully synchronized to the secondary copy, so that the coordination node decides whether to continue to execute the write request based on the consensus result.

[0091] The write operation execution module is configured to, after receiving the commit instruction sent by the coordination node, execute a write operation corresponding to the stored primary copy in the write request, and synchronize write data corresponding to the write operation to all secondary copies corresponding to the primary copy.

[0092] The implementation process of the functions and roles of the modules in the apparatus is specifically described in the implementation process of the corresponding steps in the above method, and will not be described here.

[0093] Based on the same idea as the above method, the present specification also provides a database system, comprising a coordination node and at least two database nodes; the coordination node is a node independent of the at least two database nodes or a node selected from the at least two database nodes.

[0094] Based on the same idea as the above method, the present specification also provides a computer-readable storage medium having computer instructions stored thereon, the instructions being executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0095] The computer-readable medium includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, disk storage, quantum memory, graphene-based storage medium or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition in this paper, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carriers.

[0096] Based on the same idea as the above method, the present specification also provides a computer program product, comprising computer program / instructions, which are executed by a processor to implement the steps of the method according to any one of the above embodiments.

[0097] The above is only the preferred embodiment of one or more embodiments of the present specification, and does not limit one or more embodiments of the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the protection scope of one or more embodiments of the present specification.

Claims

1. A method for processing a write request, applied to a database system, the database system comprising a coordination node, the method comprising: sending, by the coordination node, a prepare instruction corresponding to a write request involving at least two partitions to database nodes where primary replicas of the partitions involved in the write request are located after receiving the write request; performing, by the database nodes where the primary replicas are located, a consensus process for the write request with database nodes where all the secondary replicas corresponding to the primary replicas are located after receiving the prepare instruction, and feeding back a consensus result to the coordination node, the consensus result indicating whether write data corresponding to the write request can be successfully synchronized to the secondary replicas; generating, by the coordination node, a commit instruction and sending the commit instruction to all the database nodes where the primary replicas are located if all the consensus results received indicate that the write data corresponding to the write request can be successfully synchronized; and performing, by the database nodes where the primary replicas are located, a write operation corresponding to the write request stored in the primary replicas after receiving the commit instruction, and synchronizing write data corresponding to the write operation to all the secondary replicas corresponding to the primary replicas. 2.The method of claim 1, wherein performing, by the database nodes where the primary replicas are located, a consensus process for the write request with database nodes where all the secondary replicas corresponding to the primary replicas are located after receiving the prepare instruction comprises: sending, by the database nodes where the primary replicas are located, a voting request to database nodes where all the secondary replicas corresponding to the primary replicas are located after receiving the prepare instruction, the voting request indicating whether write data corresponding to the write request can be synchronized; receiving, by the database nodes where the primary replicas are located, a voting response returned by each of the database nodes where the secondary replicas are located, the voting response comprising a first voting response indicating that the write data can be synchronized and a second voting response indicating that the write data cannot be synchronized; counting, by the database nodes where the primary replicas are located, a number of the first voting responses; and wherein the consensus result comprises the number of the first voting responses, and indicates that the write data corresponding to the write request can be successfully synchronized to the secondary replicas if the number of the first voting responses is not less than a preset number threshold. 3.The method of claim 2, wherein the voting request comprises a globally unique timestamp for identifying the write request; and wherein the database nodes where the secondary replicas are located return the first voting response to the database nodes where the primary replicas are located if the timestamp of the voting request is greater than a timestamp of a historical voting request that has been responded to previously, and otherwise return the second voting response to the database nodes where the primary replicas are located. 4.The method of claim 1, further comprising: returning, by the coordination node, a write success prompt to a client sending the write request after sending the commit instruction to all the database nodes where the primary replicas are located. 5.The method of claim 1, further comprising: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The coordination node generates a rollback instruction and sends it to all the database nodes where the primary replicas are located in the case that at least one of the consensus results indicates that the write data corresponding to the write request cannot be synchronized. The database node where the primary replica is located terminates the write operation process corresponding to the write request and releases the resources allocated for the write request after receiving the rollback instruction.

6. The method of claim 5, further comprising: The coordination node re-sends the prepare instruction corresponding to the write request to all the database nodes where the primary replicas are located in the case that the rollback instruction is sent to all the database nodes where the primary replicas are located and the number of rollbacks does not reach a preset number of retries; or, The coordination node returns a write failure prompt to the client that sent the write request in the case that the rollback instruction is sent to all the database nodes where the primary replicas are located and the number of rollbacks reaches the preset number of retries.

7. The method of claim 1, wherein the database system comprises at least two database nodes, and the database nodes are used to store primary replicas or secondary replicas of at least one partition; wherein, The coordination node is a node that is distinguished from the at least two database nodes, or the coordination node is a node selected in any of the following ways: Among the at least two database nodes, a database node whose load meets a first preset condition, wherein the first preset condition comprises that the load of the coordination node is the smallest among the at least two database nodes, or the load of the coordination node is less than a preset load threshold; or Among the database nodes where the primary replicas of the partitions involved in the write request are located, a database node whose load meets a second preset condition, wherein the second preset condition comprises that the load of the coordination node is the smallest among all the database nodes where the primary replicas are located, or the load of the coordination node is less than a preset load threshold.

8. A write request processing method, the method comprising: After receiving a write request involving two partitions, sending a prepare instruction corresponding to the write request to the database nodes where the primary replicas of the partitions involved in the write request are located, so that the database nodes where the primary replicas are located and the database nodes where all the secondary replicas corresponding to the primary replicas are located perform a consensus process for the write request and return consensus results; the consensus results are used to indicate whether the write data corresponding to the write request can be successfully synchronized to the secondary replicas; After receiving the consensus results returned by the database nodes where the primary replicas are located, in the case that all the consensus results indicate that the write data corresponding to the write request can be successfully synchronized, a commit instruction is generated and sent to all the database nodes where the primary replicas are located, so that the database nodes where the primary replicas are located perform the write operation corresponding to the write request in the write request and synchronize the write data corresponding to the write operation to all the secondary replicas corresponding to the primary replicas.

9. The method of claim 8, further comprising: After sending the commit instruction to all the database nodes where the primary copy is located, a write success prompt information is returned to the client sending the write request.

10. The method of claim 8, further comprising: In a case where at least one of the consensus results indicates that the write data corresponding to the write request cannot be synchronized, a rollback instruction is generated and sent to all the database nodes where the primary copy is located.

11. The method of claim 10, further comprising: In a case where the rollback instruction is sent to all the database nodes where the primary copy is located and the number of rollbacks does not reach a preset retry number, the prepare instruction corresponding to the write request is re-sent to all the database nodes where the primary copy is located. Alternatively, In a case where the rollback instruction is sent to all the database nodes where the primary copy is located and the number of rollbacks reaches the preset retry number, a write failure prompt information is returned to the client sending the write request.

12. A write request processing method, the method comprising: After receiving a prepare instruction sent by a coordination node and corresponding to a write request involving two partitions, a primary copy of the partition stored by itself is determined, and a consensus process for the write request is performed with all the database nodes where the slaves corresponding to the primary copy are located, a consensus result is fed back to the coordination node, the consensus result is used to indicate whether the write data corresponding to the write request can be successfully synchronized to the slaves, so that the coordination node decides whether to continue to execute the write request based on the consensus result; After receiving a commit instruction sent by the coordination node, a write operation corresponding to the write request and corresponding to the stored primary copy is executed, and the write data of the write operation is synchronized to all the slaves corresponding to the primary copy.

13. The method of claim 12, wherein the database nodes where all the slaves corresponding to the primary copy are located perform the consensus process for the write request, comprising: sending a voting request to all the database nodes where the slaves corresponding to the primary copy are located, the voting request being about whether the write data corresponding to the write request can be synchronized; receiving a voting response returned by each of the database nodes where the slaves are located, the voting response including a first voting response indicating that the write data can be synchronized and a second voting response indicating that the write data cannot be synchronized; counting the number of the first voting responses; wherein the consensus result includes the number of votes, and in a case where the number of votes is not less than a preset number threshold, it indicates that the write data corresponding to the write request can be successfully synchronized to the slaves.

14. The method of claim 12, further comprising: After receiving a rollback instruction sent by the coordination node, terminating the write operation process corresponding to the write request and releasing the resources allocated for the write request.

15. A database system comprising a coordination node and at least two database nodes; the coordination node being a node independent of the at least two database nodes or a node selected from the at least two database nodes; The coordination node is configured to perform the method of any one of claims 8-11 to coordinate handling of a write request involving at least two partitions; and the database nodes hosting the primary replicas of the partitions involved in the write request are configured to perform the method of any one of claims 12-14.

16. An electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor, by running the executable instructions, implements the steps of the method of any one of claims 1-14.

17. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method of any one of claims 1-14.

18. A computer program product comprising computer program / instructions that, when executed by a processor, implement the steps of the method of any one of claims 1-14.

Images