Consistency point location acquisition method, system and device, storage medium and program product

Abstract

The invention relates to a consistency point location obtaining method, system and device, a storage medium and a program product. The method comprises the steps that under the condition that a main processing node obtains a distributed lock of a coordination server side in a distributed database system, self-locking operation is executed for the main processing node; under the condition that the current transaction execution of the main processing node is completed, a preparation completion signal of the main processing node is fed back to the coordination server; under the condition that all processing nodes in the distributed database system feed back preparation completion signals to the coordination server side, a point location obtaining instruction is sent to all data storage nodes in the distributed database system, and the point location obtaining instruction is used for instructing the data storage nodes to feed back point location information; and receiving point location information returned by each data storage node, and determining a consistent point location of the distributed database system. By adopting the method, the obtaining efficiency of the consistent point locations can be improved.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to methods, systems, devices, storage media, and program products for obtaining consistency points. Background Technology

[0002] For databases, consistent endpoints are a crucial foundation for achieving a highly available architecture. By ensuring data consistency across different nodes, high availability features such as failover and load balancing can be supported, guaranteeing business continuity.

[0003] In related technologies, since multiple transactions may access and modify the same data simultaneously in a distributed database, in order to ensure the accuracy of the consistency point, log information from each data storage node in the distributed database is collected and synchronized to obtain the consistency point of the distributed database.

[0004] However, the above method requires the collection and processing of log information, and its efficiency in obtaining consistency points is relatively low. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for obtaining consistency points, which can improve the efficiency of obtaining consistency points, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for obtaining consistency points, applied to the main processing node in a distributed database system, wherein the main processing node is any processing node in the distributed database system; the method includes:

[0007] When the main processing node acquires the distributed lock of the coordinating server in the distributed database system, a self-locking operation is performed on the main processing node.

[0008] Once the current transaction on the primary processing node is completed, a preparation completion signal for the primary processing node is sent back to the coordination server.

[0009] In a distributed database system, when all processing nodes have sent a preparation completion signal to the coordination server, a point acquisition instruction is sent to all data storage nodes in the distributed database system. The point acquisition instruction is used to instruct the data storage nodes to send back point information.

[0010] Receive point information returned by each data storage node to determine the consistency points of the distributed database system.

[0011] In one embodiment, performing a self-locking operation on the master processing node includes:

[0012] Modify the atomic variables cached on the primary processing node to a specified value, which indicates that the primary processing node is in the process of executing a command, thereby preventing new transactions from being committed to the primary processing node.

[0013] In one embodiment, receiving the point information returned by each data storage node to determine the consistency point of the distributed database system includes:

[0014] After receiving the location information returned by each data storage node, the location information is spliced ​​together to obtain a location set;

[0015] The set of points is determined as the consistency points of the distributed database system.

[0016] In one embodiment, after receiving the point information returned by each data storage node and determining the consistency point of the distributed database system, the method further includes:

[0017] Release the distributed lock in the coordination server.

[0018] In one embodiment, the method further includes:

[0019] Perform a self-lock release operation, which is the inverse operation of the self-lock operation.

[0020] In one embodiment, sending a preparation completion signal of the main processing node to the coordination server includes:

[0021] Create the first child node under the preset listening path of the coordination server;

[0022] The first child node records the preparation completion status of the main processing node.

[0023] Secondly, this application provides a method for obtaining consistency points, applied to a slave processing node in a distributed database system. The slave processing node is any processing node in the distributed database system other than the master processing node. The method includes:

[0024] If a preparation completion signal is detected from the master processing node in the coordination server of the distributed database system, a self-locking operation is performed on the slave processing node.

[0025] Once the current transaction of the processing node is completed, it sends a preparation completion signal to the coordination server. This ensures that the master processing node sends a point acquisition command to all data storage nodes in the distributed database system after all processing nodes have sent preparation completion signals to the coordination server. The master processing node then receives point information returned by each data storage node to determine the consistency point of the distributed database system.

[0026] In one embodiment, performing a self-locking operation from the processing node includes:

[0027] The atomic variable cached by the slave node will be modified to a specified value, which indicates that the slave node is in the process of executing a command, thereby preventing new transactions from being committed to the slave node.

[0028] In one embodiment, sending a preparation completion signal from the processing node back to the coordination server includes:

[0029] Create a second child node under the preset listening path of the coordination server;

[0030] The second child node records the preparation completion status of the processing node.

[0031] In one embodiment, the method further includes:

[0032] If the distributed lock in the coordination server is detected to be released, a self-lock release operation is performed on the slave processing node. The self-lock release operation is the inverse operation of the self-lock operation.

[0033] Thirdly, this application also provides a distributed database system, which includes a master processing node and at least one slave processing node;

[0034] The main processing node is used to perform a self-locking operation on the main processing node when it acquires the distributed lock of the coordination server in the distributed database system, and to send a preparation completion signal to the coordination server when the current processing transaction of the main processing node is completed.

[0035] The slave processing node is used to perform a self-locking operation when a preparation completion signal is detected from the master processing node in the coordination server of the distributed database system, and to send a preparation completion signal from the slave processing node to the coordination server when the current processing transaction of the slave processing node is completed.

[0036] The master processing node is also used to send point acquisition instructions to all data storage nodes in the distributed database system when all processing nodes in the distributed database system have sent a ready-to-complete signal to the coordination server, and to receive point information returned by each data storage node to determine the consistent point of the distributed database system.

[0037] In one embodiment, the main processing node is also used to release the distributed lock in the coordination server and perform a self-lock release operation;

[0038] The slave processing node is also used to perform a self-lock release operation on the slave processing node when it is detected that the distributed lock in the coordination server has been released.

[0039] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the method for obtaining consistent bit information from a master processing node or a slave processing node.

[0040] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method for obtaining consistent bit information from a master processing node or a slave processing node.

[0041] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the method for obtaining consistent bit information from a master processing node or a slave processing node.

[0042] The aforementioned consistency point acquisition method, apparatus, computer device, computer-readable storage medium, and computer program product, when the main processing node acquires the distributed lock of the coordination server in the distributed database system, perform a self-locking operation on the main processing node. Therefore, this application uses the distributed lock to ensure the uniqueness of the main processing node's execution of the consistency point acquisition command, preventing other processing nodes from repeatedly executing the command to acquire the consistency point while the main processing node is acquiring it. When the main processing node completes its current transaction, it sends a preparation completion signal to the coordination server. Thus, the main processing node may refrain from processing new transactions through the self-locking operation. When all processing nodes in the distributed database system have sent preparation completion signals to the coordination server, a point acquisition instruction is sent to all data storage nodes in the distributed database system. The point acquisition instruction instructs the data storage nodes to return point information. By receiving the point information returned by each data storage node, the consistency point of the distributed database system is determined. Therefore, this application obtains the point information after all processing nodes in the distributed database system have performed self-locking operations and are locked. This ensures that the data stored in all data storage nodes remains consistent (i.e., no data is modified by the processing nodes at this moment). Thus, the point information of each data storage node can be obtained to obtain the consistent point of the distributed database without synchronization processing, which helps to improve the efficiency of obtaining the consistent point. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is an application environment diagram of a consistency point acquisition method in one embodiment;

[0045] Figure 2 This is a flowchart illustrating a method for obtaining consistency points in the main processing unit in one embodiment.

[0046] Figure 3 This is a flowchart illustrating a method for obtaining consistency points from a processing unit in one embodiment;

[0047] Figure 4 This is a schematic diagram of a distributed database system scenario in one embodiment;

[0048] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0051] The consistency point acquisition method provided in this application embodiment can be applied to, for example, Figure 1In the application environment shown, a distributed database system can include at least two processing nodes and at least two data storage nodes. The processing nodes (e.g., processing node 1 and processing node 2) can coordinate with each other through a coordination server. Each processing node can process data storage nodes (e.g., data storage node 1, data storage node 2, and data storage node 3) based on received transactions, such as modifying, reading, and transferring data. The processing nodes can be, but are not limited to, various personal computers, laptops, smartphones, tablets, servers, and portable wearable devices. IoT devices can include smart TVs, smart in-vehicle devices, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. The server can be a standalone physical server or a cloud server providing cloud computing services.

[0052] In one exemplary embodiment, such as Figure 2 As shown, a method for obtaining consistency points is provided. This method is applied to the main processing node in a distributed database system. The main processing node is... Figure 1 In any processing node of the distributed database system, the process includes the following steps S110 to S140. Wherein:

[0053] Step S110: If the main processing node acquires the distributed lock of the coordination server in the distributed database system, perform a self-locking operation on the main processing node.

[0054] A distributed database system can include at least two processing nodes and at least two data storage nodes. The processing nodes can coordinate with each other through a coordination server. Each processing node can process data storage nodes based on received transactions, such as modifying, reading, or transferring data. The master processing node can be a processing node elected by the coordination server, any processing node with a pre-designated distributed lock, or any processing node that receives a bit acquisition and execution instruction.

[0055] Among them, self-locking operation is an operation that prevents new transactions from being committed to the processing node. Exemplary self-locking operations can be CAS (Compare-and-Swap) locks, pessimistic locks (such as Java's ReentrantLock, C++'s std::mutex and other in-process locks), local state bits (i.e., using local atomic variables to store node state), etc.

[0056] For example, in this embodiment, any processing node can respond to a received point acquisition execution instruction. This processing node is the main processing node, and the main processing node can then perform a distributed lock acquisition operation on the coordination server until the processing node acquires the distributed lock from the coordination server in the distributed database system. For instance, the main processing node can connect to the coordination server, create child nodes under the lock path of the distributed lock in the coordination server, acquire and sort all child nodes under the lock path, and determine whether the child node created by the main processing node is the smallest node. If it is the smallest node, the main processing node acquires the distributed lock. If it is not the smallest node, the main processing node listens for the largest node with a sequence number smaller than the child node created by the main processing node. If the largest node is deleted, the process returns to the step of determining whether the child node created by the main processing node is the smallest node, until the distributed lock in the coordination server is acquired. In another example, if the coordination server selects a processing node as the main processing node, it creates a child node corresponding to the main processing node under the lock path of the distributed lock in the coordination server, so that the main processing node acquires the distributed lock from the coordination server in the distributed database system.

[0057] Then, after the main processing node acquires the distributed lock from the coordination server in the distributed database system, a self-locking operation is performed on the main processing node. This embodiment can implement the self-locking operation using any of the following methods: CAS lock, pessimistic lock, or local state bits. Taking CAS lock as an example, this embodiment can predefine the atomic lock state for the main processing node, such as using atomic variables (e.g., Java's AtomicInteger, C++'s std::atomic). <int>The numerical storage lock state is used to ensure read and write atomicity. Then the main processing node repeatedly executes the CAS operation, trying to change the lock state of the CAS lock from "unlocked (0)" to "locked (1)" until the CAS lock is successfully acquired, thus realizing self-locking for the main processing node.

[0058] In some embodiments, performing a self-locking operation on the master processing node includes:

[0059] Step S111: Modify the atomic variable cached by the main processing node to a specified value. The specified value indicates that the main processing node is in the state of executing a command, so as to prevent new transactions from being committed to the main processing node.

[0060] Atomic variables are used to store lock states to describe the current node state. For example, an atomic variable of 0 can indicate that the main processing node is not executing a command, while an atomic variable of 1 can indicate that the main processing node is executing a command.

[0061] Therefore, the atomic variables cached by the main processing node are modified to a specified value. This specified value indicates that the main processing node is in the process of executing a command. Thus, the main processing node can use the modified atomic variables to indicate that it is currently occupied, preventing new transactions from being committed to it. This embodiment achieves self-locking of the main processing node by using CAS locks, which are originally used for multi-threaded contention of local shared resources. By modifying the values ​​of atomic variables, fast self-locking can be achieved, making it more efficient than heavyweight locks.

[0062] Step S120: When the current transaction of the main processing node is completed, send a preparation completion signal to the coordination server.

[0063] The current transaction of the primary processing node is the transaction that is being executed by the primary processing node. The readiness-to-complete signal is a signal indicating that the primary processing node has completed all transactions.

[0064] Since self-locking generally only prevents new transactions from being committed to the primary processing node, it does not restrict the primary processing node from continuing to execute the currently executing transaction. Therefore, this embodiment can detect the execution status of the primary processing node's current transaction. If the primary processing node's current transaction is completed, it sends a preparation completion signal to the coordination server. If the primary processing node's current transaction is not completed, it waits for a predetermined time before returning to the step of detecting the execution status of the primary processing node's current transaction until the primary processing node's current transaction is completed, at which point it sends a preparation completion signal to the coordination server.

[0065] In some embodiments, sending a preparation completion signal of the main processing node to the coordination server includes:

[0066] Step S121: Establish the first child node under the preset listening path of the coordination server.

[0067] Step S122: Enter the preparation completion status of the main processing node in the first child node.

[0068] The preset listening path is the path where the processing node registers the listening coordination server.

[0069] Since the coordination server provides coordination services for all processing nodes in the distributed database system, all processing nodes can register to listen to the coordination server's path (i.e., the preset listening path) upon startup. Assume the preset listening path is ` / dbproxy_cluster / dbproxy_cluster_xxx / binlog_pause`. When the current transaction of the primary processing node is completed, it can create a first child node under the coordination server's preset listening path ` / dbproxy_cluster / dbproxy_cluster_xxx / binlog_pause` and record the primary processing node's transaction status (i.e., the ready-to-complete state) on this first child node. Because this preset listening path is listened to by all processing nodes in the distributed database system, when any slave processing node in the distributed database system detects the addition of a new node to this preset listening path, it can perform a self-locking operation on the slave processing node, achieving self-locking for all processing nodes in the distributed database system. Similarly, when the current transaction of a slave processing node is completed, a second child node can be created under the coordination server's preset listening path, and the ready-to-complete state of the slave processing node can be recorded on the second child node. Therefore, this embodiment utilizes the preset listening path of the coordination server to trigger the self-locking operation of all slave processing nodes in the distributed database system, without requiring the master processing node or the coordination server to notify each slave processing node individually, which helps to improve the efficiency of obtaining consistency points.

[0070] Step S130: When all processing nodes in the distributed database system have sent a preparation completion signal to the coordination server, a point acquisition instruction is sent to all data storage nodes in the distributed database system. The point acquisition instruction is used to instruct the data storage nodes to send back point information.

[0071] The point acquisition instruction is used to instruct data storage nodes to provide point information. The point information describes the data status of the data storage node, such as node identifier (e.g., node identifier, shard to which it belongs), core point identifier (e.g., log offset, transaction identifier), auxiliary metadata (e.g., generation time, data version), etc.

[0072] When all processing nodes in a distributed database system send a preparation completion signal to the coordination server, it indicates that all processing nodes in the distributed database system have completed self-locking, and the current transaction has been completed. At this time, the stored data on all data storage nodes in the distributed database system can maintain consistency, meaning that no transaction has modified the stored data. Therefore, this embodiment can send a point acquisition command to all data storage nodes in the distributed database system, so that all data storage nodes in the distributed database system can send the corresponding point information back to the main processing node.

[0073] Step S140: Receive the location information returned by each data storage node and determine the consistency location of the distributed database system.

[0074] In this embodiment, since the point information returned by each data storage node is fed back under the condition that all data storage nodes in the distributed database system maintain consistency, this embodiment can directly determine the point information returned by each data storage node as the consistency point of the distributed database system. To standardize the data format of the consistency point, this embodiment can also convert the point information returned by each data storage node into a predetermined format and then concatenate them to obtain the consistency point of the distributed database system. For example, this embodiment can convert the point information returned by the data storage node into a key-value pair format of "node identifier-point information" to obtain the original point list of the data storage node. The original point lists of each data storage node are concatenated in sequence to obtain the point set, and the point set is determined as the consistency point of the distributed database system.

[0075] In some embodiments, receiving point information returned by each data storage node to determine the consistency point of the distributed database system includes:

[0076] Step S141: After receiving the location information returned by each data storage node, the location information is spliced ​​together to obtain a location set.

[0077] Step S142: Determine the set of points as the consistency points of the distributed database system.

[0078] Among them, the point information is information describing the data status of the data storage node, such as node identifier (e.g., node identifier, shard to which it belongs), core point identifier (e.g., log offset, transaction identifier), auxiliary metadata (e.g., generation time, data version), etc.

[0079] This embodiment can directly concatenate the location information returned by each data storage node to obtain a location set if the location information is returned in a predetermined format. If the location information is not returned in a predetermined format, the location information can be converted to a predetermined format before concatenating the location information in the predetermined format to obtain a location set.

[0080] Furthermore, the set of points is determined as the consistency points of the distributed database system.

[0081] This embodiment determines the set of points obtained by splicing the information of each point as the consistency point of the distributed database system, thus ensuring the data standardization of the consistency point.

[0082] In some embodiments, after receiving the point information returned by each data storage node and determining the consistency point of the distributed database system, the method further includes:

[0083] Step S210: Release the distributed lock in the coordination server.

[0084] This embodiment can release the distributed lock in the coordination server by deleting the child node created by the main processing node under the lock path of the distributed lock in the coordination server. This makes it easier for the main processing unit or other selected main processing units to obtain the distributed lock again when it is necessary to obtain the consistency point again.

[0085] In some embodiments, the method further includes:

[0086] Step S310: Perform the self-lock release operation, which is the reverse operation of the self-lock operation.

[0087] This embodiment can select the corresponding inverse operation as the self-lock release operation based on the selected self-locking operation, and perform the self-lock release operation on the main processing node. Taking CAS lock as an example, this embodiment can cache atomic variables (such as Java's AtomicInteger, C++'s std::atomic) on the main processing node. <int>The value of ) is changed from the specified value to the initial value. The initial value indicates that the main processing node is in a state of not executing commands, so that the main processing node can execute the new transaction that was committed normally. That is, the main processing node changes the lock status of the CAS lock from "locked (1)" to "unlocked (0)", thereby realizing the release of the self-locking of the main processing node.

[0088] In the aforementioned method for obtaining consistency points, when the primary processing node acquires the distributed lock from the coordination server in the distributed database system, a self-locking operation is performed on the primary processing node. This application uses the distributed lock to ensure the uniqueness of the primary processing node's execution of the consistency point acquisition command, preventing other processing nodes from repeatedly executing the command to acquire the consistency point while the primary processing node is acquiring it. When the primary processing node completes its current transaction, it sends a readiness completion signal to the coordination server. Thus, the primary processing node may refrain from processing new transactions through the self-locking operation. When all processing nodes in the distributed database system have sent readiness completion signals to the coordination server, a consistency point acquisition instruction is sent to all data storage nodes in the distributed database system. This instruction instructs the data storage nodes to provide consistency point information. By receiving the consistency point information returned by each data storage node, the consistency point of the distributed database system is determined. Therefore, this embodiment obtains the point information after all processing nodes in the distributed database system have performed self-locking operations and are locked. This ensures that the data stored in all data storage nodes remains consistent (i.e., no data is modified by the processing nodes at this moment). Thus, the point information of each data storage node can be obtained to obtain the consistent point of the distributed database without synchronization processing, which helps to improve the efficiency of obtaining the consistent point.

[0089] In one exemplary embodiment, such as Figure 3 As shown, this embodiment also proposes a consistency point acquisition method, applied to a slave processing node in a distributed database system. The slave processing node is any processing node in the distributed database system other than the master processing node. The method includes:

[0090] Step S410: If a preparation completion signal is detected from the master processing node in the coordination server of the distributed database system, a self-locking operation is performed on the slave processing node.

[0091] Step S420: When the current transaction of the processing node is completed, the master processing node sends a preparation completion signal to the coordination server so that when all processing nodes send preparation completion signals to the coordination server, the master processing node sends a point acquisition instruction to all data storage nodes in the distributed database system and receives point information returned by each data storage node to determine the consistency point of the distributed database system.

[0092] Since the slave processing node cannot trigger the consistency point acquisition process when the master processing node acquires the distributed lock of the coordination server, the slave processing node can monitor the coordination server to determine the progress of the consistency point acquisition process. In this embodiment, if a "ready to complete" signal is detected from the coordination server of the distributed database system, it can be determined that the master processing unit has triggered the consistency point acquisition process, and a self-locking operation can be performed on the slave processing node. For example, taking the slave processing node monitoring the coordination server through a preset monitoring path as an example, when the current transaction of the master processing node is completed, it can create a first child node under the preset monitoring path / dbproxy_cluster / dbproxy_cluster_xxx / binlog_pause of the coordination server, and record the transaction status of the master processing node, i.e., the ready to complete state, in the first child node. Since this preset monitoring path is monitored by all processing nodes in the distributed database system, after all slave processing nodes in the distributed database system detect the event of a new node on the preset monitoring path, they can perform a self-locking operation on the slave processing node, realizing self-locking of all processing nodes in the distributed database system.

[0093] Then, once the current transaction on the slave processing node is completed, it sends a preparation completion signal to the coordination server. The master processing node can also monitor the coordination server to determine the status of each processing node in the distributed database system. When all processing nodes send preparation completion signals to the coordination server, it indicates that all processing nodes in the distributed database system have completed self-locking, and the current transaction has been completed. At this point, the stored data on all data storage nodes in the distributed database system can maintain consistency; that is, no transaction has modified the stored data. The master processing unit can then send point acquisition commands to all data storage nodes in the distributed database system and receive point information returned by each data storage node to determine the consistency point of the distributed database system.

[0094] In some embodiments, performing a self-locking operation from a processing node includes:

[0095] Step S411 modifies the atomic variable cached by the slave processing node to a specified value, which indicates that the slave processing node is in the state of executing a command, so as to prevent new transactions from being committed to the slave processing node.

[0096] Atomic variables are used to store lock states to describe the current node state. For example, an atomic variable of 0 can indicate that the slave node is not executing a command, while an atomic variable of 1 can indicate that the slave node is executing a command.

[0097] Therefore, the atomic variable cached by the slave processing node is modified to a specified value. This specified value indicates that the slave processing node is in the process of executing a command. Thus, the slave processing node can use the modified atomic variable to indicate that it is already occupied, preventing new transactions from being committed to it. This embodiment achieves self-locking of the slave processing node by using CAS locks, which are originally used for multi-threaded contention of local shared resources. By modifying the value of the atomic variable, fast self-locking can be achieved, making it more efficient than heavyweight locks.

[0098] In some embodiments, sending a preparation completion signal from the processing node back to the coordination server includes:

[0099] Step S421: Establish a second child node under the preset listening path of the coordination server.

[0100] Step S422: Enter the preparation completion status of the processing node in the second child node.

[0101] Since the coordination server provides coordination services for all processing nodes in the distributed database system, all processing nodes can register a listening path for the coordination server (i.e., a preset listening path) upon startup. Assume the preset listening path is / dbproxy_cluster / dbproxy_cluster_xxx / binlog_pause. When a processing node completes its current transaction, a second child node can be created under the preset listening path of the coordination server, and the preparation completion status of the second child node can be recorded. Thus, this embodiment utilizes the preset listening path of the coordination server to achieve status feedback for all processing nodes in the distributed database system. The master processing node can determine the status of each processing node in the distributed database system without the master processing node or the coordination server querying each processing node individually, which helps improve the efficiency of obtaining consistency points.

[0102] In some embodiments, the method further includes:

[0103] Step S510: If it is detected that the distributed lock in the coordination server has been released, a self-lock release operation is performed on the processing node. The self-lock release operation is the reverse operation of the self-lock operation.

[0104] Since the master processing node acquires a distributed lock in the coordinating server during the consistency point acquisition process, the release of this distributed lock in the coordinating server indicates that the consistency point acquisition process triggered by the master processing node has been completed. Therefore, this embodiment can select the corresponding reverse operation as the self-lock release operation based on the chosen self-locking operation, and perform the self-lock release operation on the slave processing node. Taking a CAS lock as an example, this embodiment can cache atomic variables (such as Java's AtomicInteger, C++'s std::atomic) on the slave processing node. <int>The value of ) is modified from the specified value to the initial value. The initial value indicates that the slave processing node is in a state of not executing commands, so that the slave processing node can normally execute the new transaction to be committed. That is, the slave processing node changes the lock state of the CAS lock from "locked (1)" to "unlocked (0)", realizing the release of the self-lock for the slave processing node.

[0105] In this embodiment, when a readiness completion signal is detected from the master processing node in the coordination server of the distributed database system, a self-locking operation is performed on the slave processing node. Once the current transaction of the slave processing node is completed, it sends a readiness completion signal back to the coordination server. This ensures that the master processing node, after all processing nodes have sent readiness completion signals to the coordination server, sends a point acquisition command to all data storage nodes in the distributed database system and receives point information returned by each data storage node, thus determining the consistent point of the distributed database system. Therefore, this embodiment acquires point information only after all processing nodes in the distributed database system have performed self-locking operations and are locked. This ensures that the data stored in all data storage nodes remains consistent (i.e., no data has been modified by processing nodes at this moment). The obtained point information from each data storage node can be used to obtain the consistent point of the distributed database without synchronization processing, which helps improve the efficiency of acquiring the consistent point.

[0106] In one exemplary embodiment, such as Figure 4 As shown, this embodiment also proposes a distributed database system, which includes a master processing node 610 and at least one slave processing node 620.

[0107] The main processing node 610 is used to perform a self-locking operation on the main processing node 610 when it acquires the distributed lock of the coordination server in the distributed database system, and to send a preparation completion signal of the main processing node 610 to the coordination server when the current processing transaction of the main processing node 610 is completed.

[0108] The slave processing node 620 is used to perform a self-locking operation when it detects a ready-to-complete signal from the master processing node 610 in the coordination server of the distributed database system, and to send a ready-to-complete signal from the slave processing node 620 to the coordination server when the current processing transaction of the slave processing node 620 is completed.

[0109] The main processing node 610 is also used to send a point acquisition instruction to all data storage nodes in the distributed database system when all processing nodes in the distributed database system have sent a ready-to-complete signal to the coordination server, and to receive point information returned by each data storage node to determine the consistent point of the distributed database system.

[0110] In some embodiments, the main processing node 610 is also used to release the distributed lock in the coordination server and perform a self-lock release operation, which is the reverse operation of the self-lock operation.

[0111] The slave processing node 620 is also used to perform a self-lock release operation on the slave processing node 620 when it is detected that the distributed lock in the coordination server has been released.

[0112] In some embodiments, the master processing node 610 is further configured to modify the atomic variables cached by the master processing node 610 to a specified value, the specified value indicating that the master processing node 610 is in the state of executing a command, so as to prevent new transactions from being committed to the master processing node 610.

[0113] In some embodiments, the master processing node 610 is further configured to, after receiving the location information returned by each data storage node, concatenate the location information to obtain a location set, and determine the location set as the consistency location of the distributed database system.

[0114] In some embodiments, the main processing node 610 is further configured to establish a first child node under a preset listening path of the coordination server, and record the preparation completion status of the main processing node 610 in the first child node.

[0115] In some embodiments, the slave processing node 620 is further configured to modify an atomic variable cached by the slave processing node 620 to a specified value, the specified value representing that the slave processing node 620 is in the state of executing a command, so as to prevent new transactions from being committed to the slave processing node 620.

[0116] In some embodiments, the processing node 620 is further configured to establish a second child node under a preset listening path of the coordination server, and to record the preparation completion status of the processing node 620 in the second child node.

[0117] To more intuitively describe the technical solutions of the embodiments of this application, the following are the steps of a specific embodiment of this application.

[0118] This invention is verified based on a distributed database, which is deployed with two instances, udal1 and udal2. Each instance has two processing nodes and is associated with two databases, rds (data storage nodes), using ZooKeeper as the message center (coordination server). Before implementation, it is necessary to ensure that the network between the nodes of the instances is smooth and that the nodes are running normally. The method mainly includes the following steps:

[0119] A. Check the status of all shards and transactions, and prohibit cross-shard transactions;

[0120] B. The processing node that executes the consistency point acquisition process is designated as the main processing node. The main processing node acquires the distributed lock and begins executing the process of acquiring the consistency point.

[0121] C. The primary processing node acquires a CAS lock on itself to prevent it from executing new transactions.

[0122] D. The main processing node waits for all current transactions to finish before registering a readiness completion signal in ZooKeeper.

[0123] E. After other processing nodes hear the ready-to-complete signal registered by the main processing node in ZooKeeper, they acquire a CAS lock on themselves and wait for the current transaction to complete before registering the ready-to-complete signal in ZooKeeper as well.

[0124] F. After the main processing node hears the ready-to-complete signal that all processing nodes have registered in ZooKeeper, it sends a command to each database (data storage node) to obtain the set of positions, which serve as the consistency points of the distributed database system.

[0125] G. End. The main processing node releases the distributed lock, and each processing node releases its CAS lock.

[0126] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0127] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a consistent point acquisition method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0128] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the embodiments described above.

[0130] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above embodiments.

[0131] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above embodiments.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0133] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0134] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0135] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.< / int> < / int> < / int>

Claims

1. A method for acquiring a consistent point, characterized by, A master processing node applied to a distributed database system, the master processing node being any processing node in the distributed database system; the method comprising: In a case where the master processing node obtains a distributed lock of a coordination server in the distributed database system, performing a self-locking operation on the master processing node; In a case where a current transaction of the master processing node is executed, feeding back a preparation completion signal of the master processing node to the coordination server; In a case where all processing nodes in the distributed database system feed back preparation completion signals to the coordination server, sending a point position acquisition instruction to all data storage nodes in the distributed database system, the point position acquisition instruction being used to instruct the data storage nodes to feed back point position information; Receiving the point position information returned by each data storage node, and determining a consistent point position of the distributed database system.

2. The method of claim 1, wherein, The performing of the self-locking operation on the master processing node comprises: Modifying an atomic variable cached by the master processing node to a specified value, the specified value representing that the master processing node is in a state of executing a command, so as to prevent new transactions from being submitted to the master processing node.

3. The method of claim 1, wherein, The receiving of the point position information returned by each data storage node and the determining of the consistent point position of the distributed database system comprise: After receiving the point position information returned by each data storage node, splicing each piece of point position information to obtain a point position set; Determining the point position set as the consistent point position of the distributed database system.

4. The method of claim 1, wherein, After the receiving of the point position information returned by each data storage node and the determining of the consistent point position of the distributed database system, the method further comprises: Releasing the distributed lock in the coordination server.

5. The method of claim 4, wherein, The method further comprises: Performing a self-locking release operation, the self-locking release operation being a reverse operation of the self-locking operation.

6. The method according to any one of claims 1 to 5, characterized in that, The feeding back of the preparation completion signal of the master processing node to the coordination server comprises: Establishing a first sub-node under a preset listening path of the coordination server; Recording a preparation completion state of the master processing node in the first sub-node.

7. A method for acquiring a consistent point, characterized by, A slave processing node applied to a distributed database system, the slave processing node being any processing node in the distributed database system except the master processing node, the method comprising: In a case where a preparation completion signal fed back by the master processing node exists in a coordination server of the distributed database system, performing a self-locking operation on the slave processing node; In a case where a current transaction of the slave processing node is executed, feeding back a preparation completion signal of the slave processing node to the coordination server, so that the master processing node sends a point position acquisition instruction to all data storage nodes in the distributed database system in a case where all processing nodes feed back preparation completion signals to the coordination server, and receives point position information returned by each data storage node to determine a consistent point position of the distributed database system.

8. The method of claim 7, wherein, The performing of the self-locking operation on the slave processing node comprises: modifying the atomic variable cached by the slave processing node to a specified value, the specified value representing that the slave processing node is in a state of executing a command to prevent a new transaction from being committed to the slave processing node.

9. The method of claim 7, wherein, The feedback of the preparation completion signal of the slave processing node to the coordination server includes: establishing a second sub-node under a preset listening path of the coordination server; recording the preparation completion state of the slave processing node in the second sub-node.

10. The method according to any one of claims 7 to 9, characterized in that, The method further includes: In a case where the distributed lock in the coordination server is released, performing a self-lock release operation on the slave processing node, the self-lock release operation being an inverse operation of the self-lock operation.

11. A distributed database system, characterized by The distributed database system includes a master processing node and at least one slave processing node; The master processing node is configured to, in a case where the master processing node obtains a distributed lock of a coordination server in the distributed database system, perform a self-lock operation on the master processing node, and in a case where a current processing transaction of the master processing node is completed, feed back a preparation completion signal of the master processing node to the coordination server; The slave processing node is configured to, in a case where the coordination server in the distributed database system monitors a preparation completion signal fed back by the master processing node, perform a self-lock operation on the slave processing node, and in a case where a current processing transaction of the slave processing node is completed, feed back a preparation completion signal of the slave processing node to the coordination server; The master processing node is further configured to, in a case where all processing nodes in the distributed database system feed back preparation completion signals to the coordination server, send a point acquisition instruction to all data storage nodes in the distributed database system, receive point information returned by each data storage node, and determine a consistency point of the distributed database system.

12. The system of claim 11, wherein, The master processing node is further configured to release the distributed lock in the coordination server and perform a self-lock release operation, the self-lock release operation being an inverse operation of the self-lock operation. The slave processing node is further configured to, in a case where the coordination server monitors that the distributed lock is released, perform a self-lock release operation on the slave processing node.

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 10.

14. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 10.

15. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 10.

Images