Distributed database rollback log cleaning method and related device

The minimum allowed transaction number of the computing node is obtained through the global transaction node and forwarded to the data node, which solves the problem of rollback log cleaning inconsistency in distributed databases and improves the accuracy and reliability of the consistency query of distributed databases.

CN118981499APending Publication Date: 2024-11-19JINZHUAN INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411007518.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

In a distributed database, the rollback log cleaning between different data nodes is inconsistent, resulting in inaccurate consistency query results, and even the problem of probability errors in distributed consistency query may occur.

Method used

The global transaction node sends a query request to the computing node cluster, obtains the minimum allowed transaction number of each computing node, and determines the target transaction number based on this, forwards it to the data node cluster to perform the rollback log cleaning operation, ensuring that the data nodes are Rollback log consistency.

Benefits of technology

It effectively solves the problem of rollback log cleaning inconsistency in distributed databases, and improves the accuracy of consistency queries and system reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118981499A_ABST
    Figure CN118981499A_ABST
Patent Text Reader

Abstract

The invention relates to a rollback log cleaning method of a distributed database and a related device.The method comprises the steps that a query request sent by a global transaction node is received, and the query request is used for requesting a target computing node to query the minimum transaction number allowed to be used of the target computing node; in response to the query request, querying the minimum allowed transaction number of the target computing node, and returning the minimum allowed transaction number to the global transaction node; and receiving a target transaction number sent by the global transaction node, and forwarding the target transaction number to a data node cluster connected with the target computing node, so that each data node in the data node cluster executes a rollback log cleaning operation based on the target transaction number, the target transaction number is determined and obtained by the global transaction node based on the minimum allowed transaction number returned by each computing node in the computing node cluster. Therefore, the consistency of the rollback logs among different data nodes in the distributed database can be ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of distributed databases, and in particular to a rollback log cleaning method and related devices for a distributed database. Background Art

[0002] With the development of economy and technology, traditional centralized databases are increasingly unable to meet the needs of growing data volumes. In order to adapt to the development of the digital economy, distributed databases with high concurrency, high performance, and high availability have emerged.

[0003] In a distributed database, distributed transaction logic is usually implemented on computing nodes, and data nodes have the ability to perform stand-alone transactions. Therefore, transactions on data nodes in different shards are independent, and rollback log cleaning is also independent. If the rollback log cleaning between data nodes in different shards is inconsistent, or the rollback log cleaning between different primary and standby data nodes in the same shard is inconsistent, it will lead to inaccurate consistency query results for the distributed database, and even the problem of probabilistic errors in distributed consistency queries after an abnormal switch of a data node. Therefore, how to ensure the consistency of rollback log cleaning between different data nodes in a distributed database has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides a rollback log cleaning method and related devices for a distributed database to solve the problem that the existing distributed database consistency query results are inaccurate, and even the problem of probabilistic errors in distributed consistency queries after an abnormal switch of a certain data node occurs.

[0005] In a first aspect, the present application provides a rollback log cleaning method for a distributed database, which is applied to a target computing node, where the target computing node is any computing node in a computing node cluster, and the method includes:

[0006] Receiving a query request sent by a global transaction node, wherein the query request is used to request the target computing node to query its own minimum allowed transaction number;

[0007] In response to the query request, query the target computing node for a minimum allowed transaction number, and return the minimum allowed transaction number to the global transaction node;

[0008] Receive a target transaction number sent by the global transaction node, and forward the target transaction number to a data node cluster connected to the target computing node, so that each data node in the data node cluster performs a rollback log cleanup operation based on the target transaction number, wherein the target transaction number is determined by the global transaction node based on the minimum allowed transaction number returned by each computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node.

[0009] Optionally, forwarding the target transaction number to a data node cluster connected to the target computing node includes:

[0010] In the case where the data node cluster includes multiple data nodes corresponding to multiple shards, the target transaction number is forwarded to the data nodes in each of the multiple shards respectively, so that the data nodes in each of the multiple shards perform a rollback log cleanup operation based on the target transaction number.

[0011] Optionally, forwarding the target transaction number to a data node in each of the multiple shards respectively includes:

[0012] In the case where each of the multiple shards includes a primary data node and multiple backup data nodes, the target transaction number is forwarded to the primary data node in each of the multiple shards respectively, so that the primary data node performs a rollback log cleanup operation according to the target transaction number and forwards the target transaction number to the multiple backup data nodes in the shard.

[0013] In a second aspect, the present application also provides a rollback log cleaning method for a distributed database, which is applied to a global transaction node, and the method includes:

[0014] Sending a query request to each computing node in the computing node cluster, and receiving a minimum allowed transaction number returned by each computing node, wherein the minimum allowed transaction number returned by each computing node is the minimum allowed transaction number queried by each computing node in response to the query request;

[0015] Determine a target transaction number according to the minimum allowed transaction number returned by each computing node, wherein the target transaction number is used to ensure the consistency of the rollback logs of each data node;

[0016] The target transaction number is sent to a target computing node so that the target computing node forwards the target transaction number to a data node cluster connected to the target computing node, wherein the data node cluster is used to perform a rollback log cleanup operation according to the target transaction number, and the target computing node is any computing node in the computing node cluster.

[0017] Optionally, determining the target transaction number according to the minimum allowed transaction number returned by each computing node includes:

[0018] Compare the minimum allowed transaction numbers returned by each computing node to determine the minimum transaction number among the minimum allowed transaction numbers returned by each computing node;

[0019] The minimum transaction number is determined as the target transaction number.

[0020] In a third aspect, the present application further provides a rollback log cleaning device for a distributed database, which is applied to a target computing node, wherein the target computing node is any computing node in a computing node cluster, and the device comprises:

[0021] A first receiving module is used to receive a query request sent by a global transaction node, wherein the query request is used to request the target computing node to query its own minimum allowed transaction number;

[0022] A return module, configured to query the minimum allowed transaction number of the target computing node itself in response to the query request, and return the minimum allowed transaction number to the global transaction node;

[0023] The second receiving module is used to receive the target transaction number sent by the global transaction node, and forward the target transaction number to the data node cluster connected to the target computing node, so that each data node in the data node cluster performs a rollback log cleanup operation based on the target transaction number, wherein the target transaction number is determined by the global transaction node based on the minimum allowed transaction number returned by each computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node.

[0024] In a fourth aspect, the present application further provides a rollback log cleaning device for a distributed database, which is applied to a global transaction node, and the device includes:

[0025] A first sending module is used to send a query request to each computing node in the computing node cluster, and receive a minimum allowed transaction number returned by each computing node, wherein the minimum allowed transaction number returned by each computing node is the minimum allowed transaction number queried by each computing node in response to the query request;

[0026] A determination module, used to determine a target transaction number according to the minimum allowed transaction number returned by each computing node, wherein the target transaction number is used to ensure the consistency of the rollback logs of each data node;

[0027] The second sending module is used to send the target transaction number to the target computing node, so that the target computing node forwards the target transaction number to the data node cluster connected to the target computing node, wherein the data node cluster is used to perform a rollback log cleanup operation according to the target transaction number, and the target computing node is any computing node in the computing node cluster.

[0028] In a fifth aspect, the present application also provides a rollback log cleaning system for a distributed database, the system comprising a global transaction node, a computing node cluster, and a data node cluster;

[0029] The global transaction node is used to send a query request to each computing node in the computing node cluster;

[0030] Each computing node in the computing node cluster is used to query the respective minimum allowed transaction number in response to the query request, and return the respective minimum allowed transaction number obtained by the query to the global transaction node;

[0031] The global transaction node is further used to determine a target transaction number according to the minimum allowed transaction number returned by each computing node, and send the target transaction number to a target computing node, wherein the target computing node is any computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node;

[0032] The target computing node in the computing node cluster is used to forward the target transaction number to the data node cluster;

[0033] The data node cluster is used to perform a rollback log cleanup operation according to the target transaction number.

[0034] In a sixth aspect, the present application further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0035] Memory, used to store computer programs;

[0036] The processor is used to implement the rollback log cleaning method of the distributed database described in any one of the first aspects above, or implement the rollback log cleaning method of the distributed database described in any one of the second aspects above, when executing the program stored in the memory.

[0037] In the seventh aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the rollback log cleaning method for a distributed database described in any one of the first aspects above is implemented, or the rollback log cleaning method for a distributed database described in any one of the second aspects above is implemented.

[0038] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application, by receiving a query request sent by a global transaction node, wherein the query request is used to request the target computing node to query its own minimum allowed transaction number; in response to the query request, query the minimum allowed transaction number of the target computing node itself, and return the minimum allowed transaction number to the global transaction node; receive the target transaction number sent by the global transaction node, and forward the target transaction number to the data node cluster connected to the target computing node, so that each data node in the data node cluster performs a rollback log cleanup operation based on the target transaction number, wherein the target transaction number is determined by the global transaction node based on the minimum allowed transaction number returned by each computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node. Through the above method, the global transaction node can be used to determine the target transaction number based on the minimum allowed transaction number returned by each computing node in the computing node cluster, so that each data node in the data node cluster can perform a rollback log cleanup operation based on the target transaction number, thereby ensuring the consistency of the rollback logs between different data nodes in the distributed database, and effectively solving the problem of inaccurate consistency query results in existing distributed databases, and even probabilistic errors in distributed consistency queries after an abnormal switch of a data node. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0042] Figure 1 A schematic diagram of the structure of a distributed database provided in an embodiment of the present application;

[0043] Figure 2 A flow chart of a rollback log cleaning method for a distributed database provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of interaction between nodes in a distributed database provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of interaction between nodes in another distributed database provided in an embodiment of the present application;

[0046] Figure 5 A flowchart of another rollback log cleaning method for a distributed database provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of interaction between nodes in another distributed database provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of a rollback log cleaning device for a distributed database provided in an embodiment of the present application;

[0049] Figure 8 A schematic diagram of the structure of another distributed database rollback log cleaning device provided in an embodiment of the present application;

[0050] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0052] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0053] Before introducing the rollback log cleaning method of the distributed database provided by this application, the structure of the distributed database is briefly introduced. Figure 1 As shown, the distributed database may include a client access layer, a computing node cluster, a management node, a global transaction management center (i.e., a global transaction node), a post-middleware, and a data node cluster. Among them, the client access layer, i.e., the business end, is composed of multiple applications (Application, referred to as APP), which supports the general open database connection (Open Database Connectivity, referred to as ODBC) interface and Java Database Connection (JavaDatabase Connectivity, referred to as JDBC) interface. Users use the distributed database through the client access layer. The computing node cluster is composed of multiple middleware DBProxy, and the structured query language (Structured Query Language, referred to as SQL) statements complete basic processing and distribution in the computing nodes. The management node is responsible for the cluster management process in the distributed database, does not involve the business access process, and is the unified operation and maintenance entrance of the distributed database. Users can perform user and permission management, metadata management, computing node management, data node management, DDL execution, node expansion, backup and recovery, system installation, statistics and alarm management, etc. on Insight. The management node may include multiple components, such as insight, MDS, PM, CM and other components. The Global Transaction Manager (GTM) maintains the entire life cycle of global transactions in a distributed database, provides the ability to apply, release, and query global transactions, and is deployed in an active-active manner. It can also be used to generate and maintain global transaction numbers and distributed sequences for distributed transactions. A data node cluster can include multiple shards, each of which can include one primary data node and multiple backup data nodes. The data node cluster is mainly used for data reading, writing, storage, and synchronization. Post-middleware can monitor data nodes and manage high availability.

[0054] See also Figure 2 , Figure 2 A flow chart of a rollback log cleaning method for a distributed database provided in an embodiment of the present application. Figure 2 As shown, the rollback log cleaning method of the distributed database may include the following steps:

[0055] Step 201: Receive a query request sent by a global transaction node, wherein the query request is used to request a target computing node to query its own minimum allowed transaction number.

[0056] It should be noted that the rollback log cleaning method for a distributed database provided in an embodiment of the present application can be applied to a target computing node, which can be any computing node in a computing node cluster. In a distributed database, a global transaction node will periodically send a query request to each computing node (including the target computing node) in the computing node cluster, and at this time, each computing node (including the target computing node) can receive the query request sent by the global transaction node.

[0057] Step 202: In response to the query request, query the target computing node's own minimum allowed transaction number, and return the minimum allowed transaction number to the global transaction node.

[0058] After receiving the query request sent by the global transaction node, each computing node (including the target computing node) responds to the query request, queries its own minimum allowed transaction number, and returns the minimum allowed transaction number to the global transaction node.

[0059] It should be noted that the minimum allowed transaction number here means that all transaction numbers smaller than this transaction number can no longer be used for transaction processing, that is, transaction logs earlier than this transaction number cannot be accessed.

[0060] Step 203: Receive the target transaction number sent by the global transaction node, and forward the target transaction number to the data node cluster connected to the target computing node, so that each data node in the data node cluster performs a rollback log cleanup operation based on the target transaction number, wherein the target transaction number is determined by the global transaction node based on the minimum allowed transaction number returned by each computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node.

[0061] In this step, after receiving the minimum allowed transaction number returned by each computing node in the computing node cluster, the global transaction node can determine the target transaction number based on the minimum allowed transaction number returned by each computing node, and then send the target transaction number to the target computing node. In this way, the target computing node can receive the target transaction number sent by the global transaction node, and forward the target transaction number to the data node cluster connected to the target computing node. At this time, each data node in the data node cluster can perform a rollback log cleanup operation based on the target transaction number.

[0062] It should be noted that since each computing node in the computing node cluster is connected to all data nodes in the data node cluster, when the global transaction node sends the target transaction number, it only needs to select any computing node to forward the target transaction number, without selecting multiple computing nodes to forward the target transaction number.

[0063] In this embodiment, the global transaction node can be used to determine the target transaction number based on the minimum allowed transaction number returned by each computing node in the computing node cluster, so that each data node in the data node cluster can perform a rollback log cleanup operation based on the target transaction number, thereby ensuring the consistency of the rollback logs between different data nodes in the distributed database, and effectively solving the problem of inaccurate consistency query results in existing distributed databases, and even probabilistic errors in distributed consistency queries after an abnormal switch of a data node.

[0064] Furthermore, the above step 203, forwarding the target transaction number to the data node cluster connected to the target computing node, includes:

[0065] In the case where the data node cluster includes multiple data nodes corresponding to multiple shards, the target transaction number is forwarded to the data node in each of the multiple shards respectively, so that the data node in each of the multiple shards performs a rollback log cleanup operation based on the target transaction number.

[0066] In one embodiment, the data node cluster may include multiple shards, and the number of data nodes in each shard may be one or more, which is not specifically limited in the embodiment of the present application. In order to achieve consistency in rollback log cleaning between data nodes in different shards, the target computing node may forward the target transaction number to the data node in each shard in the multiple shards, so that the data node in each shard in the multiple shards can perform the rollback log cleaning operation based on the target transaction number.

[0067] For example, Figure 3 As shown in the figure, assuming that the computing node cluster in a distributed database includes computing node 1 and computing node 2, the data node cluster includes three shards, and data node 1, data node 2 and data node 3 are located in these three shards respectively, then the rollback log cleaning process of data node 1, data node 2 and data node 3 is:

[0068] Step 1: As shown in ①, the global transaction node queries computing node 1 for the minimum transaction number allowed on computing node 1;

[0069] Step 2: As shown in ②, the global transaction node queries computing node 2 for the minimum transaction number allowed on computing node 2;

[0070] Step 3: As shown in ③, computing node 1 returns the minimum allowed transaction number on computing node 1 to the global transaction node;

[0071] Step 4: As shown in ④, computing node 2 returns the minimum allowed transaction number on computing node 2 to the global transaction node;

[0072] Step 5: The global transaction node compares the minimum allowed transaction number of computing node 1 with the minimum allowed transaction number of computing node 2, and determines the minimum value of the two as the target transaction number, which is the global transaction ID (GTID) number that can be cleaned up globally and consistently.

[0073] Step 6: As shown in ⑤, the global transaction node sends the target transaction number to computing node 2.

[0074] Step 7: As shown in ⑥, compute node 2 sends the target transaction number to data node 1. After receiving the message, data node 1 performs rollback log consistency cleanup of data node 1 according to the target transaction number.

[0075] Step 8: As shown in ⑦, compute node 2 sends the target transaction number to data node 2. After receiving the message, data node 2 performs rollback log consistency cleanup of data node 2 according to the target transaction number.

[0076] Step 9: As shown in ⑧, computing node 2 sends the target transaction number to data node 3. After receiving the message, data node 3 performs rollback log consistency cleanup of data node 3 according to the target transaction number.

[0077] It should be noted that the above steps 1 and 2 can be executed simultaneously or one after another; the above steps 3 and 4 can be executed simultaneously or one after another; the above steps 6, 7 and 8 can be executed simultaneously or one after another, and the embodiments of the present application do not make specific limitations.

[0078] Through the above method, data nodes in different shards can clean up their respective rollback logs according to the same target transaction number, thereby ensuring the consistency of rollback log cleaning of data nodes between different shards.

[0079] Furthermore, the above step of forwarding the target transaction number to the data node in each of the multiple shards includes:

[0080] In the case where each of the multiple shards includes a primary data node and multiple standby data nodes, the target transaction number is forwarded to the primary data node in each of the multiple shards respectively, so that the primary data node performs a rollback log cleanup operation according to the target transaction number and forwards the target transaction number to the multiple standby data nodes in the shard.

[0081] In one embodiment, when each of the multiple shards in the data node cluster includes a primary data node and multiple backup data nodes,

[0082] In order to achieve consistency in rollback log cleaning between different primary and standby data nodes in the same shard, the target computing node can forward the target transaction number to the primary data node in each shard respectively. In this way, the primary data node in each shard can perform rollback log cleaning operations according to the target transaction number, and forward the target transaction number to multiple standby data nodes in the shard, so that multiple standby data nodes in the shard can also perform rollback log cleaning operations according to the target transaction number.

[0083] For example, Figure 4 As shown in the figure, assuming that a computing node cluster in a distributed database includes computing node 1, and a shard in the data node cluster includes data node 1 host, data node 1 standby 1, data node 1 standby 2, and data node 1 standby 3, then the rollback log cleaning process of data node 1 host, data node 1 standby 1, data node 1 standby 2, and data node 1 standby 3 is as follows:

[0084] Step 1: As shown in ①, the global transaction node sends the target transaction number to computing node 1, where the target transaction number is the minimum allowed transaction number returned by computing node 1.

[0085] Step 2: As shown in ②, compute node 1 sends the target transaction number to the data node 1 host.

[0086] Step 3: Data node 1 host writes the target transaction number to the logical log of data node 1 host, and data node 1 host performs consistency cleanup of the rollback log according to the target transaction number.

[0087] Step 4: As shown in ③, the data node 1 host sends the logical log to the data node 1 standby machine 1. The data node 1 standby machine 1 parses the target transaction number from the received logical log and performs consistency cleanup of the rollback log of the data node 1 standby machine 1 according to the target transaction number.

[0088] Step 5: As shown in ④, the data node 1 host sends the logical log to the data node 1 standby 2, and the data node 1 standby 2 parses the target transaction number from the received logical log, and performs consistency cleanup of the rollback log of the data node 1 standby 2 according to the target transaction number.

[0089] Step 6: As shown in step ⑤, the data node 1 host sends the logical log to the data node 1 standby machine 3. The data node 1 standby machine 3 parses the target transaction number from the received logical log and performs consistency cleanup of the rollback log of the data node 1 standby machine 3 according to the target transaction number.

[0090] It should be noted that the above steps 4, 5 and 6 can be executed simultaneously or sequentially, and the embodiments of the present application do not make specific limitations.

[0091] Through the above method, different master and standby data nodes in the same shard can clean up their respective rollback logs according to the same target transaction number, thereby ensuring the consistency of rollback log cleaning of different master and standby data nodes in the same shard.

[0092] See also Figure 5 , Figure 5 A flow chart of another method for cleaning rollback logs of a distributed database provided in an embodiment of the present application. Figure 5 As shown, the rollback log cleaning method of the distributed database may include the following steps:

[0093] Step 501: Send a query request to each computing node in the computing node cluster, and receive the minimum allowed transaction number returned by each computing node, wherein the minimum allowed transaction number returned by each computing node is the minimum allowed transaction number queried by each computing node in response to the query request.

[0094] It should be noted that the rollback log cleaning method for a distributed database provided in an embodiment of the present application can be applied to a global transaction node. The global transaction node can periodically send a query request to each computing node (including a target computing node) in a computing node cluster. At this time, each computing node (including a target computing node) can receive the query request sent by the global transaction node, and in response to the query request, query its own minimum allowed transaction number, and then return the minimum allowed transaction number to the global transaction node.

[0095] Step 502: Determine a target transaction number based on the minimum allowed transaction number returned by each computing node, wherein the target transaction number is used to ensure the consistency of the rollback logs of each data node.

[0096] After receiving the minimum allowed transaction number returned by each computing node in the computing node cluster, the global transaction node can determine the target transaction number based on the minimum allowed transaction number returned by each computing node. Specifically, the minimum transaction number among the minimum allowed transaction numbers returned by each computing node can be determined as the target transaction number.

[0097] Step 503: Send the target transaction number to the target computing node, so that the target computing node forwards the target transaction number to the data node cluster connected to the target computing node, wherein the data node cluster is used to perform the rollback log cleanup operation according to the target transaction number, and the target computing node is any computing node in the computing node cluster.

[0098] The global transaction node can send the target transaction number to the target computing node. In this way, the target computing node can receive the target transaction number sent by the global transaction node and forward the target transaction number to the data node cluster connected to the target computing node. At this time, each data node in the data node cluster can perform a rollback log cleanup operation based on the target transaction number.

[0099] It should be noted that since each computing node in the computing node cluster is connected to all data nodes in the data node cluster, when the global transaction node sends the target transaction number, it only needs to select any computing node to forward the target transaction number, without selecting multiple computing nodes to forward the target transaction number.

[0100] In this embodiment, the global transaction node can be used to determine the target transaction number based on the minimum allowed transaction number returned by each computing node in the computing node cluster, so that each data node in the data node cluster can perform a rollback log cleanup operation based on the target transaction number, thereby ensuring the consistency of the rollback logs between different data nodes in the distributed database, and effectively solving the problem of inaccurate consistency query results in existing distributed databases, and even probabilistic errors in distributed consistency queries after an abnormal switch of a data node.

[0101] Furthermore, the above step 502, determining the target transaction number according to the minimum allowed transaction number returned by each computing node, includes:

[0102] Compare the minimum allowed transaction numbers returned by each computing node to determine the minimum transaction number among the minimum allowed transaction numbers returned by each computing node;

[0103] The minimum transaction number is determined as the target transaction number.

[0104] In one embodiment, after receiving the minimum allowed transaction number returned by each computing node in the computing node cluster, the global transaction node can compare the minimum allowed transaction numbers returned by each computing node, determine the minimum transaction number among the minimum allowed transaction numbers returned by each computing node, and then determine the minimum transaction number as the target transaction number. In this way, the consistency of the rollback log cleaning of each data node can be guaranteed.

[0105] In one embodiment, it is assumed that a computing node cluster in a distributed database includes computing node 1 and computing node 2, and a data node cluster includes three shards, each shard includes one primary data node and three backup data nodes, such as Figure 6 As shown, the rollback log cleaning process of each data node is as follows:

[0106] 1) The global transaction node sends query requests to computing node 1 and computing node 2 respectively.

[0107] 2) The global transaction node obtains the minimum allowed transaction number of computing node 1 and the minimum allowed transaction number of computing node 2, compares the two, and selects the minimum value as the target transaction number.

[0108] 3) The global transaction node sends the target transaction number to computing node 2.

[0109] 4) Compute node 2 sends the target transaction number to data node 1, data node 2, and data node 3 respectively;

[0110] 5) Data nodes 1, 2, and 3 receive the target transaction number, and clean up the rollback log of the current data node host according to the target transaction number, and write the target transaction number into the logical log of the current data node host;

[0111] 6) Data node 1, data node 2, and data node 3 send the logical log to all standby machines of the current data node host respectively;

[0112] 7) All data node standby machines parse the target transaction number from the received logical log of the data node host, and clean up the rollback log of the current data node standby machine according to the target transaction number.

[0113] Compared with the prior art, the rollback log cleaning method provided in the embodiment of the present application can ensure the consistency of rollback log cleaning between data nodes in different shards of a distributed database and between different primary and standby data nodes in the same shard in any scenario, thereby improving the consistency query capability of the distributed database in various scenarios and greatly improving the consistency and reliability of the distributed database system.

[0114] See also Figure 7 , Figure 7 A structural diagram of a rollback log cleaning device for a distributed database provided in an embodiment of the present application. Figure 7 As shown, the rollback log cleaning device of the distributed database is applied to a target computing node, and the target computing node is any computing node in a computing node cluster. The device 700 may include:

[0115] The first receiving module 701 is used to receive a query request sent by a global transaction node, wherein the query request is used to request a target computing node to query its own minimum allowed transaction number;

[0116] A return module 702 is used to query the minimum allowed transaction number of the target computing node itself in response to the query request, and return the minimum allowed transaction number to the global transaction node;

[0117] The second receiving module 703 is used to receive the target transaction number sent by the global transaction node, and forward the target transaction number to the data node cluster connected to the target computing node, so that each data node in the data node cluster performs a rollback log cleanup operation based on the target transaction number, wherein the target transaction number is determined by the global transaction node based on the minimum allowed transaction number returned by each computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node.

[0118] Further, the second receiving module 703 includes:

[0119] A forwarding submodule is used to forward the target transaction number to the data nodes in each of the multiple shards respectively when the data node cluster includes multiple data nodes corresponding to the multiple shards, so that the data nodes in each of the multiple shards perform a rollback log cleanup operation based on the target transaction number.

[0120] Furthermore, the forwarding submodule includes:

[0121] A forwarding unit is used to forward the target transaction number to the primary data node in each of the multiple shards, respectively, when each of the multiple shards includes a primary data node and multiple backup data nodes, so that the primary data node performs a rollback log cleanup operation according to the target transaction number and forwards the target transaction number to the multiple backup data nodes in the shard.

[0122] It should be noted that the device 700 can realize the above Figure 2 The steps of the rollback log cleaning method of the distributed database shown in the figure can achieve the same technical effect, and will not be repeated here.

[0123] See also Figure 8 , Figure 8 A schematic diagram of the structure of another distributed database rollback log cleaning device provided in an embodiment of the present application. Figure 8 As shown, the rollback log cleaning device of the distributed database is applied to the global transaction node, and the device 800 may include:

[0124] The first sending module 801 is used to send a query request to each computing node in the computing node cluster, and receive a minimum allowed transaction number returned by each computing node, wherein the minimum allowed transaction number returned by each computing node is the minimum allowed transaction number queried by each computing node in response to the query request;

[0125] A determination module 802 is used to determine a target transaction number according to the minimum allowed transaction number returned by each computing node, wherein the target transaction number is used to ensure the consistency of the rollback logs of each data node;

[0126] The second sending module 803 is used to send the target transaction number to the target computing node, so that the target computing node forwards the target transaction number to the data node cluster connected to the target computing node, wherein the data node cluster is used to perform the rollback log cleanup operation according to the target transaction number, and the target computing node is any computing node in the computing node cluster.

[0127] Further, the determination module 802 includes:

[0128] A comparison submodule, used to compare the minimum allowed transaction numbers returned by each computing node, and determine the minimum transaction number among the minimum allowed transaction numbers returned by each computing node;

[0129] The determination submodule is used to determine the minimum transaction number as the target transaction number.

[0130] It should be noted that the device 800 can realize the above Figure 5 The steps of the rollback log cleaning method of the distributed database shown in the figure can achieve the same technical effect, and will not be repeated here.

[0131] See also Figure 1 , the present application also provides a rollback log cleaning system for a distributed database, the system comprising a global transaction node, a computing node cluster and a data node cluster;

[0132] The global transaction node is used to send query requests to each computing node in the computing node cluster;

[0133] Each computing node in the computing node cluster is used to respond to the query request, query the respective minimum allowed transaction number, and return the minimum allowed transaction number obtained by the query to the global transaction node;

[0134] The global transaction node is also used to determine the target transaction number according to the minimum allowed transaction number returned by each computing node, and send the target transaction number to the target computing node, wherein the target computing node is any computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node;

[0135] The target computing node in the computing node cluster is used to forward the target transaction number to the data node cluster;

[0136] The data node cluster is used to perform rollback log cleanup operations based on the target transaction number.

[0137] It should be noted that the above steps have been described in detail in the above embodiments and will not be repeated here.

[0138] like Fig. 9 As shown, the embodiment of the present application further provides an electronic device, including a processor 911, a communication interface 912, a memory 913 and a communication bus 914, wherein the processor 911, the communication interface 912, and the memory 913 communicate with each other through the communication bus 914;

[0139] Memory 913, used for storing computer programs;

[0140] In one embodiment of the present application, the processor 911 is used to implement the rollback log cleaning method of the distributed database provided by any one of the aforementioned method embodiments when executing the program stored in the memory 913.

[0141] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, a rollback log cleaning method for a distributed database as provided in any of the aforementioned method embodiments is implemented.

[0142] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0144] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0145] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A rollback log cleaning method for a distributed database, characterized in that: Applied to a target computing node, the target computing node being any computing node in a computing node cluster, the method comprises: Receiving a query request sent by a global transaction node, wherein the query request is used to request the target computing node to query its own minimum allowed transaction number; In response to the query request, query the target computing node for a minimum allowed transaction number, and return the minimum allowed transaction number to the global transaction node; Receive a target transaction number sent by the global transaction node, and forward the target transaction number to a data node cluster connected to the target computing node, so that each data node in the data node cluster performs a rollback log cleanup operation based on the target transaction number, wherein the target transaction number is determined by the global transaction node based on the minimum allowed transaction number returned by each computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node.

2. The method according to claim 1, characterized in that: The step of forwarding the target transaction number to a data node cluster connected to the target computing node includes: In the case where the data node cluster includes multiple data nodes corresponding to multiple shards, the target transaction number is forwarded to the data nodes in each of the multiple shards respectively, so that the data nodes in each of the multiple shards perform a rollback log cleanup operation based on the target transaction number.

3. The method according to claim 2, characterized in that The step of forwarding the target transaction number to a data node in each of the plurality of shards comprises: In the case where each of the multiple shards includes a primary data node and multiple backup data nodes, the target transaction number is forwarded to the primary data node in each of the multiple shards respectively, so that the primary data node performs a rollback log cleanup operation according to the target transaction number and forwards the target transaction number to the multiple backup data nodes in the shard.

4. A rollback log cleaning method for a distributed database, characterized in that: Applied to a global transaction node, the method comprises: Sending a query request to each computing node in the computing node cluster, and receiving a minimum allowed transaction number returned by each computing node, wherein the minimum allowed transaction number returned by each computing node is the minimum allowed transaction number queried by each computing node in response to the query request; Determine a target transaction number according to the minimum allowed transaction number returned by each computing node, wherein the target transaction number is used to ensure the consistency of the rollback logs of each data node; The target transaction number is sent to a target computing node so that the target computing node forwards the target transaction number to a data node cluster connected to the target computing node, wherein the data node cluster is used to perform a rollback log cleanup operation according to the target transaction number, and the target computing node is any computing node in the computing node cluster.

5. The method according to claim 4, characterized in that The step of determining the target transaction number according to the minimum allowed transaction number returned by each computing node includes: Compare the minimum allowed transaction numbers returned by each computing node to determine the minimum transaction number among the minimum allowed transaction numbers returned by each computing node; The minimum transaction number is determined as the target transaction number.

6. A rollback log cleaning device for a distributed database, characterized in that: Applied to a target computing node, the target computing node being any computing node in a computing node cluster, the device comprises: A first receiving module is used to receive a query request sent by a global transaction node, wherein the query request is used to request the target computing node to query its own minimum allowed transaction number; A return module, configured to query the minimum allowed transaction number of the target computing node itself in response to the query request, and return the minimum allowed transaction number to the global transaction node; The second receiving module is used to receive the target transaction number sent by the global transaction node, and forward the target transaction number to the data node cluster connected to the target computing node, so that each data node in the data node cluster performs a rollback log cleanup operation based on the target transaction number, wherein the target transaction number is determined by the global transaction node based on the minimum allowed transaction number returned by each computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node.

7. A rollback log cleaning device for a distributed database, characterized in that: Applied to a global transaction node, the device comprises: A first sending module is used to send a query request to each computing node in the computing node cluster, and receive a minimum allowed transaction number returned by each computing node, wherein the minimum allowed transaction number returned by each computing node is the minimum allowed transaction number queried by each computing node in response to the query request; A determination module, used to determine a target transaction number according to the minimum allowed transaction number returned by each computing node, wherein the target transaction number is used to ensure the consistency of the rollback logs of each data node; The second sending module is used to send the target transaction number to the target computing node, so that the target computing node forwards the target transaction number to the data node cluster connected to the target computing node, wherein the data node cluster is used to perform a rollback log cleanup operation according to the target transaction number, and the target computing node is any computing node in the computing node cluster.

8. A rollback log cleaning system for a distributed database, characterized in that: The system includes a global transaction node, a computing node cluster and a data node cluster; The global transaction node is used to send a query request to each computing node in the computing node cluster; Each computing node in the computing node cluster is used to query the respective minimum allowed transaction number in response to the query request, and return the respective minimum allowed transaction number obtained by the query to the global transaction node; The global transaction node is further used to determine a target transaction number according to the minimum allowed transaction number returned by each computing node, and send the target transaction number to a target computing node, wherein the target computing node is any computing node in the computing node cluster, and the target transaction number is used to ensure the consistency of the rollback logs of each data node; The target computing node in the computing node cluster is used to forward the target transaction number to the data node cluster; The data node cluster is used to perform a rollback log cleanup operation according to the target transaction number.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the rollback log cleaning method of the distributed database described in any one of claims 1-3, or implement the rollback log cleaning method of the distributed database described in any one of claims 4-5 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the rollback log cleaning method for a distributed database as described in any one of claims 1-3, or implements the rollback log cleaning method for a distributed database as described in any one of claims 4-5.