A method, apparatus, and medium for maintaining distributed metadata consistency

Abstract

This invention relates to the field of distributed database technology, and more particularly to a method, apparatus, and medium for maintaining the consistency of distributed metadata. In this application, during the execution of a data definition language in a distributed database, the central node and participating nodes operate on the target object sequentially. Based on the operation results of the target object on all nodes, it is determined whether the central node meets the commit conditions. If the central node meets the commit conditions, then the participating nodes are also considered to meet the commit conditions, simplifying the operation of the distributed data definition language into a single-node problem. Simultaneously, by using the metadata version number of the target object, the operation of the data object's stored data is restricted to nodes that need modification in the event of metadata inconsistency, acting as a local lock and improving the metadata operation process of the distributed database.

Description

Technical Field

[0001] This invention relates to the field of distributed database technology, and in particular to a method, apparatus and medium for maintaining the consistency of distributed metadata. Background Technology

[0002] In distributed database systems, metadata consistency is crucial because metadata, which describes the data itself, is a vital component of the entire database system. It includes data structure, storage location, index information, and data sharding. Inconsistent metadata across data nodes can lead to data inconsistencies, affecting the correctness of operations such as insertion and querying. For example, a table might be accessible on one node but deleted on others, resulting in users only accessing partial results. Furthermore, database systems often optimize queries based on metadata information, such as finding suitable indexes. Inconsistent metadata across nodes can cause performance degradation. Therefore, ensuring metadata consistency across all nodes is essential for the normal operation, data accuracy, and consistency of the entire distributed database system. Current technologies manage metadata through distributed transactions to maintain consistency across data nodes. However, distributed transaction operations are complex. Therefore, improving the operational flow of distributed databases to reduce resource consumption and improve operational efficiency while ensuring distributed metadata consistency is a pressing issue that needs to be addressed. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method, apparatus, and medium for maintaining the consistency of distributed metadata, in order to solve the problem that distributed transaction operations are complex in the process of keeping metadata consistent across data nodes.

[0004] In a first aspect, embodiments of this application provide a method for maintaining distributed metadata consistency, applied to a distributed database. The distributed database includes a central node, a coordinating node, and at least one participating node. The central node communicates with each participating node. The method for maintaining distributed metadata consistency includes:

[0005] The central node receives the target statement, determines the identifier of the target object in the target statement, and locks the target object. Since all metadata modifications will first attempt to lock the target object at the central node, this process is repeated.

[0006] If the central node successfully operates on the target object, the central node obtains the first version number of the target object;

[0007] The central node sends the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node;

[0008] For any participating node, obtain the second version number of the target object corresponding to the identifier in the participating node and the coordinating node. If the first version number is 1 greater than the second version number, then perform an operation on the target object on the participating node and the coordinating node.

[0009] If each participating node and the coordinating node successfully operate on the target object, the central node submits the target transaction that operated on the target object.

[0010] If the central node successfully submits the target transaction, then the participating nodes and the coordinating node submit the target transaction.

[0011] If the participating node and the coordinating node successfully submit the target transaction, the central node releases the lock on the target object.

[0012] Secondly, embodiments of this application provide an apparatus for maintaining the consistency of distributed metadata, applied to a distributed database. The distributed database includes a central node, a coordinating node, and at least one participating node. The central node communicates with each participating node. The apparatus for maintaining the consistency of distributed metadata includes:

[0013] The first operation module is used for the central node to receive the target statement, determine the identifier of the target object in the target statement, lock the target object, and operate on the target object according to the identifier of the target object and the target statement;

[0014] The module is configured to obtain the first version number of the target object if the central node successfully operates on the target object;

[0015] The sending module is used by the central node to send the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node;

[0016] The second operation module is used to obtain the second version number of the target object corresponding to the participating node and the coordinating node for any participating node. If the first version number is 1 greater than the second version number, then the target object is operated on the participating node and the coordinating node.

[0017] The first submission module is used so that if all participating nodes and the coordinating node successfully operate on the target object, the central node submits the target transaction that operates on the target object.

[0018] The second submission module is used so that if the central node successfully submits the target transaction, the participating nodes and the coordinating node submit the target transaction.

[0019] The lock release module is used so that if the participating node and the coordinating node successfully submit the target transaction, the central node releases the lock on the target object.

[0020] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for maintaining distributed metadata consistency as described in the first aspect.

[0021] The advantages of this invention compared to the prior art are:

[0022] The central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and performs operations on the target object according to the identifier and the target statement. If the central node successfully operates on the target object, it obtains the first version number of the target object. The central node sends the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node. For each participating node, it obtains the second version number of the target object corresponding to the identifier in the participating node and the coordinating node. If the first version number is 1 greater than the second version number, it performs operations on the target object on the participating node and the coordinating node. If all participating nodes and the coordinating node successfully operate on the target object, the central node commits the target transaction that operated on the target object. If the central node successfully commits the target transaction, the participating nodes and the coordinating node commit the target transaction. If the participating nodes and the coordinating node successfully commit the target transaction, the central node releases the lock on the target object. In this application, during the execution of the Data Definition Language (DDL) in a distributed database, the central node and participating nodes operate on the target object sequentially. Based on the operation results of the target object across all nodes, it is determined whether the central node meets the commit conditions. If the central node meets the commit conditions, the coordinating node and participating nodes are also considered to meet the commit conditions. This simplifies the operation of the distributed DDL into a single-node problem, improving the metadata operation process of the distributed database. Furthermore, in this application, when executing DDL and data operation statements, it is only necessary to check the version number of the metadata in the participating nodes to be executed. If the version numbers of the metadata in the participating nodes to be executed are equal, execution can proceed normally without waiting for the metadata version numbers of the entire cluster to reach consistency. This achieves the effect of local locking through the metadata version number, maximizing the support for concurrent execution of DDL and data operation statements. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an application environment for a method for maintaining distributed metadata consistency provided in Embodiment 1 of this application;

[0025] Figure 2 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment 2 of this application;

[0026] Figure 3 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment 3 of this application;

[0027] Figure 4 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment 4 of this application;

[0028] Figure 5 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment 5 of this application;

[0029] Figure 6 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment Six of this application;

[0030] Figure 7 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment 7 of this application;

[0031] Figure 8 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment 8 of this application;

[0032] Figure 9 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment 9 of this application;

[0033] Figure 10 This is a schematic diagram of the structure of a device for maintaining the consistency of distributed metadata provided in Embodiment 10 of this application;

[0034] Figure 11 This is a schematic diagram of the structure of a computer device provided in Embodiment Eleven of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0037] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0038] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0039] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0041] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0042] To illustrate the technical solution of this application, specific embodiments are described below.

[0043] The method for maintaining distributed metadata consistency provided in Embodiment 1 of this application can be applied to, for example, Figure 1 In a distributed database, the distributed database includes a central node, a coordinating node and at least one participating node, a coordinating node and a server, and communication connections between the central node, each participating node, the coordinating node and the server, wherein the central node, participating nodes and the coordinating node are different clients.

[0044] See Figure 2 This is a flowchart illustrating a method for maintaining distributed metadata consistency provided in Embodiment 2 of this application. Figure 2 As shown, the method for maintaining distributed metadata consistency may include the following steps:

[0045] S201: The central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and performs operations on the target object according to the identifier of the target object and the target statement.

[0046] In step S201, the central node receives the target statement, where the central node is a client in the distributed database, the target statement is a data definition language statement, the identifier of the target object in the target statement is determined, where the target object is the object being operated on, the identifier of the target object is the unique identifier of the object being operated on, and the target object is operated on according to the identifier of the target object and the target statement.

[0047] In this embodiment, the method for maintaining distributed metadata consistency is applied to a distributed database, which includes a server and multiple clients. Among the clients, a node is selected as the central node. The metadata of an object in the database is based on the metadata status of that object at the central node. The central node receives target statements, which are data definition language statements. Data definition language statements can be operation statements on existing objects or operation statements on newly created objects. For example, inserting a column of data into an existing data table A, or creating a new data table B, etc.

[0048] The central node identifies the target object in the target statement so that when other clients (i.e., participating nodes) perform operations on the target object, the corresponding identifier can be used to determine the target object being operated on. The target object is locked to prevent it from being operated on by other clients and participating nodes, which could cause inconsistencies in the object's metadata status in the database. At the central node, operations are performed on the target object based on its identifier and the target statement.

[0049] It should be noted that locking the target object intercepts metadata changes of the target object globally. Since all metadata changes first go to the central node, the object lock on the central node can achieve the effect of a global metadata lock.

[0050] See Figure 3 This is a flowchart illustrating the method for maintaining distributed metadata consistency provided in Embodiment 3 of this application. Figure 3 As shown, in step S201, the central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and before operating on the target object according to the identifier of the target object and the target statement, the following steps may also be included:

[0051] S301: The coordinating node receives the target statement, verifies the target statement, obtains the verification result, and if the verification result is correct, sends the target statement to the central node.

[0052] In this embodiment, the coordinating node is one of the multiple clients. The coordinating node is used to control the operations of the central node and participating nodes based on the operation results or submission results of the central node and participating nodes, so as to ensure that the metadata status of the target object in the central node and participating nodes is consistent.

[0053] Before the central node receives the target statement, the coordinating node receives the target statement, verifies it, and determines whether the target statement is correct. For example, it verifies whether the target statement conforms to the correct syntax. If the verification result is correct, the target statement is sent to the central node so that the central node can operate on the target object in the target statement according to the correct target statement.

[0054] It should be noted that the coordinating node receives the target statement, validates it, and obtains the validation result. If the validation result is incorrect, for example, if the syntax of the target statement is incorrect, the coordinating node sends the incorrect target statement information to the server. The server then terminates the operation to prevent the target node and participating nodes from starting the operation and consuming resources.

[0055] S202: If the central node successfully operates on the target object, the central node obtains the first version number of the target object.

[0056] In step S202, if the central node successfully operates on the target object, the central node obtains the first version number of the target object, where the first version number is the number of times the target object has been operated on.

[0057] In this embodiment, if the central node successfully operates on the target object, the central node obtains the first version number of the target object. The version corresponding to the first version number is the version after the central node executes the target statement. The first version number is an accumulated result, that is, the version number after the central node's last operation on the target object is incremented by 1. If the central node fails to operate on the target object, the first version number is not incremented.

[0058] It should be noted that after performing operations on the target object based on its identifier and target statement, no commit is made. This is to ensure that the node has the conditions for changing the target object's metadata, preventing the node from committing when the target object does not meet the conditions for modification, thereby consuming the corresponding resources.

[0059] It should be noted that if the data definition language corresponding to the target statement is to create a new object, and the newly created object does not have a corresponding identifier in the central node, an identifier needs to be assigned to the newly created object so that when the participating nodes execute the corresponding target statement, they can refer to the corresponding identifier and assign the same identifier to the newly created object. Specifically, if it is a newly created object, the first version number is 0.

[0060] It should be noted that if the operation performed by the central node on the target object is a deletion operation, that is, deleting an existing target object, then the first version number of the target object is a temporary version number, which is a temporary first version number and is unrelated to the version number after the central node's last operation on the target object. After the operation is successful, all records of the target object will be deleted, and the version number will not be persisted.

[0061] See Figure 4 This is a flowchart illustrating the method for maintaining distributed metadata consistency provided in Embodiment 4 of this application. Figure 4 As shown, after operating on the target object according to the target object's identifier and target statement in step S201, the following steps may also be included:

[0062] S401: If the central node fails to operate on the target object, it releases the lock on the target object and sends the first operation failure information to the coordinating node.

[0063] S402: The coordinating node sends the first operation failure information to the server, and the server terminates the operation on the target object.

[0064] In this embodiment, the distributed database also includes a server. The server is used to end the current operation based on the operation results or submission results of the central node and participating nodes. For example, if the operation or submission of the central node and participating nodes fails, the server ends the current operation to prevent the central node or participating nodes from continuing to execute the corresponding operation.

[0065] If the central node fails to operate on the target object, for example, if the target object in the target statement does not exist, or if the central node has insufficient space to operate on the target object, the central node releases the lock on the target object and sends the first operation failure message to the coordinating node. The coordinating node then sends the first operation failure message to the server, and the server terminates the operation on the target object to prevent other participating nodes from operating on the target statement.

[0066] S203: The central node sends the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node.

[0067] In step S203, after the central node successfully operates on the target object, the central node sends the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node, so that the coordinating node will perform the same operation as the participating nodes in the subsequent entire metadata change process. The participating nodes and the coordinating node are other clients in the database that have the target object.

[0068] In this embodiment, the central node sends the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node, so that the participating nodes and the coordinating node can determine the target object in the participating node based on the identifier of the target object. For example, if the identifier of the target object is 001, the participating nodes and the coordinating node will operate on the target object with the identifier 001 respectively, so as to prevent the target object's name from being the same as the name of other objects, which would cause erroneous operation on other objects.

[0069] It should be noted that the central node sends the first version number to the participating nodes and the coordinating node so that the participating nodes and the coordinating node can determine whether the metadata of the target object after executing the corresponding target statement is consistent with the metadata of the target object in the central node.

[0070] It should be noted that the coordinating node performs the same operation as the participating nodes, enabling the coordinating node to also save the version number of the target object. This is so that when executing data manipulation statements later, the coordinating node can lock the target object, allowing the corresponding participating nodes to execute the data manipulation statements only after the coordinating node acquires the corresponding lock. This prevents some participating nodes from changing the version number of the object after executing the data manipulation statements.

[0071] S204: For any participating node, obtain the second version number of the target object corresponding to the identifier in the participating node and the coordinating node. If the first version number is 1 greater than the second version number, then perform the operation on the target object on the participating node and the coordinating node.

[0072] In step S204, for any participating node, the second version number of the target object corresponding to the identifier in the participating node and the coordinating node is obtained, and it is determined whether the first version number is 1 greater than the second version number. If the first version number is 1 greater than the second version number, the target object is operated on the participating node and the coordinating node.

[0073] In this embodiment, in any participating node, the participating node and the coordinating node determine the corresponding target object based on the received target object identifier. Based on the corresponding target object, they determine the second version number of the target object in that participating node and the coordinating node. Based on the first version number and the second version number, they determine whether the participating node and the coordinating node have performed an operation on the target object. If the first version number is 1 greater than the second version number, it is considered that the version number of the target object in the participating node and the coordinating node is the version number after the previous successful operation, and the current operation can be performed. Therefore, the target object can be operated on at the participating node and the coordinating node. After the participating node and the coordinating node operate on the target object, the corresponding version number is incremented by 1 until it reaches the same version number as the target object in the central node. If any participating node or the coordinating node fails to operate on the target object, the corresponding version number is not incremented.

[0074] It should be noted that during operations on the target object on participating and coordinating nodes, object locks are applied to the target object to prevent data manipulation statements from being executed on the corresponding participating or coordinating nodes. The operation process involves applying exclusive locks to the target object on participating and coordinating nodes, modifying the target object's metadata but not persisting it. The locks applied to the target object here differ from those applied to the central node; they are locks that only apply to a single node, aiming to restrict data operations such as adding, deleting, and modifying data on the target object.

[0075] In this embodiment, the second version number of the target object corresponding to the identifier in the participating node and the coordinating node is obtained, so as to compare the second version number with the first version number and determine whether the participating node and the coordinating node have performed operations on the target object, so as to ensure that after the participating node and the coordinating node perform operations on the target object, the metadata of the target object is equal to the metadata of the target object in the central node.

[0076] It should be noted that if the target statement is to create a new target object, when the participating nodes and the coordinating nodes create a new target object, they will use the identifier of the target object assigned by the central node as the identifier of the new target object, and will not generate a new identifier.

[0077] It should be noted that participating nodes and coordinating nodes execute the corresponding target statements without committing them. This is to ensure that the node has the conditions to change the target object's metadata, and to prevent the node from committing when the target object does not meet the conditions for modification, thereby consuming the corresponding resources.

[0078] See Figure 5 This is a flowchart illustrating the method for maintaining distributed metadata consistency provided in Embodiment 5 of this application. Figure 5 As shown, in step S204, for any participating node, the second version number of the target object corresponding to the identifier in the participating node and the coordinating node is obtained. If the first version number is 1 greater than the second version number, then after operating on the target object on the participating node and the coordinating node, the following steps may also be included:

[0079] S501: If a participating node or coordinating node fails to operate on the target object, the participating node will send a second operation failure message to the coordinating node.

[0080] S502: The coordinating node controls the central node, the coordinating node, and the participating nodes to roll back the target transaction;

[0081] S503: The coordinating node sends the second operation failure information to the server, and the server terminates the operation on the target object.

[0082] In this embodiment, if a participating node or coordinating node fails to operate on the target object, for example, if the conditions that the target object depends on are not met, or if the space itself is insufficient, the participating node or coordinating node generates operation failure information, namely the second operation failure information. The coordinating node controls the central node, the coordinating node, and the participating nodes to perform rollback processing on the target transaction, so that the central node, the coordinating node, and the participating nodes can roll back the target object to the version before the operation. The corresponding first version number will also be rolled back to the version before the operation. The coordinating node sends the second operation failure information to the server, and the server ends the operation on the target object.

[0083] S205: If all participating nodes and coordinating nodes successfully operate on the target object, the central node submits the target transaction that operated on the target object.

[0084] In step S205, it is determined whether all participating nodes have successfully operated on the target object. If all participating nodes and the coordinating node have successfully operated on the target object, it is considered that the metadata of the target object in the central node is consistent with the metadata in the participating nodes and the coordinating node. The target object in the central node and the target object in the participating nodes can be submitted. After the submission is completed, the central node, participating nodes and the coordinating node have completed the operation on the target object, and the metadata of the target object has changed in the central node, participating nodes and the coordinating node.

[0085] In this embodiment, it is determined whether all participating nodes and coordinating nodes have successfully operated on the target object. If all participating nodes and coordinating nodes have successfully operated on the target object, then the metadata of the target object in the central node is considered to be consistent with the metadata in the participating nodes and the coordinating node. Therefore, when all participating nodes and coordinating nodes have successfully operated on the target object, each participating node and coordinating node sends the operation success information along with the corresponding second version number plus 1 to the central node. After receiving the information sent by the participating nodes and coordinating nodes, the central node submits the target transaction that operated on the target object.

[0086] It should be noted that when the central node submits a target transaction that operates on the target object, the central node does not perform lock release processing.

[0087] S206: If the central node successfully submits the target transaction, then the participating nodes and the coordinating node submit the target transaction.

[0088] In step S206, it is determined whether the central node has successfully committed. If the central node successfully commits the target transaction, the participating nodes and the coordinating node commit the target transaction. During the process of the participating nodes and the coordinating node committing the target transaction, the participating nodes and the coordinating node release the lock on the target object.

[0089] In this embodiment, if the central node successfully submits the target transaction, it sends a submission success message to each participating node and the coordinating node. The participating nodes and the coordinating node then begin to submit the corresponding target transaction, that is, to submit the target object with changed metadata, so that the metadata of the target object remains consistent among the central node, participating nodes, and coordinating nodes.

[0090] See Figure 6 This is a flowchart illustrating the method for maintaining distributed metadata consistency provided in Embodiment Six of this application. Figure 6 As shown, after the central node submits the target transaction for operating on the target object in step S205 if all participating nodes and the coordinating node successfully operate on the target object, the following steps may also be included:

[0091] S601: If the central node fails to commit the target transaction, the central node will send the first commit failure information of the failed target transaction to the coordinating node.

[0092] S602: The coordinating node controls the central node, participating nodes, and coordinating node to roll back the target transaction;

[0093] S603: The coordinating node sends the first commit failure message to the server, and the server ends the operation on the target object.

[0094] In this embodiment, if the central node fails to commit the target transaction, for example, due to a power outage, the central node generates a commit failure message, namely the first commit failure message. The central node sends this first commit failure message to the coordinating node. The coordinating node then controls the central node, participating nodes, and the coordinating node to perform a rollback of the target transaction. This rolls the target object back to its metadata state before the operation, and also rolls the corresponding version number back to its version number before the operation. The coordinating node then sends the first commit failure message to the server, and the server terminates its operation on the target object.

[0095] S207: If the participating nodes and the coordinating node successfully commit the target transaction, the central node releases the lock on the target object.

[0096] In step S207, it is determined whether the participating nodes and the coordinating node have successfully committed the target transaction. If all participating nodes and the coordinating node have successfully committed, the central node releases the lock on the target object, and operations can continue on the target object after the lock is released.

[0097] In this embodiment, after all participating nodes and coordinating nodes have successfully submitted, a submission success message is sent to the central node. The central node releases the lock on the target object, and the central node, participating nodes, and coordinating nodes send submission success information to the server. The server then ends its operation on the target object.

[0098] See Figure 7 This is a flowchart illustrating the method for maintaining distributed metadata consistency provided in Embodiment 7 of this application. Figure 7 As shown, if the central node successfully submits the target transaction in step S206, the following steps may also be included after the participating nodes and the coordinating node submit the target transaction:

[0099] S701: If a coordinating node or participating node fails to commit the target transaction, push the information to the coordinating node or participating node to commit the target transaction.

[0100] S702: The coordinating node or participating node commits the target transaction based on the information received regarding the target transaction.

[0101] In this embodiment, if a participating node fails to commit the target transaction (e.g., one participating node experiences a power outage and fails to commit), the participating node sends a failure message to the target node. The exception recovery module in the target node then sends a commit message to the node that failed to commit, continuing until the participating node successfully commits. After all participating nodes have successfully committed the target transaction, the central node releases the lock on the target object.

[0102] See Figure 8This is a flowchart illustrating the method for maintaining distributed metadata consistency provided in Embodiment 8 of this application.

[0103] like Figure 8 As shown, before the central node unlocks the target object in step S207, the following steps may also be included:

[0104] S801: Lock the target object in the participating nodes and coordinating nodes according to the target object's identifier;

[0105] S802: Obtain the N target participating nodes that execute data manipulation statements on the target object, where N is an integer greater than zero;

[0106] S803: For any target participating node, obtain the version number of the target object in the target participating node and the version number of the target object in the coordinating node. If the version number of the target object in the target participating node is equal to the version number of the target object in the coordinating node, then the target participating node executes data manipulation statements on the target object. Otherwise, re-execute the step of locking the target object in the participating node and the coordinating node according to the identifier of the target object.

[0107] In this embodiment, after the metadata of the target object in the participating node is changed, data manipulation statements can also be executed on the target object in the participating node at the same time, that is, operations such as adding, deleting, modifying and querying are performed on the target object.

[0108] Based on the identifier of the target object, the target object is locked in both the participating nodes and the coordinating node. Locking is to enable the participating node to acquire the object lock in the coordinating node when executing data manipulation statements. After the participating node acquires the object lock, it determines the version number of the target object to prevent the version number of the target object from changing.

[0109] Retrieves N target participating nodes from which data manipulation statements are executed on the target object, where N is a positive integer.

[0110] For any target participating node, obtain the version number of the target object in the target participating node and the version number of the target object in the coordinating node. If the version number of the target object in the target participating node is equal to the version number of the target object in the coordinating node, then the target participating node executes data manipulation statements on the target object; otherwise, re-execute the step of locking the target object based on the target object's identifier.

[0111] It should be noted that if the target participating node cannot acquire the object lock from the coordinating node, the coordinating node will intercept the data manipulation statements.

[0112] In this embodiment, the effect of local locking is achieved through the metadata version number. When executing the data definition language, data operation statements can be executed simultaneously. It is only necessary to judge the version number of the target object in the participating node to be executed. If the version number of the target object in the participating node to be executed is equal, it can be executed normally without waiting for the metadata version number of the entire cluster to reach consistency, thus maximizing the support for concurrent execution of data definition language and data operation statements.

[0113] See Figure 9 This is a flowchart illustrating the method for maintaining distributed metadata consistency provided in Embodiment 9 of this application.

[0114] like Figure 9 As shown, after the central node releases the lock on the target object in step S207, the following steps may also be included:

[0115] S901: If the central node fails to release the lock on the target object, then determine whether the central node, participating nodes, and coordinating node have successfully committed the target transaction.

[0116] S902: If the central node, participating nodes, and coordinating node successfully commit the target transaction, the central node releases the lock on the target object.

[0117] In this embodiment, if the central node fails to release the lock on the target object, for example, due to a power outage, the process involves checking whether the central node, participating nodes, and coordinating node successfully committed the target transaction once the central node recovers. If all three nodes successfully committed the target transaction, the central node releases the lock on the target object. This ensures that the central node can continue to release the lock on the target object even if all nodes have successfully committed.

[0118] It should be noted that in this embodiment, the exception response module can determine that after the central node and participating nodes have successfully submitted the target transaction, it can control the central node to release the lock on the target object.

[0119] In this embodiment, when the target statement is executed, the central node, participating nodes, and coordinating node all execute successfully. When the central node successfully commits the target transaction that operates on the target object, the participating and coordinating nodes will always commit successfully. This is because if a participating or coordinating node fails to commit, the exception recovery module in the central node will send a commit message to that participating or coordinating node until it successfully commits. Participating or coordinating nodes do not need to acquire locks from the central node; they only need to perform consistency checks on the metadata version of the target object on the involved nodes to ensure the metadata consistency of the target object. This local constraint method reduces resource waste and improves the performance and efficiency of the distributed database while ensuring the consistency of the target object's metadata.

[0120] The central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and performs operations on the target object according to the identifier and the target statement. If the central node successfully operates on the target object, it obtains the first version number of the target object. The central node sends the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node. For each participating node, it obtains the second version number of the target object corresponding to the identifier in the participating node and the coordinating node. If the first version number is 1 greater than the second version number, it performs operations on the target object on the participating node and the coordinating node. If all participating nodes and the coordinating node successfully operate on the target object, the central node commits the target transaction that operated on the target object. If the central node successfully commits the target transaction, the participating nodes and the coordinating node commit the target transaction. If the participating nodes and the coordinating node successfully commit the target transaction, the central node releases the lock on the target object. In this application, during the execution of the Data Definition Language (DDL) in a distributed database, the central node and participating nodes operate on the target object sequentially. Based on the operation results of the target object across all nodes, it is determined whether the central node meets the commit conditions. If the central node meets the commit conditions, the coordinating node and participating nodes are also considered to meet the commit conditions. This simplifies the operation of the distributed DDL into a single-node problem, improving the metadata operation process of the distributed database. Furthermore, in this application, when executing DDL and data operation statements, it is only necessary to check the version number of the metadata in the participating nodes to be executed. If the version numbers of the metadata in the participating nodes to be executed are equal, execution can proceed normally without waiting for the metadata version numbers of the entire cluster to reach consistency. This achieves the effect of local locking through the metadata version number, maximizing the support for concurrent execution of DDL and data operation statements.

[0121] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a device for maintaining distributed metadata consistency provided in Embodiment 10 of the present invention. Please refer to the following for details. Figures 2-9 as well as Figures 2-9 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. Figure 9 As shown, the device for maintaining the consistency of distributed metadata is applied to a distributed database. The distributed database includes a central node and at least one participating node. The central node communicates with each participating node. The device 100 for maintaining the consistency of distributed metadata includes: a first operation module 101, a receiving module 102, a sending module 103, a second operation module 104, a first submission module 105, a second submission module 106, and a locking module 107.

[0122] The first operation module 101 is used by the central node to receive the target statement, determine the identifier of the target object in the target statement, lock the target object, and perform operations on the target object according to the identifier of the target object and the target statement.

[0123] Module 102 is used to obtain the first version number of the target object if the central node successfully operates on the target object.

[0124] The sending module 103 is used by the central node to send the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node.

[0125] The second operation module 104 is used to obtain the second version number of the target object corresponding to the identifier in the participating node and the coordinating node for any participating node. If the first version number is 1 greater than the second version number, then the target object is operated on in the participating node and the coordinating node.

[0126] The first submission module 105 is used to submit the target transaction that operated on the target object if all participating nodes and the coordinating node successfully operate on the target object.

[0127] The second submission module 106 is used to submit the target transaction if the central node successfully submits the target transaction.

[0128] The lock release module 107 is used to release the lock on the target object if the participating nodes and the coordinating node successfully commit the target transaction.

[0129] Optionally, the device 100 for maintaining distributed metadata consistency also includes:

[0130] The verification module is used to coordinate the nodes to receive the target statement, verify the target statement, obtain the verification result, and if the verification result is correct, send the target statement to the central node.

[0131] Optionally, the device 100 for maintaining distributed metadata consistency also includes:

[0132] The first sending module is used to release the lock on the target object if the central node fails to operate on the target object, and send the first operation failure information of the failed operation on the target object to the coordinating node.

[0133] The first termination module is used to coordinate the node to send the first operation failure information to the server, and the server terminates the operation on the target object.

[0134] Optionally, the device 100 for maintaining distributed metadata consistency also includes:

[0135] The second sending module is used to send a second operation failure message to the coordinating node if a participating node or the coordinating node fails to operate on the target object.

[0136] The first rollback module is used to coordinate the control center node, coordinating node, and participating nodes to roll back the target transaction.

[0137] The second termination module is used to coordinate the node to send the second operation failure information to the server, and the server terminates the operation on the target object.

[0138] Optionally, the device 100 for maintaining distributed metadata consistency also includes:

[0139] The third sending module is used to send the first commit failure information of the target transaction to the coordinating node if the central node fails to commit the target transaction.

[0140] The second rollback module is used to coordinate the control center node, participating nodes, and coordinating node to roll back the target transaction.

[0141] The third termination module is used to coordinate the node to send the first submission failure information to the server, and the server ends the operation on the target object.

[0142] Optionally, the device 100 for maintaining distributed metadata consistency also includes:

[0143] The push module is used to push information about the completion of the target transaction to the coordinating node or participating node if the coordinating node or participating node fails to complete the target transaction.

[0144] The commit module is used by the coordinating node or participating node to commit the target transaction based on the information received regarding the target transaction.

[0145] Optionally, the device 100 for maintaining distributed metadata consistency also includes:

[0146] The locking module is used to lock the target object in participating nodes and coordinating nodes based on the target object's identifier.

[0147] The acquisition module is used to acquire N target participating nodes that execute data manipulation statements on the target object, where N is an integer greater than zero.

[0148] The judgment module is used to obtain the version number of the target object in the target participating node and the version number of the target object in the coordinating node for any target participating node. If the version number of the target object in the target participating node is equal to the version number of the target object in the coordinating node, the target participating node executes data manipulation statements on the target object. Otherwise, the step of locking the target object in the participating node and the coordinating node based on the target object's identifier is re-executed.

[0149] Optionally, the device 100 for maintaining distributed metadata consistency also includes:

[0150] The first judgment module is used to determine whether the central node, participating nodes, and coordinating node have successfully committed the target transaction if the central node fails to release the lock on the target object.

[0151] The second judgment module is used to release the lock on the target object if the central node, participating nodes, and coordinating node successfully submit the target transaction.

[0152] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0153] Figure 11 This is a schematic diagram of the structure of a computer device provided in Embodiment Eleven of this application. Figure 11 As shown, the computer device of this embodiment includes: at least one processor ( Figure 11 The diagram shows only one of the following: a memory and a computer program stored in the memory and executable on at least one processor. When the processor executes the computer program, it implements the steps in any of the above-described methods for maintaining the consistency of distributed metadata.

[0154] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 11 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces.

[0155] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0156] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0157] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0158] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a computer device, it enables the computer device to execute the steps in the above method embodiments.

[0159] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0160] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0161] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for maintaining distributed metadata consistency, characterized in that, Applied to a distributed database, the distributed database includes a central node, a coordinating node, and at least one participating node, wherein the central node communicates with each participating node, and the method for maintaining the consistency of distributed metadata includes: The central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and performs operations on the target object according to the identifier of the target object and the target statement; If the central node successfully operates on the target object, the central node obtains the first version number of the target object; The central node sends the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node; For any participating node, obtain the second version number of the target object corresponding to the identifier in the participating node and the coordinating node. If the first version number is 1 greater than the second version number, then perform an operation on the target object on the participating node and the coordinating node. If each participating node and the coordinating node successfully operate on the target object, the central node submits the target transaction that operated on the target object. If the central node successfully submits the target transaction, then the participating nodes and the coordinating node submit the target transaction. If the participating node and the coordinating node successfully submit the target transaction, the central node releases the lock on the target object.

2. The method for maintaining distributed metadata consistency as described in claim 1, characterized in that, The central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and before performing operations on the target object based on the identifier of the target object and the target statement, it also includes: The coordinating node receives the target statement, verifies the target statement, obtains the verification result, and if the verification result is correct, sends the target statement to the central node.

3. The method for maintaining distributed metadata consistency as described in claim 2, characterized in that, The distributed database also includes a server, and the central node, each participating node, the coordinating node, and the server communicate with each other. After operating on the target object based on the target object's identifier and the target statement, the process further includes: If the central node fails to operate on the target object, it unlocks the target object and sends the first operation failure information for the failed operation on the target object to the coordinating node. The coordinating node sends the first operation failure information to the server, and the server terminates the operation on the target object.

4. The method for maintaining distributed metadata consistency as described in claim 3, characterized in that, For any participating node, the method of obtaining the second version number of the target object corresponding to the identifier in the participating node and the coordinating node, and if the first version number is 1 greater than the second version number, then after performing operations on the target object on the participating node and the coordinating node, the method further includes: If either the participating node or the coordinating node fails to operate on the target object, the participating node will send a second operation failure message to the coordinating node. The coordinating node controls the central node, the coordinating node, and the participating nodes to roll back the target transaction; The coordinating node sends the second operation failure information to the server, and the server terminates the operation on the target object.

5. The method for maintaining distributed metadata consistency as described in claim 3, characterized in that, After the central node submits the target transaction to operate on the target object, it also includes: If the central node fails to commit the target transaction, the central node will send a first commit failure message to the coordinating node. The coordinating node controls the central node, the participating nodes, and the coordinating node to perform rollback processing on the target transaction; The coordinating node sends the first submission failure information to the server, and the server terminates its operation on the target object.

6. The method for maintaining distributed metadata consistency as described in claim 3, characterized in that, After the participating nodes and the coordinating node submit the target transaction, the process further includes: If either the coordinating node or a participating node fails to submit the target transaction, information about submitting the target transaction is pushed to the coordinating node or participating node. The coordinating node or the participating node submits the target transaction based on the received information about submitting the target transaction.

7. The method for maintaining distributed metadata consistency as described in claim 3, characterized in that, Before the central node unlocks the target object, the process also includes: Based on the identifier of the target object, the target object is locked in the participating nodes and the coordinating nodes; Obtain N target participating nodes that execute data manipulation statements on the target object, where N is an integer greater than zero; For any target participating node, obtain the version number of the target object in the target participating node and the version number of the target object in the coordinating node. If the version number of the target object in the target participating node is equal to the version number of the target object in the coordinating node, then the target participating node executes a data manipulation statement on the target object. Otherwise, re-execute the step of locking the target object in the participating node and the coordinating node based on the identifier of the target object.

8. The method for maintaining distributed metadata consistency as described in claim 3, characterized in that, After the central node releases the lock on the target object, it also includes: If the central node fails to release the lock on the target object, then a determination is made as to whether the central node, the participating nodes, and the coordinating node successfully submitted the target transaction. If the central node, the participating nodes, and the coordinating node successfully submit the target transaction, then the central node releases the lock on the target object.

9. An apparatus for maintaining the consistency of distributed metadata, characterized in that, Applied to a distributed database, the distributed database includes a central node, a coordinating node, and at least one participating node, wherein the central node communicates with each participating node, and the apparatus for maintaining the consistency of distributed metadata includes: The first operation module is used for the central node to receive the target statement, determine the identifier of the target object in the target statement, lock the target object, and operate on the target object according to the identifier of the target object and the target statement; The module is configured to obtain the first version number of the target object if the central node successfully operates on the target object; The sending module is used by the central node to send the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node; The second operation module is used to obtain the second version number of the target object corresponding to the participating node and the coordinating node for any participating node. If the first version number is 1 greater than the second version number, then the target object is operated on the participating node and the coordinating node. The first submission module is used so that if all participating nodes and the coordinating node successfully operate on the target object, the central node submits the target transaction that operates on the target object. The second submission module is used so that if the central node successfully submits the target transaction, the participating nodes and the coordinating node submit the target transaction. The lock release module is used so that if the participating node and the coordinating node successfully submit the target transaction, the central node releases the lock on the target object.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for maintaining distributed metadata consistency as described in claims 1 to 8.

Images