Method and device for maintaining consistency of distributed metadata and medium

By introducing central nodes, coordinating points and participating nodes in distributed databases, and using target statement locking and version number management mechanisms, the problem of difficult to maintain metadata consistency in distributed databases is solved, and efficient metadata operations and Concurrent execution improves system performance and efficiency.

CN119088785AActive Publication Date: 2024-12-06SHENZHEN INST OF COMPUTING SCI
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411032447.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-12-06
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

In distributed database systems, metadata consistency is difficult to maintain, resulting in data inconsistency and performance degradation. The existing distributed transaction operations are complex, making it difficult to improve operational efficiency and reduce resource usage.

Method used

By introducing central nodes, coordinating points and participating nodes in distributed databases, target statement locking, version number management and transaction submission mechanisms are adopted to simplify the metadata operation process and implement local locking to support concurrent execution.

Benefits of technology

It effectively maintains distributed metadata consistency, simplifies the execution of data definition language and data operation statements, improves concurrent execution capabilities, and reduces resource occupation and operation complexity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088785A_ABST
    Figure CN119088785A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of distributed databases, in particular to a method and device for maintaining the consistency of distributed metadata and a medium. In the application, in an execution process of a data definition language in a distributed database, a central node and participating nodes operate a target object in sequence, and according to operation results of the target object on all nodes, whether the central node has a submission condition is determined, and if the central node has the submission condition, the target object is submitted. And if not, considering that the participating node also has a submission condition, and simplifying the operation of the distributed data definition language into a single-node problem. Meanwhile, through the metadata version number of the target object, all the nodes needing to be modified are limited to operate the data stored in the data object under the condition that the metadata is inconsistent, the function of a local lock is achieved, and the metadata operation process of the distributed database is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of distributed databases, and in particular to a method, device and medium for maintaining distributed metadata consistency. Background Art

[0002] In a distributed database system, metadata consistency is very important, because metadata is used to describe data and is an important part of the entire database system, including data structure, storage location, index information, data sharding, etc. If the metadata on each data node is inconsistent, it may cause data inconsistency, affecting the correctness of the results of operations such as insertion and query. For example, a table can be accessed on a certain node, but the table has been deleted on other nodes, resulting in users only being able to read part of the results. In addition, the database system usually performs query optimization based on metadata information, such as finding a suitable index. If the metadata on each node is inconsistent, it may cause performance degradation. Therefore, ensuring the consistency of metadata on each node is an important guarantee for ensuring the normal operation of the entire distributed database system, data accuracy and consistency. In the prior art, metadata is managed through distributed transactions to keep the metadata on each data node consistent, but distributed transaction operations are more complicated. Therefore, in the process of keeping the metadata on each data node consistent, how to improve the operation process of the distributed database to reduce resource usage and improve operation efficiency while ensuring the consistency of distributed metadata has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, embodiments of the present application provide a method, device, and medium for maintaining distributed metadata consistency to solve the problem of complex distributed transaction operations in the process of maintaining consistency of metadata on each data node.

[0004] In a first aspect, an embodiment of the present application provides a method for maintaining distributed metadata consistency, which is applied to a distributed database, wherein the distributed database includes a central node, a coordinating node, and at least one participating node, and the central node is connected to each participating node for communication. The method for maintaining distributed metadata consistency includes:

[0005] The central node receives the target statement, determines the identifier of the target object in the target statement, and locks the target object. Since all metadata modifications will first attempt to lock the target object at the central node,

[0006] If the central node successfully operates the target object, the central node obtains the first version number of the target object;

[0007] The central node sends the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node;

[0008] For any participating node, obtain the second version number of the target object corresponding to the identification in the participating node and the coordinating node, and if the first version number is greater than the second version number by 1, perform operations on the target object on the participating node and the coordinating node;

[0009] If all participating nodes and the coordinating node successfully operate the target object, the central node submits the target transaction for operating the target object;

[0010] If the central node successfully submits the target transaction, the participating nodes and the coordinating node submit the target transaction;

[0011] If the participating nodes and the coordinating node successfully submit the target transaction, the central node releases the lock on the target object.

[0012] In a second aspect, an embodiment of the present application provides a device for maintaining distributed metadata consistency, which is applied to a distributed database, wherein the distributed database includes a central node, a coordinating node, and at least one participating node, and the central node is communicated with each participating node. The device for maintaining distributed metadata consistency includes:

[0013] A first operation module, configured for the central node to receive a target statement, determine an identifier of a target object in the target statement, lock the target object, and operate the target object according to the identifier of the target object and the target statement;

[0014] an obtaining module, configured to obtain, if the central node successfully operates the target object, a first version number of the target object by the central node;

[0015] A sending module, used for the central node to send the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node;

[0016] A second operation module is used to obtain, for any participating node, a second version number of a target object correspondingly identified in the participating node and the coordinating node, and if the first version number is 1 greater than the second version number, operate the target object on the participating node and the coordinating node;

[0017] A first submission module, configured to enable the central node to submit a target transaction for operating the target object if all participating nodes and the coordinating node successfully operate the target object;

[0018] A second submission module, configured to enable the participating nodes and the coordinating node to submit the target transaction if the central node submits the target transaction successfully;

[0019] The lock releasing module is used for, if the participating nodes and the coordinating node successfully submit the target transaction, the central node then releases the lock on the target object.

[0020] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for maintaining distributed metadata consistency as described in the first aspect is implemented.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and operates on the target object according to the identifier of the target object and the target statement. If the central node successfully operates on the target object, the central node obtains the first version number of the target object. The central node sends the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node. For any participating node, the central node obtains the second version number of the target object corresponding to the identifier in the participating node and the coordinating node. If the first version number is greater than the second version number by 1, the target object is operated on the participating node and the coordinating node. If all participating nodes and the coordinating node successfully operate on the target object, the central node submits the target transaction for operating on the target object. If the central node submits the target transaction successfully, the participating nodes and the coordinating node submit the target transaction. If the participating nodes and the coordinating node submit the target transaction successfully, the central node releases the lock on the target object. In the present application, during the execution of the data definition language in the distributed database, the central node and the participating nodes successively operate on the target object, and determine whether the central node meets the submission conditions based on the operation results of the target object on all nodes. If the central node meets the submission conditions, it is considered that the coordinating node and the participating nodes also meet the submission conditions, simplifying the operation of the distributed data definition language to a single-node problem, and improving the metadata operation process of the distributed database. In addition, in the present application, when executing the data definition language and data operation statements, it is only necessary to judge the version number of the metadata in the participating nodes to be executed. When the version numbers of the metadata in the participating nodes to be executed are equal, it can be executed normally without waiting for the metadata version number of the entire cluster to be consistent. The metadata version number is used to achieve the effect of local locking, and the concurrent execution of the data definition language and data operation statements is supported to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0024] Figure 1 This is a schematic diagram of an application environment of a method for maintaining distributed metadata consistency provided in Embodiment 1 of the present application;

[0025] Figure 2 This is a flow chart of a method for maintaining distributed metadata consistency provided in Embodiment 2 of the present application;

[0026] Figure 3 This is a flowchart of a method for maintaining distributed metadata consistency provided in Embodiment 3 of the present application;

[0027] Figure 4 This is a flowchart of a method for maintaining distributed metadata consistency provided in Embodiment 4 of the present application;

[0028] Figure 5 This is a flowchart of a method for maintaining distributed metadata consistency provided in Embodiment 5 of the present application;

[0029] Figure 6 This is a flowchart of a method for maintaining distributed metadata consistency provided in Example 6 of the present application;

[0030] Figure 7 This is a flow chart of a method for maintaining distributed metadata consistency provided in Embodiment 7 of the present application;

[0031] Figure 8 This is a flowchart of a method for maintaining distributed metadata consistency provided in Embodiment 8 of the present application;

[0032] Fig. 9 This is a flowchart of a method for maintaining distributed metadata consistency provided in Embodiment 9 of the present application;

[0033] Fig.10 This is a schematic diagram of the structure of a device for maintaining distributed metadata consistency provided in Embodiment 10 of the present application;

[0034] Fig.11 It is a structural diagram of a computer device provided in Example 11 of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0037] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0038] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0039] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0041] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0042] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0043] A method for maintaining distributed metadata consistency provided in the first embodiment of the present application can be applied in the following situations: Figure 1 In a distributed database, the distributed database includes a central node, a coordinating node and at least one participating node, a coordinating node and a server, and the central node, each participating node, the coordinating node and the server are communicated with each other, wherein the central node, the participating node and the coordinating node are different clients.

[0044] See also Figure 2 , is a flow chart of a method for maintaining distributed metadata consistency provided in Embodiment 2 of the present application. Figure 2 As shown, the method for maintaining distributed metadata consistency may include the following steps:

[0045] S201: The central node receives a target statement, determines an identifier of a target object in the target statement, locks the target object, and operates the target object according to the identifier of the target object and the target statement.

[0046] In step S201, the central node receives a target statement, wherein the central node is a client in a distributed database, the target statement is a data definition language statement, and the identifier of a target object in the target statement is determined, wherein the target object is an operated object, the identifier of the target object is a unique identifier of the operated object, and the target object is operated according to the identifier of the target object and the target statement.

[0047] In this embodiment, in this embodiment, the method for maintaining distributed metadata consistency is applied to a distributed database, which includes a server and multiple clients, wherein a node is selected in the client as a central node, and the metadata of the object in the database is based on the metadata state of the object in the central node. The central node receives a target statement, wherein the target statement is a data definition language statement, and the data definition language statement can be an operation statement for an existing object, or an operation statement for a newly created object, for example, inserting a column of data into an existing data table A, or creating a new data table B, etc.

[0048] The identifier of the target object in the target statement is determined at the central node, so that when the target object in other clients, i.e., participating nodes, is operated, the target object to be operated is determined according to the corresponding identifier. The target object is locked to prevent the object from being operated in other clients and participating nodes, which would cause the metadata status of the object to be inconsistent in the database. At the central node, the target object is operated according to the identifier of the target object and the target statement.

[0049] It should be noted that the target object is locked. The purpose of this lock is to intercept metadata changes of the target object globally. Because all metadata changes will first go to the central node, the object lock on the central node can have the effect of a global metadata lock.

[0050] See also Figure 3 , is a flow chart of a method for maintaining distributed metadata consistency provided in Embodiment 3 of the present application. Figure 3 As shown, in step S201, the central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and before operating the target object according to the identifier of the target object and the target statement, the following steps may also be included:

[0051] S301: The coordination node receives the target statement, verifies the target statement, obtains the verification result, and sends the target statement to the central node if the verification result is correct.

[0052] In this embodiment, the coordinating node is a node among multiple clients, and the coordinating node is used to control the operations of the central node and the participating nodes according to the operation results or submission results of the central node and the participating nodes to ensure that the status of the metadata of the target object in the central node and the participating nodes is consistent.

[0053] Before the central node receives the target statement, the coordinating node receives the target statement and verifies the target statement to determine whether the target statement is correct, for example, whether the target statement conforms to the correct grammar, etc., and obtains the verification result. If the verification result is correct, the target statement is sent to the central node so that the central node can operate the target object in the target statement according to the correct target statement.

[0054] It should be noted that the coordinating node receives the target statement, verifies the target statement, and obtains the verification result. If the verification result is incorrect, for example, the syntax of the target statement is incorrect, the coordinating node sends the incorrect target statement information to the server, and the server ends the operation to prevent the target node and the participating nodes from starting to execute the operation and occupying resources.

[0055] S202: If the central node successfully operates the target object, the central node obtains the first version number of the target object.

[0056] In step S202, if the central node successfully operates the target object, the central node obtains a first version number of the target object, where the first version number is the number of times the target object has been operated.

[0057] In this embodiment, if the central node successfully operates the target object, the central node obtains the first version number of the target object, wherein the version corresponding to the first version number is the version after the central node executes the target statement. The first version number is the result of accumulation, that is, the version number after the central node last operated on the target object plus 1. If the central node fails to operate on the target object, the first version number is not accumulated.

[0058] It should be noted that after operating the target object according to the target object's identifier and target statement, no submission is performed. This is to ensure that the node has the conditions for changing the target object's metadata, so as to prevent the node from submitting when the target object does not have the conditions for changing, thereby occupying the corresponding resources.

[0059] It should be noted that if the corresponding data definition language in the target statement is to create a new object, the newly created object has no corresponding identifier in the central node, and an identifier needs to be assigned to the newly created object so that when the participating nodes execute the corresponding target statement, they can refer to the corresponding identifier and assign the same identifier to the newly created object. If it is a newly created object, the first version number is 0.

[0060] It should be noted that if the central node's operation on the target object is a deletion operation, that is, deleting an existing target object, the first version number of the corresponding target object is a temporary version number, that is, a temporary first version number, which has nothing to do with the version number after the central node's last operation on the target object. After the operation is successful, all records of the target object will be deleted and the version number will not be persistent.

[0061] See also Figure 4 , is a flow chart of a method for maintaining distributed metadata consistency provided in Embodiment 4 of the present application. Figure 4 As shown, after operating the target object according to the identifier of the target object and the target sentence in step S201, the following steps may also be included:

[0062] S401: If the central node fails to operate on the target object, the central node releases the lock on the target object and sends first operation failure information of the failed operation on the target object to the coordination node;

[0063] S402: The coordination node sends the first operation failure information to the server, and the server ends the operation on the target object.

[0064] In this embodiment, the distributed database also includes a server, which is used to terminate the operation based on the operation results or submission results of the central node and the participating nodes. For example, when the operation or submission of the central node and the participating nodes fails, the server terminates the operation to prevent the central node or the participating nodes from continuing to execute the corresponding operation.

[0065] If the central node fails to operate on the target object, for example, the target object being operated in the target statement does not exist, or there is insufficient space on the central node, the target object cannot be operated on. The central node releases the lock on the target object and sends the first operation failure information of the failed operation on the target object to the coordination node. The coordination node sends the first operation failure information to the server, and the server ends the operation on the target object to prevent other participating nodes from operating on the target statement.

[0066] S203: The central node sends the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node.

[0067] In step S203, after the central node successfully operates on the target object, the central node sends the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node, so that the coordinating node will perform the same operation as the participating node in the subsequent entire metadata change process, where the participating node and the coordinating node are other clients that have the target object in the database.

[0068] In this embodiment, the central node sends the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node, so that the participating nodes and the coordinating node can determine the target object in the participating node according to the identifier of the target object. For example, if the identifier of the target object is 001, the participating node and the coordinating node respectively operate on the target object identified as 001 to prevent the name of the target object from being the same as the name of other objects, which may cause erroneous operation on other objects.

[0069] It should be noted that the central node sends the first version number to the participating nodes and the coordinating node so that the participating nodes and the coordinating node can determine whether the metadata of the target object after executing the corresponding target statement is consistent with the metadata of the target object in the central node based on the first version number.

[0070] It should be noted that the coordinating node and the participating nodes perform the same operation, so that the coordinating node also saves the version number of the target object, so that when the data manipulation statement is executed subsequently, the coordinating node locks the target object, so that the corresponding participating node can execute the data manipulation statement after the coordinating node obtains the corresponding lock, to prevent some participating nodes from changing the version number of the object after executing the data manipulation statement.

[0071] S204: For any participating node, obtain the second version number of the target object corresponding to the identification in the participating node and the coordinating node, and if the first version number is 1 greater than the second version number, operate the target object on the participating node and the coordinating node.

[0072] In step S204, for any participating node, obtain the second version number of the target object corresponding to the identification in the participating node and the coordinating node, and determine whether the first version number is 1 greater than the second version number. If the first version number is 1 greater than the second version number, operate on the target object on the participating node and the coordinating node.

[0073] In this embodiment, in any participating node, the participating node and the coordinating node determine the corresponding target object based on the identifier of the received target object, determine the second version number of the corresponding target object in the participating node and the coordinating node based on the corresponding target object, and judge whether the corresponding participating node and the coordinating node perform operations on the target object based on the first version number and the second version number. If the first version number is 1 greater than the second version number, it is considered that the version number corresponding to the target object in the participating node and the coordinating node is the version number after the last successful operation, and the current operation can be performed, so the target object can be operated on the participating node and the coordinating node. After the participating node and the coordinating node operate on the target object, the corresponding version number is increased by 1 to reach the same version number as the target object in the central node. If there is a participating node and the coordinating node that fails to operate on the target object, the corresponding version number is not accumulated.

[0074] It should be noted that the operation process on the target object on the participating node and the coordinating node includes adding an object lock to the target object to prevent the execution of data operation statements on the corresponding participating node or coordinating node. The operation process includes adding an exclusive lock to the target object on the participating node and the coordinating node, modifying the metadata of the target object but not persisting it. The lock added to the target object here is different from the lock on the central node. It is a lock that only acts on a single node range. The purpose is to limit the execution of data operations such as addition, deletion, and modification on the target object;

[0075] In this embodiment, the second version number of the target object corresponding to the identification in the participating nodes and the coordinating node is obtained to compare the second version number with the first version number to determine whether the participating nodes and the coordinating node operate on the target object, so as to ensure that after the participating nodes and the coordinating node operate on the target object, the metadata of the target object is equal to the metadata of the target object in the central node.

[0076] It should be noted that if the target statement is to create a new target object, when the participating nodes and the coordinating node create the new target object, the identifier of the target object allocated by the central node is used as the identifier of the new target object, and no new identifier is generated.

[0077] It should be noted that the participating nodes and the coordinating node execute the corresponding target statement without submitting it in order to ensure that the node has the conditions for changing the metadata of the target object, so as to prevent the node from submitting when the target object does not have the conditions for change, thereby occupying the corresponding resources.

[0078] See also Figure 5 , is a flow chart of a method for maintaining distributed metadata consistency provided in Embodiment 5 of the present application. Figure 5 As shown, in step S204, for any participating node, the second version number of the target object corresponding to the identification in the participating node and the coordinating node is obtained. If the first version number is 1 greater than the second version number, after the target object is operated on the participating node and the coordinating node, the following steps may also be included:

[0079] S501: If a participating node or a coordinating node fails to operate a target object, the participating node sends second operation failure information of the failed operation on the target object to the coordinating node;

[0080] S502: The coordinating node controls the central node, the coordinating node and the participating nodes to roll back the target transaction;

[0081] S503: The coordination node sends the second operation failure information to the server, and the server ends the operation on the target object.

[0082] In this embodiment, if a participating node or a coordinating node fails to operate on the target object, for example, the conditions on which the target object depends are not met, or there is insufficient space, etc., the participating node or the coordinating node generates information about the operation failure, i.e., the second operation failure information, and the coordinating node controls the central node, the coordinating node, and the participating nodes to roll back the target transaction, so that the central node, the coordinating node, and the participating nodes roll back the target object to the version before the operation, and the corresponding first version number will also be rolled back to the version before the operation. The coordinating node sends the second operation failure information to the server, and the server ends the operation on the target object.

[0083] S205: If all participating nodes and the coordinating node successfully operate on the target object, the central node submits the target transaction for operating on the target object.

[0084] In step S205, it is determined whether all participating nodes have successfully operated on the target object. If all participating nodes and the coordinating node have successfully operated on the target object, it is considered that the metadata of the target object in the central node is consistent with the metadata in the participating nodes and the coordinating nodes, and the target object in the central node and the target object in the participating nodes can be submitted. After the submission is completed, the central node, the participating nodes and the coordinating nodes have completed the operation on the target object, and the target object metadata has been changed in the central node, the participating nodes and the coordinating nodes.

[0085] In this embodiment, it is determined whether all participating nodes and the coordinating node have successfully operated on the target object. If all participating nodes and the coordinating node have successfully operated on the target object, it is considered that the metadata of the target object in the central node is consistent with the metadata in the participating nodes and the metadata in the coordinating node. Therefore, when the participating nodes and the coordinating node have successfully operated on the target object, each participating node and the coordinating node sends the information of successful operation and the corresponding second version number plus 1 to the central node. After receiving the information sent by the participating nodes and the coordinating node, the central node submits the target transaction for operating the target object.

[0086] It should be noted that when the central node submits a target transaction for operating a target object, the central node does not release the lock.

[0087] S206: If the central node submits the target transaction successfully, the participating nodes and the coordinating node submit the target transaction.

[0088] In step S206, it is determined whether the central node has submitted the target transaction successfully. If the central node has submitted the target transaction successfully, the participating nodes and the coordinating node submit the target transaction. During the process of the participating nodes and the coordinating node submitting the target transaction, the participating nodes and the coordinating node release the lock on the target object.

[0089] In this embodiment, if the central node successfully submits the target transaction, it sends a submission success message to each participating node and coordinating node, and the participating nodes and coordinating nodes begin to submit the corresponding target transaction, that is, submit the target object after the metadata is changed, so that the metadata of the target object remains consistent in the central node, participating nodes and coordinating nodes.

[0090] See also Figure 6 , is a flow chart of a method for maintaining distributed metadata consistency provided in Example 6 of the present application. Figure 6 As shown, in step S205, if all participating nodes and the coordinating node successfully operate the target object, after the central node submits the target transaction for operating the target object, the following steps may also be included:

[0091] S601: If the central node fails to submit the target transaction, the central node sends first submission failure information of the failure to submit the target transaction to the coordination node;

[0092] S602: The coordinating node controls the central node, participating nodes and the coordinating node to roll back the target transaction;

[0093] S603: The coordination node sends the first submission failure information to the server, and the server ends the operation on the target object.

[0094] In this embodiment, if the central node fails to submit the target transaction, for example, the central node is out of power, etc., the central node generates submission failure information, i.e., the first submission failure information, and the central node sends the first submission failure information of the failure to submit the target transaction to the coordination node, and the coordination node controls the central node, the participating nodes and the coordination node to roll back the target transaction, i.e., roll back the target object to the metadata state before the operation, and also roll the corresponding version number to the version number before the operation. The coordination node sends the first submission failure information to the server, and the server ends the operation on the target object.

[0095] S207: If the participating nodes and the coordinating node successfully submit the target transaction, the central node releases the lock on the target object.

[0096] In step S207, it is determined whether the participating nodes and the coordinating node have successfully submitted the target transaction. If all participating nodes and the coordinating node have successfully submitted the transaction, the central node releases the lock on the target object, and the target object can continue to be operated after the lock is released.

[0097] In this embodiment, after all participating nodes and the coordinating node have submitted successfully, a message of successful submission is sent to the central node, the central node releases the lock on the target object, and the central node, participating nodes and coordinating node send the information of successful submission to the server, and the server ends the operation on the target object.

[0098] See also Figure 7 , is a flow chart of a method for maintaining distributed metadata consistency provided in Embodiment 7 of the present application. Figure 7 As shown, in step S206, if the central node successfully submits the target transaction, after the participating nodes and the coordinating node submit the target transaction, the following steps may also be included:

[0099] S701: If a coordinating node or a participating node fails to submit a target transaction, information on submitting the target transaction is pushed to the coordinating node or the participating node;

[0100] S702: The coordinating node or the participating node commits the target transaction according to the received information on committing the target transaction.

[0101] In this embodiment, if there is a participating node or a participating node fails to submit the target transaction, for example, one of the participating nodes is out of power and fails to submit successfully, then the participating node sends the submission failure information to the target node, and the abnormal recovery module in the target node sends the submission information to the node that failed to submit until the participating node submits successfully. After each participating node and participating node successfully submits the target transaction, the central node releases the lock on the target object.

[0102] See also Figure 8, which is a flow chart of the method for maintaining distributed metadata consistency provided in Example 8 of the present application.

[0103] like Figure 8 As shown, before the central node releases the lock on the target object in step S207, the following steps may also be included:

[0104] S801: According to the identifier of the target object, the target object is locked in the participating node and the coordinating node;

[0105] S802: Obtain N target participating nodes that execute data manipulation statements on the target object, where N is an integer greater than zero;

[0106] S803: For any target participating node, obtain the version number of the target object in the target participating node and the version number of the target object in the coordinating node. If the version number of the target object in the target participating node is equal to the version number of the target object in the coordinating node, the target participating node executes the data manipulation statement on the target object. Otherwise, re-execute the step of locking the target object in the participating node and the coordinating node according to the identification of the target object.

[0107] In this embodiment, after the metadata of the target object in the participating node is changed, data manipulation statements can also be executed on the target object in the participating node at the same time, that is, operations such as adding, deleting, modifying, and checking are performed on the target object.

[0108] According to the identifier of the target object, the target object is locked in the participating node and the coordinating node. The purpose of locking is to enable the target participating node to obtain the object lock in the coordinating node and the target participating node to obtain the object lock when executing data manipulation statements, and then determine the version number of the target object to prevent the target object version number from changing.

[0109] Gets N target participating nodes that execute data manipulation statements on the target object, where N is an integer greater than zero.

[0110] For any target participating node, obtain the version number of the target object in the target participating node and the version number of the target object in the coordination node. If the version number of the target object in the target participating node is equal to the version number of the target object in the coordination node, the target participating node executes the data manipulation statement on the target object. Otherwise, re-execute the step of locking the target object according to the identifier of the target object.

[0111] It should be noted that if the target participating node cannot obtain the object lock at the coordinating node, the coordinating node will intercept the data manipulation statement.

[0112] In this embodiment, the effect of local locking is achieved through the metadata version number. When executing the data definition language, data operation statements can be executed at the same time. It is only necessary to judge the version number of the target object in the participating nodes to be executed. When the version numbers of the target objects in the participating nodes to be executed are equal, normal execution can be achieved without waiting for the metadata version number of the entire cluster to be consistent, thereby supporting the concurrent execution of data definition language and data operation statements to the greatest extent.

[0113] See also Fig. 9 , which is a flow chart of the method for maintaining distributed metadata consistency provided in Example 9 of the present application.

[0114] like Fig. 9 As shown, after the central node releases the lock on the target object in step S207, the following steps may also be included:

[0115] S901: If the central node fails to lock the target object, it is determined whether the central node, participating nodes and coordinating nodes successfully submit the target transaction;

[0116] S902: If the central node, participating nodes and coordinating node successfully submit the target transaction, the central node releases the lock on the target object.

[0117] In this embodiment, if the central node fails to lock the target object, for example, the machine loses power, etc., then when the central node returns to normal, it determines whether the central node, participating nodes, and coordinating nodes successfully submit the target transaction. If the central node, participating nodes, and coordinating nodes successfully submit the target transaction, the central node releases the lock on the target object. In this way, when the central node, participating nodes, and coordinating nodes all submit successfully, the central node continues to lock the target object.

[0118] It should be noted that, in this embodiment, after the abnormal response module determines that the central node and the participating nodes have successfully submitted the target transaction, the central node can be controlled to release the lock on the target object.

[0119] In this embodiment, when executing the target statement, the central node, participating nodes and coordinating nodes are all executed successfully, and when the central node successfully submits the target transaction for operating the target object, the participating nodes and coordinating nodes will always submit successfully, because after the participating node or the coordinating node fails to submit, the abnormal recovery module in the central node will send a submission message to the participating node or the coordinating node until the participating node or the coordinating node submits successfully. The participating node or the coordinating node does not need to take the lock from the central node. As long as the metadata version of the target object on the node involved is checked for consistency, the metadata consistency of the target object can be guaranteed. Through this local restriction method, while ensuring the consistency of the target object metadata, the waste of resources is reduced and the performance and efficiency of the distributed database are improved.

[0120] The central node receives the target statement, determines the identifier of the target object in the target statement, locks the target object, and operates on the target object according to the identifier of the target object and the target statement. If the central node successfully operates on the target object, the central node obtains the first version number of the target object. The central node sends the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node. For any participating node, the central node obtains the second version number of the target object corresponding to the identifier in the participating node and the coordinating node. If the first version number is greater than the second version number by 1, the target object is operated on the participating node and the coordinating node. If all participating nodes and the coordinating node successfully operate on the target object, the central node submits the target transaction for operating on the target object. If the central node submits the target transaction successfully, the participating nodes and the coordinating node submit the target transaction. If the participating nodes and the coordinating node submit the target transaction successfully, the central node releases the lock on the target object. In the present application, during the execution of the data definition language in the distributed database, the central node and the participating nodes successively operate on the target object, and determine whether the central node meets the submission conditions based on the operation results of the target object on all nodes. If the central node meets the submission conditions, it is considered that the coordinating node and the participating nodes also meet the submission conditions, simplifying the operation of the distributed data definition language to a single-node problem, and improving the metadata operation process of the distributed database. In addition, in the present application, when executing the data definition language and data operation statements, it is only necessary to judge the version number of the metadata in the participating nodes to be executed. When the version numbers of the metadata in the participating nodes to be executed are equal, it can be executed normally without waiting for the metadata version number of the entire cluster to be consistent. The metadata version number is used to achieve the effect of local locking, and the concurrent execution of the data definition language and data operation statements is supported to the greatest extent.

[0121] See also Fig.10 , Fig.10 This is a schematic diagram of the structure of a device for maintaining distributed metadata consistency provided by Embodiment 10 of the present invention. Figure 2-Figure 9 as well as Figure 2-Figure 9 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 9 As shown, the device for maintaining distributed metadata consistency is applied to a distributed database, the distributed database includes a central node and at least one participating node, the central node is communicated with each participating node, and the device 100 for maintaining distributed metadata consistency includes: a first operation module 101, a obtaining module 102, a sending module 103, a second operation module 104, a first submission module 105, a second submission module 106, and a lock release module 107.

[0122] The first operation module 101 is used for the central node to receive the target statement, determine the identifier of the target object in the target statement, lock the target object, and operate the target object according to the identifier of the target object and the target statement.

[0123] The obtaining module 102 is used for the central node to obtain the first version number of the target object if the central node successfully operates the target object.

[0124] The sending module 103 is used for the central node to send the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node.

[0125] The second operation module 104 is used to obtain the second version number of the target object corresponding to the identification in the participating node and the coordinating node for any participating node, and if the first version number is 1 greater than the second version number, operate the target object on the participating node and the coordinating node;.

[0126] The first submission module 105 is used for the central node to submit the target transaction for operating the target object if all participating nodes and the coordinating node successfully operate the target object.

[0127] The second submission module 106 is configured to enable the participating nodes and the coordinating node to submit the target transaction if the central node submits the target transaction successfully.

[0128] The lock releasing module 107 is used for the central node to release the lock on the target object if the participating nodes and the coordinating node successfully submit the target transaction.

[0129] Optionally, the apparatus 100 for maintaining distributed metadata consistency further includes:

[0130] The verification module is used to coordinate the node to receive the target sentence, verify the target sentence, and obtain the verification result. If the verification result is correct, the target sentence is sent to the central node.

[0131] Optionally, the apparatus 100 for maintaining distributed metadata consistency further includes:

[0132] The first sending module is used to release the lock on the target object if the central node fails to operate on the target object, and send first operation failure information of the failure to operate on the target object to the coordination node.

[0133] The first ending module is used to coordinate the node to send the first operation failure information to the server, so that the server ends the operation on the target object.

[0134] Optionally, the apparatus 100 for maintaining distributed metadata consistency further includes:

[0135] The second sending module is used for, if there is a participating node or a coordinating node that fails to operate the target object, the participating node sends second operation failure information of the failed operation on the target object to the coordinating node.

[0136] The first rollback module is used for the coordination node to control the central node, the coordination node and the participating nodes to rollback the target transaction.

[0137] The second ending module is used to coordinate the node to send the second operation failure information to the server, so that the server ends the operation on the target object.

[0138] Optionally, the apparatus 100 for maintaining distributed metadata consistency further includes:

[0139] The third sending module is used for, if the central node fails to submit the target transaction, the central node sends first submission failure information of the failure to submit the target transaction to the coordination node.

[0140] The second rollback module is used for the coordination node to control the central node, the participating nodes and the coordination node to roll back the target transaction.

[0141] The third ending module is used to coordinate the node to send the first submission failure information to the server, and the server ends the operation on the target object.

[0142] Optionally, the apparatus 100 for maintaining distributed metadata consistency further includes:

[0143] The push module is used to push information of submitting the target transaction to the coordinating node or participating node if there is a coordinating node or participating node that fails to submit the target transaction.

[0144] The submission module is used for the coordinating node or the participating node to submit the target transaction according to the received information of submitting the target transaction.

[0145] Optionally, the apparatus 100 for maintaining distributed metadata consistency further includes:

[0146] The locking module is used to lock the target object in the participating nodes and the coordinating node according to the identifier of the target object.

[0147] The acquisition module is used to acquire N target participating nodes that execute data manipulation statements on the target object, where N is an integer greater than zero.

[0148] The judgment module is used to obtain the version number of the target object in the target participating node and the version number of the target object in the coordination node for any target participating node. If the version number of the target object in the target participating node is equal to the version number of the target object in the coordination node, the target participating node executes the data manipulation statement on the target object. Otherwise, the step of locking the target object in the participating node and the coordination node according to the identification of the target object is re-executed.

[0149] Optionally, the apparatus 100 for maintaining distributed metadata consistency further includes:

[0150] The first judgment module is used to judge whether the central node, the participating nodes and the coordinating node successfully submit the target transaction if the central node fails to lock the target object.

[0151] The second judgment module is used for the central node to release the lock on the target object if the central node, the participating nodes and the coordinating node successfully submit the target transaction.

[0152] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0153] Fig.11 This is a schematic diagram of the structure of a computer device provided in Example 11 of the present application. Fig.11 As shown, the computer device of this embodiment includes: at least one processor ( Fig.11 Only one is shown), a memory, and a computer program stored in the memory and executable on at least one processor, wherein when the processor executes the computer program, the steps in any of the above-mentioned method embodiments for maintaining distributed metadata consistency are implemented.

[0154] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Fig.11 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components, for example, it may also include a network interface, etc.

[0155] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0156] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of an operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments may also be an external storage device of a computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on a computer device. Further, the memory may also include both an internal storage unit of a computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as program codes of computer programs, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0157] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiment when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, USB flash drive, mobile hard disk, magnetic disk or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0158] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.

[0159] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0160] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0161] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0162] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for maintaining distributed metadata consistency, characterized in that: Applied to a distributed database, the distributed database includes a central node, a coordinating node, and at least one participating node, the central node being in communication with each participating node, the method for maintaining distributed metadata consistency includes: The central node receives a target statement, determines an identifier of a target object in the target statement, locks the target object, and operates the target object according to the identifier of the target object and the target statement; If the central node successfully operates the target object, the central node obtains the first version number of the target object; The central node sends the target statement, the identifier of the target object and the first version number to the participating nodes and the coordinating node; For any participating node, obtain the second version number of the target object correspondingly identified in the participating node and the coordinating node, and if the first version number is one greater than the second version number, perform an operation on the target object on the participating node and the coordinating node; If all participating nodes and the coordinating node successfully operate on the target object, the central node commits the target transaction for operating on the target object; If the central node successfully submits the target transaction, the participating nodes and the coordinating node submit the target transaction; If the participating nodes and the coordinating node successfully submit the target transaction, the central node releases the lock on the target object.

2. The method for maintaining distributed metadata consistency according to claim 1, wherein: The central node receives a target statement, determines an identifier of a target object in the target statement, locks the target object, and before operating the target object according to the identifier of the target object and the target statement, further includes: The coordination node receives the target statement, verifies the target statement, and obtains a verification result. If the verification result is correct, the coordination node sends the target statement to the central node.

3. The method for maintaining distributed metadata consistency according to claim 2, wherein: The distributed database further includes a server, the central node, each participating node, the coordinating node and the server are in communication connection with each other, and after the target object is operated according to the identifier of the target object and the target statement, the following steps are further included: If the central node fails to operate on the target object, the central node releases the lock on the target object and sends first operation failure information indicating that the operation on the target object fails to be performed to the coordination node; The coordination node sends the first operation failure information to the server, and the server ends the operation on the target object.

4. The method for maintaining distributed metadata consistency according to claim 3, wherein: For any participating node, obtaining a second version number of a target object corresponding to an identifier in the participating node and the coordinating node, and if the first version number is one greater than the second version number, then after operating the target object on the participating node and the coordinating node, further comprising: If the participating node or the coordinating node fails to operate the target object, the participating node sends second operation failure information indicating that the operation on the target object fails to the coordinating node; The coordinating node controls the central node, the coordinating node and the participating nodes to roll back the target transaction; The coordination node sends the second operation failure information to the server, and the server ends the operation on the target object.

5. The method for maintaining distributed metadata consistency according to claim 3, wherein: After the central node submits the target transaction for operating the target object, the method further includes: If the central node fails to submit the target transaction, the central node sends first submission failure information of the failure to submit the target transaction to the coordination node; The coordinating node controls the central node, the participating nodes and the coordinating node to roll back the target transaction; The coordination node sends the first submission failure information to the server, and the server ends the operation on the target object.

6. The method for maintaining distributed metadata consistency according to claim 3, wherein: After the participating nodes and the coordinating node submit the target transaction, the method further includes: If the coordinating node or the participating node fails to submit the target transaction, push the information of submitting the target transaction to the coordinating node or the participating node; The coordinating node or the participating node submits the target transaction according to the received information on submitting the target transaction.

7. The method for maintaining distributed metadata consistency according to claim 3, wherein: Before the central node releases the lock on the target object, the method further includes: According to the identifier of the target object, locking the target object in the participating node and the coordinating node; Obtain N target participating nodes that execute a data manipulation statement on the target object, where N is an integer greater than zero; For any target participating node, obtain the version number of the target object in the target participating node and the version number of the target object in the coordination node. If the version number of the target object in the target participating node is equal to the version number of the target object in the coordination node, the target participating node executes the data manipulation statement on the target object. Otherwise, re-execute the step of locking the target object in the participating node and the coordination node according to the identification of the target object.

8. The method for maintaining distributed metadata consistency according to claim 3, wherein: After the central node releases the lock on the target object, the method further includes: If the central node fails to lock the target object, determining whether the central node, the participating nodes, and the coordinating node successfully submit the target transaction; If the central node, the participating nodes, and the coordinating node successfully commit the target transaction, the central node releases the lock on the target object.

9. A device for maintaining distributed metadata consistency, characterized in that: Applied to a distributed database, the distributed database includes a central node, a coordinating node, and at least one participating node, the central node is in communication with each participating node, and the apparatus for maintaining distributed metadata consistency includes: A first operation module is configured to receive a target statement at the central node, determine an identifier of a target object in the target statement, lock the target object, and operate the target object according to the identifier of the target object and the target statement; an obtaining module, configured to obtain, by the central node, a first version number of the target object if the central node successfully operates the target object; A sending module, configured for the central node to send the target statement, the identifier of the target object, and the first version number to the participating nodes and the coordinating node; A second operation module is configured to obtain, for any participating node, a second version number of a target object correspondingly identified in the participating node and the coordinating node, and perform an operation on the target object on the participating node and the coordinating node if the first version number is one greater than the second version number; A first submission module is configured to enable the central node to submit a target transaction for operating the target object if all participating nodes and the coordinating node successfully operate the target object; A second submission module, configured to cause the participating nodes and the coordinating node to submit the target transaction if the central node successfully submits the target transaction; The lock releasing module is configured to enable the central node to release the lock on the target object if the participating nodes and the coordinating node successfully submit the target transaction.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for maintaining distributed metadata consistency according to claims 1 to 8 is implemented.

Citation Information

Patent Citations

  • Method and system for dynamically managing metadata of distributed file system

    CN101697168A

  • Method for controlling consistency of non-relational data after reading and writing through random sequence

    CN113535746A

  • Cloud Storage System with Distributed Metadata

    EP2557514A1

  • Method and apparatus for reconstructing and checking the consistency of deduplication metadata of a deduplication file system

    US20170031768A1

  • Enhanced locking mechanism for b+ tree data structures

    US20220261386A1