Metadata maintenance method, database cluster, program product and medium

Abstract

The embodiment of the invention discloses a metadata maintenance method, a database cluster, a program product and a medium, which are used for realizing access and maintenance of metadata in a multi-master database cluster. The method is applied to a database cluster, the database cluster comprises a first database node and a metadatabase node, and the method comprises the steps that the first database node processes a first data transaction; the first database node sends a first message to the metadatabase node; the metadatabase node processes a first metadata transaction in response to the first message, the first metadata transaction including operating on the first metadata; after the first database node submits the first data transaction, the first database node sends a second message to the metadatabase node; the metadatabase node submits the first metadata transaction in response to the second message. According to the embodiment of the invention, the metadata access and maintenance of the multi-master database cluster can be realized under the conditions that the database kernel is changed as little as possible and the cost is low.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on September 12, 2023, with application number 202311176547.2 and invention name “A metadata management method, apparatus and computing device cluster”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The embodiments of the present application relate to the technical field of database clusters, and in particular to a metadata maintenance method, a database cluster, a program product, and a medium. Background Art

[0003] In a multi-master database cluster based on shared storage and distributed databases, in a software as a service (SAAS) multi-tenant scenario, in order to provide high-concurrency data reading and writing capabilities, user database management and maintenance capabilities within the cluster, and user database rapid migration capabilities within the cluster, the database metadata needs to be managed in a unified and centralized manner, and each node in the cluster needs to have the ability to read and write metadata. Metadata is information used to describe data attributes, mainly used to describe the structure of a data table, indicate storage locations, record historical data, or find resources, etc.

[0004] A multi-master database cluster can use database middleware to maintain metadata. The database middleware configures the metadata table for each database node, and the metadata is included in the metadata table. Each database node maintains local metadata through the storage engine, and the database middleware is responsible for recording the database node corresponding to each metadata table.

[0005] Since the storage engine of the database node can only access local data, when the metadata in the database node changes, the database middleware needs to reconfigure the metadata table maintained by each database node, making the solution of maintaining metadata using the database middleware complex and difficult to implement. Summary of the invention

[0006] The embodiments of the present application provide a metadata maintenance method, a database cluster, a program product, and a medium, which can implement metadata access and maintenance of a multi-master database cluster with minimal changes to the database kernel and at low cost.

[0007] The first aspect of the embodiment of the present application provides a metadata maintenance method, which is applied to a database cluster, wherein the database cluster includes a first database node and a metadata database node, wherein the first database node is used to perform read and write operations on local data, and the metadata database node is used to perform read and write operations on metadata corresponding to the local data. In the method, the first database node processes a first data transaction, and the first data transaction includes operating on first data in the local data. The first database node determines the operation of the first metadata corresponding to the first data according to the operation of the first data, and the first database node sends a first message to the metadata database node. After receiving the first message, the metadata database node processes the first metadata transaction according to the operation determined by the first database node, and the first metadata transaction includes operating on the first metadata. When the metadata database node completes the operation on the first metadata, the first database node submits the first data transaction. When the first database node successfully submits the first data transaction, the first database node sends a second message to the metadata database node. After receiving the second message, the metadata database node submits the first metadata transaction.

[0008] In the embodiment of the present application, the first database node instructs the metadata database node to operate on the metadata. Therefore, when the metadata changes, the metadata does not need to be reconfigured, but is operated by the metadata database node, so that metadata access and maintenance of the database cluster is achieved with as little change to the database kernel as possible and at a low cost.

[0009] In some optional embodiments, the database cluster also includes a second database node. In the method, the second database node processes a second data transaction, and the second data transaction includes operating on second data in the local data. The second database node sends a third message to the metadata database node, and the metadata database node responds to the third message and processes a second metadata transaction, and the second metadata transaction includes operating on the second metadata. When the metadata database node completes the operation on the second metadata, the second database node submits the second data transaction. When the second database node successfully submits the second data transaction, the second database node sends a fourth message to the metadata database node. After receiving the fourth message, the metadata database node submits the second metadata transaction.

[0010] In this implementation, the metadata database node can process multiple metadata transactions in parallel, and process metadata according to instructions from different database nodes, thereby improving the processing efficiency of metadata transactions.

[0011] In some optional implementations, the first database node may also record an identifier of the first metadata transaction, where the identifier is used to indicate the first metadata transaction.

[0012] In some optional implementations, after the metadata database node successfully submits the first metadata transaction, the metadata database node sends a feedback message to the first database node. After receiving the feedback message, the database node deletes the identifier of the first metadata transaction recorded locally.

[0013] In the embodiment of the present application, the database node can instruct the metadata database node to commit or roll back the metadata transaction that was not successfully committed according to the identifier of the local metadata transaction when a failure occurs, thereby ensuring the atomicity of the metadata transaction.

[0014] In some optional implementations, the metadata repository node places the identifier of the first metadata transaction in the first message. After the metadata repository node receives the first message, the metadata repository node records the identifier of the first metadata transaction included in the first message.

[0015] In some optional implementations, after the metadata database node successfully submits the first metadata transaction, the metadata database node deletes the identifier of the first metadata transaction that has been persisted.

[0016] In some optional implementations, when the first database node fails, the first database node will restart, and then obtain the identifier of the third metadata transaction that has not been successfully submitted on the metadata database node. The first database node compares the obtained identifier of the third metadata transaction with the identifiers in the local metadata transaction list one by one, and determines whether the metadata transaction list includes the identifier of the third metadata transaction. If included, the first database node sends a fifth message to the metadata database node, instructing the metadata database node to submit the third metadata transaction. If not included, the first database node sends a sixth message to the metadata database node, instructing the metadata database node to roll back the third metadata transaction.

[0017] In the embodiment of the present application, by recording the metadata transaction identifier in the database node and the metadata database node, it is determined whether the metadata transaction has been successfully submitted before the database node fails. The database node instructs the metadata database node to submit or roll back the third metadata transaction according to the result of the determination, thereby ensuring the atomicity of the transaction and supporting atomic data definition language (DDL) transactions like a stand-alone database.

[0018] A second aspect of an embodiment of the present application provides a database cluster, the database cluster comprising:

[0019] A first database node, configured to process a first data transaction, wherein the first data transaction includes operating on first data;

[0020] The first database node is further used to send a first message to the metadata database node;

[0021] The metadata database node is used to process a first metadata transaction in response to the first message, where the first metadata transaction includes operating on first metadata, where the first metadata corresponds to the first data;

[0022] The first database node is further configured to send a second message to the metadata database node after the first database node commits the first data transaction;

[0023] The metadata database node is further configured to submit the first metadata transaction in response to the second message.

[0024] In some optional implementations, the database cluster further includes:

[0025] A second database node, used for processing a second data transaction, where the second data transaction includes operating on second data;

[0026] The second database node is further used to send a third message to the metadata database node;

[0027] The metadata database node is further used to process a second metadata transaction in response to the third message, where the second metadata transaction includes operating on second metadata, and the second metadata corresponds to the second data;

[0028] The second database node is further configured to send a fourth message to the metadata database node after the second database node commits the second data transaction;

[0029] The metadata database node is further configured to submit the second metadata transaction in response to the fourth message.

[0030] In some optional implementations, the first database node is further used to record an identifier of the first metadata transaction, where the identifier of the first metadata transaction is used to indicate the first metadata transaction.

[0031] In some optional implementations, the first database node is further configured to receive a feedback message from the metadata database node, the feedback message being used to indicate that the metadata database node has successfully submitted the first metadata transaction; and in response to the feedback message, delete the identifier of the first metadata transaction.

[0032] In some optional implementations, the first message includes an identifier of the first metadata transaction, and the metadata repository node is further configured to record the identifier of the first metadata transaction.

[0033] In some optional implementations, the metadata database node is further configured to delete an identifier of the first metadata transaction after the first metadata transaction is successfully submitted.

[0034] In some optional implementations, the first database node is further used to obtain an identifier of a third metadata transaction that has not been successfully submitted on the metadata database node; when the metadata transaction list of the first database node includes the identifier of the third metadata transaction, a fifth message is sent to the metadata database node, and the fifth message is used to instruct the metadata database node to submit the third metadata transaction; when the metadata transaction list of the first database node does not include the identifier of the third metadata transaction, a sixth message is sent to the metadata database node, and the sixth message instructs the metadata database node to roll back the third metadata transaction.

[0035] A third aspect of an embodiment of the present application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method described in the first aspect above.

[0036] A fourth aspect of an embodiment of the present application provides a computer program product comprising instructions, wherein when the instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect above.

[0037] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, characterized in that it includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A system architecture diagram of a metadata maintenance method in an embodiment of the present application;

[0039] Figure 2 This is a schematic diagram of an embodiment of a metadata maintenance method in an embodiment of the present application;

[0040] Figure 3 This is a schematic diagram of another embodiment of a metadata maintenance method in an embodiment of the present application;

[0041] Figure 4 This is a schematic diagram of another embodiment of a metadata maintenance method in an embodiment of the present application;

[0042] Figure 5 This is a schematic diagram of another embodiment of a metadata maintenance method in an embodiment of the present application;

[0043] Figure 6 Another system architecture diagram of a metadata maintenance method in an embodiment of the present application;

[0044] Figure 7This is a schematic diagram of the structure of a computing device in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of the structure of a computing device cluster in an embodiment of the present application;

[0046] Fig. 9 Another structural diagram of a computing device cluster in an embodiment of the present application. DETAILED DESCRIPTION

[0047] Embodiments of the present application provide a metadata maintenance method, a database cluster, a program product, and a medium for implementing access to and maintenance of metadata in a multi-master database cluster.

[0048] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0049] The terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0050] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0051] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0052] See also Figure 1 , the following briefly describes the system architecture based on which the metadata maintenance method in the embodiment of the present application is based:

[0053] The system architecture is a database cluster architecture, which includes multiple master nodes, at least one metadata database node and shared storage. The master node is also called a database node. In practical applications, the database cluster may include multiple database nodes or multiple metadata database nodes. In the embodiment of the present application, two database nodes and one metadata database node are taken as an example. Figure 1 As shown, the data in the shared storage is divided into multiple databases, including database A, database B, database C and database D. Each database belongs to only one database node. For example, database A and database B are accessed and maintained by database node 1, and database C and database D are accessed and maintained by database node 2.

[0054] The metadata corresponding to the data in the database is uniformly stored in the metadata repository, and is accessed and maintained by the metadata repository nodes. In this multi-master database cluster, the database nodes are responsible for operating the data, the metadata repository nodes are responsible for operating the metadata corresponding to the data, and the shared storage is responsible for storing the data and the corresponding metadata. The operation on the metadata is defined by the database node and implemented by the metadata repository node. In the embodiment of the present application, the operation on the data includes one or more of writing data, deleting data or modifying data, and the operation on the metadata includes one or more of writing metadata, deleting metadata or modifying metadata.

[0055] The following is a description of a metadata maintenance method in an embodiment of the present application:

[0056] See also Figure 2 , Figure 2 A schematic diagram of a metadata maintenance method in an embodiment of the present application.

[0057] The first database node 201 and the second database node 202 operate on the data. The database node includes a metadata storage engine, which implements part of the storage engine interface required by the structured query language (SQL), meets the needs of each node to read and write metadata tables, and enables the database node to instruct the metadata database node to operate the metadata. The meaning of the metadata storage engine is the same as that of the original storage engine, but the implementation method is different from the original storage engine. The metadata storage engine can read data from other databases like a client, so that when the database needs to be migrated, the metadata corresponding to the database does not need to be fully reallocated, but can be used to instruct the metadata database node to implement one or more operations of adding, deleting or modifying through the metadata storage engine.

[0058] In the specific implementation, since there is no need to modify the code logic of the existing storage engine, but directly replace the storage engine for the metadata table, there are few changes to the parsing execution layer of the database kernel, and the focus is on the implementation of the metadata storage engine, which makes it easier to develop and maintain during the development process, and the possibility of affecting existing functions due to the introduction of problems is low. Therefore, metadata access and maintenance of multi-master database clusters can be achieved with as little change to the database kernel as possible and at a low cost.

[0059] When a new data table is added to the database, new metadata corresponding to the data table needs to be written, or when a data table is deleted from the database, the metadata corresponding to the data table needs to be deleted. At this time, the database node obtains the metadata in the metadata database through the metadata database node, and the database node determines whether metadata can be added or deleted in the metadata database.

[0060] When the structure of a data table is changed, such as adding or deleting a column in the data table, the metadata corresponding to the data table needs to be modified. At this time, the database node obtains the corresponding metadata in the metadata database through the metadata database node, and the database node instructs the metadata database node to modify the metadata.

[0061] The first database node 201 or the second database node 202 obtains metadata from the shared storage 204 through the metadata database node 203. The first database node 201 or the second database node 202 instructs the metadata database node 203 to operate the metadata, including adding new metadata, deleting metadata, or modifying metadata, and then the metadata database node 203 submits the operation. This operation is a metadata transaction.

[0062] The following takes the first database node as an example to describe the operation process of the database node and the metadata database node on the metadata. Figure 3 , Figure 3 This is a schematic diagram of an embodiment of a database node obtaining metadata in an embodiment of the present application.

[0063] 301. A first database node processes a first data transaction;

[0064] The first database node processes a first data transaction, and the first data transaction includes operating the local first data, including one or more of adding the first data, deleting the first data, or modifying the first data. When the operation of the first database node on the first data involves a change in the structure of a data table, such as adding a data table, deleting a data table, or modifying the structure of a data table, the first metadata corresponding to the data table also needs to be written, deleted, or modified.

[0065] 302. The first database node sends a request message to the metadata database node;

[0066] The first database node sends a request message to the metadata database node, requesting the metadata database node to send the first metadata to the first database node.

[0067] 303. The metadata database node obtains first metadata;

[0068] The metadatabase node obtains the first metadata from the shared storage in response to the request message and stores the first metadata in the local node.

[0069] 304. The metadata database node sends first metadata to the first database node;

[0070] The metadata database node sends the first metadata to the first database node. After receiving the first metadata, the first database node saves the first metadata to the local node. The first database node determines whether it can be modified based on the first metadata. For example, the first database node needs to add a data table named user. At this time, the first metadata is all the metadata in the metadata database. The first database node determines whether there is metadata in the first metadata that is the same as the metadata corresponding to the data table. If it exists, the metadata cannot be written and the database will not perform subsequent operations; if it does not exist, the first database node can operate on the first metadata.

[0071] See also Figure 4 , Figure 4 This is a schematic diagram of an embodiment of the database node operating the first metadata in an embodiment of the present application.

[0072] 401. A first database node generates an identifier of a first metadata transaction;

[0073] The first database node operates the first metadata through the metadata database node, and the operation is a first metadata transaction. The first database node generates an identifier corresponding to the first metadata transaction and records the identifier.

[0074] 402. The first database node sends a first message to the metadata database node;

[0075] The first message includes the identifier of the first metadata transaction, and the first message is used to instruct the metadata database node to start the first metadata transaction. The metadata database node records the identifier of the first metadata transaction and establishes a connection, through which the first database node instructs the metadata database node to operate on the first metadata.

[0076] 403. The first database node sends a target message to the metadata database node;

[0077] The first metadata transaction is initiated by the first database node, so the operation of the first metadata is defined by the first database node, that is, the first database node determines how to operate the first metadata. For example, if the first metadata needs to be modified to obtain the target metadata, the first database node needs to determine the target metadata and send the modification requirement to the metadata database node through a target message.

[0078] In actual applications, since the operation on the first metadata includes one or more of adding the first metadata, deleting the first metadata, or modifying the first metadata, the target message may include multiple messages, which is not specifically limited here.

[0079] 404. The metadata database node processes the first metadata transaction;

[0080] After receiving the target message, the metadata database node operates on the first metadata according to the target message, and the operation on the first metadata corresponds to the operation on the first data. For example, if the first database node needs to add a column of age in the data table user, the metadata database node increases the number of columns in the first metadata by one, and adds the column name and data type of the column. The column name of the column is age, and the data type is integer.

[0081] 405. The first database node sends a persistence message to the metadata database node;

[0082] When all target messages have been sent, the first data node sends a persistence message to the metadata repository node, where the persistence message is used to instruct the metadata repository node to persist the first metadata transaction.

[0083] 406. The metadata database node persists the first metadata transaction;

[0084] When the metadata database node receives the persistence message and completes the first metadata transaction, the metadata database node persists the first metadata transaction, that is, saves the operation on the first metadata to the permanent storage medium of the metadata database node. In actual applications, the metadata database node persists the operation on the first metadata to the log layer of the shared storage. At this time, the identifier of the first metadata transaction is also persisted, and the first metadata transaction belongs to a transaction that has not been successfully submitted.

[0085] In some possible implementations, the metadata repository node marks the first metadata transaction as being in a prepare state, indicating that the transaction has not been committed.

[0086] 407. The metadata database node sends a response message to the first database node;

[0087] After making the first metadata transaction persistent, the metadata database node sends a response message to the first database node to instruct the first database node to make the first metadata transaction persistent.

[0088] 408. The first database node persists the first data transaction;

[0089] After receiving the response message, the first database node persists the first data transaction to the shared storage. At this time, the identifier of the first metadata transaction is persisted to the metadata transaction list of the first database node. In actual applications, the identifier of the first metadata transaction can also be persisted to other carriers such as files, which is not limited here.

[0090] In some possible implementations, the first database node marks the first data transaction as being in a prepare state, indicating that the transaction has not been committed.

[0091] 409. The first database node submits the first data transaction;

[0092] The first database node submits the first data transaction, which makes the first data transaction formally effective and visible to the outside.

[0093] In some possible implementations, the first database node marks the first data transaction as a committed state, indicating that the transaction has been committed.

[0094] 410. The first database node sends a second message to the metadata database node;

[0095] After the first database node successfully submits the first data transaction, the first database node sends a second message to the metadata database node to instruct the metadata database node to submit the first metadata transaction.

[0096] 411. The metadata database node submits the first metadata transaction;

[0097] After receiving the second message, the metadata database node submits the first metadata transaction in response to the second message, so that the first metadata transaction is officially effective and visible to the outside.

[0098] In some possible implementations, the first metadata database node marks the first metadata transaction as a committed state, indicating that the transaction has been committed.

[0099] 412. The metadata database node deletes the identifier of the first metadata transaction;

[0100] After the metadata database node successfully submits the first metadata transaction, the metadata database node deletes the identifier of the first metadata transaction. At this time, the first metadata transaction does not belong to the transaction that has not been successfully submitted.

[0101] 413. The metadata database node sends a feedback message to the first database node;

[0102] After the metadata database node successfully submits the first metadata transaction, the metadata database node sends a feedback message to the first database node to instruct the first database node to delete the identifier of the first metadata transaction.

[0103] In this embodiment, the order of step 412 and step 413 is not limited. In practical applications, step 413 can be performed first, and then step 412, which is not limited here.

[0104] 414. The first database node deletes the identifier of the first metadata transaction;

[0105] After receiving the feedback message from the metadata database node, the first database node deletes the identifier of the first metadata transaction.

[0106] In the embodiment of the present application, the first database node instructs the metadata database node to operate on the metadata. Therefore, when the metadata changes, the metadata does not need to be completely reconfigured. Instead, the metadata database node operates to add, delete or modify the first metadata, thereby achieving metadata access and maintenance of the database cluster with as little change to the database kernel as possible and at a low cost.

[0107] When the first database node is running, a failure may occur, causing the database node to restart. After the first database node restarts, the non-persistent transactions will be lost. At this time, the first database node decides to commit or roll back the transaction on the metadata database node based on the restored metadata transaction identifier list.

[0108] See also Figure 5 , Figure 5 This is a schematic diagram of an embodiment of recovery when a database node fails in an embodiment of the present application;

[0109] 501. Restore local affairs;

[0110] When the first database node restarts, it reads the local transaction from the shared storage log layer. If the local transaction has not been fully persisted before the failure is sent, the database node can only read part of the local transaction from the shared storage log layer. At this time, the database node rolls back the local transaction, that is, restores the local transaction to a state that has not yet occurred. If the local transaction has been persisted before the failure occurs, the first database node obtains the local transaction from the shared storage.

[0111] 502. Obtain a metadata transaction identifier list;

[0112] After the first database node recovers the local transaction, the first database node obtains a metadata transaction identifier list, where the metadata transaction identifier list is used to store identifiers of metadata transactions that have been persisted and have not been successfully committed.

[0113] In practical applications, the identifier of the metadata transaction may be stored in a file or other carrier, and the first database node obtains the file.

[0114] 503. Obtain an identifier of a third metadata transaction that has not been successfully submitted on the metadata repository node;

[0115] The first database node requests the metadata database node to send a metadata transaction identifier list of the metadata database node to the first database node, and the metadata transaction identifier list is used to store identifiers of metadata transactions that have been persisted and have not been successfully submitted. If the metadata database node has persisted the third metadata transaction when the database node fails, the identifier of the third metadata transaction is included in the list.

[0116] 504. Determine whether the metadata transaction list includes an identifier of the third metadata transaction;

[0117] When the first database node fails, step 408 has not been executed, that is, the local transaction has not been persisted, and the metadata transaction list of the first database node does not include the identifier of the third metadata transaction. At this time, the first database node executes step 506.

[0118] When the first database node fails, step 408 has been executed, that is, the local transaction has been persisted, and the metadata transaction list of the first database node includes the identifier of the third metadata transaction. At this time, the first database node executes step 505.

[0119] The identifier of the third metadata transaction includes the identifier of the first database node, so the first database node can obtain the corresponding identifier of the third metadata transaction from the metadata transaction identifier list of the metadata node.

[0120] 505. Instruct the metadata database node to submit the third metadata transaction;

[0121] The first database node sends a fifth message to the metadata database node, and the metadata database node submits the third metadata transaction in response to the fifth message, so that the third metadata transaction is officially effective and visible to the outside.

[0122] 506. Instruct the metadata database node to roll back the third metadata transaction;

[0123] The first database node sends a sixth message to the metadata database node. In response to the sixth message, the metadata database node obtains a record of the third metadata transaction in the log layer of the shared storage, and rolls back the third metadata transaction based on the operation on the third metadata in the record, that is, restores the third metadata transaction to a state that has not yet occurred.

[0124] 507. Delete the identifier of the persistent metadata transaction.

[0125] When the third metadata transaction is successfully committed or rolled back, the first database node instructs the metadata database node to delete the identifier of the third metadata transaction, and the recovery is completed.

[0126] In the embodiment of the present application, the consistency of the first database node and the metadata database node on the metadata transaction is guaranteed through two-phase commit, so that the metadata transaction that has become a distributed transaction still maintains atomicity, that is, all operations of a transaction are either executed uninterruptedly or none of them are executed.

[0127] The present application also provides a database cluster, such as Figure 6 As shown, including:

[0128] A first database node 601 is used to process a first data transaction, where the first data transaction includes operating on first data;

[0129] The first database node 601 is further used to send a first message to the metadata database node;

[0130] The metadata database node 602 is used to process a first metadata transaction in response to the first message, where the first metadata transaction includes operating on first metadata, where the first metadata corresponds to the first data;

[0131] The first database node 601 is further configured to send a second message to the metadata database node after the first database node commits the first data transaction;

[0132] The metadata database node 602 is further configured to submit the first metadata transaction in response to the second message.

[0133] The first database node and the metadata database node can be implemented by software or hardware. Exemplarily, the implementation of the first database node is described below. Similarly, the implementation of the metadata database node can refer to the implementation of the first database node.

[0134] As an example of a software functional unit, the first database node may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the above computing instance may be one or more. For example, the first database node may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0135] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0136] As an example of a hardware functional unit, the first database node may include at least one computing device, such as a server, etc. Alternatively, the first database node may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0137] The multiple computing devices included in the first database node can be distributed in the same region or in different regions. The multiple computing devices included in the first database node can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first database node can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0138] The present application also provides a computing device 700. Figure 7 As shown, computing device 700 includes: bus 701, processor 702, memory 703 and communication interface 704. Processor 702, memory 703 and communication interface 704 communicate through bus 701. Computing device 700 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 700.

[0139] The bus 701 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus 702 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 702 may include a path for transmitting information between various components of the computing device 700 (eg, the memory 703, the processor 702, and the communication interface 704).

[0140] The processor 702 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0141] The memory 703 may include a volatile memory, such as a random access memory (RAM). The processor 702 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0142] The memory 703 stores executable codes, and the processor 702 executes the executable codes to respectively implement the functions of the first database node and the metadata database node, thereby implementing the metadata maintenance method. That is, the memory 703 stores instructions for executing the metadata maintenance method.

[0143] The communication interface 704 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 700 and other devices or a communication network.

[0144] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0145] like Figure 8 As shown, the computing device cluster includes at least one computing device 700. The memory 703 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the metadata maintenance method.

[0146] In some possible implementations, the memory 703 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the metadata maintenance method. In other words, the combination of one or more computing devices 700 may jointly execute instructions for executing the metadata maintenance method.

[0147] It should be noted that the memory 703 in different computing devices 700 in the computing device cluster can store different instructions for executing part of the functions of the database cluster. That is, the instructions stored in the memory 703 in different computing devices 700 can implement the functions of one or more nodes in the first database node and the metadata database node.

[0148] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig. 9 A possible implementation is shown. Fig. 9 As shown, two computing devices 700A and 700B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device. In this type of possible implementation, the memory 703 in the computing device 700A stores instructions for executing the functions of the first database node. At the same time, the memory 703 in the computing device 700B stores instructions for executing the functions of the metadata database node.

[0149] It should be understood that Fig. 9 The functions of the computing device 700A shown in FIG. 7 may also be completed by multiple computing devices 700. Similarly, the functions of the computing device 700B may also be completed by multiple computing devices 700.

[0150] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the metadata maintenance method, or the metadata maintenance method.

[0151] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct a computing device to execute a metadata maintenance method, or instructs a computing device to execute a metadata maintenance method.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A metadata maintenance method, characterized in that: The method is applied to a database cluster, the database cluster includes a first database node and a meta-database node, and the method includes: The first database node processes a first data transaction, where the first data transaction includes operating on first data; The first database node sends a first message to the metadata database node; The metadata database node processes a first metadata transaction in response to the first message, where the first metadata transaction includes operating on first metadata, where the first metadata corresponds to the first data; After the first database node submits the first data transaction, the first database node sends a second message to the metadata database node; The metadata database node commits the first metadata transaction in response to the second message.

2. The method according to claim 1, characterized in that The database cluster further includes a second database node, and the method further includes: The second database node processes a second data transaction, where the second data transaction includes operating on second data; The second database node sends a third message to the metadata database node; The metadata repository node processes a second metadata transaction in response to the third message, the second metadata transaction comprising operating on second metadata, the second metadata corresponding to the second data; After the second database node submits the second data transaction, the second database node sends a fourth message to the metadata database node; The metadata database node commits the second metadata transaction in response to the fourth message.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: The first database node records an identifier of the first metadata transaction, where the identifier of the first metadata transaction is used to indicate the first metadata transaction.

4. The method according to claim 3, characterized in that The method further comprises: The first database node receives a feedback message from the metadata database node, where the feedback message is used to indicate that the metadata database node has successfully submitted the first metadata transaction; The first database node deletes the identifier of the first metadata transaction in response to the feedback message.

5. The method according to any one of claims 1 to 4, characterized in that The first message includes an identification of the first metadata transaction; After the metadata database node receives the first message, the method further includes: The metadata repository node records the identifier of the first metadata transaction.

6. The method according to claim 5, characterized in that The method further comprises: After successfully submitting the first metadata transaction, the metadata database node deletes the identifier of the first metadata transaction.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: The first database node obtains an identifier of a third metadata transaction that has not been successfully submitted on the metadata database node; When the metadata transaction list of the first database node includes the identifier of the third metadata transaction, the first database node sends a fifth message to the metadata database node, where the fifth message is used to instruct the metadata database node to submit the third metadata transaction; When the metadata transaction list of the first database node does not include the identifier of the third metadata transaction, the first database node sends a sixth message to the metadata database node, where the sixth message instructs the metadata database node to roll back the third metadata transaction.

8. A database cluster, characterized in that: The database cluster includes: A first database node, configured to process a first data transaction, wherein the first data transaction includes operating on first data; The first database node is further used to send a first message to the metadata database node; The metadata database node is configured to process a first metadata transaction in response to the first message, where the first metadata transaction includes operating on first metadata, where the first metadata corresponds to the first data; The first database node is further configured to send a second message to the metadata database node after the first database node submits the first data transaction; The metadata database node is further configured to submit the first metadata transaction in response to the second message.

9. The database cluster according to claim 8, characterized in that: The database cluster also includes: A second database node, configured to process a second data transaction, wherein the second data transaction includes operating on second data; The second database node is further used to send a third message to the metadata database node; The metadata database node is further configured to process a second metadata transaction in response to the third message, where the second metadata transaction includes operating on second metadata, and the second metadata corresponds to the second data; The second database node is further configured to send a fourth message to the metadata database node after the second database node commits the second data transaction; The metadata database node is further configured to submit the second metadata transaction in response to the fourth message.

10. The database cluster according to claim 8 or 9, characterized in that: The first database node is also used for: An identifier of the first metadata transaction is recorded, where the identifier of the first metadata transaction is used to indicate the first metadata transaction.

11. The database cluster according to claim 10, characterized in that: The first database node is also used for: receiving a feedback message from the metadata repository node, wherein the feedback message is used to indicate that the metadata repository node has successfully submitted the first metadata transaction; In response to the feedback message, an identifier of the first metadata transaction is deleted.

12. The database cluster according to any one of claims 8 to 11, characterized in that: The first message includes an identification of the first metadata transaction; The metadata database node is further used to record the identifier of the first metadata transaction.

13. The database cluster according to claim 12, characterized in that: The metadata repository node is also used to: After the metadata database node successfully submits the first metadata transaction, the identifier of the first metadata transaction is deleted.

14. The database cluster according to any one of claims 8 to 13, characterized in that: The first database node is also used for: Acquire an identifier of a third metadata transaction that has not been successfully submitted on the metadata repository node; When the metadata transaction list of the first database node includes the identifier of the third metadata transaction, sending a fifth message to the metadata database node, where the fifth message is used to instruct the metadata database node to submit the third metadata transaction; When the metadata transaction list of the first database node does not include the identifier of the third metadata transaction, a sixth message is sent to the metadata database node, wherein the sixth message instructs the metadata database node to roll back the third metadata transaction.

15. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 7.

16. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to any one of claims 1 to 7.

17. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method as claimed in any one of claims 1 to 7.

Images