Metadata processing method, device and electronic device

Abstract

The present invention provides a metadata processing method, device and electronic device, which relates to the field of data processing technology. The method includes receiving a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table; sending a data table creation instruction to the database to create the target data table through the database; if the database returns a successful data table creation message, synchronizing the metadata information to each computing node in the database; if the database returns a message indicating that the metadata information synchronization is completed, sending confirmation information that the target data table is successfully created to the database. The embodiment of the present invention can ensure the view consistency of each computing node in a distributed database on the metadata of the library table, and improve the service quality of the database.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a metadata processing method, device and electronic device. Background Art

[0002] In a distributed database, the consistency of computing node metadata is related to the integrity of business access data and the consistency of views.

[0003] At present, in order to ensure the consistency of metadata of newly created library tables on each computing node in a distributed database, one of the solutions is based on metadata center synchronization. Specifically, this solution requires the computing node to synchronize the library table metadata to the metadata center when creating the library table, and then other computing nodes synchronize the newly created library table metadata information from the metadata center. In this way, the visible time of the newly created library table metadata on each computing node may be inconsistent. For example, after the user creates a library table on one of the computing nodes A, if a network anomaly occurs and reconnects to another computing node B, the computing node B may not have synchronized the metadata in time, which makes the user unable to access the newly created library table through the computing node B, resulting in a flashing problem.

[0004] In general, the way that metadata of newly created libraries and tables is processed in existing distributed databases may cause inconsistent visibility time of metadata of newly created libraries and tables on various computing nodes, resulting in reduced consistency of library and table views between computing nodes. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a metadata processing method, device and electronic device, which can ensure that each computing node in a distributed database has a consistent view of the metadata of the library table. Figure 1 Consistency and improve the service quality of the database.

[0006] In a first aspect, an embodiment of the present invention provides a metadata processing method, which is applied to a metadata server, and the metadata server is communicatively connected to a preset database, and a plurality of computing nodes are running on the database, the method comprising: receiving a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table; sending a data table creation instruction to the database to create the target data table through the database; if information indicating successful creation of the data table is received from the database, synchronizing the metadata information to each computing node in the database; if information indicating completion of synchronization of the metadata information is received from the database, sending confirmation information indicating successful creation of the target data table to the database.

[0007] In an optional embodiment of the present invention, the above-mentioned target data table includes a main table and multiple sub-tables split from the main table, and multiple storage nodes are running on the database; the step of sending a data table creation instruction to the database to create the target data table through the database includes: starting a first distributed transaction, and creating the multiple sub-tables in the above-mentioned multiple storage nodes through the first distributed transaction.

[0008] In an optional embodiment of the present invention, after the above-mentioned step of sending a data table creation instruction to the database to create the target data table through the database, the method further includes: if information indicating that the data table creation failed is received from the database, sending confirmation information indicating that the target data table creation failed to the database.

[0009] In an optional embodiment of the present invention, the above step of synchronizing the metadata information to each computing node in the database includes: starting a second distributed transaction, and synchronizing the metadata information to each computing node in the database through the second distributed transaction.

[0010] In an optional embodiment of the present invention, the above method also includes: if a registration request for a target node is received from a database, determining whether there is a data table creation process running in the metadata server; if not, adding a metadata change lock to the metadata information in the metadata server; registering the target node with the metadata server; synchronizing the metadata information in the metadata server to the successfully registered target node, and unlocking the metadata change lock.

[0011] In an optional embodiment of the present invention, before the above-mentioned step of registering the target node with the metadata server, the method further includes: if it is determined that there is a data table creation process running in the metadata server, monitoring the status of the running data table creation process; if it is monitored that the data table creation process ends, adding a metadata change lock to the metadata information in the metadata server.

[0012] In an optional embodiment of the present invention, after the above-mentioned step of adding a metadata change lock to the metadata information in the metadata server, the method further includes: if a data table creation request is received, caching the data table creation request in a request queue; and, after the step of unlocking the metadata change lock, the method further includes: continuing to process the data table creation requests that have been cached in the request queue.

[0013] In an optional embodiment of the present invention, the target node is a database computing node or a database storage node.

[0014] In a second aspect, an embodiment of the present invention further provides a metadata processing method, which is applied to a database, the database is communicatively connected to a preset metadata server, and a plurality of computing nodes are running on the database, the method comprising: if a user's creation request for a target data table is received, the creation request is forwarded to the metadata server; wherein the creation request carries metadata information of the target data table; if a data table creation instruction sent by the metadata server is received, the target data table is created through the database; if the target data table is successfully created, information on successful creation of the data table is sent to the metadata server, so that the metadata server synchronizes the metadata information to each computing node in the database after receiving the information on successful creation of the data table; if it is monitored that each computing node in the database has completed synchronization of the metadata information, information on completion of synchronization of the metadata information is sent to the metadata server; if confirmation information on successful creation of the target data table sent by the metadata server is received, confirmation information on successful creation of the target data table is returned to the user.

[0015] In an optional embodiment of the present invention, the above-mentioned target data table includes a main table and multiple sub-tables split from the main table, and multiple storage nodes are running on the database; the above-mentioned step of creating the target data table through the database includes: creating the multiple sub-tables to the multiple storage nodes through a third distributed transaction; the above-mentioned step of synchronizing the metadata information to each computing node in the database includes: synchronizing the metadata information to each computing node in the database through a fourth distributed transaction.

[0016] In a third aspect, an embodiment of the present invention further provides a metadata processing device, which is applied to a metadata server, and the metadata server is communicatively connected to a preset database, and a plurality of computing nodes are running on the database. The device comprises: a data table creation request receiving module, which is used to receive a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table; a data table creation instruction sending module, which is used to send a data table creation instruction to the database to create the target data table through the database; a metadata information synchronization module, which is used to synchronize the metadata information to each computing node in the database if information indicating successful creation of the data table is received from the database; and a confirmation information sending module, which is used to send confirmation information indicating successful creation of the target data table to the database if information indicating completion of synchronization of the metadata information is received from the database.

[0017] In a fourth aspect, an embodiment of the present invention further provides a metadata processing device, which is applied to a database, the database is communicatively connected to a preset metadata server, and a plurality of computing nodes are running on the database, the device comprising: a creation request forwarding module, for forwarding a creation request for a target data table to the metadata server if a creation request for the target data table is received from a user; wherein the creation request carries metadata information of the target data table; a target data table creation module, for creating the target data table through the database if a data table creation instruction sent by the metadata server is received; a first information sending module, for sending information of successful creation of the data table to the metadata server if the target data table is successfully created, so that the metadata server synchronizes the metadata information to each computing node in the database after receiving the information of successful creation of the data table; a second information sending module, for sending information of completion of synchronization of the metadata information to the metadata server if it is monitored that each computing node in the database has completed synchronization of the metadata information; and a third information sending module, for returning confirmation information of successful creation of the target data table to the user if confirmation information of successful creation of the target data table is received from the metadata server.

[0018] In a fifth aspect, an embodiment of the present invention further provides an electronic device, which includes a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-mentioned metadata processing method.

[0019] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned metadata processing method.

[0020] The embodiments of the present invention bring the following beneficial effects:

[0021] The embodiments of the present invention provide a metadata processing method, device and electronic device, which receive a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table; sends a data table creation instruction to the database to create the target data table through the database; if the database receives information indicating that the data table has been successfully created, synchronizes the metadata information to each computing node in the database; if the database receives information indicating that the metadata information synchronization is complete, sends confirmation information indicating that the target data table has been successfully created to the database. In this manner, when it is necessary to create a target data table on the database, the metadata server coordinates the creation process of the target data table. Only when the target data table is successfully created in the database, the metadata server starts synchronizing the cloud data information to each computing node in the database, and only when each computing node in the database completes the synchronization of the cloud data information, is it confirmed that the data table has been successfully created. In this way, the visibility of the library table metadata by each computing node in the distributed database can be guaranteed. Figure 1 Consistency and improve the service quality of the database.

[0022] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0023] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A schematic diagram of a process flow of a metadata processing method provided by an embodiment of the present invention;

[0026] Figure 2 A flowchart of a method for dynamically expanding an engine in a database provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of a flow chart of another metadata processing method provided by an embodiment of the present invention;

[0028] Figure 4A schematic diagram of an application scenario of a metadata processing method provided by an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of the structure of a metadata processing device provided by an embodiment of the present invention;

[0030] Figure 6 A schematic diagram of the structure of another metadata processing device provided by an embodiment of the present invention;

[0031] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.

[0032] Icons: 51-data table creation request receiving module; 52-data table creation instruction sending module; 53-metadata information synchronization module; 54-confirmation information sending module; 61-creation request forwarding module; 62-target data table creation module; 63-first information sending module; 64-second information sending module; 65-third information sending module; 71-processor; 72-memory; 73-bus; 74-communication interface. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] During the deployment of a distributed database cluster, several computing node instances are generally deployed. When a user creates a library table through any of the computing nodes, it is necessary to ensure that all computing node instances in the cluster can synchronize the metadata information of the newly created library table to ensure that all users can read, write and access the newly created library table through each computing node.

[0035] One of the methods in the related art is a single-point library table creation solution, which requires that all library table creation operations can only be performed through a single-point computing node (that is, the main computing node), and other computing nodes then synchronize the metadata of the newly created library table from the main computing node. However, this method cannot meet the needs of non-main computing nodes to dynamically create library tables during operation. Another method is a solution based on metadata center synchronization, as described in the background technology, which may cause the visible time of the metadata of the newly created library table to be inconsistent for each computing node, resulting in reduced consistency of the library table view between computing nodes.

[0036] Based on this, the embodiments of the present invention provide a metadata processing method, device and electronic device, which can be applied to application scenarios where metadata information of a data table needs to be maintained, such as creating and modifying a database table. To facilitate understanding of this embodiment, a metadata processing method disclosed in an embodiment of the present invention is first introduced in detail.

[0037] See also Figure 1 , which is a flowchart of a metadata processing method provided by an embodiment of the present invention, wherein the method is applied to a metadata server, and the metadata server is connected to a preset database in communication, and a plurality of computing nodes are running on the database. Figure 1 As shown, the method comprises the following steps:

[0038] Step S102: receiving a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table.

[0039] Here, the metadata server can be a metadata server, or a data table or service process with interactive functions running on a computer / server, and provides metadata services through the data table or service process. In some embodiments, the metadata server can act as a metadata management center.

[0040] When a user initiates a creation request for a target data table to any of the multiple computing nodes on the database, the database forwards the creation request to the metadata server, which receives the creation request and obtains metadata information of the target data table from the creation request. The metadata information is information used to describe the attributes of the data table, and is used to support functions such as indicating storage location, historical data, resource search, and file records.

[0041] Step S104: sending a data table creation instruction to the database to create a target data table through the database.

[0042] In one possible implementation manner, the target data table includes a main table and multiple sub-tables split from the main table, and multiple storage nodes are running on the database; the target data table can be created on the database in the following manner:

[0043] The metadata server starts a first distributed transaction, and creates the above-mentioned multiple sub-tables in the multiple storage nodes through the first distributed transaction. Here, since multiple sub-tables are created through distributed transactions, either all sub-tables are created successfully, or if any sub-table fails to be created, all sub-tables fail to be created. Moreover, when any sub-table in the above-mentioned multiple sub-tables fails to be created, the sub-tables in the above-mentioned multiple sub-tables created in other storage nodes are all deleted.

[0044] Step S106: If the information indicating that the data table is successfully created is received from the database, the metadata information is synchronized to each computing node in the database.

[0045] When the database successfully creates the target data table, it returns information about the successful creation of the data table to the metadata server. At this time, the metadata server receives the information about the successful creation of the data table and triggers the synchronization of the metadata information of the target data table to each computing node in the database.

[0046] Here, if the metadata server receives the data table creation failure information returned by the database, it indicates that the target data table is not successfully created on the database. At this time, the metadata server sends confirmation information (Acknowledge character, ACK) of the target data table creation failure to the database.

[0047] In at least one possible implementation, the metadata server may synchronize the metadata information to each computing node in the database in the following manner:

[0048] A second distributed transaction is started, and the metadata information is synchronized to each computing node in the database through the second distributed transaction. Here, similar to the aforementioned method of creating multiple sub-tables to multiple storage nodes through the first distributed transaction, when synchronizing the metadata information to each computing node in the multiple computing nodes, either all synchronizations are successful, and the database returns the metadata information synchronization completion / success information to the metadata server; or all synchronizations fail. Here, if one computing node fails to synchronize the metadata information, all synchronizations fail, and the metadata information that has been synchronized to the computing node is deleted accordingly, and the database returns the metadata information synchronization failure information to the metadata server.

[0049] Step S108: If the metadata information synchronization completion information is received from the database, a confirmation message that the target data table is successfully created is sent to the database.

[0050] When the metadata information of the above-mentioned target data table has been synchronized to every computing node in the database, the database sends a message that the synchronization of the metadata information is completed to the metadata server. After receiving the message that the synchronization of the metadata information is completed, the metadata server sends a confirmation message that the target data table has been successfully created to the database.

[0051] In this way, after the above steps S102 to S108, when it is necessary to create a target data table on the database, the metadata server coordinates the creation process of the target data table. Only when the target data table is successfully created in the database, the metadata server starts to synchronize the cloud data information to each computing node in the database, and only when each computing node in the database completes the synchronization of the cloud data information, it is confirmed that the data table is finally created successfully. In this way, it can ensure that each computing node in the distributed database has a consistent view of the library table metadata. Figure 1 Consistency and improve the service quality of the database.

[0052] Compared with the method of performing metadata synchronization based on a metadata center in the related art, the metadata processing method in this embodiment is coordinated and controlled by a metadata server. When the target data table is successfully created, the metadata server controls the synchronization of the metadata information of the target data table to each computing node in the database. The metadata information synchronization here is performed uniformly, and only when each computing node has synchronized the metadata information is the successful creation of the target data table confirmed, thereby ensuring that after the user receives the information that the data table has been successfully created, each computing node in the database has successfully synchronized the metadata information of the target database, thereby avoiding the problem in the related art that each computing node synchronizes the metadata information of the newly created library table from the metadata center, resulting in inconsistent synchronization of information by each computing node, resulting in inconsistent visible time of the metadata of the newly created library table by each computing node.

[0053] The embodiment of the present invention provides a metadata processing method, which receives a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table; sends a data table creation instruction to the database to create the target data table through the database; if the database returns a message indicating that the data table has been successfully created, synchronizes the metadata information to each computing node in the database; if the database returns a message indicating that the metadata information synchronization is complete, sends confirmation information that the target data table has been successfully created to the database. This method supports concurrent creation of library tables by multiple nodes, and can ensure that each computing node in the distributed database has consistent visibility of the library table metadata. Figure 1 Consistency and improve the service quality of the database.

[0054] exist Figure 1 Based on the metadata processing method shown in FIG. 1 , this embodiment also provides a better method for extending the engine in the database, see Figure 2 , shown is a flow chart of a method for dynamically expanding an engine in a database, the method is applied to a metadata server, the metadata server is communicated with a preset database, and multiple computing nodes are running on the database.

[0055] Depend on Figure 2 It can be seen that the method comprises the following steps:

[0056] Step S202: If a registration request for a target node is received from a database, determine whether there is a data table creation process running in the metadata server; if not, execute step S204; if yes, execute step S206.

[0057] The target node may be a database computing node or a database storage node.

[0058] When a new engine (target node) needs to be added to the database, the database sends a registration request for the target node to the metadata server. At this time, the metadata server determines whether there is a data table creation process currently running (that is, an unfinished table creation process). If not, a metadata change lock is added to the metadata information in the metadata server; if so, the status of the running data table creation process is monitored, and when the data table creation process is detected to have ended, a metadata change lock is added to the metadata information in the metadata server.

[0059] Step S204: adding a metadata change lock to the metadata information in the metadata server.

[0060] After the metadata information is locked with a metadata change lock, the metadata information of the database table cannot be changed until the metadata change lock is revoked.

[0061] After adding the metadata change lock to the metadata information, if the metadata service still receives a new data table creation request, the received data table creation request is cached in the request queue, and after unlocking the metadata change lock, the cached data table creation request in the request queue is continued to be processed.

[0062] Step S206: monitoring the status of the running data table creation process.

[0063] Step S208: If it is detected that the data table creation process is finished, a metadata change lock is added to the metadata information in the metadata server.

[0064] Step S210: register the target node to the metadata server.

[0065] Step S212: Synchronize the metadata information in the metadata server to the successfully registered target node, and unlock the metadata change lock.

[0066] After adding the metadata change lock to the metadata information, the target node is registered with the metadata server, and then the metadata information in the metadata server is synchronized to the successfully registered target node, so that the metadata information in the newly registered engine is consistent with the metadata information in other engines of the same type in the database (for example, the same computing node or the same storage node, etc.). Finally, the aforementioned metadata change lock is unlocked, so that operations such as table creation or table modification can continue, and the metadata information of the database table can be operated.

[0067] The metadata processing method provided in this embodiment can support the consistency of the library table view of the computing node in the scenario of dynamically expanding the computing node, and can also support the consistency of the library table view of the computing node in the scenario of dynamically expanding the storage node. Figure 1 Consistency.

[0068] The above-mentioned embodiment describes the metadata processing method from the metadata server side, and this embodiment describes the metadata processing method from the database side.

[0069] See also Figure 3 , which is a flow chart of another metadata processing method provided by an embodiment of the present invention, the method is applied to a database, the database is connected to a preset metadata server, and multiple computing nodes are running on the database. Figure 3 It can be seen that the method comprises the following steps:

[0070] Step S302: if a creation request for a target data table is received from the user, the creation request is forwarded to the metadata server; wherein the creation request carries metadata information of the target data table.

[0071] Step S304: if a data table creation instruction sent by the metadata server is received, the target data table is created through the database.

[0072] In one possible implementation manner, the target data table includes a main table and multiple sub-tables split from the main table, and multiple storage nodes are running on the database; and the multiple sub-tables can be created in the multiple storage nodes through a third distributed transaction.

[0073] Step S306: If the target data table is created successfully, send the information that the data table is created successfully to the metadata server, so that the metadata server synchronizes the metadata information to each computing node in the database after receiving the information that the data table is created successfully.

[0074] Here, the metadata information can be synchronized to each computing node in the database through the fourth distributed transaction. The above "third" and "fourth" are only used for the purpose of description and distinction, and cannot be understood as indicating or implying relative importance.

[0075] Step S308: If it is monitored that each computing node in the database has completed the synchronization of the metadata information, a message indicating that the synchronization of the metadata information is completed is sent to the metadata server.

[0076] Step S310: if the confirmation information of the successful creation of the target data table sent by the metadata server is received, the confirmation information of the successful creation of the target data table is returned to the user.

[0077] The method for processing metadata described on the database side provided in the embodiment of the present invention has the same technical features as the method for processing metadata described on the metadata server side provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0078] In order to more clearly understand the metadata processing method provided by this embodiment, an application example is introduced below.

[0079] like Figure 4 As shown, it is a schematic diagram of an application scenario of a metadata processing method. Figure 4 In the illustrated implementation, when user 1 initiates a request to create a data table Table1 to the DS1 computing node in the database, the following steps are performed:

[0080] (1) At time T1: User 1 initiates a request to create table Table1 through computing node DS 1.

[0081] (2) Time T2: Compute node DS1 registers with the metadata server to create metadata information about Table1 and its sub-table distribution;

[0082] (3) Time T3: The metadata server starts a distributed transaction and creates a sub-table of Table1 on each database storage node.

[0083] (4) If step (3) fails, delete the subtable of Table1 on each storage node and then jump to step (6); otherwise, jump to step (5);

[0084] (5) Time T4: Start the distributed transaction and synchronize the metadata information of Table1 and its subtables to each computing node;

[0085] (6) Time T5: The metadata server returns an ACK message of creating table Table1 to computing node DS1;

[0086] (7) Time T6: Computing node DS1 returns ACK information of creating table DS1 to the user.

[0087] Similarly, the process of user 2 initiating creation of table Table2 through computing node DS2 is similar to the process of creating table Table1, which is not repeated here.

[0088] Corresponds to Figure 1 In addition to the metadata processing method shown in , the embodiment of the present invention also provides a metadata processing device, which is applied to a metadata server, and the metadata server is connected to a preset database in communication, and a plurality of computing nodes are running on the database. Figure 5 , which is a schematic diagram of the structure of a metadata processing device, wherein the device includes a data table creation request receiving module 51, a data table creation instruction sending module 52, a metadata information synchronization module 53 and a confirmation information sending module 54 connected in sequence, and the functions of each module are as follows:

[0089] The data table creation request receiving module 51 is used to receive a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table;

[0090] A data table creation instruction sending module 52 is used to send a data table creation instruction to the database to create the target data table through the database;

[0091] The metadata information synchronization module 53 is used to synchronize the metadata information to each computing node in the database if the information indicating that the data table is successfully created is received from the database;

[0092] The confirmation information sending module 54 is used to send confirmation information of successful creation of the target data table to the database if the metadata information synchronization completion information is received from the database.

[0093] In one possible implementation manner, the target data table includes a main table and multiple sub-tables split from the main table, and multiple storage nodes are running on the database; the data table creation instruction sending module 52 is also used to: start a first distributed transaction, and create the multiple sub-tables in the multiple storage nodes through the first distributed transaction.

[0094] In another possible implementation, the apparatus further includes a confirmation information sending module 54, which is used to: if the information indicating that the data table creation failed is received from the database, send confirmation information indicating that the target data table creation failed to the database.

[0095] In another possible implementation, the metadata information synchronization module 53 is further used to: start a second distributed transaction, and synchronize the metadata information to each computing node in the database through the second distributed transaction.

[0096] In another possible implementation, the above-mentioned device also includes an engine extension module, which is used to: if a registration request for a target node is received from a database, determine whether there is a data table creation process running in the metadata server; if not, add a metadata change lock to the metadata information in the metadata server; register the target node with the metadata server; synchronize the metadata information in the metadata server to the successfully registered target node, and unlock the metadata change lock.

[0097] In another possible implementation, the engine extension module is further used to: if it is determined that there is a data table creation process running in the metadata server, monitor the status of the running data table creation process; if it is detected that the data table creation process ends, add a metadata change lock to the metadata information in the metadata server.

[0098] In another possible implementation, the apparatus further includes a creation request cache module, which is used to: if a data table creation request is received, cache the data table creation request in a request queue; and after the step of unlocking the metadata change lock, the method further includes: continuing to process the data table creation requests that have been cached in the request queue.

[0099] In another possible implementation, the target node is a database computing node or a database storage node.

[0100] In addition, corresponding to Figure 3 In addition to the metadata processing method shown in the figure, this embodiment also provides another metadata processing device, which is applied to a database, the database is connected to a preset metadata server, and multiple computing nodes are running on the database. Figure 6 As shown, it is a schematic diagram of the structure of another metadata processing device, wherein the device includes a creation request forwarding module 61, a target data table creation module 62, a first information sending module 63, a second information sending module 64 and a third information sending module 65 connected in sequence, wherein the functions of each module are as follows:

[0101] A creation request forwarding module 61 is used for forwarding a creation request for a target data table from a user to the metadata server if the creation request is received; wherein the creation request carries metadata information of the target data table;

[0102] A target data table creation module 62 is used to create the target data table through the database if a data table creation instruction sent by the metadata server is received;

[0103] The first information sending module 63 is used to send information of successful creation of the data table to the metadata server if the target data table is successfully created, so that the metadata server synchronizes the metadata information to each computing node in the database after receiving the information of successful creation of the data table;

[0104] The second information sending module 64 is used to send the metadata information synchronization completion information to the metadata server if it is monitored that each computing node in the database has completed the synchronization of the metadata information;

[0105] The third information sending module 65 is used to return the confirmation information of the successful creation of the target data table to the user if the confirmation information of the successful creation of the target data table sent by the metadata server is received.

[0106] In one possible implementation, the target data table includes a main table and multiple sub-tables split from the main table, and multiple storage nodes are running on the database; the target data table creation module 62 is also used to: create the multiple sub-tables to the multiple storage nodes through a third distributed transaction; the first information sending module 63 is also used to: synchronize the metadata information to each computing node in the database through a fourth distributed transaction.

[0107] The metadata processing device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned metadata processing method embodiment. For the sake of brief description, for matters not mentioned in the embodiment of the metadata processing device, reference may be made to the corresponding contents in the aforementioned metadata processing method embodiment.

[0108] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it is a schematic diagram of the structure of the electronic device, wherein the electronic device includes a processor 71 and a memory 72, the memory 72 stores machine executable instructions that can be executed by the processor 71, and the processor 71 executes the machine executable instructions to implement the above-mentioned metadata processing method.

[0109] exist Figure 7 In the illustrated embodiment, the electronic device further includes a bus 73 and a communication interface 74 , wherein the processor 71 , the communication interface 74 and the memory 72 are connected via the bus.

[0110] The memory 72 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 74 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0111] The processor 71 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 71 or the instructions in the form of software. The above processor 71 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 71 reads the information in the memory 72 and completes the steps of the metadata processing method of the above-mentioned embodiment in combination with its hardware.

[0112] An embodiment of the present invention also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned metadata processing method. The specific implementation can be found in the aforementioned method embodiment, which will not be repeated here.

[0113] The metadata processing method, metadata processing device, and computer program product of an electronic device provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the metadata processing method described in the previous method embodiment. The specific implementation can be referred to the method embodiment, which will not be repeated here.

[0114] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0115] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0116] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0117] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for processing metadata, characterized in that: Applied to a metadata server, the metadata server is connected to a preset database, and a plurality of computing nodes are running on the database, the method includes: Receiving a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table; Sending a data table creation instruction to the database to create the target data table through the database; the target data table includes a main table and a plurality of sub-tables obtained by splitting the main table; If the information returned by the database that the data table is successfully created is received, the metadata information is synchronized to each of the computing nodes in the database; If the metadata information synchronization completion information returned by the database is received, confirmation information that the target data table is successfully created is sent to the database.

2. The metadata processing method according to claim 1, characterized in that: A plurality of storage nodes are running on the database; The step of sending a data table creation instruction to the database to create the target data table through the database includes: A first distributed transaction is started, and the plurality of sub-tables are created in the plurality of storage nodes through the first distributed transaction.

3. The metadata processing method according to claim 1, characterized in that: After the step of sending a data table creation instruction to the database to create the target data table through the database, the method further includes: If the information that the data table creation fails is received from the database, confirmation information that the target data table creation fails is sent to the database.

4. The metadata processing method according to claim 1, characterized in that: The step of synchronizing the metadata information to each of the computing nodes in the database includes: A second distributed transaction is started, and the metadata information is synchronized to each of the computing nodes in the database through the second distributed transaction.

5. The metadata processing method according to claim 1, characterized in that: The method further comprises: If a registration request for a target node is received from a database, determining whether there is a data table creation process running in the metadata server; If not, adding a metadata change lock to the metadata information in the metadata server; Registering the target node to the metadata server; The metadata information in the metadata server is synchronized to the successfully registered target node, and the metadata change lock is unlocked.

6. The metadata processing method according to claim 5, characterized in that: Before the step of registering the target node with the metadata server, the method further includes: If it is determined that there is a data table creation process running on the metadata server, monitoring the status of the running data table creation process; If it is detected that the data table creation process is finished, a metadata change lock is added to the metadata information in the metadata server.

7. The metadata processing method according to claim 5, characterized in that: After the step of adding a metadata change lock to the metadata information in the metadata server, the method further includes: If a data table creation request is received, the data table creation request is cached in a request queue; Furthermore, after the step of unlocking the metadata change lock, the method further includes: Continue to process the data table creation requests that have been cached in the request queue.

8. The metadata processing method according to claim 5, characterized in that: The target node is a database computing node or a database storage node.

9. A method for processing metadata, characterized in that: Applied to a database, the database is connected to a preset metadata server for communication, and a plurality of computing nodes are running on the database, the method includes: If a user's creation request for a target data table is received, the creation request is forwarded to the metadata server; wherein the creation request carries metadata information of the target data table; If a data table creation instruction sent by the metadata server is received, the target data table is created through the database; the target data table includes a main table and a plurality of sub-tables obtained by splitting the main table; If the target data table is created successfully, sending information indicating that the data table is created successfully to the metadata server, so that the metadata server synchronizes the metadata information to each of the computing nodes in the database after receiving the information indicating that the data table is created successfully; If it is monitored that each of the computing nodes in the database has completed the synchronization of the metadata information, sending information that the synchronization of the metadata information is completed to the metadata server; If confirmation information indicating that the target data table is successfully created is received from the metadata server, confirmation information indicating that the target data table is successfully created is returned to the user.

10. The metadata processing method according to claim 9, characterized in that: A plurality of storage nodes are running on the database; The step of creating the target data table through the database includes: Creating the plurality of sub-tables into the plurality of storage nodes through a third distributed transaction; The step of synchronizing the metadata information to each of the computing nodes in the database includes: The metadata information is synchronized to each of the computing nodes in the database through a fourth distributed transaction.

11. A metadata processing device, characterized in that: Applied to a metadata server, the metadata server is connected to a preset database, a plurality of computing nodes are running on the database, and the device comprises: A data table creation request receiving module, used to receive a creation request for a target data table sent by the database; wherein the creation request carries metadata information of the target data table; A data table creation instruction sending module, used for sending a data table creation instruction to the database, so as to create the target data table through the database; the target data table includes a main table and a plurality of sub-tables obtained by splitting the main table; A metadata information synchronization module, configured to synchronize the metadata information to each computing node in the database if receiving information that the data table is successfully created returned by the database; The confirmation information sending module is used to send confirmation information of successful creation of the target data table to the database if the information of completion of synchronization of the metadata information is received from the database.

12. A metadata processing device, characterized in that: Applied to a database, the database is connected to a preset metadata server for communication, a plurality of computing nodes are running on the database, and the device comprises: A creation request forwarding module, configured to forward a creation request for a target data table from a user to the metadata server if the creation request is received; wherein the creation request carries metadata information of the target data table; A target data table creation module is used to create the target data table through the database if a data table creation instruction sent by the metadata server is received; the target data table includes a main table and multiple sub-tables obtained by splitting the main table; A first information sending module is used to send information of successful creation of the data table to the metadata server if the target data table is successfully created, so that the metadata server synchronizes the metadata information to each of the computing nodes in the database after receiving the information of successful creation of the data table; A second information sending module is used to send information that the synchronization of the metadata information is completed to the metadata server if it is monitored that each of the computing nodes in the database has completed the synchronization of the metadata information; The third information sending module is used to return the confirmation information of successful creation of the target data table to the user if the confirmation information of successful creation of the target data table sent by the metadata server is received.

13. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the metadata processing method according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the metadata processing method according to any one of claims 1 to 10.

Images