Distributed database catalog management method, system, electronic device and readable storage medium

Abstract

The present application relates to the field of computer technology, and in particular to a distributed database catalog management method, system, electronic device and readable storage medium, wherein the method includes a database agent sending updated catalog information to a catalog server; wherein the database agent is a front-end middleware for accessing a database, and the database agent stores the catalog of the database; the catalog server sends the received database catalog update to a catalog database; the catalog database receives the database catalog update and updates the stored database catalog. The technical solution of the present application is more flexible in maintaining the catalog data of a distributed database, and can be based on both the catalog server and the database agent node, with stronger fault tolerance, ensuring the timeliness and consistency of the catalog data of the distributed database on each node, and ensuring the continuous availability of the distributed database under a mixed online transaction load.

Description

Technical Field

[0001] The present application relates to the technical field of computer data processing, and in particular to a distributed database catalog management method, system, electronic device and readable storage medium. Background Art

[0002] In the architecture of distributed databases, in order to shield the complexity brought by sharding, applications usually do not directly access distributed databases, but access them through database proxies (DBProxy). DBProxy is responsible for verifying and parsing SQL statement requests from applications, routing them to the correct database shards, and assembling responses from one or more database shards before returning them to the application. In distributed databases, in order to complete the verification, parsing, routing and assembly functions of SQL statements, the distributed database catalog (Database Catalog) occupies an important position. It is used to describe the information of various objects in the database, such as describing database structure information such as tables and indexes. In distributed databases, catalog data is constantly changing. For example, executing DDL statements such as Create table will add structural information of related tables and indexes to the catalog data. Executing DDL statements such as Alter table will modify the structural information of related tables in the catalog data, so distributed database catalog management is very important. Currently, there are two main methods for distributed database catalog management:

[0003] The first management method is to store catalog data only in the catalog server. When DBProxy executes SQL statements, it needs to request catalog data from the catalog server. As applications become increasingly complex, distributed databases need to support mixed online transaction loads (Mixed OLTP) in a high-concurrency environment. This load includes both daily transactions and batch processing, which will result in a large number of DML statement requests and DDL statement requests. A large number of DML statement requests will initiate a large number of catalog data read requests to the catalog server, and a large number of DDL statement requests may initiate a large number of write requests to the catalog server. The catalog server will become a bottleneck, so this method cannot meet the extremely high requirements of mixed online transaction loads in terms of performance.

[0004] The second management method is to store catalog data persistently in files or local databases on all DBProxy nodes, and persistently store it in the database on the catalog server, and achieve consistency of catalog data between DBProxy nodes and catalog servers through a trigger-broadcast mechanism. In this mechanism, during the synchronization process between the catalog server and DBProxy nodes, catalog data can only be based on the catalog server, and DBProxy is not supported. Summary of the invention

[0005] The purpose of this application is to solve at least one of the above technical defects. The technical solution adopted by this application is as follows:

[0006] In a first aspect, an embodiment of the present application provides a distributed database catalog management method, which is applied to a catalog management system. The method includes:

[0007] The database agent sends the updated catalog information to the catalog server; wherein the database agent is a front-end middleware for accessing the database, and the database agent stores the catalog of the database;

[0008] The catalog server sends the received database catalog update to the catalog database;

[0009] The catalog database receives the database catalog update and updates the stored database catalog.

[0010] Optionally, the method further comprises:

[0011] There are multiple database agents;

[0012] The multiple database agents respectively send their updated catalog information to the same catalog server.

[0013] Optionally, the method further comprises:

[0014] The catalog database sends the stored database catalog information to the catalog server;

[0015] The catalog server sends the database catalog received from the catalog database to the plurality of database agents;

[0016] The plurality of database agents respectively update stored catalogs according to database catalog information received from the catalog server.

[0017] Optionally, the database agent sending the updated catalog information to the catalog server includes:

[0018] The database agent includes a catalog management module and a catalog module; wherein the catalog module is used to store the database catalog stored locally by the database agent;

[0019] After the catalog management module reads the catalog update information in the catalog module;

[0020] The catalog management module sends the read catalog update information to the catalog synchronization server.

[0021] Optionally, the communication method between the database proxy and the catalog server, and between the catalog server and the catalog database includes:

[0022] Socket long connection communication, socket short connection communication or message queue communication.

[0023] Optionally, the method further comprises:

[0024] The database is a relational sub-library and sub-table database;

[0025] The database agent is a data processing middle layer that can access a relational sub-library and sub-table database.

[0026] Optionally, the method further comprises: when the database proxy is different from the database catalog stored in the catalog server, receiving a request for a user-specified database catalog; the request is used to control the database proxy to access the database according to the target database catalog specified by the user.

[0027] In a second aspect, an embodiment of the present application provides a distributed database catalog management system, the system comprising: a database, a database agent, a catalog server and a catalog database; wherein the database agent is a front-end middleware for accessing the database, and the database agent stores the catalog of the database;

[0028] The database agent sends the updated catalog information to the catalog server;

[0029] The catalog server sends the received database catalog update to a catalog database;

[0030] The catalog database receives the database catalog update and updates the stored database catalog.

[0031] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory;

[0032] The memory is used to store operation instructions;

[0033] The processor is used to execute the above-mentioned distributed database catalog management method by calling the operation instruction.

[0034] In a fourth aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the distributed database catalog management method is implemented.

[0035] The distributed database catalog management solution disclosed in the embodiment of the present application stores catalog data persistently in all database proxy nodes and persistently in the catalog database on the catalog server side, and synchronizes catalog data between the database proxy and the catalog server through a two-way two-stage synchronization method. If a conflict occurs, it is automatically handled through an arbitration mechanism. The technical solution provided in the embodiment of the present application brings at least one of the following beneficial effects:

[0036] (1) This method allows the catalog data to be based on both the catalog server and the DBProxy node during the synchronization process, which is more flexible and convenient.

[0037] (2) This method supports mixed online transaction load scenarios in a high-concurrency environment. Since a large number of DML statement requests and a large number of DDL statement requests only read and write the local catalog data of the current DBProxy node when they are executed, there is no performance pressure on the catalog server.

[0038] (3) In extreme scenarios, if the local catalog data of a DBProxy node is damaged, it can be automatically repaired through the synchronization mechanism without affecting the availability of the current DBProxy node.

[0039] (4) The catalog data maintenance mechanism of the DBProxy node is diverse, which can be written through passive persistence by the catalog server or active persistence by the DBProxy node.

[0040] (5) In the scenario where the catalog data of the catalog server needs to be modified, after the catalog data of the catalog server is modified, it can be quickly sent to all DBProxy nodes through the synchronization mechanism according to the established configuration strategy.

[0041] (6) Synchronization initiation is flexible and can be triggered by events, timed initiation, or customized conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0043] Figure 1 A schematic diagram of a process flow of a distributed database catalog management method provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of the structure of a distributed database catalog management system provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0047] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0048] To more clearly describe the embodiments of the present application, some definitions, concepts or devices that may be used in the embodiments are introduced below:

[0049] MySQL is an open source relational database management system (RDBMS) that uses the most commonly used database management language - Structured Query Language (SQL) for database management.

[0050] MariaDB database management system is a branch of MySQL, mainly maintained by the open source community. It uses the GPL license. The purpose of MariaDB is to be fully compatible with MySQL, including API and command line, so that it can easily become a substitute for MySQL.

[0051] PostgreSQL is a free software object-relational database management system (ORDBMS) with a very complete set of features. It is an object-relational database management system based on POSTGRES version 4.2 developed by the Department of Computer Science at the University of California.

[0052] Kafka is an open source stream processing platform developed by the Apache Software Foundation and written in Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all the action stream data of consumers in the website.

[0053] DBProxy is a data middleware based on the MySQL protocol. It fixes some bugs and adds many features based on the open source Atlas. Its main functions include read-write separation, slave database load balancing, IP filtering, smooth online and offline DB by table-splitting DBA, automatic removal of downtime DB, complete monitoring information, SQL filtering, and slave database traffic configuration.

[0054] Database schema definition language DDL (Data Definition Language) is a language used to describe real-world entities to be stored in a database.

[0055] Structured Query Language (SQL) is a special-purpose programming language. It is a database query and programming language used to access data and query, update and manage relational database systems.

[0056] Open Database Connectivity (ODBC) was created to solve the problem of data sharing between heterogeneous databases. It has now become a major part of WOSA (The Windows Open System Architecture) and a database access interface standard based on the Windows environment. ODBC provides a unified interface for heterogeneous database access, allowing applications to use SQL as the data access standard to access data managed by different DBMS; it enables applications to directly manipulate data in the DB, eliminating the need to change with changes in the DB. ODBC can be used to access DB files on various computers, and even non-database objects such as Excel tables and ASCII data files.

[0057] Java Database Connectivity (JDBC) is an application programming interface in the Java language that regulates how client programs access databases. It provides methods such as querying and updating data in the database. JDBC is also a trademark of Sun Microsystems. The JDBC we usually refer to is oriented to relational databases.

[0058] DBProxy is a front-end middleware cluster responsible for SQL statement parsing and execution in distributed databases. It completes the parsing, routing, and assembly functions of SQL statements by reading the local database catalog.

[0059] In a distributed database, objects such as databases and data tables are sharded to break through the performance limit of a single centralized database and support higher concurrent SQL requests through sharding clusters. SQL requests to access distributed databases can be data manipulation languages ​​(DML) such as query (select), insert (insert), delete (delete) or update (update) for data operations; they can also be data definition languages ​​(DDL) such as create (create), modify (alter), rename (rename), and delete (drop) for database objects such as tables, views, and indexes. In the architecture of distributed databases, in order to shield the complexity brought by sharding, applications usually do not directly access distributed databases, but access them through proxy middleware (DBProxy). DBProxy middleware is responsible for verifying and parsing SQL statement requests from applications, routing them to the correct database shards, and assembling responses from one or more database shards and returning them to the application. In a distributed database, in order to complete the verification, parsing, routing and assembly functions of SQL statements, the distributed database catalog plays an important role. It is used to describe the information of various objects in the database, such as describing the database structure information such as tables and indexes. In a distributed database, the catalog data is constantly changing. For example, executing a DDL statement such as Create table will add the structural information of related tables and indexes to the catalog data. Executing a DDL statement such as Alter table will modify the structural information of related tables in the catalog data.

[0060] As mentioned above, both current catalog data management methods have certain limitations. The first management method cannot meet the extremely high requirements of hybrid online transaction loads in terms of performance. In the second management method, during the synchronization process between the catalog server and the DBProxy node, the catalog data can only be based on the catalog server, and DBProxy is not supported. In addition, in the scenario where the catalog data version of the DBProxy node is inconsistent with the version on the catalog server, the DBProxy node needs to periodically compare the version number with the catalog server (the comparison period is configurable). If the version is inconsistent, it is necessary to actively pull the catalog data on the catalog server to the DBProxy local and replace the local version. The DBProxy node cannot push the local version of the catalog data to the catalog server. Based on this, the present application discloses a distributed database catalog management solution to at least solve one of the above technical problems.

[0061] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems will be described in detail below with reference to the accompanying drawings. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0062] In order to make the purpose, technical solutions and advantages of this application clearer, Figure 1 A flowchart of a distributed database catalog management method provided by an embodiment of the present application is disclosed. The method is applied to a catalog management system, such as Figure 1 As shown, the method includes:

[0063] S101, the database agent sends updated catalog information to the catalog server; wherein the database agent is a front-end middleware for accessing the database, and the database agent stores the catalog of the database;

[0064] In an optional embodiment of the present application, the database agent includes a catalog management module and a catalog module; the catalog management module may also be referred to as a catalog synchronization client, and the catalog module may also be referred to as a local catalog; wherein the catalog module is used to store the database catalog stored locally by the database agent, that is, the database catalog is stored in the file system where the database agent node is located in XML format, and is mapped to the catalog tables system.table_info, system.column_info, and system.index_info on the catalog database;

[0065] After the catalog management module reads the catalog update information in the catalog module;

[0066] The catalog management module sends the read catalog update information to the catalog synchronization server.

[0067] S102, the catalog server sends the received database catalog update to the catalog database; the catalog server may also be referred to as a catalog synchronization server, that is, used to synchronize the catalog update information stored in the database agent to the catalog server and the catalog database;

[0068] S103, the catalog database receives the database catalog update and updates the stored database catalog. The catalog database is used to store the distributed database global catalog information, the catalog database stores table information, column information and index information, which can be stored in the three tables system.table_info, system.column_info and system.index_info in actual applications.

[0069] In an optional embodiment of the present application, the method further includes:

[0070] There are multiple database proxies, and each of the multiple database proxies sends its own updated catalog information to the same catalog server. In actual application scenarios, there are multiple target applications to access the database, or referred to as target application clusters, and correspondingly there is a database proxy cluster, that is, each of the multiple target applications corresponds to a database proxy, and each database proxy includes at least a catalog management module and a catalog module.

[0071] In an optional embodiment of the present application, the method further includes:

[0072] The catalog database sends the stored database catalog information to the catalog server;

[0073] The catalog server sends the database catalog received from the catalog database to the plurality of database agents;

[0074] The plurality of database agents respectively update stored catalogs according to database catalog information received from the catalog server.

[0075] Based on the embodiment of the present application, bidirectional synchronization of catalog data between the database proxy and the catalog server is achieved, so that during the synchronization process between the catalog server and the DBProxy node, the catalog data can be based on both the catalog server and the DBProxy node.

[0076] In an optional embodiment of the present application, the communication method between the database agent and the catalog server, and the catalog server and the catalog database includes: socket long connection communication, socket short connection communication or message queue communication.

[0077] In an optional embodiment of the present application, the method further includes: the target application cluster accesses the database proxy cluster through an ODBC or JDBC interface, and distributes the SQL request to a node in the database proxy cluster through a load balancing device for SQL request processing;

[0078] The database proxy cluster receives SQL processing requests initiated by the target application and completes the parsing, routing and assembly functions of SQL statements by reading the local database catalog;

[0079] In an optional embodiment of the present application, the method further includes:

[0080] The database is a relational database with sub-libraries and sub-tables; the database proxy is a data processing middle layer that can access the relational database with sub-libraries and sub-tables. That is, the database data sharding cluster receives and responds to SQL requests from the database proxy node. The nodes in the shards usually use open source relational databases such as MySQL, MariaDB or PostgreSQL.

[0081] In an optional embodiment of the present application, the system also includes a cluster manager to manage database proxy nodes and data sharding nodes, etc.

[0082] In an optional embodiment of the present application, the method further includes: when the database proxy is different from the database catalog stored in the catalog server, receiving a user-specified database catalog request; the request is used to control the database proxy to access the database according to the target database catalog specified by the user. That is, if a logical conflict occurs between the local dynamic catalog and the global catalog data, it will be executed according to the pre-set conflict handling strategy, and the specific handling strategies include two types, namely, client-side (clientwin) and server-side (serverwin). In the specific implementation, the setting of the conflict handling strategy can be stored in the ConflictPolicy tag of the local dynamic catalog XML file and the ConflictPolicy field of the global catalog database related table respectively.

[0083] In order to more clearly introduce the distributed database catalog management method disclosed in this application, the following embodiment will introduce in detail the bidirectional two-stage synchronization process of catalog information between the database agent and the catalog database as follows:

[0084] Phase 1: The database agent synchronizes the updated local dynamic catalog data to the catalog database. The specific process of the synchronization is as follows:

[0085] Step 1: The catalog management module initiates a catalog data synchronization request to the catalog server;

[0086] Step 2: The catalog server responds that synchronization is possible;

[0087] Step 3: The catalog management module initiates a catalog data phase 1 synchronization request to the catalog server;

[0088] Step 4, the catalog management module obtains the update data from the catalog module;

[0089] Step 5: The catalog management module sends the local catalog update to the catalog server;

[0090] Step 6: The catalog server applies the received local catalog update to the catalog database. If there is a conflict, it executes according to the policy configured by ConflictPolicy.

[0091] Step 7: The catalog server responds to the catalog management module, and the catalog synchronization to the server is completed;

[0092] Step 8: The catalog management module notifies the catalog server that the first phase of synchronization is completed.

[0093] Phase 2: Synchronize the updated catalog database data to the catalog module of the database agent. The specific process of the synchronization is as follows:

[0094] Step 1: The catalog management module initiates a catalog data synchronization request to the catalog server;

[0095] Step 2: The catalog server obtains updated data from the catalog database;

[0096] Step 3: The catalog server packages the acquired catalog update data and sends it to the catalog management module;

[0097] Step 4: The catalog management module applies the received catalog update to the catalog module. If there is a conflict, it is executed according to the policy configured by ConflictPolicy.

[0098] Step 5: The catalog management module notifies the catalog server and the catalog application is completed locally;

[0099] Step 6: The catalog management module notifies the catalog server that synchronization is complete;

[0100] Step 7: The catalog management module initiates a catalog data end synchronization request to the catalog server;

[0101] Step 8: The catalog server responds and the process ends.

[0102] In an optional embodiment of the present application, the method for capturing catalog updates in the database proxy catalog module is as follows: the database proxy node receives a DDL statement request initiated by the application and modifies the catalog information in the catalog module. The following is an example of adding a field prft_cter to the table AccountTab:

[0103] alter table AccountTab add prft_cter char(6)

[0104] The above statement adds the prft_cter field to the local dynamic catalog XML file.<SQL_Content> In the tag, the value of the SyncStatus tag in the XML file will be changed from YES to NO, indicating that the current catalog update needs to be synchronized with the server.

[0105] In an optional embodiment of the present application, the method for capturing updates to the server-side catalog database is:

[0106] The catalog server obtains catalog data updates from the system.column_info_track table of the catalog database, and synchronizes the updated catalog information to the catalog management module of the database agent. This embodiment uses the tracking table method to track changes in catalog information in the catalog database, that is, three tracking tables are created for the three tables system.table_info, system.column_info and system.index_info, respectively, with the names system.table_info_track, system.column_info_track and system.index_info_track. The fields of the above three tracking tables are basically the same as those of the source table, except that there are three more fields:

[0107] Operation field: records the change operation types of the three tables (insert, delete or update);

[0108] Clientid field: indicates the client ID that the catalog synchronization server needs to synchronize;

[0109] syncstatus field: indicates whether the synchronization is completed, the value is YES or NO.

[0110] Create insert, update and delete triggers on the system.table_info, system.column_info and system.index_info tables respectively. In this way, the insert, update and delete operations on these three tables will be recorded in the system.table_info_track, system.column_info_track and system.index_info_track tables.

[0111] Modify catalog information in the catalog database, for example, add a field prft_cter to the AccountTab table, so that a new record about the prft_cter field will be inserted into the system.column_info table in the global catalog database. At this time, the column_info_insert_trigger trigger on system.column_info will be triggered. Assuming that there are M database agent management modules, the trigger will insert M records into the column_info_track table. The management module ID in each record is the corresponding catalog management module ID. Since it is a newly added field, the operation field value is insert, and sycstatus is YES, indicating that the corresponding database agent is to synchronize data.

[0112] In an optional embodiment of the present application, the method further includes obtaining the synchronization status of the catalog management module and the catalog server of each database agent based on the current synchronization sequence number (SyncID). The specific implementation process is: when the current database agent performs a field addition operation in the catalog module, the current catalog management module initiates synchronization, and the synchronization status of each client is obtained through the following method.

[0113] 1) The current catalog management module sends the catalog information of the added field to the catalog server;

[0114] 2) The catalog server inserts the added field information into the column_info table. The trigger column_info_insert_trigger on the column_info table will be triggered, and M records will be inserted into the column_info_track table. The syncstatus field value of each record is NO, indicating that the catalog server will synchronize this change information to M database agents.

[0115] 3) Since the catalog information of the current table in the current catalog management module does not need to be sent down again by the catalog server, the corresponding syncstatus field is marked as YES.

[0116] 4) The catalog server synchronizes with other database agents. After the synchronization is completed, the syncstatus field corresponding to the field identified by the corresponding catalog management module ID is marked as YES.

[0117] 5) In this way, the synchronization status can be determined from the catalog server according to the status of syncstatus corresponding to each catalog management module ID.

[0118] In the same principle, when the catalog server synchronizes DDL change information to the client, it also uses the catalog management module ID field and the syncstatus field to check the synchronization status of each catalog management module at a certain synchronization (SyncID identifies a certain synchronization ID).

[0119] In an optional embodiment of the present application, the method also includes that the catalog data synchronization between the database agent and the catalog server can implement a breakpoint retransmission mechanism based on the maximum synchronization sequence number (MaxSyncID), and the implementation process is: a synchronization location file LocalSync.xml is stored in each catalog management module, and the file records the ID of each catalog management module and the current maximum synchronization sequence number (described by SyncID), which identifies the position where the current catalog management module and the server have been synchronized, and the ID of the catalog management module is unique within the cluster.

[0120] There is also a synchronization location table ServerSync_Info on the server side. This table is stored in the catalog database and is used to record the ID of each catalog management module and the current maximum synchronization sequence number SyncID synchronized with each catalog management module. The following takes the bidirectional two-stage synchronization of catalog data as an example:

[0121] 1) When the catalog management module of the database agent initiates synchronization with the catalog server for the first time, a unique incremental serial number is generated locally and written into the MaxSyncID of the LocalSync.xml file.

[0122] 2) During the first phase of synchronization, the catalog management module initiates catalog data update synchronization to the catalog server, and sends SyncID (value is MaxSyncID+1) according to the current MaxSyncID value in the LocalSync.xml file. The server updates the catalog data to the catalog database system.table_info, system.column_info, system.index_info and other related tables.

[0123] 3) During the second phase of synchronization, the catalog synchronization server sends the server-side catalog update to the catalog management module, and the client updates the local catalog data.

[0124] 4) After the two phases are successfully synchronized, the catalog management module updates the MaxSyncID value in the LocalSync.xml file to the current SyncID value and persists it, and the catalog server updates the MaxSyncID field in the ServerSync_Info table to the current SyncID value, completing the entire synchronization process.

[0125] 5) Entering the continuous synchronization state, the MaxSyncID field value in the LocalSync.xml file of the catalog management module and the MaxSyncID field value in the ServerSync_Info table of the catalog server side continuously increases.

[0126] 6) If a synchronization exception occurs, if the MaxSyncID in the LocalSync.xml file is greater than the MaxSyncID of the catalog management module in the ServerSync_Info table, the catalog management module needs to synchronize the full catalog XML file of the corresponding table to the catalog server, and the catalog server updates the MaxSyncID field in the ServerSync_Info table; if the MaxSyncID in the LocalSync.xml file is less than the MaxSyncID of the catalog management module in the ServerSync_Info table, the catalog server needs to synchronize the full catalog data of the corresponding table to the catalog management module, and the catalog management module updates the MaxSyncID value in the LocalSync.xml file.

[0127] based on Figure 1 The distributed database catalog management method provided by the embodiment shown, Figure 2 A distributed database catalog management system provided by an embodiment of the present application is shown. Figure 2 As shown, the system may mainly include: 201 database, 202 database agent, 203 catalog server and 204 catalog database; wherein the database agent is a front-end middleware for accessing the database, and the database agent stores the catalog of the database;

[0128] The 202 database agent sends the updated catalog information to the 203 catalog server;

[0129] The catalog server 203 sends the received database catalog update to the catalog database 204;

[0130] The 204 catalog database receives the database catalog update and updates the stored database catalog.

[0131] In an optional embodiment of the present application, the 202 database proxy further includes the database proxy including a 2021 catalog management module and a 2022 catalog module; wherein the 2022 catalog module is used to store the database catalog stored locally by the database proxy;

[0132] After the 2021 catalog management module reads the catalog update information in the 2022 catalog module; the 2022 catalog management module is used to initiate two-way synchronization with the 203 catalog server, and send the update of the 2022 catalog module to the 203 catalog server; receive the response of the 203 catalog server, download the update of the 204 catalog database to the local, and update the 2022 catalog module.

[0133] The 2021 catalog management module sends the read catalog update information to the catalog synchronization server.

[0134] In an optional embodiment of the present application, the association system also includes 205 a cluster manager managing 202 database proxy nodes and 201 database shard nodes, etc.

[0135] It can be understood that the above modules of the distributed database catalog management system in this embodiment have the functions of implementing Figure 1 The functions of the corresponding steps of the method in the embodiment shown in . The function can be implemented by hardware, or by hardware executing the corresponding software. The hardware or software includes one or more modules corresponding to the above functions. The above modules can be software and / or hardware, and the above modules can be implemented separately or integrated with multiple modules. For the functional description of the above modules, please refer to Figure 1 The corresponding description of the method in the embodiment shown in will not be repeated here.

[0136] An embodiment of the present application provides an electronic device, including a processor and a memory;

[0137] A memory, used for storing operation instructions;

[0138] The processor is used to execute the distributed database catalog management method provided in any embodiment of the present application by calling an operation instruction.

[0139] As an example, Figure 3 A schematic diagram of the structure of an electronic device applicable to the embodiment of the present application is shown. Figure 3 As shown, the electronic device 2000 includes: a processor 2001 and a memory 2003. The processor 2001 and the memory 2003 are connected, such as through a bus 2002. Optionally, the electronic device 2000 may also include a transceiver 2004. It should be noted that in actual applications, the transceiver 2004 is not limited to one, and the structure of the electronic device 2000 does not constitute a limitation on the embodiments of the present application.

[0140] The processor 2001 is applied in the embodiment of the present application to implement the method shown in the above method embodiment. The transceiver 2004 may include a receiver and a transmitter. The transceiver 2004 is applied in the embodiment of the present application to implement the function of the electronic device of the embodiment of the present application communicating with other devices when executed.

[0141] Processor 2001 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0142] The bus 2002 may include a path to transmit information between the above components. The bus 2002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 2002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0143] The memory 2003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0144] Optionally, the memory 2003 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 2001. The processor 2001 is used to execute the application code stored in the memory 2003 to implement the distributed database catalog management method provided in any embodiment of the present application.

[0145] The electronic device provided in the embodiment of the present application is applicable to any embodiment of the above method and will not be described in detail here.

[0146] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the distributed database catalog management method shown in the above method embodiment is implemented.

[0147] The computer-readable storage medium provided in the embodiments of the present application is applicable to any embodiment of the above method and will not be described in detail here.

[0148] The distributed database catalog management solution disclosed in the embodiment of the present application sends updated catalog information to the catalog server through a database agent; wherein the database agent is a front-end middleware for accessing a database, and the database agent stores the catalog of the database; the catalog server sends the received database catalog update to the catalog database; the catalog database receives the database catalog update and updates the stored database catalog. The technical solution of the present application is more flexible for maintaining the catalog data of the distributed database, and can be based on both the catalog server and the database agent node, with stronger fault tolerance, ensuring the timeliness and consistency of the catalog data of the distributed database on each node, and ensuring the continuous availability of the distributed database under mixed online transaction loads.

[0149] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0150] The above are only some embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A distributed database catalog management method, applied to a catalog management system, characterized in that: The method comprises: The database agent sends the updated catalog information to the catalog server; wherein the database agent is a front-end middleware for accessing the database, and the database agent stores the catalog of the database; The catalog server sends the received database catalog update to the catalog database; The catalog database receives the database catalog update and updates the stored database catalog; The method further comprises: There are multiple database agents; The multiple database agents respectively send their updated catalog information to the same catalog server; The catalog database sends the stored database catalog information to the catalog server; The catalog server sends the database catalog received from the catalog database to the plurality of database agents; The plurality of database agents respectively update stored catalogs according to database catalog information received from the catalog server; The database agent sends the updated catalog information to the catalog server including: The database agent includes a catalog management module and a catalog module; wherein the catalog module is used to store the database catalog stored locally by the database agent; After the catalog management module reads the catalog update information in the catalog module, the catalog management module sends the read catalog update information to the catalog synchronization server.

2. The distributed database catalog management method according to claim 1, characterized in that: The communication method between the database agent and the catalog server, the catalog server and the catalog database includes: Socket long connection communication, socket short connection communication or message queue communication.

3. The distributed database catalog management method according to claim 2, characterized in that: The method further comprises: The database is a relational sub-library and sub-table database; The database agent is a data processing middle layer that can access a relational sub-library and sub-table database.

4. The distributed database catalog management method according to any one of claims 1 to 3, characterized in that: The method further comprises: When the database agent is different from the database catalog stored in the catalog server, a user-specified database catalog request is received; the request is used to control the database agent to access the database according to the target database catalog specified by the user.

5. A distributed database catalog management system, characterized in that: The system comprises: a database, a database agent, a catalog server and a catalog database; wherein the database agent is a front-end middleware for accessing the database, and the database agent stores the catalog of the database; The database agent sends the updated catalog information to the catalog server; The catalog server sends the received database catalog update to the catalog database; The catalog database receives the database catalog update and updates the stored database catalog; The system further comprises: There are multiple database agents; The multiple database agents respectively send their updated catalog information to the same catalog server; The catalog database sends the stored database catalog information to the catalog server; The catalog server sends the database catalog received from the catalog database to the plurality of database agents; The plurality of database agents respectively update stored catalogs according to database catalog information received from the catalog server; The database agent sends the updated catalog information to the catalog server including: The database agent includes a catalog management module and a catalog module; wherein the catalog module is used to store the database catalog stored locally by the database agent; After the catalog management module reads the catalog update information in the catalog module, the catalog management module sends the read catalog update information to the catalog synchronization server.

6. An electronic device, characterized in that: including a processor and a memory; The memory is used to store operation instructions; The processor is used to execute the method according to any one of claims 1 to 4 by calling the operation instruction.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images