Logical decoding method, computing device, and database system

Abstract

Embodiments of the present application provide a logical decoding method, a computing device, and a database system. Data table metadata required for logical decoding can be synchronized to a decoding dictionary of a shared storage medium, and logical logs are generated on the basis of the data table metadata. In this way, data table metadata does not need to be acquired from a system table, thereby achieving rapid reclamation of system tables, and reducing database space bloat. The method comprises: receiving a creation instruction for a logical replication slot; then acquiring first data table metadata from a database on the basis of the creation instruction; storing the first data table metadata in a decoding dictionary of a shared storage medium; sequentially acquiring a transaction log from a write-ahead log; and generating a logical log on the basis of the transaction log and the first data table metadata.

Description

A logic decoding method, computing device, and database system

[0001] This application claims priority to Chinese Patent Application No. 202411657575.0, filed on November 18, 2024, entitled "A Logic Decoding Method, Computing Device and Database System", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of databases, and more particularly to a logic decoding method, a computing device, and a database system. Background Technology

[0003] Logical decoding is a technique that converts the database's write-ahead log (WAL) into a logical log.

[0004] One current logical decoding method is roughly as follows: After the computing device obtains the transaction log from the write-ahead log, it sends a query request to the storage node based on the transaction log. The storage node obtains the data table metadata from the system table based on the query request and sends the data table metadata to the computing device. The computing device generates a logical log based on the transaction log and the data table metadata.

[0005] The above logical decoding process requires obtaining data table metadata from the system table. Therefore, the storage node must retain the system table during the logical decoding process, which affects the speed of reclaiming the system table. Summary of the Invention

[0006] This application provides a logical decoding method that synchronizes the data table metadata required for logical decoding to a decoding dictionary on a shared storage medium. Then, a logical log is generated based on the transaction log of the write-ahead log and the data table metadata obtained from the decoding dictionary. This eliminates the need to retrieve data table metadata from system tables, thus preventing the blocking of system table reclamation and reducing database space bloat. This application also provides a computing device, database system, computer-readable storage medium, and computer program product capable of implementing the above method.

[0007] The first aspect provides a logical decoding method, which includes: receiving a logical replication slot creation instruction, obtaining first data table metadata from the database according to the creation instruction, storing the first data table metadata in a decoding dictionary of a shared storage medium, obtaining transaction logs sequentially from the write-ahead log, and generating a logical log based on the transaction logs and the first data table metadata.

[0008] By implementing this method, during logical decoding, the metadata of the data tables required for logical decoding can be synchronized to the decoding dictionary on the shared storage medium. Then, logical logs are generated based on the transaction logs of the write-ahead log and the metadata of the data tables obtained from the decoding dictionary. This eliminates the need to obtain the metadata of the data tables from the system tables, thus avoiding blocking the reclamation of the system tables and reducing database space expansion.

[0009] In conjunction with the first aspect, in the first possible implementation, the above logical decoding method further includes: when the transaction log includes data definition language (DDL) statements, determining the second data table metadata based on the DDL statements and the first data table metadata, and then updating the first data table metadata in the decoding dictionary to the second data table metadata. After the data table is modified, the data table metadata in the decoding dictionary can be updated, and logical decoding of subsequent transaction logs can be performed based on the updated data table metadata. This way, retaining only one version of the data table metadata allows for continuous logical decoding, eliminating the need to retain multiple versions of the data table in the database. This significantly reduces the storage resources occupied by the data table and effectively reduces database space expansion.

[0010] In conjunction with the first aspect or the first possible implementation of the first aspect, in the second possible implementation, retrieving the first data table metadata from the database according to the creation instruction includes: obtaining the consistency point of the computing device according to the creation instruction; receiving consistency points from other computing devices in the database system; determining the target snapshot corresponding to the maximum value among all consistency points of the computing devices; and retrieving the first data table metadata from the database based on the target snapshot. Here, the database is a distributed database. The consistency point is the snapshot commit sequence number obtained when the logical replication slots are in a consistent state. Based on this target snapshot, the latest version of the data table metadata can be obtained, enabling logical decoding of all transaction logs of a user on different nodes.

[0011] In a third possible implementation, combining the first aspect or the second possible implementation of the first aspect, obtaining the consistency point of the computing device according to the creation instruction includes: creating a logical replication slot according to the creation instruction and setting the logical replication slot to a start state; obtaining a first active transaction set of the computing device while the logical replication slot is in the start state; obtaining a second active transaction set of the computing device while the logical replication slot is in a waiting-for-active-transactions-to-complete state; setting the state of the logical replication slot to a consistent state when all active transactions in the first active transaction set have ended and all active transactions in the second active transaction set have ended; and determining the consistency point of the computing device as the snapshot commit sequence number obtained in the consistent state. This allows for the acquisition of a local security snapshot, and the data table metadata obtained based on this security snapshot can support logical decoding by the current computing node.

[0012] In conjunction with the first aspect or the first possible implementation of the first aspect, in another possible implementation, the database is a distributed database. Optionally, the distributed database may be a distributed database that includes pluggable databases.

[0013] The second aspect provides a logical decoding method applied to a first database system, which includes a proxy node, multiple computing devices, and a shared storage medium. The logical decoding method includes: after receiving a logical replication slot creation instruction, the proxy node sends the creation instruction to the multiple computing devices; each computing device retrieves first data table metadata from the distributed database according to the creation instruction; writes the first data table metadata into a decoding dictionary in the shared storage medium; sequentially retrieves transaction logs from a first write-ahead log; when the transaction logs do not include DDL statements, a logical log for the computing device is generated based on the transaction logs and the first data table metadata. By implementing this method, multiple computing devices in the distributed database can simultaneously execute the logical decoding task of the same user, thereby improving the logical decoding speed.

[0014] In conjunction with the second aspect, in the first possible implementation, when the transaction log includes DDL statements, the computing device obtains the second data table metadata based on the DDL statements and the first data table metadata, and updates the first data table metadata in the decoding dictionary to the second data table metadata. This allows for updating the data table metadata, and subsequent transaction logs can be logically decoded based on the updated data table metadata. This eliminates the need to retain multiple versions of the data table in the database, significantly reducing the storage resources occupied by the data table and effectively minimizing database space expansion.

[0015] In conjunction with the second aspect, in the first possible implementation, after the first database system is upgraded, the computing device generates a second write-ahead log and sends it to the agent node. When the log format of the second write-ahead log differs from that of the first write-ahead log, the agent node sends the logical log corresponding to the second write-ahead log to the second database system. The logical log corresponding to the second write-ahead log is used by the second database system for data synchronization. The second database system can be, but is not limited to, a disaster recovery database system. When the upgraded write-ahead log has a different log format than the pre-upgrade write-ahead log, the second database system cannot perform data synchronization based on the upgraded write-ahead log, but can perform data synchronization based on the logical log corresponding to the upgraded write-ahead log, thus solving the data synchronization problem caused by the database upgrade.

[0016] In conjunction with the second aspect or other possible implementations, in another possible implementation, the database is a distributed database. Optionally, the distributed database can be a distributed database that includes pluggable databases.

[0017] The third aspect provides a logical decoding method applied to the main computing device of a database system. The method includes: receiving a logical replication slot creation instruction; retrieving first data table metadata from the database according to the creation instruction; storing the first data table metadata in a write-ahead log; sending the write-ahead log to a standby computing device; sequentially retrieving transaction logs from the write-ahead log; and generating a logical log based on the transaction logs and the first data table metadata. The main computing device provides the primary database, and the database system also includes a standby computing device providing a standby database.

[0018] This allows the data table metadata required for logical decoding to be stored in the write-ahead log. During logical decoding, the data table metadata is retrieved from the write-ahead log, eliminating the need to retrieve data table metadata from the system table. Therefore, this method does not block the reclamation of the system table and can improve the reclamation speed of the system table.

[0019] In conjunction with the third aspect, in the first possible implementation, the logical decoding method of this application further includes: when the transaction log includes DDL statements, determining the second data table metadata based on the DDL statements and the first data table metadata; and updating the first data table metadata in the write-ahead log to the second data table metadata. After a data table is modified, the data table metadata in the write-ahead log can be updated, and logical decoding of subsequent transaction logs can be performed based on the updated data table metadata. This way, retaining only one version of the data table metadata allows for continuous logical decoding, eliminating the need to retain multiple versions of the data table in the database. This significantly reduces and saves storage resources occupied by the data table, effectively reducing database space expansion.

[0020] The fourth aspect provides a logical decoding method applied to a standby computing device in a database system. The method includes: obtaining a write-ahead log sent by the primary computing device, then sequentially retrieving a transaction log from the write-ahead log; and generating a logical log based on the transaction log and the metadata of a first data table in the write-ahead log. The standby computing device provides a standby database, and the database system also includes a primary computing device that provides the primary database.

[0021] In this manner, the standby computing unit can perform logical decoding based on the data table metadata in the write-ahead log. Compared to the standby computing unit generating a system table based on the write-ahead log and then performing logical decoding based on the data table metadata obtained from the system table, this significantly improves the logical decoding speed.

[0022] In conjunction with the fourth aspect, in the first possible implementation, when the transaction log includes DDL statements, the second data table metadata is obtained based on the DDL statements and the first data table metadata, and the first data table metadata in the write-ahead log is updated to the second data table metadata.

[0023] A fifth aspect provides a computing device including a communication module and a processing module. The communication module is used to receive a creation instruction for a logical replication slot; the processing module is used to retrieve metadata of a first data table from a database according to the creation instruction; store the metadata of the first data table in a decoding dictionary of a shared storage medium; retrieve transaction logs sequentially from a write-ahead log; and generate a logical log based on the transaction logs and the metadata of the first data table.

[0024] In conjunction with the fifth aspect, in the first possible implementation, when the transaction log includes DDL statements, the processing module is also used to determine the second data table metadata based on the DDL statements and the first data table metadata, and update the first data table metadata of the decoding dictionary to the second data table metadata.

[0025] In conjunction with the fifth aspect or the first possible implementation of the fifth aspect, in the second possible implementation, the processing module is specifically used to obtain the consistency point of the computing device according to the creation instruction; receive the consistency points of other computing devices in the database system; determine the target snapshot corresponding to the maximum value among the consistency points of all computing devices; and obtain the first data table metadata from the distributed database according to the target snapshot.

[0026] In conjunction with the second possible implementation of the fifth aspect, in the third possible implementation, the processing module is specifically used to create a logical replication slot according to the creation instruction; when the logical replication slot is in the start state, it obtains the first active transaction set of the computing device; when the logical replication slot is in the waiting active transaction completion state, it obtains the second active transaction set of the computing device; when all active transactions in the first active transaction set have ended and all active transactions in the second active transaction set have ended, it sets the state of the logical replication slot to a consistent state; and it determines the consistency point of the computing device as the snapshot commit sequence number obtained in the consistent state.

[0027] For the explanation of terms, the steps for each module, and the beneficial effects in the fifth aspect, please refer to the corresponding descriptions in the first aspect.

[0028] A sixth aspect provides a database system, which serves as a first database system. The first database system includes a proxy node, multiple computing devices, and a shared storage medium. The proxy node is used to receive a creation instruction for a logical replication slot and send the creation instruction to the multiple computing devices. The computing devices are used to retrieve first data table metadata from the database according to the creation instruction; write the first data table metadata into a decoding dictionary of the shared storage medium; sequentially retrieve transaction logs from a first write-ahead log; and generate a logical log for the computing devices based on the transaction logs and the first data table metadata.

[0029] In conjunction with the sixth aspect, in the first possible implementation, when the transaction log includes DDL statements, the computing device is further configured to obtain the second data table metadata based on the DDL statements and the first data table metadata; and update the first data table metadata of the decoding dictionary to the second data table metadata.

[0030] In conjunction with the sixth aspect or the first possible implementation of the sixth aspect, in the second possible implementation, logical logs sent by other computing devices are obtained, and the logical logs of all computing devices are sorted into a logical log sequence; the proxy node is also used to receive the logical log sequence sent by the computing devices.

[0031] In conjunction with the sixth aspect or the above possible implementations, in the third possible implementation, the proxy node is also used to obtain the second write-ahead log sent by the computing device; when the log format of the second write-ahead log is different from that of the first write-ahead log, the logical log corresponding to the second write-ahead log is sent to the second database system, and the logical log corresponding to the second write-ahead log is used by the second database system for data synchronization.

[0032] For the explanation of terms, the steps for each module, and the beneficial effects in the sixth aspect, please refer to the corresponding descriptions in the second aspect.

[0033] A seventh aspect provides a computing device, including a communication module and a processing module. The communication module is used to receive a creation instruction for a logical replication slot; the processing module is used to retrieve first data table metadata from a database according to the creation instruction; store the first data table metadata in a write-ahead log; and sequentially retrieve transaction logs from the write-ahead log. The communication module is also used to send the write-ahead log to a standby computing device and generate a logical log based on the transaction logs and the first data table metadata. The computing device is used as the main computing device of a database system.

[0034] In conjunction with the seventh aspect, in the first possible implementation, when the transaction log includes DDL statements, the processing module is also used to determine the second data table metadata based on the DDL statements and the first data table metadata; and update the first data table metadata in the write-ahead log to the second data table metadata.

[0035] For the explanation of terms, the steps for each module, and the beneficial effects in the seventh aspect, please refer to the corresponding descriptions in the third aspect.

[0036] An eighth aspect provides a computing device including a communication module and a processing module. The communication module is used to receive a write-ahead log sent by a main computing device, and the processing module is used to sequentially retrieve a transaction log from the write-ahead log; and generate a logical log based on the transaction log and first data table metadata in the write-ahead log. The computing device is used as a backup computing device for a database system.

[0037] In conjunction with the eighth aspect, in the first possible implementation, when the transaction log includes DDL statements, the processing module is also used to obtain the second data table metadata based on the DDL statements and the first data table metadata, and update the first data table metadata in the write-ahead log to the second data table metadata.

[0038] For the explanation of terms, the steps for each module, and the beneficial effects in the eighth section, please refer to the corresponding descriptions in the fourth section.

[0039] A ninth aspect provides a computing device cluster, the computing device cluster including at least one computing device, each computing device including a processor and a memory, the memory storing computer-readable instructions, the processor of the at least one computing device executing the computer-readable instructions to cause the computing device cluster to perform the logical decoding method in the foregoing aspect or any possible implementation of the foregoing aspect.

[0040] A tenth aspect provides a computer-readable storage medium including computer-readable instructions; the computer-readable instructions are used to implement the logical decoding method in the foregoing aspect or any possible implementation of the foregoing aspect.

[0041] The eleventh aspect provides a computer program product comprising computer-readable instructions; the computer-readable instructions are used to implement the logical decoding method in the foregoing aspect or any possible implementation of the foregoing aspect. Attached Figure Description

[0042] Figure 1 is a schematic diagram of a distributed database in an embodiment of this application;

[0043] Figure 2 is another schematic diagram of the distributed database in an embodiment of this application;

[0044] Figure 3 is a flowchart of a logic decoding method in an embodiment of this application;

[0045] Figure 4 is a flowchart of determining a consistency point in an embodiment of this application;

[0046] Figure 5 is a schematic diagram of logical decoding in a distributed database in an embodiment of this application;

[0047] Figure 6A is a schematic diagram of a database upgrade causing logical decoding failure in an embodiment of this application;

[0048] Figure 6B is a schematic diagram of successful logical decoding after database upgrade in an embodiment of this application;

[0049] Figure 7 is another flowchart of the logic decoding method in an embodiment of this application;

[0050] Figure 8 is a flowchart of logic decoding performed by the main computing device and the backup computing device in an embodiment of this application.

[0051] Figure 9 is another flowchart of the logic decoding performed by the main computing device and the backup computing device in an embodiment of this application;

[0052] Figure 10 is a structural diagram of a computing device in an embodiment of this application;

[0053] Figure 11 is a structural diagram of a computing device in an embodiment of this application;

[0054] Figure 12 is a structural diagram of a computing device cluster in an embodiment of this application. Detailed Implementation

[0055] The logical decoding method of this application can be applied to database systems, such as distributed database systems, master-slave database systems, or heterogeneous database systems.

[0056] The following describes the scenario of logical decoding in a distributed database system. Referring to Figure 1, in one embodiment, the distributed database system of this application includes a proxy node 110, computing devices 121 to 12n, and a shared storage medium 130.

[0057] The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone. The computing devices 121 to 12n can communicate with each other via a remote procedure call protocol (RPC). The computing devices 121 to 12n are respectively connected to the shared storage medium 130 via a network to access the shared storage medium 130. The shared storage medium 130 can be, but is not limited to, a shared disk. The shared storage medium 130 can store write-ahead logs 1301 to 130n and a decoding dictionary 131, etc.

[0058] Agent node 110 can provide a data replication service (DRS). The functionality of agent node 110 can be implemented by any computing device within the database system. The data replication service is used for database migration and synchronization. This service can be used for data synchronization between primary and backup nodes, critical data backup, and overall data migration. The data replication service may include logical replication functionality, which refers to replicating target data or target data change information based on logical logs.

[0059] In logical replication, the process of converting the write-ahead log into the logical log is called logical decoding. Logical decoding relies on the write-ahead log and table metadata. The logical log records database modification operations, and its format can be, but is not limited to, Structured Query Language (SQL) or JSON format. SQL is used to manage and manipulate relational databases. SQL includes, but is not limited to, Data Manipulation Language (DML) and Data Definition Language (DDL). DML includes insert, update, and delete statements, used to add, delete, and modify data in the database. DDL is used to create, delete, and modify database objects, such as tables and databases.

[0060] Multi-version concurrency control (MVCC) supports concurrent reads and writes by storing multiple versions of data. Each transaction sees a specific version of the data when reading it, rather than the current value. It achieves read-write separation and does not require locking during data reads. Before making persistent modifications to data files, the changes are recorded in a write-ahead log. In the event of a database system failure before actual data changes, the write-ahead log can maintain data consistency.

[0061] To ensure the correctness and reliability of transactions during the writing or updating of data, database management systems must possess four properties: atomicity, consistency, isolation, and durability, collectively known as ACID. Atomicity means that a transaction is indivisible and irreducible. All operations within a transaction either complete entirely or not at all; it cannot end prematurely. If an error occurs during transaction execution, it will be rolled back to the state before the transaction began, as if the transaction had never been executed. Consistency means that the integrity of the database is not compromised before and after a transaction. This means that the written data conforms to all preset constraints, triggers, cascading rollbacks, etc. Isolation means allowing multiple concurrent transactions to read, write, and modify data in the database simultaneously. Isolation prevents data inconsistency caused by overlapping execution of multiple transactions. Transaction isolation is divided into different levels, including read uncommitted, read committed, repeatable read, and serializable. Durability means that after a transaction is completed, the changes to the data are permanent and will not be lost even in the event of a system failure.

[0062] After a user selects the data replication service of proxy node 110, proxy node 110 sends commands to computing devices 121 to 12n. Each computing device retrieves data table metadata from the decoding dictionary according to the commands, and then generates a logical log based on the write-ahead log and the data table metadata. Computing device 121 can retrieve the logical log from computing devices 122 to 12n, sorts the logical logs of all computing devices, and sends the sorted logical log to proxy node 110, which displays the sorted logical log. Furthermore, computing device 121 can synchronize decoding progress information to other computing devices and store the decoding progress information in the logical replication slot of the shared storage medium 130. Computing device 121 can also update the decoding dictionary based on data change information in the write-ahead log, so that the decoding dictionary includes different versions of data table metadata. Similarly, computing devices 122 to 12n can also update the decoding dictionary.

[0063] Referring to Figure 2, in one embodiment, the proxy node 110 can connect to computing devices 121 to 12n via a database interface. The database interface can be, but is not limited to, Java Database Connectivity (JDBC). Computing device 121 can process database tasks in storage instance 22 via threads 211, 212, and 213. It should be understood that the number of threads in computing device 121, and the number of threads processing tasks, is not limited to those shown in Figure 2. The databases corresponding to storage instance 22 include pluggable database (PDB) 231 and pluggable database 232. Storage instance 22 allows multiple pluggable databases to run, each with its own data files, control files, redo log files, etc. A PDB can be unplugged from one storage instance and inserted into another; this architecture enables hot-swapping of databases, i.e., inserting a new database without affecting existing databases. The logical replication management unit 241 in pluggable database 231 can manage logical replication slots, such as slot1, slot2, ..., slotn. The write-ahead log management unit 242 can manage the log stream. The architecture of other computing devices is similar to that of computing device 121.

[0064] The following describes a scenario where logical decoding is performed between primary and standby database systems. In another embodiment, both the primary and standby database systems can provide logical replication functionality, i.e., a dual-active replication scenario. The primary database system can send write-ahead logs to the standby database system. The standby database system performs logical replay based on the write-ahead logs to obtain the system tables. Then, the standby database system retrieves the table metadata from the system tables and performs logical decoding based on the table metadata. Alternatively, the primary database system can send write-ahead logs, including table metadata, to the standby database system, which then performs logical decoding based on the write-ahead logs.

[0065] The following describes a scenario for logical decoding in a heterogeneous database system. In one embodiment, the online transaction processing (OLTP) computing device records data change information of row-store data in a write-ahead log, then converts the write-ahead log into a logical log, and sends the logical log to the online analytical processing (OLAP) computing device. The OLAP computing device converts the logical log into SQL statements, and then executes the SQL statements on the column storage engine to change the column-store data. This allows for the synchronization of data in heterogeneous databases.

[0066] When different versions of a database store data in different formats, these different versions of the database can also be considered heterogeneous databases. In another embodiment, a first database system and a second database system are used to provide different versions of the database. After the computing device of the first database system converts the write-ahead log into a logical log, it sends the logical log to the computing device of the second database system. The computing device of the second database system converts the logical log into SQL statements, and then executes the SQL statements in its local database to achieve data synchronization.

[0067] Current logical decoding methods require retrieving data table metadata from system tables, thus preventing the reclamation of system tables during the logical decoding process. When performing logical decoding on multiple transaction logs in the write-ahead log, the computing device queries different versions of data table metadata, requiring storage nodes to retain multiple versions of the system table. This leads to rapid expansion of the database space, especially when logical decoding is slow. To address this issue, this application can copy the data table metadata required for logical decoding from the system table and store it in a shared storage medium. This eliminates the need to retain the system table during logical decoding, allowing for rapid reclamation of the system table. A detailed description follows, referring to Figure 3. In one embodiment, the logical decoding method of this application includes the following steps:

[0068] S301, Receive the creation command for the logical replication slot.

[0069] After the user inputs the creation command for the logical replication slot, the computing device creates the logical replication slot according to the creation command. The logical replication slot is used to stream the transaction log in the write-ahead log. It can identify the user's logical decoding task and track the progress of logical decoding. For example, Zhang San's logical replication slot is denoted as slot1, which represents Zhang San's logical decoding task.

[0070] S302. Obtain the metadata of the first data table from the database according to the creation instruction.

[0071] When the database provided by the database system is a distributed database, the user's logical decoding task can be executed simultaneously on multiple computing devices. In this case, logical decoding of different write-ahead logs may be required. This embodiment can obtain the latest version of the data table metadata (i.e., the first data table metadata). The latest version of the data table metadata includes the data table metadata that different write-ahead logs depend on. In this way, storing only one data table metadata in the decoding dictionary can meet the logical decoding needs of multiple computing devices, saving the storage resources occupied by the data table metadata.

[0072] When the database provided by the database system is not a distributed database, the computing device can determine the target snapshot based on the consistency point of the computing device, and obtain the first data table metadata from the database based on the target snapshot.

[0073] S303. Store the metadata of the first data table in the decoding dictionary of the shared storage medium.

[0074] Specifically, after copying the metadata of the first data table from the system tables of the database, the copied metadata is stored in a decoding dictionary on shared storage media. The decoding dictionary stores the metadata of the data tables that logical decoding depends on. Data table metadata is also known as system table information, table definition information, or table structure information. Data table metadata includes, but is not limited to, the following information: table name and column names.

[0075] S304. Obtain the transaction log from the write-ahead log.

[0076] The write-ahead log includes one or more transaction logs, which are used to record database change information.

[0077] S305. Generate the first logical log based on the transaction log and the metadata of the first data table.

[0078] The first data table metadata is read from the decoding dictionary based on the snapshot in the transaction log. If the status of the first data table metadata is "existing but not collected" (which can be recorded as EXIST_BUT_UNCOLLECTED), the decoding thread waits. If the status of the first data table metadata is "collected" (which can be recorded as COLLECTED), the first logical log is generated based on the transaction log and the first data table metadata. For example, if the transaction log includes `insert INTO t1 VALUES(1,2)`, and the first data table metadata includes `t1`, `r1`, `int`, `r2`, and `int`, the resulting logical log includes: `(t1 INSERT r1[int]:1r2[int]:2)`, which indicates that (1,2) is inserted into `t1`.

[0079] S306. Determine whether the transaction log includes DDL statements. If not, execute S309; ​​if yes, execute S307.

[0080] When the transaction log does not contain DDL statements, it indicates that the transaction did not modify the table metadata. When the transaction log includes DDL statements, it indicates that the transaction modified the table metadata, and the table metadata in the decoding dictionary needs to be updated.

[0081] S307. Determine the metadata of the second data table based on the DDL statement and the metadata of the first data table.

[0082] When a DDL statement adds a table, it adds metadata to the first table's metadata to obtain the second table's metadata. When a DDL statement deletes a table, it removes the table's metadata from the first table's metadata to obtain the second table's metadata. When a DDL statement modifies a table, it modifies the table's metadata according to the DDL statement, such as changing the table's columns or name, to obtain the second table's metadata.

[0083] S308. Update the metadata of the first data table of the decoding dictionary to the metadata of the second data table.

[0084] Create new data table metadata in the decoding dictionary and set its status to "existing but not collected". After obtaining the second data table metadata, populate it into the new data table metadata and set its status to "collected", i.e., incrementally collected.

[0085] S309. Determine if a stop command has been received. If not, execute S304. If yes, then end.

[0086] When a stop command is received, logical decoding is stopped. It should be noted that the conditions for stopping logical decoding are not limited to the stop command; they can be set according to the actual situation. For example, logical decoding can be stopped when all transaction logs in the write-ahead log have been logically decoded.

[0087] In this embodiment, the metadata of the data table required for logic decoding can be copied from the system table and stored in the shared storage medium. This way, there is no need to retain the system table during logic decoding, and the system table can be quickly reclaimed.

[0088] Secondly, data table metadata can be updated based on transaction logs, and subsequent transaction logs can be logically decoded based on the updated metadata. Compared to storing multiple versions of a data table, this significantly reduces the storage resources occupied by the data table and mitigates database space bloat.

[0089] When the database provided by the database system is a distributed database, a user's logical decoding task can be executed simultaneously on multiple computing devices. In this case, logical decoding of different write-ahead logs may be required, which typically relies on different versions of data table metadata. This application can perform logical decoding of different write-ahead logs based on a single data table metadata. The following describes this process. In an optional embodiment, S302 includes: obtaining the consistency point of the computing device according to a creation instruction; receiving consistency points from other computing devices in the database system; determining the target snapshot corresponding to the maximum value among all consistency points of the computing devices; and obtaining the first data table metadata from the distributed database based on the target snapshot.

[0090] In this embodiment, after receiving a creation instruction, one computing device in the database system can preempt the distributed lock and send the creation instruction to other computing devices. Each computing device obtains its local consistency point, which is the snapshot commit sequence number at the end of an active transaction. The maximum value among the consistency points of all computing devices is the global consistency point. Based on the global consistency point, the latest version of the data table metadata (i.e., the first data table metadata) can be obtained. The latest version of the data table metadata includes multiple data table metadata that the write-ahead log depends on. Thus, storing only one data table metadata in the decoding dictionary can satisfy the logical decoding needs of multiple computing devices. This eliminates the need to store multiple versions of the system table, thereby saving storage resources occupied by the system table.

[0091] The process of obtaining a local consistency point is described below. Referring to Figure 4, in an optional embodiment, obtaining the consistency point of the computing device according to the creation instruction may include the following steps:

[0092] S401. Create a logical copy slot according to the creation command.

[0093] When creating a logical replication slot, the current transaction log of the write-ahead log is used as the starting point. At this time, the reclamation of the write-ahead log and the minimum CSN of the reclamation are blocked.

[0094] S402. Set the state of the logical copy slot to the start state.

[0095] S403: Obtain the first set of active transactions of the computing device.

[0096] The first set of active transactions includes active transactions acquired when the logical replication slot is in the initial state.

[0097] S404. Set the state of the logical replication slot to the waiting for active transaction completion state.

[0098] S405. Determine whether all active transactions in the first active transaction set have ended. If yes, execute S407; otherwise, execute S406.

[0099] When all active transactions in the first active transaction set have ended, the logical replication slot is set to the waiting state for a full snapshot. If all active transactions in the first active transaction set have not ended, the active transactions are added to the second active transaction set.

[0100] S406. Add the active transaction to the second active transaction set of the computing device.

[0101] The second set of active transactions includes active transactions acquired while the logical replication slot is in a state of waiting for active transactions to complete.

[0102] S407. Set the state of the logical replication slot to the waiting for complete snapshot state, and stop adding active transactions to the second active transaction set of the computing device.

[0103] S408. When all active transactions in the second active transaction set have ended, set the state of the logical replication slot to a consistent state.

[0104] S409. Determine the consistency point of the computing device as the first snapshot commit sequence number obtained in the consistency state.

[0105] The snapshot commit sequence number, or CSN for short, is a sequence number generated when a transaction is committed. It corresponds one-to-one with the transaction and can be used to check the decoding progress.

[0106] In this embodiment, the computing device can obtain a local consistency point, which can be considered as a consistency point of different transaction logs in the computing device.

[0107] The above describes a method for logic decoding using a computing device. The following describes a method for logic decoding using a database system as shown in Figure 1. Referring to Figure 5, in another embodiment, the logic decoding method of this application includes the following steps:

[0108] S501, the agent node 110 sends the creation command for the logical replication slot to the computing device 121.

[0109] After receiving the instruction to create a logical replication slot, the agent node 110 sends the creation instruction to the computing device 121.

[0110] S502, computing device 121 sends the creation command to computing devices 122 to 12n.

[0111] S503, the computing device 121 retrieves the metadata of the first data table from the database according to the creation instruction.

[0112] S504, the computing device 121 stores the metadata of the first data table in the decoding dictionary 131 of the shared storage medium 130.

[0113] S505, the computing device 121 sequentially retrieves the transaction log from the write-ahead log 1301.

[0114] S506, The computing device 121 generates a logical log based on the transaction log and the metadata of the first data table.

[0115] S503-S506 are similar to S302-S305, and specific details can be found in the corresponding descriptions in the embodiment shown in Figure 3. When the transaction log includes DDL statements, computing device 121 obtains the second data table metadata based on the DDL statements and the first data table metadata. Computing device 121 updates the first data table metadata of the decoding dictionary to the second data table metadata. It should be noted that the logical decoding process performed by computing devices 122-12n is similar to the logical decoding process performed by computing device 121.

[0116] S507, Computing device 121 obtains logic logs sent by other computing devices.

[0117] S508, computing device 121 sorts the logical logs of all computing devices into a logical log sequence.

[0118] The computing device 121 can also synchronize the CSN of the logical log to other computing devices and persist the CSN of the logical log to the logical replication slot.

[0119] S509, Agent node 110 receives the logical log sequence sent by computing device 121.

[0120] In this embodiment, multiple computing devices in a distributed database can simultaneously execute the logic decoding task of the same user, thereby improving the logic decoding speed.

[0121] Secondly, it can update data table metadata, and based on the updated metadata, subsequent transaction logs can be logically decoded. This eliminates the need to retain multiple versions of the data table in the database, significantly reducing the storage resources occupied by the data table and effectively minimizing database space bloat.

[0122] In an optional embodiment, the logical decoding method of this application further includes: the proxy node 110 obtaining the second write-ahead log sent by the computing device 121; when the log format of the second write-ahead log is different from that of the first write-ahead log, the proxy node 110 sends the logical log corresponding to the second write-ahead log to the second database system, and the logical log corresponding to the second write-ahead log is used by the second database system for data synchronization.

[0123] In this embodiment, when the upgraded write-ahead log has a different log format than the original write-ahead log, the second database system cannot perform data synchronization based on the upgraded write-ahead log. Instead, it can perform data synchronization based on the logical log corresponding to the upgraded write-ahead log, thereby solving the data synchronization problem caused by the database upgrade.

[0124] Referring to Figure 6A, in one embodiment, a first database system provides a primary database, and a second database system provides a disaster recovery database. The data in the primary database and the disaster recovery database need to be kept synchronized. For example, after the first and second database systems synchronize the instance flashback point, both systems include a first version of the data, which includes, but is not limited to, undo segments. When the primary database is upgraded, the first database system generates a second version of the write-ahead log, meaning the log format has changed. After the first database system sends the second version of the write-ahead log to the second database system, the second system's logical playback component can logically replay the first version of the write-ahead log, but it cannot perform logical playback based on the second version of the write-ahead log; therefore, logical decoding fails.

[0125] Referring to Figure 6B, in another embodiment, after the first database system is upgraded, it is configured with a second version of the logic decoding component, and after the second database system is upgraded, it is configured with a second version of the logic playback component. After the first database system sends the logic log to the second database system, the logic playback component of the second database system can convert the logic log into SQL statements, i.e., the logic decoding is successful. Executing the SQL statements can achieve data synchronization between the heterogeneous databases.

[0126] The logical decoding method of this application will be described below with reference to the database system shown in Figure 1. Referring to Figure 7, in one embodiment, the logical decoding method of this application includes the following steps:

[0127] S701. Create table t1 in computing device 121.

[0128] In this embodiment, the user connects to the computing device 121 via gsql, and the operation to create table t1 in the computing device 121 is create table t1(r1 int,r2 int). r1 and r2 are the two columns of t1.

[0129] S702. Create slot1 in computing device 121.

[0130] A user creates slot1 in computing device 121. The logical replication manager in the database sends a create command to computing devices including write-ahead logs, such as computing devices 122, ..., 12n. The computing device receiving the create command obtains the local consistency point and creates the logical replication slot. It should be understood that a user can create a logical replication slot on any computing device.

[0131] Once all computing devices have obtained a consistency point, computing device 121 obtains a global consistency point based on the consistency points of all computing devices, and then scans the database based on the global consistency point, synchronizing the data table metadata to the decoding dictionary. Other computing devices obtain the data table metadata from the decoding dictionary.

[0132] S703, Insert data into t1.

[0133] For example, the INSERT INTO t1 VALUES(1,2); INSERT INTO t1 VALUES(2,3); INSERT INTO t1 VALUES(3,4).

[0134] S704. Delete table t1 in computing device 121.

[0135] For example, the statement to delete table 1 includes: drop table t1.

[0136] S705, execute vacuum.

[0137] Vacuum is a database maintenance operation used to clean and optimize a database. Its purposes include freeing up unused space, organizing data pages, improving query performance, and preventing data bloat.

[0138] S706. Extract the logic log in slot 1 of computing device 122.

[0139] S707, The database returns logical logs.

[0140] For example, the database returns three logical logs:

[0141] ("(11,11)",320000003,BEGIN)("(11,11)",320000003,"table public.t1 INSERT:r1[integer]:1 r2[integer]:2")("(11,11)",320000003,COMMIT);

[0142] ("(20,1)",320000004,BEGIN)("(20,1)",320000003,"table public.t1 INSERT:r1[integer]:2 r2[integer]:3")("(20,1)",320000004,COMMIT);

[0143] ("(34,1)",320000005,BEGIN)("(34,1)",320000003,"table public.t1 INSERT:r1[integer]:3 r2[integer]:4")("(34,1)",320000005,COMMIT).

[0144] In the first logical log, the first field is the transaction identifier, the second field is the transaction commit sequence number, and the third field is the data modification operation. For example, (11,11) is the transaction identifier, 320000003 is the transaction commit sequence number, and in `table public.t1 INSERT:r1[integer]:1 r2[integer]:2`, `table public.t1` identifies the operation object as `t1`, `INSERT` indicates an insert operation, and `r1[integer]:1 r2[integer]:2` indicates that both `r1` and `r2` are of type integer, inserting a value of 1 into `r1` and a value of 2 into `r2`. This logical log reflects the user operation: `INSERT INTO t1 VALUES(1,2)`.

[0145] In this embodiment, the computing device can quickly delete table t1 without affecting logical decoding. This enables rapid data table reclamation and reduces database space bloat.

[0146] In a master-slave database scenario, the master database system sends a write-ahead log to the slave database system. The slave database system performs logical replay based on the write-ahead log, then retrieves data table metadata from the system table obtained from the logical replay, and performs logical decoding based on this data table metadata. The logical decoding process in the slave database system involves multiple steps, resulting in low efficiency. This application simplifies the steps in the slave database system, thereby improving logical decoding efficiency. A detailed description follows, referring to Figure 8. In one embodiment, the logical decoding method of this application includes the following steps:

[0147] S801, The main computing device receives the command to create a logical copy slot.

[0148] S802, The main computing device retrieves the metadata of the first data table from the database according to the creation instruction.

[0149] S803, The main computing device saves the metadata of the first data table in the write-ahead log.

[0150] S804. The primary computing unit sends the write-ahead log to the backup computing unit.

[0151] S805. The main computing unit retrieves the transaction logs sequentially from the write-ahead log.

[0152] S806. The main computing device generates a logical log based on the transaction log and the metadata of the first data table.

[0153] S805 to S806 is the logic decoding process of the main computing device, and S804 to S808 is the logic decoding process of the backup computing device. There is no fixed order between the two processes.

[0154] S807, The standby computing unit obtains the transaction log from the write-ahead log.

[0155] S808, the standby computing device generates a logical log based on the transaction log and the metadata of the first data table.

[0156] In this embodiment, after the primary database system sends the write-ahead log, which includes data table metadata, to the backup database system, the backup database system can perform logical decoding based on the data table metadata in the write-ahead log. This eliminates the need for logical replay based on the write-ahead log, thus simplifying the logical decoding steps and improving logical decoding efficiency.

[0157] In an optional embodiment, the logical decoding method of this application further includes: when the transaction log includes DDL statements, the main computing device determines the second data table metadata based on the DDL statements and the first data table metadata; updates the first data table metadata in the write-ahead log to the second data table metadata; when the transaction log includes DDL statements, the standby computing device determines the second data table metadata based on the DDL statements and the first data table metadata; updates the first data table metadata in the write-ahead log to the second data table metadata.

[0158] In this embodiment, after modifying the data table, the metadata of the write-ahead log can be updated. Based on the updated metadata, subsequent transaction logs can be logically decoded. This allows for continuous logical decoding while retaining only one version of the data table metadata, eliminating the need to store multiple versions of the data table in the database. This significantly reduces the storage resources occupied by the data table and effectively minimizes database space bloat.

[0159] Referring to Figure 9, in one embodiment, the logic decoding method of this application includes the following steps:

[0160] S901. Create table t1 in the main computing device.

[0161] S902. Create slot1 in the main computing device.

[0162] S903, Insert data into t1.

[0163] When inserting data into t1, the data changes can be recorded in the write-ahead log. Then, the main computing device retrieves the data table metadata of t1 from the database and saves it in the write-ahead log.

[0164] S904. Extract the logic log from slot1 of the standby computing device.

[0165] After the primary computing device creates slot1, the standby computing device can also create slot1, and their slot1s correspond to the same user's logical decoding task. When the primary computing device sends the write-ahead log to the standby computing device, the standby computing device retrieves the transaction log and data table metadata from the write-ahead log, and generates the logical log based on the transaction log and data table metadata.

[0166] S905, The database returns a logical log.

[0167] Returning logical logs from a database means returning logical logs to the application or user.

[0168] In this embodiment, the standby computing device can obtain transaction logs and data table metadata from the write-ahead log, and then perform logical decoding based on the transaction logs and data table metadata. Furthermore, it can update the data table metadata based on the transaction logs, and then perform logical decoding on subsequent transaction logs. This allows for logical decoding without retrieving data table metadata from system tables or other storage locations; therefore, this logical decoding method can be considered a self-closing logical decoding method.

[0169] Referring to Figure 10, this application provides a computing device 1000, which includes a communication module 1001 and a processing module 1002.

[0170] The computing device 1000 can implement the functions of the computing device in the embodiment shown in FIG3. In one embodiment, the communication module 1001 is used to receive the creation instruction of the logical replication slot; the processing module 1002 is used to obtain the metadata of the first data table from the database according to the creation instruction; store the metadata of the first data table in the decoding dictionary of the shared storage medium; obtain the transaction log sequentially from the write-ahead log, and generate the logical log according to the transaction log and the metadata of the first data table.

[0171] In an optional embodiment, when the transaction log includes DDL statements, the processing module 1002 is further configured to determine the second data table metadata based on the DDL statements and the first data table metadata, and update the first data table metadata of the decoding dictionary to the second data table metadata.

[0172] In another alternative embodiment, the processing module 1002 is specifically configured to obtain the consistency point of the computing device according to the creation instruction; receive the consistency points of other computing devices in the database system; determine the target snapshot corresponding to the maximum value among the consistency points of all computing devices; and obtain the first data table metadata from the distributed database according to the target snapshot.

[0173] In another optional embodiment, the processing module 1002 is specifically configured to create a logical replication slot according to the creation instruction; when the logical replication slot is in the start state, obtain a first active transaction set of the computing device; when the logical replication slot is in the waiting active transaction completion state, obtain a second active transaction set of the computing device; when all active transactions in the first active transaction set have ended and all active transactions in the second active transaction set have ended, set the state of the logical replication slot to a consistent state; and determine the consistency point of the computing device as the snapshot commit sequence number obtained in the consistent state.

[0174] The computing device 1000 can also implement the functions of the main computing device in the embodiment shown in FIG8. In one embodiment, the communication module 1001 is used to receive the creation instruction of the logical replication slot; the processing module 1002 is used to obtain the first data table metadata from the database according to the creation instruction; save the first data table metadata in the write-ahead log; and obtain the transaction log sequentially from the write-ahead log; the communication module 1001 is also used to send the write-ahead log to the backup computing device and generate the logical log according to the transaction log and the first data table metadata.

[0175] In an optional embodiment, when the transaction log includes DDL statements, the processing module 1002 is further configured to determine the second data table metadata based on the DDL statements and the first data table metadata; and update the first data table metadata in the write-ahead log to the second data table metadata.

[0176] The computing device 1000 can also implement the functions of the backup computing device in the embodiment shown in FIG8. In one embodiment, the communication module 1001 is used to receive the write-ahead log sent by the main computing device, and the processing module 1002 is used to sequentially obtain the transaction log from the write-ahead log; and generate a logical log based on the transaction log and the first data table metadata in the write-ahead log.

[0177] In an optional embodiment, when the transaction log includes DDL statements, the processing module 1002 is further configured to obtain the second data table metadata based on the DDL statements and the first data table metadata, and update the first data table metadata in the write-ahead log to the second data table metadata.

[0178] This application also provides a database system, which includes a proxy node 110, computing devices 121 to 12n, and a shared storage medium 130 as shown in FIG1. ​​In one embodiment, the proxy node 110 is used to receive a creation instruction for a logical replication slot and send the creation instruction to the computing devices 121 to 12n. Each computing device is used to obtain first data table metadata from the database according to the creation instruction; write the first data table metadata into a decoding dictionary of the shared storage medium 1103; sequentially obtain transaction logs from a first write-ahead log; and generate a logical log of the computing device according to the transaction logs and the first data table metadata.

[0179] In an optional embodiment, when the transaction log includes DDL statements, each computing device is further configured to obtain second data table metadata based on the DDL statements and first data table metadata; and update the first data table metadata of the decoding dictionary to the second data table metadata.

[0180] In another optional embodiment, computing device 121 is further configured to acquire logical logs sent by other computing devices and sort the logical logs of all computing devices into a logical log sequence; the proxy node is further configured to receive the logical log sequence sent by the computing devices. It should be noted that the computing device that sorts the logical logs of all computing devices may be, but is not limited to, computing device 121, and can be configured according to actual conditions.

[0181] In another optional embodiment, the proxy node 110 is also used to obtain the second write-ahead log sent by the computing device 121; when the log format of the second write-ahead log is different from that of the first write-ahead log, the logical log corresponding to the second write-ahead log is sent to the second database system, and the logical log corresponding to the second write-ahead log is used by the second database system for data synchronization.

[0182] This application also provides a computing device 1100. As shown in FIG11, the computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate with each other via the bus 1102. It should be understood that this application does not limit the number of processors and memory in the computing device 1100.

[0183] Bus 1102 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 11, but this does not imply that there is only one bus or one type of bus. Bus 1104 can include pathways for transmitting information between various components of computing device 1100 (e.g., memory 1106, processor 1104, communication interface 1108).

[0184] Processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP). The processor includes multiple processing cores.

[0185] The memory 1106 may include volatile memory, such as random access memory (RAM). The memory 1106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD). In some embodiments, the memory 1106 stores executable program code, which the processor 1104 executes to implement the functions of the aforementioned processing module 1002 and communication module 1001, thereby implementing the aforementioned logic decoding method. In other embodiments, the memory 1106 stores executable program code, which the processor 1104 executes to implement the functions of the aforementioned proxy node 110 or computing devices 121 to 12n, thereby implementing the aforementioned logic decoding method.

[0186] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.

[0187] This application also provides a computing device cluster capable of implementing the logical decoding method of this application. Referring to FIG12, the computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0188] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may store the same instructions for executing the logic decoding method. In some possible implementations, the memory 1006 of one or more computing devices 1000 in the computing device cluster may each store a portion of the instructions for executing the logic decoding method. In other words, a combination of one or more computing devices 1000 can jointly execute the instructions for executing the logic decoding method.

[0189] This application also provides a computer program product containing instructions. The computer program product may be software or program products containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on at least one computing device, it causes the at least one computing device to execute the logical decoding method of this application. The code of the computer program product may reside within a computing node, storage engine, or SQL interface layer.

[0190] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to execute the logical decoding method of this application.

[0191] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0192] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of logic decoding, characterized by, The method is applied to a computing device of a database system, the database system further including a shared storage medium, and the method includes: Receive the creation command for the logical replication slot; The first data table metadata is retrieved from the database according to the creation instruction; The metadata of the first data table is stored in the decoding dictionary of the shared storage medium; Retrieve the transaction logs sequentially from the write-ahead log; A logical log is generated based on the transaction log and the metadata of the first data table.

2. The method of claim 1, wherein, The method further includes: When the transaction log includes Data Definition Language (DDL) statements, the second data table metadata is determined based on the DDL statements and the first data table metadata. Update the metadata of the first data table of the decoding dictionary to the metadata of the second data table.

3. The method according to claim 1 or 2, characterized in that, The database is a distributed database, and the step of obtaining the first data table metadata from the database according to the creation instruction includes: The consistency point of the computing device is obtained according to the creation instructions; Receive consistency points from other computing devices in the database system; Determine the target snapshot corresponding to the maximum value among the consistency points of all computing devices; The first data table metadata is obtained from the distributed database based on the target snapshot.

4. The method of claim 3, wherein, The step of obtaining the consistency point of the computing device according to the creation instruction includes: Create a logical replication slot according to the creation instructions; With the logical replication slot in the start state, acquire the first active transaction set of the computing device; When the logical replication slot is in a state of waiting for active transactions to complete, obtain the second set of active transactions of the computing device; When all active transactions in the first active transaction set have ended and all active transactions in the second active transaction set have ended, the state of the logical replication slot is set to a consistent state. The consistency point of the computing device is determined to be the snapshot submission sequence number obtained when the consistency state is reached.

5. A method of logic decoding, characterized by, The method is applied to a first database system, the first database system including a proxy node, multiple computing devices, and a shared storage medium, the method comprising: The proxy node receives the creation instruction for the logical replication slot and sends the creation instruction to the plurality of computing devices; The computing device retrieves first data table metadata from the distributed database according to the creation instruction; stores the first data table metadata in the decoding dictionary of the shared storage medium; retrieves transaction logs sequentially from the first write-ahead log; and generates the logical log of the computing device based on the transaction logs and the first data table metadata.

6. The method of claim 5, wherein, The method further includes: When the transaction log includes Data Definition Language (DDL) statements, the computing device obtains the second data table metadata based on the DDL statements and the first data table metadata. The computing device updates the metadata of the first data table of the decoding dictionary to the metadata of the second data table.

7. The method according to claim 5 or 6, characterized in that, The method further includes: The computing device receives logical logs sent by other computing devices and sorts the logical logs of all computing devices into a logical log sequence. The proxy node receives the logical log sequence sent by the computing device.

8. The method according to any one of claims 5 to 7, characterized in that, The method further includes: The proxy node obtains the second write-ahead log sent by the computing device; When the log format of the second write-ahead log is different from that of the first write-ahead log, the proxy node sends the logical log corresponding to the second write-ahead log to the second database system. The logical log corresponding to the second write-ahead log is used by the second database system for data synchronization.

9. A method of logic decoding, characterized by, The method is applied to the main computing device of a database system, the main computing device being used to provide a main database, and the database system further including a backup computing device being used to provide a backup database, the method comprising: Receive the creation command for the logical replication slot; The first data table metadata is retrieved from the database according to the creation instruction; The metadata of the first data table is saved in the write-ahead log; Send the write-ahead log to the backup computing device; Retrieve the transaction logs sequentially from the write-ahead log; A logical log is generated based on the transaction log and the metadata of the first data table.

10. The method of claim 9, wherein, The method further includes: When the transaction log includes Data Definition Language (DDL) statements, the second data table metadata is determined based on the DDL statements and the first data table metadata. Update the metadata of the first data table in the write-ahead log to the metadata of the second data table.

11. A method of logic decoding, characterized by, The method is applied to a backup computing device in a database system, the backup computing device being used to provide a backup database. The database system further includes a primary computing device, the primary computing device being used to provide a primary database. The method includes: Obtain the write-ahead log sent by the main computing device, the write-ahead log including metadata of the first data table; Retrieve the transaction logs sequentially from the write-ahead log; A logical log is generated based on the transaction log and the metadata of the first data table.

12. The method of claim 11, wherein, The method further includes: When the transaction log includes Data Definition Language (DDL) statements, the second data table metadata is determined based on the DDL statements and the first data table metadata. Update the metadata of the first data table in the write-ahead log to the metadata of the second data table.

13. A computing device, comprising: include: The communication module is used to receive the creation command for the logical replication slot; The processing module is used to retrieve the metadata of the first data table from the database according to the creation instruction; The metadata of the first data table is stored in the decoding dictionary of the shared storage medium; Transaction logs are sequentially retrieved from the write-ahead log; logical logs are generated based on the transaction logs and the metadata of the first data table.

14. The apparatus according to claim 13, characterized in that, When the transaction log includes Data Definition Language (DDL) statements, the processing module is further configured to determine the second data table metadata based on the DDL statements and the first data table metadata. Update the metadata of the first data table of the decoding dictionary to the metadata of the second data table.

15. The apparatus of claim 13 or 14, wherein, The processing module is specifically used to obtain the consistency point of the computing device according to the creation instruction; receive the consistency points of other computing devices in the database system; determine the target snapshot corresponding to the maximum value among the consistency points of all computing devices; and obtain the first data table metadata from the distributed database according to the target snapshot.

16. The apparatus of claim 15, wherein, The processing module is specifically used to create a logical replication slot according to the creation instruction; and, when the logical replication slot is in the start state, to obtain the first active transaction set of the computing device. When the logical replication slot is in a state of waiting for active transactions to complete, obtain the second set of active transactions of the computing device; When all active transactions in the first active transaction set have ended and all active transactions in the second active transaction set have ended, the state of the logical replication slot is set to a consistent state. The consistency point of the computing device is determined to be the snapshot commit sequence number obtained when the consistency state is reached.

17. A database system, characterized in that The database system is used as a first database system, which includes proxy nodes, multiple computing devices, and a shared storage medium. The proxy node is used to receive the creation instruction of the logical replication slot and send the creation instruction to the plurality of computing devices; The computing device is used to retrieve first data table metadata from the database according to the creation instruction; Write the metadata of the first data table into the decoding dictionary of the shared storage medium; The transaction logs are sequentially obtained from the first write-ahead log; the logical logs of the computing device are generated based on the transaction logs and the metadata of the first data table.

18. The database system of claim 17, wherein, When the transaction log includes Data Definition Language (DDL) statements, the computing device is further configured to obtain second data table metadata based on the DDL statements and the first data table metadata. Update the metadata of the first data table of the decoding dictionary to the metadata of the second data table.

19. The database system according to claim 17 or 18, characterized in that, The computing device is also used to acquire logical logs sent by other computing devices and sort the logical logs of all computing devices into a logical log sequence. The proxy node is also used to receive logical log sequences sent by the computing device.

20. The database system according to any one of claims 17 to 19, characterized in that, The proxy node is also used to obtain the second write-ahead log sent by the computing device; when the log format of the second write-ahead log is different from that of the first write-ahead log, the logical log corresponding to the second write-ahead log is sent to the second database system, and the logical log corresponding to the second write-ahead log is used by the second database system for data synchronization.

21. A computing device, comprising: The computing device is used as the main computing device of the database system, and the computing device includes: The communication module is used to receive the creation command for the logical replication slot; The processing module is configured to retrieve the metadata of the first data table from the database according to the creation instruction; and store the metadata of the first data table in a write-ahead log. The communication module is also used to send the write-ahead log to a backup computing device; The processing module is further configured to sequentially obtain transaction logs from the write-ahead logs and generate logical logs based on the transaction logs and the metadata of the first data table.

22. The apparatus of claim 21, wherein, When the transaction log includes Data Definition Language (DDL) statements, the processing module is further configured to determine the second data table metadata based on the DDL statements and the first data table metadata. Update the metadata of the first data table in the write-ahead log to the metadata of the second data table.

23. A computing device, comprising: include: A communication module is used to receive a write-ahead log sent by the main computing device, the write-ahead log including metadata of a first data table; The processing module is used to sequentially obtain transaction logs from the write-ahead logs and generate logical logs based on the transaction logs and the metadata of the first data table.

24. The apparatus according to claim 23, characterized in that, When the transaction log includes Data Definition Language (DDL) statements, the processing module is further configured to obtain second data table metadata based on the DDL statements and the first data table metadata, and update the first data table metadata in the write-ahead log to the second data table metadata.

25. A cluster of computing devices, characterized in that, The system includes at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 12.

26. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a cluster of computing devices, perform the method as described in any one of claims 1 to 12.

27. A computer program product comprising instructions, wherein: When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1 to 12.

Images