Heterogeneous database synchronization method and device, electronic equipment and storage medium

Abstract

The invention relates to a heterogeneous database synchronization method and device, electronic equipment and a storage medium, and the method comprises the steps: analyzing a transaction log of a source database, extracting and generating an abstract syntax tree of a DDL event, and carrying out the node-level syntax conversion of the abstract syntax tree according to the database type of a target database and a preset syntax mapping rule base, and obtaining an equivalent DDL operation sequence of the target database, blocking a data processing operation corresponding to the database object associated with the DDL event, and executing the equivalent DDL operation sequence on the target database. The whole-process intelligent processing of the isomerous database DDL is realized without depending on manpower, the equivalent DDL operation sequence of the target database having semantic consistency with the DDL event of the source database is obtained through dynamic replenishment and deep analysis of the DDL event and the context semantics of the DDL event in combination with the grammar mapping rule base, the processing efficiency of isomerous database DDL synchronization is improved, and the processing efficiency of isomerous database DDL synchronization is improved. And meanwhile, the safety and reliability of synchronous processing can be ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database synchronization, and particularly relates to a heterogeneous data synchronization method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the development of the Internet, enterprises have an increasing demand for heterogeneous database synchronization, and the requirements for synchronization technology are also increasingly high. As an important part of enterprise data integration and data governance, the development of heterogeneous database synchronization technology is of great significance for breaking down data barriers and improving data value.

[0003] However, the compatibility of DDL (Data Definition Language) between heterogeneous databases is poor, the semantic difference is large, and the execution is complex, which makes it difficult for existing synchronization technology to achieve efficient, safe and intelligent synchronization processing of DDL of heterogeneous databases. SUMMARY

[0004] The present application provides a heterogeneous database synchronization method, device, electronic equipment and storage medium to solve the problem that existing synchronization technology cannot achieve efficient, safe and intelligent synchronization processing of DDL of heterogeneous databases.

[0005] In a first aspect, the present application provides a heterogeneous database synchronization method, which comprises:

[0006] parsing a transaction log of a source database, extracting a data definition language (DDL) event and context information of the DDL event, and generating an abstract syntax tree of the DDL event;

[0007] performing node-level syntax conversion on the abstract syntax tree according to a database type of a target database and a preset syntax mapping rule library, to obtain an equivalent DDL operation sequence of the target database;

[0008] blocking data processing operations corresponding to database objects associated with the DDL event, and executing the equivalent DDL operation sequence on the target database.

[0009] In a possible implementation, the node-level syntax conversion on the abstract syntax tree according to the database type of the target database and the preset syntax mapping rule library to obtain the equivalent DDL operation sequence of the target database comprises:

[0010] determining a target syntax mapping rule set from the preset syntax mapping rule library according to the database type of the target database, wherein the target syntax mapping rule set includes syntax mapping rules between the source database and the target database;

[0011] Traverse the nodes in the abstract syntax tree, and perform the following processing for the traversed nodes to obtain a reconstructed abstract syntax tree:

[0012] Search the target syntax mapping rule set for a syntax mapping rule containing the syntax component in the node; in the case of finding, replace the syntax component in the node with a target syntax component according to the found syntax mapping rule; in the case of not finding, trigger semantic conflict detection, and perform syntax compensation operation on the syntax component in the node according to the semantic conflict detection result;

[0013] According to the reconstructed abstract syntax tree, obtain an equivalent DDL operation sequence of the target database.

[0014] In a possible implementation, the triggering of the semantic conflict detection and the performing of the syntax compensation operation on the syntax component in the node according to the semantic conflict detection result include:

[0015] Parsing the syntax component in the node to obtain keyword information;

[0016] In the case where the keyword information indicates that the syntax component in the node belongs to a constraint mechanism, performing syntax compensation operation on the syntax component in the node by using a constraint equivalent compensation mechanism;

[0017] In the case where the keyword information indicates that the syntax component in the node belongs to an index mechanism or a storage engine component, performing syntax compensation operation on the syntax component in the node by using a function simulation compensation mechanism.

[0018] In a possible implementation, the performing of the syntax compensation operation on the syntax component in the node by using the constraint equivalent compensation mechanism includes:

[0019] Generating a program logic unit replacing the original constraint mechanism;

[0020] Executing the program logic unit to perform syntax compensation operation on the syntax component in the node.

[0021] In a possible implementation, the performing of the syntax compensation operation on the syntax component in the node by using the function simulation compensation mechanism includes:

[0022] Generating a plurality of equivalent operation statements simulating the original index mechanism or the original storage engine component;

[0023] Executing the plurality of equivalent operation statements to perform syntax compensation operation on the syntax component in the node.

[0024] In a possible implementation, the data processing operation corresponding to the database object associated with the DDL event is blocked, and the sequence of equivalent DDL operations is executed on the target database, comprising:

[0025] The database object associated with the DDL event is determined as a target database object;

[0026] A global barrier is applied for the target database object, and a global barrier identifier is generated;

[0027] The data processing operation corresponding to the target database object is blocked based on the global barrier identifier;

[0028] The sequence of equivalent DDL operations is executed on the target database in the global barrier effective state.

[0029] In a possible implementation, after the sequence of equivalent DDL operations is executed on the target database, the method further comprises:

[0030] If the sequence of equivalent DDL operations is executed successfully on the target database, the global barrier for the target database object is released based on the global barrier identifier, and the data processing operation corresponding to the target database object is continued to be executed;

[0031] If the sequence of equivalent DDL operations is not executed successfully on the target database, a fault-tolerant processing mechanism is triggered for the sequence of equivalent DDL operations, and if the sequence of equivalent DDL operations is still not executed successfully on the target database after the fault-tolerant processing, the global barrier for the target database object is released according to a safe replay mechanism.

[0032] In a possible implementation, the fault-tolerant processing mechanism is triggered for the sequence of equivalent DDL operations, comprising:

[0033] The sequence of equivalent DDL operations is executed again on the target database until the execution is successful or a set number of execution times is reached.

[0034] In a possible implementation, the global barrier for the target database object is released according to the safe replay mechanism, comprising:

[0035] The data structure of the target database is rolled back;

[0036] After the rollback is completed, the global barrier for the target database object is released, and the data processing operation corresponding to the target database object is continued to be executed.

[0037] In a second aspect, the application provides a heterogeneous database synchronization device, comprising:

[0038] a log analysis module, configured to analyze a transaction log of a source database, extract a data definition language (DDL) event and context information of the DDL event, and generate an abstract syntax tree of the DDL event;

[0039] an operation sequence generation module, configured to perform node-level syntax conversion on the abstract syntax tree according to a database type of a target database and a preset syntax mapping rule library, and obtain an equivalent DDL operation sequence of the target database;

[0040] an operation sequence execution module, configured to block a data processing operation corresponding to a database object associated with the DDL event, and execute the equivalent DDL operation sequence on the target database.

[0041] In a possible implementation, the operation sequence generation module comprises:

[0042] a mapping rule determination unit, configured to determine a target syntax mapping rule set from the preset syntax mapping rule library according to the database type of the target database, the target syntax mapping rule set comprising syntax mapping rules between the source database and the target database;

[0043] a node traversal unit, configured to traverse a node in the abstract syntax tree, and perform the following processing on the traversed node to obtain a reconstructed abstract syntax tree:

[0044] searching, from the target syntax mapping rule set, for a syntax mapping rule comprising a syntax component in the node; in the case of finding, replacing the syntax component in the node with a target syntax component according to the found syntax mapping rule; in the case of not finding, triggering semantic conflict detection, and performing a syntax compensation operation on the syntax component in the node according to a result of the semantic conflict detection;

[0045] an operation sequence determination unit, configured to obtain the equivalent DDL operation sequence of the target database according to the reconstructed abstract syntax tree.

[0046] In a possible implementation, the node traversal unit comprises:

[0047] a node analysis subunit, configured to analyze a syntax component in the node to obtain keyword information;

[0048] a constraint processing subunit, configured to, in the case where the keyword information indicates that the syntax component in the node belongs to a constraint mechanism, perform a syntax compensation operation on the syntax component in the node by using a constraint equivalent compensation mechanism;

[0049] The index processing subunit is configured to, in a case where the keyword information indicates that the syntax component in the node belongs to an index mechanism or a storage engine component, perform syntax compensation operation on the syntax component in the node by using a functional simulation compensation mechanism.

[0050] In a possible implementation, the constraint processing subunit is specifically configured to:

[0051] generate a program logic unit replacing the original constraint mechanism;

[0052] execute the program logic unit to perform syntax compensation operation on the syntax component in the node.

[0053] In a possible implementation, the index processing subunit is specifically configured to:

[0054] generate a plurality of equivalent operation statements simulating the original index mechanism or the original storage engine component;

[0055] execute the plurality of equivalent operation statements to perform syntax compensation operation on the syntax component in the node.

[0056] In a possible implementation, the operation sequence execution module comprises:

[0057] a target database object determination subunit configured to determine a database object associated with the DDL event as a target database object;

[0058] a global barrier identification generation subunit configured to apply a global barrier to the target database object and generate a global barrier identification;

[0059] a data processing subunit configured to block data processing operation corresponding to the target database object based on the global barrier identification;

[0060] an operation sequence execution subunit configured to execute the equivalent DDL operation sequence on the target database in a case where the global barrier is effective.

[0061] In a possible implementation, the operation sequence execution module further comprises:

[0062] a global barrier release subunit configured to, in a case where the execution of the equivalent DDL operation sequence on the target database is successful, release the global barrier for the target database object based on the global barrier identification, and continue to perform data processing operation corresponding to the target database object;

[0063] a fault-tolerant processing subunit, configured to trigger a fault-tolerant processing mechanism for the equivalent DDL operation sequence in a case that the execution of the equivalent DDL operation sequence on the target database is unsuccessful, and release the global barrier for the target database object according to a safe replay mechanism in a case that the execution of the equivalent DDL operation sequence on the target database is still unsuccessful after the fault-tolerant processing.

[0064] In a possible implementation, the fault-tolerant processing subunit is specifically configured to:

[0065] re-execute the equivalent DDL operation sequence on the target database until the execution is successful or a set number of execution times is reached.

[0066] In a possible implementation, the fault-tolerant processing subunit is further configured to:

[0067] rollback the data structure of the target database;

[0068] after the rollback is completed, release the global barrier for the target database object, and continue to execute the data processing operation corresponding to the target database object.

[0069] In a third aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the processor is configured to execute a heterogeneous database synchronization program stored in the memory, so as to implement the heterogeneous database synchronization method in any one of the first aspect.

[0070] In a fourth aspect, the present application provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the heterogeneous database synchronization method in any one of the first aspect.

[0071] Compared with the prior art, the technical solution provided in the embodiments of the present application has the following advantages: the method provided in the embodiments of the present application extracts and generates an abstract syntax tree of a DDL event by analyzing a transaction log of a source database, performs node-level syntax conversion on the abstract syntax tree according to a database type of a target database and a preset syntax mapping rule library, obtains an equivalent DDL operation sequence of the target database, blocks data processing operations corresponding to database objects associated with the DDL event, and executes the equivalent DDL operation sequence on the target database. The full-process intelligent processing of heterogeneous database DDL is realized without relying on manual work. The equivalent DDL operation sequence of the target database having semantic consistency with the source database DDL event is obtained through dynamic replenishment and deep analysis of the DDL event and its context semantics, in combination with the syntax mapping rule, the processing efficiency of heterogeneous database DDL synchronization is improved, and the safety and reliability in the synchronization process can be ensured by analyzing the dependent data objects of the DDL event and blocking the associated data processing operations. BRIEF DESCRIPTION OF DRAWINGS

[0072] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0074] One or more embodiments are exemplarily illustrated by pictures in the drawings corresponding thereto, and these exemplary illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, unless otherwise specified. The drawings in the drawings do not constitute a proportional limitation.

[0075] Figure 1 An embodiment flowchart of a heterogeneous database synchronization method provided in the embodiments of the present application;

[0076] Figure 2 An embodiment flowchart of another heterogeneous database synchronization method provided in the embodiments of the present application;

[0077] Figure 3 An embodiment flowchart of still another heterogeneous database synchronization method provided in the embodiments of the present application;

[0078] Figure 4 A structural block diagram of a heterogeneous database synchronization device provided in the embodiments of the present application;

[0079] Figure 5 A structural schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0080] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0081] The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. For the purpose of simplifying the present application, components and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, reference numerals and / or letters can be repeated in different examples in the present application. Such repetition is for the purpose of simplification and clarity, and does not indicate the relationship between the various embodiments and / or settings discussed.

[0082] To solve the technical problem that the prior art is difficult to implement efficient, secure, and full-link intelligent synchronization processing of heterogeneous database DDL, the present application provides a heterogeneous database synchronization method and device, an electronic device, and a storage medium, which implement full-process intelligent processing of heterogeneous database DDL, do not need to rely on manual work, obtain an equivalent DDL operation sequence of a target database having semantic consistency with a source database DDL event by dynamic replenishment and deep analysis of a DDL event and its context semantics in combination with a syntax mapping rule, improve the processing efficiency of heterogeneous database DDL synchronization, and at the same time, block associated data processing operations by analyzing dependent data objects of the DDL event, which can ensure the safety and reliability in the synchronization process.

[0083] Figure 1 An embodiment flowchart of a heterogeneous database synchronization method provided for an embodiment of the present application is shown in FIG. 1, including the following steps: Figure 1

[0084] Step 101: Analyzing a transaction log of a source database, extracting a data definition language (DDL) event and context information of the DDL event, and generating an abstract syntax tree of the DDL event.

[0085] ​The source database can be any type of relational database (such as MySQL, Oracle, SQL Server) or non-relational database (such as MongoDB, if it supports DDL-like operations) that stores the original data and data structure definitions of the business system. In the data synchronization scenario, the structure or data changes of the source database are captured and synchronized to the target database (such as from the production library to the test library, from the old version library to the new version library).

[0086] The DDL event can refer to the structural change behavior and related records generated when the DDL statement is executed in the source database, which is essentially an event triggered when creating, modifying, or deleting the structure of a database object (such as a table, index, view, etc.).

[0087] The context information can refer to the execution time of the corresponding DDL data statement, the execution user (for auditing), and the current other fields of the table (for checking dependencies), for example, the foreign key association table referred to when creating the table.

[0088] The abstract syntax tree is an abstract representation of the syntax structure of the source code, which presents the syntax structure of the code in a tree-like data structure, ignoring the details in the source code (such as spaces, comments, semicolons, etc.), and only retaining the core syntax components and their relationships. Simply put, the abstract syntax tree is like the "skeleton" of the source code. In addition, the abstract syntax tree is composed of multiple nodes with a time sequence, and each node contains at least one syntax component.

[0089] The syntax component can be a basic component element (for example: keyword) or component that constitutes the syntax structure in the database statement. For example, it can be a keyword and other context information that constitutes the syntax structure in the DDL database statement.

[0090] By establishing an abstract syntax tree, the time sequence order between multiple DDL events in the transaction log can be well reflected, and in addition, the order relationship between each node in the abstract syntax tree can be well reflected, that is, the execution logic order of the syntax components within a single DDL statement. For example, the DDL events in the transaction log are sorted by the actual occurrence time (for example, "create table first → add field second → create index last"), and this time sequence directly affects the final state of the database structure (the later DDL may depend on the result of the earlier DDL). In addition, the syntax components of a single DDL statement (such as operation type, target object, property definition, etc.) have a fixed execution logic order (for example, "first locate the target table → then perform the add field operation → finally define the field constraint"), and the abstract syntax tree accurately reflects this order through the parent-child hierarchy of the nodes and the arrangement order of the sibling nodes.

[0091] In an embodiment, the specific implementation of parsing the transaction log of the source database, extracting the data definition language (DDL) event and the context information of the DDL event, and generating the abstract syntax tree of the DDL event is as follows: a third-party transaction log parsing tool is used to parse transaction logs of different formats, and then DDL database statements are filtered out from the parsed transaction logs, context information is supplemented, and the filtered DDL database statements are converted into abstract syntax trees.

[0092] For example, for the DDL database statement ALTER TABLE user ADD COLUMN age INT NOT NULL DEFAULT 0 filtered out from the transaction log, the DDL statement is split into a plurality of minimum syntax units by lexical analysis, such as operation type: ALTER TABLE, target object: table user, operation content: add field age, type INT, constraint NOT NULL, and the like, and the minimum syntax units are combined into a tree structure, i.e., an abstract syntax tree, according to the database rules by syntax analysis.

[0093] Step 102: performing node-level syntax conversion on the abstract syntax tree according to the database type of the target database and the preset syntax mapping rule library to obtain an equivalent DDL operation sequence of the target database.

[0094] The target database can refer to a database that receives data structure changes of the source database, and the essence is a "synchronization destination" of the DDL event in the source database.

[0095] The syntax mapping rule library at least includes one syntax mapping rule set, and the essence is a data storage library of a plurality of syntax mapping rule sets, which is used to store different syntax mapping rule sets between databases corresponding to any two database types.

[0096] The syntax mapping rule set can refer to a predefined rule set used to describe the syntax corresponding relationship between different databases, and is a "translation dictionary" for implementing node-level syntax conversion of the abstract syntax tree. The essence is to record the mapping relationship between the syntax components in the source database and the syntax components in the target database.

[0097] Due to the relational database (such as MySQL, Oracle), non-relational database (such as MongoDB) and even the same type of different versions of the database, there are significant differences in the core syntax elements such as data types, keywords, object operation methods. For example, the NUMBER (10, 2) of the Oracle database is used to represent the data type of the accurate value, while the DECIMAL (10, 2) of the SQL Server database is used to represent the data type of the accurate value; the MongoDB database as a non-relational database does not even have a "fixed data type definition" syntax, but the type is implied by the document structure.

[0098] Therefore, the syntax mapping rule set is not universal, but is designed for a specific "source database type-target database type" combination. Each rule explicitly states how a "source database type syntax element" should correspond to a "target database type syntax element". This specificity is the prerequisite for correct conversion.

[0099] For example, there is a syntax mapping rule set between the MySQL database and the Oracle database, such as: MySQL-Oracle syntax mapping rule set; there is a syntax mapping rule set between the SQL Server database and the PostgreSQL database, such as: SQL Server-PostgreSQL syntax mapping rule set; there is a syntax mapping rule set between the Oracle database and the SQL Server database, such as: Oracle-SQL Server syntax mapping rule set. The above multiple syntax mapping rule sets are stored in the syntax mapping rule library.

[0100] In addition, for some databases without fixed data type definitions, such as the MongoDB database, a syntax mapping rule set between it and any other database can be established separately, such as: MongoDB database-Oracle database syntax mapping rule set. The above is only an example, and the present application does not limit this.

[0101] The equivalent DDL operation sequence of the target database can achieve the same operation as the original DDL database statement of the source database in the target database, and its essence is consistent with the semantics of the original DDL database statement in the source database.

[0102] For example, the equivalent DDL operation sequence of the converted target database can ensure that the data structure (such as a table, a field, a constraint, an index, and the like) of the target database is consistent with the data structure of the source database after execution of the original DDL database statement. For example, a table is created in the source MySQL database by using CREATE TABLE user (id INT AUTO_INCREMENT PRIMARY KEY, name VARCHAR 20 NOT NULL), and the target Oracle database needs to implement the self-incrementing primary key by using a sequence (SEQUENCE) + trigger (TRIGGER), and needs to ensure the character length by using VARCHAR2 (20 CHAR), so as to finally achieve the same or equivalent effect of "id self-incrementing, name non-empty and length 20 characters".

[0103] In addition, the equivalent DDL operation sequence of the target database can reflect the time sequence of the DDL operation of the target database, that is, has a certain operation order. For example, the source database Oracle can directly support ALTER TABLE user MODIFY (age NUMBER 3) to modify the table field type, but the target database MySQL can need to first add a temporary field, migrate data, delete the original field, and rename the temporary field if the original field has data, and the four operations constitute a sequence with time sequence. The equivalent operation sequence formed by the above four operations can achieve the equivalent effect of the DDL database statement in the source data.

[0104] In an embodiment, the specific implementation of the equivalent DDL operation sequence of the target database is that, according to the database type of the target database and the preset syntax mapping rule library, node-level syntax conversion is performed on the abstract syntax tree, the specific implementation is that: according to the database type of the target database and the data type of the source database, the syntax mapping rule set of the above two database types is searched from the syntax mapping rule library, the syntax component in the node of the abstract syntax tree is replaced according to the syntax mapping rule set, the target abstract syntax tree of the target database corresponding to the abstract syntax tree in the source data is obtained, and the equivalent DDL operation sequence with time sequence of the target database is obtained according to the target abstract syntax tree.

[0105] For example, if the target database is of the Oracle database type and the source database is of the SQL Server database type, the SQL Server-Oracle syntax mapping rule set can be found from the syntax mapping rule set library. Each node on the abstract syntax tree is traversed, the syntax components in the node are parsed, the corresponding SQL Server-Oracle syntax mapping rule set is found according to the syntax components, the syntax components are replaced with target syntax components in the case where the syntax components in the node correspond to target syntax components in the Oracle database, and the entire abstract syntax tree is traversed until a target abstract syntax tree after conversion is obtained. The target abstract syntax tree is further subjected to semantic adaptation to obtain a final equivalent DDL operation sequence of the target database with timing.

[0106] Step 103, blocking the data processing operation corresponding to the database object associated with the DDL event, and executing the equivalent DDL operation sequence on the target database.

[0107] The database object can refer to a database entity directly acted on or associated with the DDL event, and is a structured unit for organizing, storing, or managing data in the database. For example, a table, a view, an index, a stored procedure, a function, a trigger, and a constraint in the database.

[0108] The data processing operation can refer to reading, writing, modifying, calculating, and other operations on data in the database object, and the core feature is "not changing the structure of the database object, but only operating the data stored in the object". These operations are usually triggered by DML (Data Manipulation Language) or business logic of an application program. In addition, the data processing operation can also refer to other DDL operation statements associated with the DDL event.

[0109] Since the DDL event is essentially a modification of the structure of the database object (such as a table, a field, an index, etc.) (such as adding a field, modifying a type, deleting a table, etc.), and the data processing operation, i.e., DML (such as query, insertion, update, deletion), depends on the current database structure, if the two are executed concurrently, it can cause structure and data mismatch. For example, the source database executes "ALTERTABLE user ADD COLUMN age INT" (adds an age field), and if the target database has a data processing operation (such as inserting a record without the age field) at the same time as synchronizing the DDL, it will cause insertion failure due to "new field exists but not assigned", or cause a syntax error due to missing field.

[0110] Therefore, in order to guarantee the data consistency, structure stability and operation safety of the target database when the equivalent DDL operation sequence is executed, avoid data errors or synchronization failures caused by concurrent operations, the data processing operations corresponding to the database objects associated with the DDL event need to be blocked, to ensure that the equivalent DDL operation sequence can be executed completely in an environment without concurrent interference, and finally realize the structure consistency of the source database and the target database, and the matching of the data and the structure is not destroyed by concurrent operations, to guarantee that the state of the database after synchronization is accurate and usable.

[0111] In an embodiment, the specific implementation manner of blocking the data processing operations corresponding to the database objects associated with the DDL event and executing the equivalent DDL operation sequence on the target database is as follows: obtaining the database objects dependent on the DDL event and the database objects dependent on the data processing operation, when the database objects dependent on the DDL event are the same as the database objects dependent on the data processing operation, it is indicated that the data processing operation shares the dependent database objects with the DDL event, and the data processing operation is blocked, and then the equivalent DDL operation sequence is executed on the target database.

[0112] For example, assuming that the database statement corresponding to the current DDL event to be synchronized is "ALTER TABLE user MODIFY COLUMN age INT" (modify the age field type of the table user), the associated database objects include: the table user (directly affected), the view (such as v_user_info) dependent on the table user, the index (such as idx_user_age) based on the age field, etc. The data processing operations "for the above associated database objects" need to be blocked, that is, the data read / write and modification operations of all the database objects associated with the above DDL event. After the data processing operations for the above database objects are completed, the equivalent DDL operation sequence is executed on the target database.

[0113] The method provided by the embodiment of the application comprises the following steps: analyzing a transaction log of a source database, extracting and generating an abstract syntax tree of a DDL event, determining a target syntax mapping rule set from a preset syntax mapping rule library according to a database type of a target database, performing node-level syntax conversion on the abstract syntax tree according to the target syntax mapping rule set, obtaining an equivalent DDL operation sequence of the target database, blocking data processing operations corresponding to database objects associated with the DDL event, and executing the equivalent DDL operation sequence on the target database. The full-process intelligent processing of the heterogeneous database DDL is implemented, and the processing efficiency of the heterogeneous database DDL synchronization is improved. Meanwhile, by analyzing the dependent data objects of the DDL event, the associated data processing operations are blocked, and the safety and reliability in the synchronization process are ensured.

[0114] Figure 2 The embodiment flowchart of another heterogeneous database synchronization method provided by the embodiment of the application mainly describes how to obtain an equivalent DDL operation sequence of a target database according to a DDL event of a source database, as shown in Figure 2

[0115] Step 201: Analyzing a transaction log of a source database, extracting data definition language (DDL) event and context information of the DDL event, and generating an abstract syntax tree of the DDL event.

[0116] For step 201, refer to the detailed description of the related embodiments.

[0117] Step 202: Determining a target syntax mapping rule set from a preset syntax mapping rule library according to a database type of a target database, and the target syntax mapping rule set comprises syntax mapping rules between the source database and the target database.

[0118] In an embodiment, the specific implementation manner that the target syntax mapping rule set comprising syntax mapping rules between the source database and the target database is determined from the preset syntax mapping rule library according to the database type of the target database is as follows: taking the source database type and the target database type as retrieval conditions, and determining the corresponding target syntax mapping rule set from the preset syntax mapping rule library.

[0119] ​For example, the current target database is of the MySQL database type, and the source database is of the Oracle database type. A MySQL-Oracle syntax mapping rule set is found from the preset syntax mapping rule library, and is determined as the target syntax mapping rule set. This is only an example, and the target syntax mapping rule between the source database and the target database can also be determined in other manners, which is not limited in the embodiments of the present application.

[0120] In step 203, the nodes in the abstract syntax tree are traversed, and the following processing is performed on the traversed nodes to obtain the reconstructed abstract syntax tree: a syntax mapping rule containing the syntax component in the node is searched from the target syntax mapping rule set; in the case of finding, the syntax component in the node is replaced with the target syntax component according to the found syntax mapping rule; in the case of not finding, semantic conflict detection is triggered, and a syntax compensation operation is performed on the syntax component in the node according to the semantic conflict detection result.

[0121] The abstract syntax tree is a tree-shaped data structure, in which each node represents a syntax structure (such as an expression, a statement, a declaration, etc.) in the code, and the relationship between the nodes reflects the syntax hierarchy and combination rules of the code.

[0122] Each node in the abstract syntax tree can be a data structure containing specific syntax component information, and can include a node type for identifying the syntax structure represented by the node, an attribute for storing specific information of the node, and a child node for representing other nodes under the current node.

[0123] The semantic conflict detection can refer to a core component specially used for detecting, identifying, and processing the semantic conflict properties and types between the syntax component in the node of the abstract syntax tree and the target syntax component when the syntax component cannot find a corresponding mapping relationship through the target syntax mapping rule set.

[0124] In an embodiment, the nodes in the abstract syntax tree are traversed, and the following processing is performed on the traversed nodes to obtain the reconstructed abstract syntax tree: a syntax mapping rule containing the syntax component in the node is searched from the target syntax mapping rule set; in the case of finding, the syntax component in the node is replaced with the target syntax component according to the found syntax mapping rule; in the case of not finding, semantic conflict detection is triggered, and a syntax compensation operation is performed on the syntax component in the node according to the semantic conflict detection result.

[0125] Specifically, for each node traversed, the information in the node is parsed and identified, and the node information is processed as follows to obtain the reconstructed abstract syntax tree: the syntax component of the node is filtered from the node information, the syntax component is searched in the target syntax mapping rule set, when the conversion rule matching the syntax component is found in the target syntax mapping rule set, the syntax component in the node is modified, replaced or reorganized according to the conversion rule, and the child nodes in the current node are recursively processed. In the case where it cannot be found, semantic conflict detection is triggered, and a syntax compensation operation is performed on the current node. After all nodes (including the root node, the child nodes at each level) are traversed, the structure, node type and attribute of the original abstract syntax tree have been updated to conform to the syntax structure of the target database. At this time, the abstract syntax tree obtained is the reconstructed abstract syntax tree.

[0126] For example, if the syntax component of the currently traversed node is "function call" and the function name is LIMIT (source database MySQL), and the target database is Oracle, there may be a mapping in the MySQL-Oracle syntax mapping rule set: LIMIT→ROWNUM, at this time, the LIMIT function in the current node is replaced by the ROWNUM function according to the above mapping; if the mapping relationship of the function LIMIT is not found from the MySQL-Oracle syntax mapping rule set, semantic conflict detection is triggered, and a syntax compensation operation is performed on the current node to achieve the effect of the above function LIMIT operation through other multiple operations in the target database.

[0127] In an embodiment, semantic conflict detection can be triggered in the following manner: the syntax component in the node is parsed to obtain keyword information; in the case where the keyword information indicates that the syntax component in the node belongs to a constraint mechanism, a constraint equivalent compensation mechanism is used to perform a syntax compensation operation on the syntax component in the node; in the case where the keyword information indicates that the syntax component in the node belongs to an index mechanism or a storage engine component, a function simulation compensation mechanism is used to perform a syntax compensation operation on the syntax component in the node.

[0128] The syntax compensation operation can be an operation that, when a semantic conflict occurs, indirectly realizes the function of the source syntax component through other syntax or mechanisms supported by the target database, and ensures the semantic consistency of the DDL database operation statements of the source database and the target database.

[0129] In the process of converting the syntax component in the current node according to the syntax mapping rule set, it can be found that the mapping relationship corresponding to the syntax component in the current node cannot be found in the syntax mapping rule set, that is, the syntax component of the source database cannot directly correspond to the syntax in the target database, resulting in that the semantics cannot be directly mapped, and then some compensation strategies need to be taken to convert the syntax component in the current node to ensure that the syntax component in the node can still maintain the original semantics and functions after conversion processing.

[0130] In the above process, the core of the compensation mechanism according to the semantic conflict detection result is to identify the type of the syntax component through the keyword, and then adopt the compensation strategy accordingly. For example, if the keyword information in the node indicates that the syntax component belongs to the constraint mechanism, the constraint equivalent compensation mechanism is adopted for syntax compensation operation, and if the keyword information in the node indicates that the syntax component belongs to the index mechanism or the storage engine component, the function simulation compensation mechanism is adopted for syntax compensation operation of the syntax component in the node.

[0131] As a possible implementation manner, the constraint equivalent compensation mechanism can be implemented by the following manner: generating a program logic unit to replace the original constraint mechanism; executing the program logic unit to perform syntax compensation operation on the syntax component in the node. The function simulation compensation mechanism can be implemented by the following manner: generating a plurality of equivalent operation statements to simulate the original index mechanism or the original storage engine component; executing the plurality of equivalent operation statements to perform syntax compensation operation on the syntax component in the node.

[0132] The core function of the constraint mechanism is to ensure data integrity (such as limiting the range of field values, ensuring uniqueness, maintaining the relationship between tables, etc.). The target of the syntax compensation of the constraint mechanism is to find the syntax or mechanism in the target database that can equivalently implement the same data integrity rule, without pursuing the consistency of the syntax form, but ensuring that the constraint logic is completely equivalent.

[0133] For example, for the constraint equivalent compensation mechanism: assuming that there is a CHECK (age>18) constraint in the node of the source database (PostgreSQL), and the target MySQL database does not support CHECK, that is, the mapping relationship related to the CHECK constraint cannot be found from the PostgreSQL-MySQL syntax mapping rule set, then the syntax compensation operation on the target database is triggered, and a program logic unit to replace the original constraint mechanism is generated: a trigger (TRIGGER) is created, and IF age<=18 THEN SIGNAL ERROR logic is executed when data is inserted / updated, to equivalently implement the check constraint. Then the above program logic unit is executed to implement the syntax compensation operation on the target database.

[0134] The core function of the index mechanism is to optimize query performance (such as accelerating retrieval and sorting); the core function of the storage engine component is to provide data storage and access features (such as transaction support, lock mechanism, storage medium). The compensation target of such components is to simulate the effect of the original function as much as possible through the functions supported by the target database (allowing a certain degree of performance difference or different implementation methods).

[0135] For example, for the functional simulation compensation mechanism: assume that there is a FULLTEXT INDEX (supporting natural language retrieval) in the node of the source database (MySQL), but full-text indexing is not supported in the target database (PostgreSQL). That is, no mapping relationship related to FULLTEXT INDEX can be found in the MySQL-PostgreSQL syntax mapping rule set, at which time the syntax compensation operation on the target database is triggered, and a plurality of equivalent operation statements simulating the original index mechanism or the original storage engine component are generated: create an index based on tsvector such as CREATE INDEX idx_ft ON articles USING gin to_tsvector 'english', content, and modify the query statement to use to_tsquery to simulate full-text retrieval function. Then execute the plurality of equivalent operation statements to implement the syntax compensation operation on the target database.

[0136] Step 204: obtaining the equivalent DDL operation sequence of the target database according to the reconstructed abstract syntax tree.

[0137] In a possible implementation, the specific implementation of obtaining the equivalent DDL operation sequence of the target database according to the reconstructed abstract syntax tree is as follows: traversing each node in the abstract syntax tree, calling the corresponding DDL syntax template of the target database according to the syntax component in each node, these templates are string templates defined in advance and conforming to the syntax specification of the target database. Then parse the child nodes in the node and fill in the template placeholders, extract and convert the information in the node into actual values in the node template placeholders to obtain the final equivalent DDL operation sequence of the target database.

[0138] Step 205: blocking the data processing operation corresponding to the database object associated with the DDL event, and executing the equivalent DDL operation sequence on the target database.

[0139] For step 205, refer to the detailed description of the related embodiments described above.

[0140] Through Figure 2The detailed description of the embodiment shown, through the semantic conflict detection + syntax compensation operation, for the syntax component without direct mapping rule, through the equivalent mechanism to realize synchronization, improves the compatibility for different database types, and the traversal recursive abstract syntax tree node mode can flexibly process complex syntax structure, avoids conversion failure caused by complex structure, and the whole process from abstract syntax tree generation, mapping rule matching, semantic conflict detection to equivalent DDL operation sequence of target data is automatically executed, compared with the traditional "manual conversion script writing" mode, greatly reduces the labor cost and human error probability, and improves the efficiency and stability of heterogeneous database synchronization.

[0141] Figure 3 The embodiment flowchart of another heterogeneous database synchronization method provided by the embodiment of the application mainly describes how to block the data processing operation corresponding to the database object associated with the DDL event, for example, Figure 3 As shown, the method comprises the following steps:

[0142] Step 301, parsing the transaction log of the source database, extracting the data definition language (DDL) event and the context information of the DDL event, and generating an abstract syntax tree of the DDL event.

[0143] Step 302, according to the database type of the target database and the preset syntax mapping rule library, performing node-level syntax conversion on the abstract syntax tree to obtain an equivalent DDL operation sequence of the target database.

[0144] For steps 301-302, refer to the detailed description of the related embodiments described above.

[0145] Step 303, determining the database object associated with the DDL event as a target database object, applying a global barrier to the target database object, and generating a global barrier identifier.

[0146] Step 304, blocking the data processing operation corresponding to the target database object based on the global barrier identifier.

[0147] For steps 303-304, the following is a unified description:

[0148] The global barrier is used to impose "operation restrictions" on a specific object in a distributed environment (such as a multi-node database cluster). The application process needs to be implemented through a centralized coordination component or a built-in lock mechanism of the database.

[0149] The global barrier identifier can be used to identify the order in which the global barrier is inserted into the database object.

[0150] In an embodiment, the database object associated with the DDL event is determined as the target database object, a global barrier is applied for the target database object, a global barrier identifier is generated, and the specific implementation mode of blocking the data processing operation corresponding to the target database object based on the global barrier identifier is as follows: according to the context information of the DDL event of the source database, the database object associated with the DDL is identified and determined as the target database object, a global barrier is applied for the target database object, and after the application is successful, a global barrier identifier is generated. The centralized coordination component broadcasts a "barrier takes effect" event to all nodes or access layers of the target database, and carries the global barrier identifier and the target database object information, so as to block the data processing operation corresponding to the target database object.

[0151] For example, assuming that the target database object is a "user table", a request for applying a global barrier is sent to the central coordination component, carrying a global barrier identifier: 001, a target database object: "user table", and an operation restriction type: data processing operation. The central coordination component checks whether the target database object has been locked by other barriers. If not, a global barrier record is created and marked as "locked", if already locked, it is waited or processed according to the priority. If the application is successful, the global barrier identifier is returned. After the global barrier takes effect, the central coordination component broadcasts a "barrier takes effect" event to all nodes.

[0152] Step 305, in the global barrier effective state, an equivalent DDL operation sequence is executed on the target database. In the case of successful execution of the equivalent DDL operation sequence on the target database, step 306 is executed, and in the case of unsuccessful execution of the equivalent DDL operation sequence on the target database, step 307 is executed.

[0153] Step 306, the global barrier is released for the target database object based on the global barrier identifier, and the data processing operation corresponding to the target database object is continued.

[0154] Step 307, a fault tolerance processing mechanism is triggered for the equivalent DDL operation sequence, and in the case that the equivalent DDL operation sequence is still not successfully executed on the target database after the fault tolerance processing, the global barrier is released for the target database object according to the safe playback mechanism.

[0155] The following is a unified description of steps 305-307:

[0156] In one embodiment, when the global barrier is active, an equivalent DDL operation sequence is executed on the target database. If the execution is successful, the global barrier is released for the target database object based on the global barrier identifier, and the data processing operations corresponding to the target database object continue to be executed. If the execution fails, a fault tolerance mechanism is triggered for the equivalent DDL operation sequence. If the execution of the equivalent DDL operation sequence on the target database still fails after fault tolerance, the global barrier is released for the target database object according to the security replay mechanism. The specific implementation of this mechanism is as follows: When the global barrier is active, the equivalent DDL operation sequence is executed on the target database. If the execution is successful, the blocking on the target database object is released, and a barrier release request is sent to the central coordination component according to the order of the global barrier identifiers. According to the "first-in, first-out" principle, the data processing operations in the blocking queue are released, and the equivalent DDL operation sequence continues to be executed in the newly updated data structure of the target database. If the execution fails, an attempt is made to repair the fault through the fault tolerance mechanism. If the execution still fails, the security barrier is released according to the security replay mechanism.

[0157] The fault tolerance mechanism is implemented by re-executing the equivalent DDL operation sequence on the target database until successful or a set number of executions is reached. For example, the set number of executions can be 3, 2, or 1.

[0158] The secure replay mechanism is implemented by: rolling back the target database's data structure; after the rollback is complete, releasing the global barrier for the target database object, and continuing to execute the data processing operations corresponding to the target database object. For example, after the data structure rollback is complete, the data processing operations corresponding to the target database object are executed based on the original data structure.

[0159] pass Figure 3 The illustrated embodiment serializes data processing operations in a distributed environment and DDL event synchronization operations in the source database through a global barrier, ensuring the atomicity and success rate of the operations and fundamentally guaranteeing the consistency of data structures in a distributed environment. Centralized coordination achieves cross-node state unification in a distributed environment, improving synchronization reliability. Fault tolerance and secure replay construct a complete anomaly handling loop, reducing the impact of failures on business operations. Ultimately, it achieves the goal of "safe and reliable heterogeneous database DDL synchronization with minimal business interference," providing crucial support for the stable operation of heterogeneous database clusters.

[0160] Figure 4 A structural block diagram of a heterogeneous database synchronization device provided in this application embodiment, the device comprising:

[0161] The log analysis module 41 is configured to analyze a transaction log of a source database, extract a data definition language (DDL) event and context information of the DDL event, and generate an abstract syntax tree of the DDL event.

[0162] The operation sequence generation module 42 is configured to perform node-level syntax conversion on the abstract syntax tree according to a database type of a target database and a preset syntax mapping rule library, to obtain an equivalent DDL operation sequence of the target database.

[0163] The operation sequence execution module 43 is configured to block a data processing operation corresponding to a database object associated with the DDL event, and execute the equivalent DDL operation sequence on the target database.

[0164] In a possible implementation, the operation sequence generation module includes:

[0165] The mapping rule determination unit is configured to determine a target syntax mapping rule set from the preset syntax mapping rule library according to a database type of a target database, and the target syntax mapping rule set includes syntax mapping rules between the source database and the target database.

[0166] The node traversal unit is configured to traverse a node in the abstract syntax tree, and perform the following processing on the traversed node to obtain a reconstructed abstract syntax tree:

[0167] searching, from the target syntax mapping rule set, for a syntax mapping rule including a syntax component in the node; in the case of finding, replacing the syntax component in the node with a target syntax component according to the found syntax mapping rule; in the case of not finding, triggering semantic conflict detection, and performing a syntax compensation operation on the syntax component in the node according to a result of the semantic conflict detection;

[0168] The operation sequence determination unit is configured to obtain an equivalent DDL operation sequence of the target database according to the reconstructed abstract syntax tree.

[0169] In a possible implementation, the node traversal unit includes:

[0170] The node parsing subunit is configured to parse a syntax component in the node to obtain keyword information.

[0171] The constraint processing subunit is configured to, in the case where the keyword information indicates that the syntax component in the node belongs to a constraint mechanism, perform a syntax compensation operation on the syntax component in the node by using a constraint equivalent compensation mechanism.

[0172] The index processing subunit is configured to, in the case that the keyword information indicates that the syntax component in the node belongs to an index mechanism or a storage engine component, perform syntax compensation operation on the syntax component in the node by using a functional simulation compensation mechanism.

[0173] In a possible implementation, the constraint processing subunit is specifically configured to:

[0174] generate a program logic unit replacing the original constraint mechanism;

[0175] execute the program logic unit to perform syntax compensation operation on the syntax component in the node.

[0176] In a possible implementation, the index processing subunit is specifically configured to:

[0177] generate a plurality of equivalent operation statements simulating the original index mechanism or the original storage engine component;

[0178] execute the plurality of equivalent operation statements to perform syntax compensation operation on the syntax component in the node.

[0179] In a possible implementation, the operation sequence execution module comprises:

[0180] a target database object determination subunit configured to determine a database object associated with the DDL event as a target database object;

[0181] a global barrier identification generation subunit configured to apply a global barrier to the target database object and generate a global barrier identification;

[0182] a data processing subunit configured to block data processing operation corresponding to the target database object based on the global barrier identification;

[0183] an operation sequence execution subunit configured to execute the equivalent DDL operation sequence on the target database in the case that the global barrier is effective.

[0184] In a possible implementation, the operation sequence execution module further comprises:

[0185] a global barrier release subunit configured to, in the case that the execution of the equivalent DDL operation sequence on the target database is successful, release the global barrier for the target database object based on the global barrier identification, and continue to perform data processing operation corresponding to the target database object;

[0186] The fault-tolerant processing subunit is configured to trigger a fault-tolerant processing mechanism for the equivalent DDL operation sequence when the execution of the equivalent DDL operation sequence on the target database fails, and release the global barrier for the target database object according to a safe replay mechanism when the execution of the equivalent DDL operation sequence on the target database still fails after the fault-tolerant processing.

[0187] In a possible implementation, the fault-tolerant processing subunit is specifically configured to:

[0188] re-execute the equivalent DDL operation sequence on the target database until the execution succeeds or a set number of execution times is reached.

[0189] In a possible implementation, the fault-tolerant processing subunit is further configured to:

[0190] rollback the data structure of the target database;

[0191] after the rollback is completed, release the global barrier for the target database object, and continue to execute the data processing operation corresponding to the target database object.

[0192] As shown in Figure 5 The embodiment of the present application provides an electronic device, which comprises a processor 111, a communication interface 112, a memory 113 and a communication bus 114, wherein the processor 111, the communication interface 112 and the memory 113 complete mutual communication through the communication bus 114,

[0193] The memory 113 is configured to store a computer program.

[0194] In an embodiment of the present application, the processor 111 is configured to execute the program stored in the memory 113, and implement the heterogeneous database synchronization method provided by any one of the preceding method embodiments, comprising:

[0195] parsing a transaction log of a source database, extracting a data definition language (DDL) event and context information of the DDL event, and generating an abstract syntax tree of the DDL event;

[0196] performing node-level syntax conversion on the abstract syntax tree according to a database type of a target database and a preset syntax mapping rule library, to obtain an equivalent DDL operation sequence of the target database;

[0197] blocking a data processing operation corresponding to a database object associated with the DDL event, and executing the equivalent DDL operation sequence on the target database.

[0198] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the heterogeneous database synchronization method provided by any one of the preceding method embodiments.

[0199] The apparatus embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0200] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0201] It should be understood that the terms used herein are for the purpose of describing particular example embodiments only and are not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "has" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order in which they are described, unless specifically identified as an order dependent step. It is also to be understood that additional or alternative steps can be employed.

[0202] The above only describes specific embodiments of the present application, so that those skilled in the art can understand or implement the present application. Various modifications of these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A method for synchronizing heterogeneous databases, characterized in that, The method includes: Parse the transaction log of the source database, extract the Data Definition Language (DDL) events and the context information of the DDL events, and generate the abstract syntax tree of the DDL events; Based on the database type of the target database and the preset syntax mapping rule base, the abstract syntax tree is subjected to node-level syntax transformation to obtain the equivalent DDL operation sequence of the target database. Block the data processing operation corresponding to the database object associated with the DDL event, and execute the equivalent DDL operation sequence on the target database.

2. The method according to claim 1, characterized in that, The step of performing node-level syntax transformation on the abstract syntax tree based on the database type of the target database and a preset syntax mapping rule base to obtain the equivalent DDL operation sequence of the target database includes: Based on the database type of the target database, a target syntax mapping rule set is determined from a preset syntax mapping rule base. The target syntax mapping rule set includes syntax mapping rules between the source database and the target database. Traverse the nodes in the abstract syntax tree and perform the following processing on the traversed nodes to obtain the reconstructed abstract syntax tree: The system searches for a grammar mapping rule that contains the grammar component in the node from the target grammar mapping rule set. If a grammar mapping rule is found, the grammar component in the node is replaced with the target grammar component according to the found grammar mapping rule. If no grammar mapping rule is found, a semantic conflict detection is triggered, and a grammar compensation operation is performed on the grammar component in the node according to the semantic conflict detection result. Based on the reconstructed abstract syntax tree, the equivalent DDL operation sequence of the target database is obtained.

3. The method according to claim 2, characterized in that, The triggering of semantic conflict detection, and the performing of syntactic compensation operations on the syntactic components in the node based on the semantic conflict detection results, includes: Parse the syntactic components in the nodes to obtain keyword information; When the keyword information indicates that the grammatical components in the node belong to the constraint mechanism, the constraint equivalent compensation mechanism is used to perform grammatical compensation operation on the grammatical components in the node. If the keyword information indicates that the syntactic components in the node belong to the indexing mechanism or storage engine component, a functional simulation compensation mechanism is used to perform syntactic compensation operations on the syntactic components in the node.

4. The method according to claim 3, characterized in that, The step of using a constraint equivalence compensation mechanism to perform syntactic compensation operations on the syntactic components in the node includes: Generate program logic units that replace the original constraint mechanism; The program logic unit is executed to perform syntax compensation operations on the syntax components in the node.

5. The method according to claim 3, characterized in that, The step of using a functional simulation compensation mechanism to perform syntactic compensation operations on the syntactic components in the node includes: Generate multiple equivalent operation statements that simulate the original indexing mechanism or original storage engine components; The multiple equivalent operation statements are executed to perform syntactic compensation operations on the syntactic components in the node.

6. The method according to claim 1, characterized in that, The blocking of data processing operations corresponding to the database object associated with the DDL event, and the execution of the equivalent DDL operation sequence on the target database, includes: The database object associated with the DDL event is identified as the target database object; Apply for a global barrier for the target database object and generate a global barrier identifier; Based on the global barrier identifier, the data processing operation corresponding to the target database object is blocked; With the global barrier in effect, the equivalent DDL operation sequence is executed on the target database.

7. The method according to claim 6, characterized in that, After performing the equivalent DDL operation sequence on the target database, the method further includes: If the equivalent DDL operation sequence is successfully executed on the target database, the global barrier is released based on the global barrier identifier for the target database object, and the data processing operation corresponding to the target database object continues to be executed. If the execution of the equivalent DDL operation sequence on the target database fails, a fault tolerance mechanism is triggered for the equivalent DDL operation sequence. If the execution of the equivalent DDL operation sequence on the target database still fails after fault tolerance, the global barrier is released for the target database object according to the safe replay mechanism.

8. The method according to claim 7, characterized in that, The fault-tolerant processing mechanism triggered for the equivalent DDL operation sequence includes: The equivalent DDL operation sequence is re-executed on the target database until execution is successful or the set number of executions is reached.

9. The method according to claim 7, characterized in that, The step of releasing the global barrier for the target database object according to the security replay mechanism includes: Roll back the data structure of the target database; After the rollback is complete, the global barrier is released for the target database object, and the data processing operations corresponding to the target database object continue to be executed.

10. A heterogeneous database synchronization device, characterized in that, The device includes: The log parsing module is used to parse the transaction logs of the source database, extract Data Definition Language (DDL) events and the context information of the DDL events, and generate the abstract syntax tree of the DDL events. The operation sequence generation module is used to perform node-level syntax transformation on the abstract syntax tree according to the database type of the target database and a preset syntax mapping rule library to obtain the equivalent DDL operation sequence of the target database. The operation sequence execution module is used to block the data processing operation corresponding to the database object associated with the DDL event, and execute the equivalent DDL operation sequence on the target database.

11. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a heterogeneous database synchronization program stored in the memory to implement the heterogeneous database synchronization method according to any one of claims 1-9.

12. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the heterogeneous database synchronization method according to any one of claims 1-9.

Images