Database data synchronization method, system and device and readable storage medium

Abstract

The invention discloses a database data synchronization method, system and device and a storage medium. The method comprises the following steps: acquiring a log, within a preset SCN range, of a first time identifier SCN in a source library; each obtained log is analyzed, and information of each log is obtained; taking the log of which the data table ID belongs to the to-be-synchronized data table ID set as a target log; for each target log, according to a data table ID and a first SCN included in the information of the target log, a target data dictionary is determined from the metadatabase, and the target data dictionary is a data dictionary corresponding to the data table ID in the metadatabase, and the SCN is smaller than the first SCN and closest to the first SCN; and based on the target data dictionary and the target log, determining an operation instruction compatible with the target database and sending the operation instruction to the target database so as to realize data synchronization of the target database and the source library, so that log operation content and a data dictionary version can be compatible during analysis, and the requirements of real-time performance and reliability of data synchronization are met.

Description

Technical Field

[0001] This application relates to the field of database technology, specifically to a database data synchronization method, system, device, and computer-readable storage medium. Background Technology

[0002] Currently, Oracle databases are widely used in finance, government affairs, education and other fields. Their business data needs to be incrementally synchronized to the target database to meet the needs of backup, business analysis and other requirements. LogMiner-based log mining solutions have become mainstream due to their accurate capture of changes.

[0003] In related technologies, LogMiner parses binary logs by executing a stored procedure to generate a complete dictionary snapshot of the table structure at a certain point in time in the source database, and writes it to the source database log. Subsequent incremental synchronization is based solely on this fixed snapshot dictionary for log parsing. However, existing solutions have significant shortcomings in continuous incremental synchronization: on the one hand, dictionary snapshot generation consumes a large amount of I / O resources, which can easily lead to generation delays during peak periods of business activity in the source database, directly blocking the incremental log parsing process; on the other hand, the complete snapshot dictionary occupies a large amount of log space in the source database, generating a large amount of redundant data and severely reducing incremental synchronization efficiency; moreover, existing solutions only use the latest version of the data dictionary to parse logs, resulting in incompatibility between the old version of the logs and the new version of the data dictionary, leading to parsing errors. In summary, existing technical solutions are unable to meet the core requirements of critical business operations for data synchronization compatibility, real-time performance, and reliability. Summary of the Invention

[0004] This application provides a database data synchronization method, system, device, and computer-readable storage medium, which can solve the technical problems of poor data synchronization compatibility, real-time performance, and reliability in the prior art.

[0005] In a first aspect, embodiments of this application provide a database data synchronization method, the database data synchronization method comprising: Obtain logs from the source database whose first-time identifier SCN falls within a preset SCN range. The starting point of the preset SCN range is the second SCN. The first SCN is determined based on the log generation time, and the second SCN is determined based on the latest synchronization time between the metadata database and the source database. The metadata database includes a data dictionary for each version of the data table to be synchronized, and each version's data dictionary is associated with its corresponding SCN. Each log entry is parsed to obtain information for each log entry, including the first SCN, data table ID, and operation type. The target log is the log whose table ID belongs to the set of table IDs to be synchronized; For each target log, a target data dictionary is determined from the metadata database based on the data table ID and the first SCN included in the target log information. The target data dictionary is the data dictionary in the metadata database that corresponds to the data table ID and whose SCN is less than the first SCN and is closest to the first SCN. Based on the target data dictionary and target log, operation instructions compatible with the target database are determined and sent to the target database so that the target database can execute the operation instructions to achieve data synchronization with the source database.

[0006] In conjunction with the first aspect, in one implementation, before obtaining the logs in the source library whose SCN is immediately identified as falling within a preset SCN range, the method further includes: Obtain the initial table structure of each table in the set of data tables to be synchronized from the source database; Based on the largest first SCN in the source database and the initial table structure, determine the initial data dictionary for each data table; The initial data dictionary for each data table is stored in the metadata database, and the largest first SCN is used as the second SCN.

[0007] In conjunction with the first aspect, in one implementation, parsing each acquired log entry to obtain information for each log entry includes: Each log entry obtained is loaded into the log parsing tool LogMiner, so that LogMiner can parse the logs and store the parsing results in the system view of the source library; Query the parsing results in the system view to obtain information for each log entry.

[0008] In conjunction with the first aspect, in one implementation, determining the operation instructions compatible with the target database based on the target data dictionary and the target log includes: If the operation type of the target log is data change, then the data change operation description information is determined based on the target data dictionary and the target log information; The data change operation description information is converted into operation instructions compatible with the target database.

[0009] In conjunction with the first aspect, in one implementation, the step of determining the operation instructions compatible with the target database based on the target data dictionary and the target log further includes: If the operation type of the target log is a table structure change, then the table structure change operation description information is obtained based on the target data dictionary and the target log information; The description information of the data table structure change operation is converted into operation instructions compatible with the target database.

[0010] In conjunction with the first aspect, in one implementation, after obtaining the table structure change operation description information, the method further includes: Based on the target data dictionary, a new version of the target data dictionary is generated according to the changes in the table structure, and the first SCN corresponding to the target log is used as the SCN corresponding to the new version of the target data dictionary.

[0011] Secondly, embodiments of this application provide a database data synchronization system, the database data synchronization system comprising: The acquisition module is used to acquire logs in the source database whose first-time identifier SCN is within a preset SCN range. The starting point of the preset SCN range is the second SCN. The first SCN is determined based on the log generation time, and the second SCN is determined based on the latest synchronization time between the metadata database and the source database. The metadata database includes a data dictionary for each version of the data table to be synchronized, and the data dictionary for each version is associated with the corresponding SCN. The parsing module is used to parse each acquired log to obtain information about each log, including the first SCN, the data table ID, and the operation type; the logs whose data table IDs belong to the set of data table IDs to be synchronized are the target logs. The determination module is used to determine the target data dictionary from the metadata database for each target log based on the data table ID and the first SCN included in the target log information. The target data dictionary is the data dictionary in the metadata database that corresponds to the data table ID and whose SCN is less than the first SCN and is closest to the first SCN. The sending module is used to determine operation instructions compatible with the target database based on the target data dictionary and target log, and send them to the target database so that the target database can execute the operation instructions to achieve data synchronization with the source database.

[0012] In conjunction with the second aspect, in one implementation, the parsing module includes: Each log entry obtained is loaded into the log parsing tool LogMiner, so that LogMiner can parse the logs and store the parsing results in the system view of the source library; Query the parsing results in the system view to obtain information for each log entry.

[0013] Thirdly, embodiments of this application provide a database data synchronization device, which includes a processor, a memory, and a database data synchronization program stored in the memory and executable by the processor, wherein when the database data synchronization program is executed by the processor, it implements the steps of the database data synchronization method as described in the first aspect.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium, characterized in that the computer-readable storage medium stores a database data synchronization program, wherein when the database data synchronization program is executed by a processor, it implements the steps of the database data synchronization method as described in the first aspect.

[0015] The beneficial effects of the technical solutions provided in this application include: Logs with a first-time identifier SCN within a preset SCN range are acquired from the source database. Each acquired log is parsed to obtain information including a first SCN, a data table ID, and an operation type. Logs whose data table IDs belong to the set of data table IDs to be synchronized are selected as target logs. For each target log, a target data dictionary is determined from the metadata database based on the data table ID and the first SCN included in the target log information. The target data dictionary is the data dictionary in the metadata database whose corresponding data table ID has an SCN less than the first SCN and is closest to the first SCN. Based on the target data dictionary and the target logs, operation instructions compatible with the target database are determined and sent to the target database so that the target database can execute the operation instructions to achieve data synchronization with the source database. This ensures that the log operation content and the data dictionary version are compatible during parsing and meets the requirements for real-time and reliable data synchronization. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an embodiment of the database data synchronization method of this application; Figure 2 This is a diagram illustrating the Oracle data synchronization process. Figure 3 This is a schematic diagram of the architecture of an embodiment of the database data synchronization system of this application; Figure 4 This is a schematic diagram of the hardware structure of the database data synchronization device involved in the embodiments of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0019] In a first aspect, embodiments of this application provide a database data synchronization method.

[0020] In one embodiment, reference is made to Figure 1 , Figure 1 This is a schematic flowchart of an embodiment of the database data synchronization method of this application. Figure 1 As shown, database data synchronization methods include: Step S10: Obtain logs in the source database whose first time identifier SCN is within the range of a preset SCN. The starting point of the preset SCN range is the second SCN. The first SCN is determined based on the log generation time, and the second SCN is determined based on the latest synchronization time between the metadata database and the source database. The metadata database includes a data dictionary for each version of the data table to be synchronized, and the data dictionary for each version is associated with the corresponding SCN. In one embodiment, the Oracle database's archive log and redo log record detailed information about all data changes in the database, such as... Figure 2 As shown, Figure 2 The Oracle data synchronization process is illustrated below. First, a metadata database is prepared to store the multi-version data dictionary of the tables to be synchronized and the incremental synchronization progress (the offset position of the incremental synchronization). The synchronization program connects to the source database and the metadata database via JDBC, remotely calling the DBMS_LOGMNR stored procedure package provided by the Oracle database to execute the log retrieval task in real time. First, it loads the data dictionary information from the metadata database, executes a query to obtain the second SCN from the latest synchronization between the metadata database and the source database, and uses this second SCN as the starting point of the range for retrieving source database logs (e.g., 8000). Simultaneously, based on local configuration and other factors, the number of logs to be retrieved in a single batch is determined, and the starting offset position `startScn` and the ending offset position `endScn` are determined. Therefore, the preset SCN range is [8000, 8500]. Then, the synchronization program performs a query operation in the source database to retrieve redo logs and archived logs within the starting and ending SCN range, and the SCN of each version of the data dictionary is associated with the corresponding log's SCN.

[0021] Furthermore, in one embodiment, before obtaining logs in the source library whose SCN is immediately identified as falling within a preset SCN range, the method further includes: Obtain the initial table structure of each table in the set of data tables to be synchronized from the source database; Based on the largest first SCN in the source database and the initial table structure, determine the initial data dictionary for each data table; The initial data dictionary for each data table is stored in the metadata database, and the largest first SCN is used as the second SCN.

[0022] In one embodiment, such as Figure 2 As shown, the preset data tables to be synchronized are Table 1, Table 2, and Table 3. Obtain the complete initial table structure for these three tables: for example, query the source database table ALL_OBJECTS to obtain table ID, table name, schema name, database name, creation time, etc.; query the source database table ALL_TABLE_COLUMNS to obtain column ID, column name, column type, precision, scale, and other table column information; obtain table index information and primary key information from the source database tables ALL_IND_COLUMNS, ALL_INDEXES, ALL_CONS_COLUMNS, and ALL_CONSTRAINTS. If this is the first time starting the Oracle data synchronization program, then use V$DATABAS... The E system view queries the CURRENT_SCN field to obtain the largest first SCN among the current redo logs and archive logs of all tables in the source database, denoted as SCN0. This SCN serves as the initial dictionary's SCN. If the data dictionary does not exist in the metadata database at the initial moment, an initial dictionary is created in the metadata database based on SCN0 and the obtained initial table structure. This results in the initial data dictionaries for Tables 1, 2, and 3, with SCN0 serving as the initial synchronization baseline SCN (second SCN), which is the time of the latest synchronization between the metadata database and the source database. Data synchronization can only begin after the initial dictionary is created, ensuring that the subsequent log SCNs are greater than the initial table dictionary's SCN. All generated data dictionaries are then stored in the metadata database.

[0023] Step S20: Parse each log entry to obtain information for each log entry, including the first SCN, data table ID, and operation type. In one embodiment, after obtaining the redo logs and archive logs within the preset SCN range, each log is parsed to obtain the information of each log.

[0024] Furthermore, in one embodiment, parsing each acquired log entry to obtain information for each log entry includes: Each log entry obtained is loaded into the log parsing tool LogMiner, so that LogMiner can parse the logs and store the parsing results in the system view of the source library; Query the parsing results in the system view to obtain information for each log entry.

[0025] In one embodiment, such as Figure 2As shown, DBMS_LOGMNR.ADD_LOGFILE is called to add the logs to be parsed to LogMiner. Then, DBMS_LOGMNR.START_LOGMNR is executed, passing the start and end SCNs of the preset SCN range as parameters to start the log mining session. After the session starts, LogMiner log information is obtained from the V$LOGMNR_CONTENS system view based on the start and end SCNs, namely SCN, ROWID, operation type, SQL statement, table name, schema name, transaction number, data table ID, time, and other information.

[0026] Step S30: Use the logs whose data table IDs belong to the set of data table IDs to be synchronized as the target logs; Step S40: For each target log, determine the target data dictionary from the metadata database based on the data table ID and the first SCN included in the target log information, wherein the target data dictionary is the data dictionary in the metadata database that corresponds to the data table ID and whose SCN is less than the first SCN and is closest to the first SCN; In one embodiment, such as Figure 2 As shown, the metadata database stores multiple versions of data dictionaries for the data tables to be synchronized. The core fields include the data table ID, the SCN associated with the data dictionary, the table structure mapping JSON, and the creation time. For each target log, the logic of sorting by SQL according to the matching rules is implemented to find the version closest to the first SCN. Assuming that the metadata database contains 3 versions of data dictionaries for table 1, corresponding to SCN1 (8100), SCN2 (8200), and SCN3 (8300) respectively, and the second SCN of the latest synchronization between the metadata and the source database is 8300, combined with the tableID (table1 ID) corresponding to target log 1 (first SCN is 8400), the target data dictionary closest to the first SCN is determined to be the version of data dictionary with SCN=SCN3.

[0027] Step S50: Based on the target data dictionary and target log, determine the operation instructions compatible with the target database and send them to the target database so that the target database can execute the operation instructions to achieve data synchronization with the source database.

[0028] Furthermore, in one embodiment, determining the operation instructions compatible with the target database based on the target data dictionary and the target log includes: If the operation type of the target log is data change, then the data change operation description information is determined based on the target data dictionary and the target log information; The data change operation description information is converted into operation instructions compatible with the target database.

[0029] In one embodiment, taking the data insertion operation type with SCN=8010 and data table ID=1001 as an example, the table ID, column ID, etc. in the log REDO_SQL are converted into actual field names in combination with the matched target data dictionary to obtain the data change operation description information; then, according to the syntax rules of the target database, the data change operation description information is converted into structured operation instructions for the target database to perform data synchronization.

[0030] Furthermore, in one embodiment, the step of determining the operation instructions compatible with the target database based on the target data dictionary and the target log further includes: If the operation type of the target log is a table structure change, then the table structure change operation description information is obtained based on the target data dictionary and the target log information; The description information of the data table structure change operation is converted into operation instructions compatible with the target database.

[0031] In one embodiment, taking the column addition operation type with SCN=8480 and data table ID=1001 as an example, the table structure change operation description information is obtained by parsing the target data dictionary. Then, according to the syntax rules of the target database, the data table structure change operation description information is converted into structured operation instructions for the target database to perform data synchronization.

[0032] Additionally, another embodiment of this application includes: If the table structure change operation DDL is a table creation, table modification, table deletion, or creation or modification of primary keys or indexes, then the data dictionary of the data table needs to be updated. If the DDL log type is table creation, then a new version of the data dictionary is created, the table creation DDL statement is parsed into a TableDict object, and added to the linked list of the multi-version data dictionary.

[0033] Furthermore, in one embodiment, after obtaining the table structure change operation description information, the method further includes: Based on the target data dictionary, a new version of the target data dictionary is generated according to the changes in the table structure, and the first SCN corresponding to the target log is used as the SCN corresponding to the new version of the target data dictionary.

[0034] In one embodiment, such as Figure 2 As shown, the changes to the data table structure also include adding a mapping relationship between the new column ID and the data table ID based on the target data dictionary, generating a new version of the data dictionary, setting the SCN of the new version of the data dictionary to the first SCN of the log (8480), and then the synchronization program takes the first SCN of the target log as the SCN corresponding to the new version of the target data dictionary, denoted as SCN1, and then returns to execute a new round of synchronization process to obtain a new starting SCN.

[0035] In this embodiment, by associating each data table structure change DDL with the corresponding data dictionary version, it can be ensured that the incremental log before the DDL operation matches the old version dictionary, and the incremental log after the DDL operation matches the new version dictionary (triggered by the DDL log), which fundamentally avoids the parsing errors caused by the new / old logs sharing the same dictionary and ensures the parsing accuracy of the entire incremental synchronization process.

[0036] Secondly, embodiments of this application also provide a database data synchronization system.

[0037] In one embodiment, reference is made to Figure 3 , Figure 3 This is a schematic diagram of the functional modules of an embodiment of the database data synchronization device of this application. Figure 3 As shown, the database data synchronization device includes: The acquisition module is used to acquire logs in the source database whose first-time identifier SCN is within a preset SCN range. The starting point of the preset SCN range is the second SCN. The first SCN is determined based on the log generation time, and the second SCN is determined based on the latest synchronization time between the metadata database and the source database. The metadata database includes a data dictionary for each version of the data table to be synchronized, and the data dictionary for each version is associated with the corresponding SCN. The parsing module is used to parse each acquired log to obtain information about each log, including the first SCN, the data table ID, and the operation type; the logs whose data table IDs belong to the set of data table IDs to be synchronized are the target logs. The determination module is used to determine the target data dictionary from the metadata database for each target log based on the data table ID and the first SCN included in the target log information. The target data dictionary is the data dictionary in the metadata database that corresponds to the data table ID and whose SCN is less than the first SCN and is closest to the first SCN. The sending module is used to determine operation instructions compatible with the target database based on the target data dictionary and target log, and send them to the target database so that the target database can execute the operation instructions to achieve data synchronization with the source database.

[0038] Furthermore, in one embodiment, the database data synchronization device further includes a determination and storage module, used for: Obtain the initial table structure of each table in the set of data tables to be synchronized from the source database; Based on the largest first SCN in the source database and the initial table structure, determine the initial data dictionary for each data table; The initial data dictionary for each data table is stored in the metadata database, and the largest first SCN is used as the second SCN.

[0039] Furthermore, in one embodiment, the parsing module is used for: Each log entry obtained is loaded into the log parsing tool LogMiner, so that LogMiner can parse the logs and store the parsing results in the system view of the source library; Query the parsing results in the system view to obtain information for each log entry.

[0040] Furthermore, in one embodiment, the determining module is used to: If the operation type of the target log is data change, then the data change operation description information is determined based on the target data dictionary and the target log information; The data change operation description information is converted into operation instructions compatible with the target database.

[0041] Furthermore, in one embodiment, the determining module is also used to: If the operation type of the target log is a table structure change, then the table structure change operation description information is obtained based on the target data dictionary and the target log information; The description information of the data table structure change operation is converted into operation instructions compatible with the target database.

[0042] Furthermore, in one embodiment, the database data synchronization device further includes a generation module, used for: Based on the target data dictionary, a new version of the target data dictionary is generated according to the changes in the table structure, and the first SCN corresponding to the target log is used as the SCN corresponding to the new version of the target data dictionary.

[0043] The functions of each module in the above-mentioned database data synchronization device correspond to the steps in the above-mentioned database data synchronization method embodiment, and their functions and implementation processes will not be described in detail here.

[0044] Thirdly, embodiments of this application provide a database data synchronization device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0045] Reference Figure 4 , Figure 4 This is a schematic diagram of the hardware structure of the database data synchronization device involved in the embodiments of this application. In the embodiments of this application, the database data synchronization device may include a processor, a memory, a communication interface, and a communication bus.

[0046] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0047] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the database data synchronization device, as well as interfaces used for interconnecting the database data synchronization device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0048] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0049] The processor can be a general-purpose processor, which can call a database data synchronization program stored in memory and execute the database data synchronization method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the database data synchronization program is called can be referred to in the various embodiments of the database data synchronization method of this application, and will not be repeated here.

[0050] Those skilled in the art will understand that the hardware structure shown in Figure m does not constitute a limitation of this application and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0051] Fourthly, embodiments of this application also provide a computer-readable storage medium.

[0052] The present application provides a computer-readable storage medium storing a database data synchronization program, wherein when the database data synchronization program is executed by a processor, it implements the steps of the database data synchronization method described above.

[0053] The method implemented when the database data synchronization program is executed can be referred to in various embodiments of the database data synchronization method of this application, and will not be repeated here.

[0054] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0055] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0056] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0057] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0058] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0059] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0060] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A database data synchronization method, characterized in that, The database data synchronization method includes: Obtain logs from the source database whose first-time identifier SCN falls within a preset SCN range. The starting point of the preset SCN range is the second SCN. The first SCN is determined based on the log generation time, and the second SCN is determined based on the latest synchronization time between the metadata database and the source database. The metadata database includes a data dictionary for each version of the data table to be synchronized, and each version's data dictionary is associated with its corresponding SCN. Each log entry is parsed to obtain information for each log entry, including the first SCN, data table ID, and operation type. The target log is the log whose table ID belongs to the set of table IDs to be synchronized; For each target log, a target data dictionary is determined from the metadata database based on the data table ID and the first SCN included in the target log information. The target data dictionary is the data dictionary in the metadata database that corresponds to the data table ID and whose SCN is less than the first SCN and is closest to the first SCN. Based on the target data dictionary and target log, operation instructions compatible with the target database are determined and sent to the target database so that the target database can execute the operation instructions to achieve data synchronization with the source database.

2. The database data synchronization method as described in claim 1, characterized in that, Before obtaining logs from the source library whose SCN is immediately identified as falling within a preset SCN range, the process also includes: Obtain the initial table structure of each table in the set of data tables to be synchronized from the source database; Based on the largest first SCN in the source database and the initial table structure, determine the initial data dictionary for each data table; The initial data dictionary for each data table is stored in the metadata database, and the largest first SCN is used as the second SCN.

3. The database data synchronization method as described in claim 1, characterized in that, The process of parsing each acquired log entry to obtain information for each log entry includes: Each log entry obtained is loaded into the log parsing tool LogMiner, so that LogMiner can parse the logs and store the parsing results in the system view of the source library; Query the parsing results in the system view to obtain information for each log entry.

4. The database data synchronization method as described in claim 1, characterized in that, The step of determining operation instructions compatible with the target database based on the target data dictionary and target log includes: If the operation type of the target log is data change, then the data change operation description information is determined based on the target data dictionary and the target log information; The data change operation description information is converted into operation instructions compatible with the target database.

5. The database data synchronization method as described in claim 1, characterized in that, The step of determining the operation instructions compatible with the target database based on the target data dictionary and target log also includes: If the operation type of the target log is a table structure change, then the table structure change operation description information is obtained based on the target data dictionary and the target log information; The description information of the data table structure change operation is converted into operation instructions compatible with the target database.

6. The database data synchronization method as described in claim 5, characterized in that, After obtaining the table structure change operation description information, it also includes: Based on the target data dictionary, a new version of the target data dictionary is generated according to the changes in the table structure, and the first SCN corresponding to the target log is used as the SCN corresponding to the new version of the target data dictionary.

7. A database data synchronization system, characterized in that, The database data synchronization system includes: The acquisition module is used to acquire logs in the source database whose first-time identifier SCN is within a preset SCN range. The starting point of the preset SCN range is the second SCN. The first SCN is determined based on the log generation time, and the second SCN is determined based on the latest synchronization time between the metadata database and the source database. The metadata database includes a data dictionary for each version of the data table to be synchronized, and the data dictionary for each version is associated with the corresponding SCN. The parsing module is used to parse each acquired log to obtain information about each log, including the first SCN, the data table ID, and the operation type; the logs whose data table IDs belong to the set of data table IDs to be synchronized are the target logs. The determination module is used to determine the target data dictionary from the metadata database for each target log based on the data table ID and the first SCN included in the target log information. The target data dictionary is the data dictionary in the metadata database that corresponds to the data table ID and whose SCN is less than the first SCN and is closest to the first SCN. The sending module is used to determine operation instructions compatible with the target database based on the target data dictionary and target log, and send them to the target database so that the target database can execute the operation instructions to achieve data synchronization with the source database.

8. The database data synchronization system as described in claim 7, characterized in that, The parsing module includes: Each log entry obtained is loaded into the log parsing tool LogMiner, so that LogMiner can parse the logs and store the parsing results in the system view of the source library; Query the parsing results in the system view to obtain information for each log entry.

9. A database data synchronization device, characterized in that, The database data synchronization device includes a processor, a memory, and a database data synchronization program stored in the memory and executable by the processor, wherein when the database data synchronization program is executed by the processor, it implements the steps of the database data synchronization method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a database data synchronization program, wherein when the database data synchronization program is executed by a processor, it implements the steps of the database data synchronization method as described in any one of claims 1 to 6.

Images