Data synchronization method, device and electronic device

Abstract

The embodiments of this specification disclose a data synchronization method, device and electronic device. The method includes: after the application version is put into production, obtaining a first data table set that has changed in the source database compared to before the application version was put into production; obtaining a data table commonly included in the first data table set and the second data table set as a data table in a third data table set, wherein the second data table set includes data tables that need to be synchronized in the source database before the application version is put into production; according to the third data table set, updating the metadata of the source database and the target database; according to the updated source database and the updated target database, performing a data synchronization operation to synchronize the data in the data table of the source database to the data table in the target database. The embodiments of this specification can realize data synchronization.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular to a data synchronization method, device and electronic device. Background Art

[0002] In banking business, it is often necessary to synchronize data from the source database to the target database. Due to the adjustment requirements of banking business or system architecture, the application version may be iterated. When the application version is put into production, the table structure of the source database may change. As a result, a lot of manual operations are required to achieve data synchronization. Summary of the invention

[0003] The embodiments of the present specification provide a data synchronization method, device, and electronic device to achieve data synchronization when the table structure of a source database changes due to the production of an application version, thereby improving data synchronization efficiency.

[0004] A first aspect of an embodiment of this specification provides a data synchronization method, including:

[0005] After the application version is put into production, a first set of data tables that have changed in the source database compared to before the application version was put into production is obtained;

[0006] Acquire data tables commonly included in the first data table set and the second data table set as data tables in the third data table set, wherein the second data table set includes data tables that need to be synchronized in the source database before the application version is put into production;

[0007] According to the third data table set, metadata of the source database and the target database are updated;

[0008] A data synchronization operation is performed according to the updated source database and the updated target database to synchronize the data in the data table of the source database to the data table of the target database.

[0009] A second aspect of the embodiments of this specification provides a data synchronization device, including:

[0010] A first acquisition unit is used to acquire, after the application version is put into production, a first data table set that has changed in the source database compared to before the application version was put into production;

[0011] A second acquisition unit is used to acquire data tables commonly included in the first data table set and the second data table set as data tables in the third data table set, wherein the second data table set includes data tables that need to be synchronized in the source database before the application version is put into production;

[0012] An updating unit, used for updating metadata of a source database and a target database according to a third data table set;

[0013] The execution unit is used to perform a data synchronization operation according to the updated source database and the updated target database, so as to synchronize the data in the data table of the source database to the data table of the target database.

[0014] According to a third aspect of the embodiments of this specification, an electronic device is provided, including:

[0015] at least one processor;

[0016] A memory storing program instructions, wherein the program instructions are configured to be suitable for being executed by the at least one processor, and the program instructions include instructions for executing the method as described in the first aspect.

[0017] The technical solutions provided in the embodiments of this specification are: BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 A flowchart of a data synchronization method in an embodiment of this specification;

[0020] Figure 2 This is a schematic diagram of the structure of a data synchronization device in an embodiment of this specification;

[0021] Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of this specification. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0023] In the related art, when an application version is put into production, the table structure of the source database may change. In this case, it is necessary to stop the data synchronization operation first, then put the application version into production, and finally refresh the source database and the target database. If the above operations are omitted when the application version is put into production, data synchronization errors will occur.

[0024] See also Figure 1 The embodiment of this specification provides a data synchronization method. The data synchronization method can be applied to a server. The server can be installed with data synchronization software. The data synchronization software can include IBM's Change Data Capture (CDC) software, etc. The data synchronization method can include the following steps.

[0025] Step S11: After the application version is put into production, a first set of data tables that have changed in the source database compared to before the application version was put into production is obtained.

[0026] In some embodiments, before the application version is put into production, it can be determined whether the source database meets the data synchronization stop condition; if so, the application version can be put into production. Specifically, for example, it can be determined whether the source database has unfinished sessions and / or unfinished transactions; if so, it can be determined that the source database does not meet the data synchronization stop condition; if not, it can be determined that the source database meets the data synchronization stop condition.

[0027] In some embodiments, after the application version is put into production, the cached data of the source database instance can be cleared.

[0028] In some embodiments, after the application version is put into production, a data table that has changed in the source database compared to before the application version is put into production can be obtained as a data table in the first data table set, wherein the changed data table may include at least one of the following: a newly added data table, a rebuilt data table, and a data table with a changed structure.

[0029] Step S13: obtaining data tables commonly included in the first data table set and the second data table set as data tables in the third data table set, wherein the second data table set includes data tables that need to be synchronized in the source database before the application version is put into production.

[0030] In some embodiments, the second data table set includes data tables that need to be synchronized in the source database before the application version is put into production. The second data table set may be pre-set. Data tables included in the first data table set and the second data table set may be obtained as data tables in the third data table set. The data tables in the third data table set may be data tables that need to be refreshed after the application version is put into production.

[0031] Step S15: Update the metadata of the source database and the target database according to the third data table set.

[0032] In some embodiments, the metadata of the database may include configuration information of the database. The metadata of the source database and the metadata of the target database may be updated according to the data table in the third data table set. For example, the metadata of the source database and the metadata of the target database may be updated by executing the CHCCLP script command in the CDC software.

[0033] Step S17: performing a data synchronization operation according to the updated source database and the updated target database to synchronize the data in the data table of the source database to the data table of the target database.

[0034] In some embodiments, a data synchronization operation may be performed based on the updated source database and the updated target database to synchronize data in a data table of the source database to a data table of the target database.

[0035] In some embodiments, the predefined mapping rules can be updated according to the third data table set, and the mapping rules are used to represent the mapping relationship between the data table in the source database and the data table in the target database. In this way, the data synchronization operation can be performed according to the updated source database, the updated target database and the updated mapping rules to synchronize the data in the data table of the source database to the data table in the target database. For example, the predefined mapping rules can be updated by executing the CHCCLP script command in the CDC software.

[0036] The data synchronization method of the embodiment of this specification can obtain the first data table set that has changed in the source database compared to before the application version is put into production after the application version is put into production; can obtain the data tables contained in the first data table set and the second data table set as the data tables in the third data table set, and the second data table set contains the data tables that need to be synchronized in the source database before the application version is put into production; can update the metadata of the source database and the target database according to the third data table set; can perform data synchronization operations according to the updated source database and the updated target database to synchronize the data in the data table of the source database to the data table in the target database. The embodiment of this specification can realize data synchronization when the table structure of the source database changes due to the production of the application version, reduce the manual operation cost, and improve the efficiency of data synchronization.

[0037] See also Figure 2 The embodiment of this specification provides a data synchronization device. The data synchronization device can be applied to a server, and specifically can include the following units.

[0038] A first acquisition unit 21 is used to acquire, after the application version is put into production, a first data table set that has changed in the source database compared to before the application version is put into production;

[0039] The second acquisition unit 23 is used to acquire data tables commonly included in the first data table set and the second data table set as data tables in the third data table set, wherein the second data table set includes data tables that need to be synchronized in the source database before the application version is put into production;

[0040] An updating unit 25, configured to update metadata of a source database and a target database according to a third data table set;

[0041] The execution unit 27 is used to perform a data synchronization operation according to the updated source database and the updated target database, so as to synchronize the data in the data table of the source database to the data table of the target database.

[0042] See also Figure 3 The embodiments of this specification also provide an electronic device.

[0043] The electronic device may include a memory and a processor.

[0044] In this embodiment, the memory includes but is not limited to dynamic random access memory (DRAM) and static random access memory (SRAM), etc. The memory can be used to store computer instructions.

[0045] In this embodiment, the processor may be implemented in any suitable manner. For example, the processor may take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc. The processor may be used to execute the computer instructions to implement Figure 1 The corresponding embodiment.

[0046] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for device embodiments and electronic device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In addition, it is understandable that after reading this specification document, those skilled in the art can think of any combination of some or all of the embodiments listed in this specification without creative work, and these combinations are also within the scope of disclosure and protection of this specification.

[0047] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0048] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0049] It can be known from the above description of the implementation mode that the technicians in this field can clearly understand that the present specification can be implemented by means of software plus the necessary general hardware platform. Based on such an understanding, the technical solution of the present specification can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present specification or some parts of the embodiments.

[0050] This specification can be used in many general or special computer system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments that include any of the above systems or devices.

[0051] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0052] Although the present specification is described through embodiments, those skilled in the art will appreciate that there are many modifications and changes to the present specification without departing from the spirit of the present specification, and it is intended that the appended claims include these modifications and changes without departing from the spirit of the present specification.

Claims

1. A data synchronization method, comprising: After the application version is put into production, a first data table set that has changed in the source database compared to before the application version is put into production is obtained, wherein the first data table includes changed data tables, and the changed data tables include data tables that have changed in the source database after the application version is put into production compared to the source database before the application version is put into production; Acquire data tables commonly included in the first data table set and the second data table set as data tables in the third data table set, wherein the second data table set includes data tables that need to be synchronized in the source database before the application version is put into production; According to the third data table set, metadata of the source database and the target database are updated; A data synchronization operation is performed according to the updated source database and the updated target database to synchronize the data in the data table of the source database to the data table of the target database.

2. The method of claim 1, further comprising: Before the application version is put into production, determine whether the source database meets the conditions for stopping data synchronization; If so, stop the data synchronization operation so that the application version can be put into production.

3. The method according to claim 2, wherein the step of determining whether the source database satisfies the data synchronization stop condition comprises: Determine whether there are any unfinished sessions and / or unfinished transactions in the source database; If it exists, it is determined that the source database does not meet the data synchronization stop conditions.

4. The method according to claim 2, wherein the step of determining whether the source database satisfies the data synchronization stop condition comprises: Determine whether there are any unfinished sessions and / or unfinished transactions in the source database; If it does not exist, determine whether the source database meets the conditions for stopping data synchronization.

5. The method according to claim 1, wherein the first set of data tables that have changed in the source database comprises at least one of the following: a newly added data table, a rebuilt data table, and a data table with a changed structure.

6. The method of claim 1, further comprising: Clear the cached data of the source DB instance.

7. The method of claim 1, further comprising: According to the third data table set, the predefined mapping rule is updated, where the mapping rule is used to represent the mapping relationship between the data table in the source database and the data table in the target database.

8. The method according to claim 7, wherein the performing of the data synchronization operation comprises: Perform data synchronization operations based on the updated mapping rules.

9. A data synchronization device, comprising: A first acquisition unit is used to acquire, after the application version is put into production, a first data table set that has changed in the source database compared to before the application version is put into production, wherein the first data table includes a changed data table, and the changed data table includes a data table that has changed in the source database after the application version is put into production compared to the source database before the application version is put into production; A second acquisition unit is used to acquire data tables commonly included in the first data table set and the second data table set as data tables in the third data table set, wherein the second data table set includes data tables that need to be synchronized in the source database before the application version is put into production; An updating unit, used for updating metadata of a source database and a target database according to a third data table set; The execution unit is used to perform a data synchronization operation according to the updated source database and the updated target database, so as to synchronize the data in the data table of the source database to the data table of the target database.

10. An electronic device comprising: at least one processor; A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, and the program instructions include instructions for executing the method according to any one of claims 1 to 8.

Images