Data synchronization method, system and device and storage medium

Abstract

The invention relates to a data synchronization method and device and a storage medium, and relates to the field of database synchronization. The method comprises the following steps: firstly, acquiring change data in a source business database; secondly, based on a preset rule, performing rule processing on the change data, and outputting a change object matched with the target business database; and finally, executing a structured query language (SQL) statement corresponding to the change object to the target business database, and displaying a data synchronization state of the source business database and the target business database at a visual front end. Through the above steps, according to the technical scheme of the disclosure, the data needing to be synchronized can be processed based on the preset rule updated at any time, heterogeneous data synchronization is realized, and a high-frequency transaction millisecond-level real-time synchronization scene is met; and meanwhile, field mapping, condition filtering and calculation rules are defined at a visual front end, so that the data synchronization efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of database synchronization, and particularly to a data synchronization method, system, device, and storage medium. Background Art

[0002] In the context of digital transformation, it has become normal for enterprises to support business operations through multiple sets of information systems, such as Enterprise Resource Planning (ERP) systems, Supply Chain Management (SCM) systems, and Customer Relationship Management (CRM) systems. Currently, database synchronization technologies are mainly divided into two categories:

[0003] Batch ETL tools (such as Informatica, DataStage): Extract, transform, and load data through scheduled tasks, but there are problems such as high latency and large resource consumption, and they cannot meet real-time scenarios.

[0004] Change Data Capture (CDC) tools (such as Debezium, Canal): Can capture database incremental change events, but lack the ability to deeply integrate dynamic data transformation and rule engines, and additional adaptation logic needs to be developed.

[0005] However, due to problems such as historical technology selection differences, collaboration barriers between departments, and asynchronous system iteration cycles, there are significant heterogeneities in the database table structures, field naming rules, and data storage logics of each system, forming "information silos". Traditional data synchronization solutions rely on manual script development or batch Extract-Transform-Load (ETL) tools, and it is difficult to meet the requirements of real-time business decision-making, cross-system data consistency, and agile response.

[0006] Insufficient real-time performance: Traditional ETL tools rely on periodic batch processing, and the latency is usually in minutes or even hours, and they cannot support millisecond-level response scenarios such as financial transactions and real-time monitoring.

[0007] Flexibility and maintenance cost: Data transformation logic needs to be implemented through hard coding, and business rule changes require re-development and deployment, resulting in a long development cycle and high maintenance cost; lack of visual debugging tools, and the rule verification efficiency is low.

[0008] Therefore, there is an urgent need for a synchronization system that can dynamically adapt to heterogeneous data models, support real-time synchronization, and have high reliability as an intermediary role to eliminate data circulation barriers and support the efficient utilization of enterprise-level data assets. Summary of the Invention

[0009] Based on the above problems, the present disclosure proposes a data synchronization method, which can process the data to be synchronized based on preset rules that are updated at any time, achieve heterogeneous data synchronization, and meet the millisecond-level real-time synchronization scenario of high-frequency trading; at the same time, define field mapping, condition filtering, and calculation rules in the visualization front-end to improve the data synchronization efficiency.

[0010] According to some embodiments of the present disclosure, there is provided a data synchronization method, including:

[0011] Obtain the changed data in the source business database;

[0012] Based on preset rules, perform rule processing on the changed data and output a changed object adapted to the target business database;

[0013] Execute the Structured Query Language (SQL) statement corresponding to the changed object into the target business database, and display the data synchronization status of the source business database and the target business database in the visualization front-end.

[0014] In some embodiments, the step of obtaining the changed data in the source business database includes:

[0015] Subscribe to the changed data and perform preprocessing on the changed data after obtaining it.

[0016] In some embodiments, the step of performing rule processing on the changed data based on preset rules and outputting a changed object adapted to the target business database includes:

[0017] According to a preset function library, hot-load the preset rules and generate a rule script corresponding to the preset rules;

[0018] Execute the rule script and output a changed object adapted to the target business database.

[0019] In some embodiments, the step of performing rule processing on the changed data based on preset rules and outputting a changed object adapted to the target business database further includes:

[0020] Based on the operation data obtained in the visualization front-end, update and load the preset rules.

[0021] In some embodiments, between the step of outputting a changed object adapted to the target business database and the step of executing the Structured Query Language (SQL) statement corresponding to the changed object into the target business database, the method further includes:

[0022] Based on an SQL standard extension adapter, identify the type of the target business database;

[0023] Generate a data manipulation DML language adapted to the SQL statement corresponding to the change object according to the type of the target business database;

[0024] Merge the SQL statements belonging to the same type of DML language.

[0025] In some embodiments, the method further includes:

[0026] Corresponding to an execution error, save an error record, and repeatedly execute the foregoing steps of performing rule processing on the changed data based on preset rules, outputting a change object adapted to the target business database; executing the structured query language SQL statement corresponding to the change object into the target business database, and displaying the data synchronization status of the source business database and the target business database on a visual front end.

[0027] According to some other embodiments of the present disclosure, there is provided a data synchronization system, including:

[0028] A data layer, including a source business database and a target business database;

[0029] A service layer for performing the following steps:

[0030] Obtain the changed data in the source business database;

[0031] Based on preset rules, perform rule processing on the changed data, and output a change object adapted to the target business database;

[0032] Execute the structured query language SQL statement corresponding to the change object into the target business database;

[0033] A front-end display layer for displaying the data synchronization status of the source business database and the target business database on a visual front end.

[0034] According to some other embodiments of the present disclosure, there is provided a data synchronization device, including:

[0035] An acquisition module for acquiring the changed data in the source business database;

[0036] A processing module for performing rule processing on the changed data based on preset rules and outputting a change object adapted to the target business database;

[0037] A synchronization module for executing the structured query language SQL statement corresponding to the change object into the target business database and displaying the data synchronization status of the source business database and the target business database on a visual front end.

[0038] According to still some other embodiments of the present disclosure, there is provided an apparatus, including: a memory; and a processor coupled to the memory, the processor being configured to execute the method in any one of the above embodiments based on instructions stored in the memory device.

[0039] According to still some other embodiments of the present disclosure, there is provided a computer-readable storage medium having computer instructions stored thereon, and when the instructions are executed by a processor, the method in any one of the above embodiments is implemented.

[0040] According to still some other embodiments of the present disclosure, there is also provided a computer program product, including instructions that, when executed by a processor, cause the processor to execute the method according to any one of the above embodiments.

[0041] In the above embodiments, first, change data in the source service database is obtained; secondly, based on preset rules, the change data is processed according to the rules to output a change object adapted to the target service database; finally, the structured query language (SQL) statement corresponding to the change object is executed into the target service database, and the data synchronization status of the source service database and the target service database is displayed on the visual front end. Through the above steps, the data to be synchronized is processed based on the preset rules that are updated at any time, so as to achieve heterogeneous data synchronization and meet the millisecond-level real-time synchronization scenario of high-frequency transactions; at the same time, field mapping, condition filtering, and calculation rules are defined on the visual front end to improve the data synchronization efficiency.

[0042] Other features, aspects, and advantages of the present disclosure will become clear through the following detailed description of the exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The embodiments of the present disclosure will be described below with reference to the accompanying drawings. It should be understood that the drawings in the following description only relate to some embodiments of the present disclosure and do not constitute a limitation to the present disclosure. In the drawings:

[0044] Figure 1 Flowcharts showing some embodiments of the data synchronization method of the present disclosure;

[0045] Figure 2 Flowcharts showing some embodiments of the data synchronization system of the present disclosure;

[0046] Figure 3 Flowcharts showing still some other embodiments of the data synchronization method of the present disclosure;

[0047] Figure 4 Flowcharts showing some embodiments of the data synchronization apparatus of the present disclosure;

[0048] Figure 5 Block diagrams showing some embodiments of the data synchronization method of the present disclosure;

[0049] Figure 6 A block diagram showing some embodiments of an electronic device for data synchronization according to the present disclosure.

[0050] It should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not necessarily drawn according to actual proportional relationships. The same or similar reference numerals are used in the various drawings to represent the same or similar components. Therefore, once an item is defined in one drawing, it may not be further discussed in subsequent drawings. Detailed embodiments

[0051] Next, the technical solutions in the embodiments of the present disclosure will be clearly and completely described in conjunction with the drawings in the embodiments of the present disclosure. It should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein.

[0052] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard. Unless otherwise specifically stated, the relative arrangements, numerical expressions, and numerical values of the components and steps set forth in these embodiments should be construed as merely exemplary and not limiting the scope of the present disclosure.

[0053] The term "comprising" and its variants used in the present disclosure mean an open term that includes at least the subsequent elements / features but does not exclude other elements / features, that is, "including but not limited to". The term "based on" means "at least partially based on".

[0054] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules, or units, and are not used to limit the order of the functions performed by these devices, modules, or units or their interdependent relationships. Unless otherwise specified, the concepts such as "first" and "second" are not intended to imply that the objects so described must be in a given order in terms of time, space, ranking, or any other way.

[0055] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise clearly stated in the context, it should be understood as "one or more".

[0056] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0057] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0058] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In addition, in one or more embodiments, specific features, structures, or characteristics can be combined in any suitable manner that will be clear to those of ordinary skill in the art from the present disclosure.

[0059] Figure 1 The flowchart showing some embodiments of the data synchronization method of the present disclosure is as Figure 1 shown:

[0060] S101, obtain the changed data in the source business database.

[0061] Among them, this step mainly includes subscribing to the changed data and preprocessing the changed data after obtaining it. Specifically, the data capturer listens for and obtains the changed data in the source business database, accepts data of a specified schema and table, filters out unnecessary data, preprocesses the changed data, and converts it into changed data in a fixed format. The changed data includes schema, table, event type (insert, update, delete), and the data before and after the change.

[0062] S102, based on preset rules, perform rule processing on the changed data and output a changed object adapted to the target business database.

[0063] This step mainly includes the following steps:

[0064] According to the preset function library, hot-load the preset rules and generate a rule script for the preset rules; execute the rule script and output a changed object adapted to the target business database.

[0065] Specifically, the preset function library is mainly set in the rule engine. The rule engine is mainly based on the QLExpress dynamic script engine parsing tool, supports Groovy / Java syntax, and realizes dynamic loading and hot update of business rules. At the same time, it is responsible for executing the rule script for data adaptation written according to the business logic, and subsequently adapting the changed data of the source business database into the changed object of the target business database.

[0066] In this method, the step of performing rule processing on the changed data and outputting the changed object adapted to the target business database further includes: updating and loading the preset rules based on the operation data obtained from the visual front end.

[0067] Among them, the visual operation front end will display the data synchronization status in real time, and can debug the preset rules based on the synchronization status in real time. The rule engine supports hot loading of rule scripts, and the new rules can take effect without restarting the service. At the same time, built-in function libraries (such as date formatting and data encryption) are provided in the rule engine to reduce the debugging difficulty and the development complexity of rule scripts.

[0068] Between steps S102 and S103, this data synchronization method further includes:

[0069] Based on the SQL standard extension adapter, identify the type of the target business database; according to the type of the target business database, generate a data manipulation language (DML) adapted to the structured query language (SQL) statement corresponding to the changed object; merge the SQL statements belonging to the same type of DML language.

[0070] Specifically, the SQL generator generates the corresponding SQL statements for the changed objects according to the configuration metadata. The types of DML languages include SQL statements of insert, update, and delete. Among them, for insert statements, they can be merged into a single statement. For example, two insert statements insert into user (id, name) values (1, 'a'); insert into user (id, name) values (2, 'b') can be merged into one statement: insert into user (id, name) values (1, 'a'), (2, 'b'). It should be noted that only insert statements are merged, and there are no similar statements for update and delete.

[0071] S103, execute the structured query language SQL statement corresponding to the changed object into the target business database, and display the data synchronization status of the source business database and the target business database on the visual front end.

[0072] Among them, the SQL execution scheduler executes the generated SQL statements to the target business database according to the configuration metadata and ensures eventual consistency.

[0073] During the execution of the foregoing steps, embodiments of the present disclosure may also perform the following steps in real time:

[0074] Corresponding to an execution error, save an error record, and repeat the steps of processing the changed data based on a preset rule, outputting a changed object adapted to the target business database; executing a structured query language (SQL) statement corresponding to the changed object to the target business database, and displaying the data synchronization status of the source business database and the target business database on the visualization front end.

[0075] Through the above steps, the present disclosure supports a dynamic rule engine, supports hot loading of rule scripts, and new preset rules can take effect without restarting the service; and the built-in function library of the preset rules (such as date formatting and data encryption) reduces the complexity of script development. Process the data to be synchronized based on the preset rules that can be updated at any time to achieve heterogeneous data synchronization. Based on the ANSI SQL standard extended adapter, automatically identify the target library dialect (such as ON DUPLICATE KEY UPDATE of MySQL), and at the same time optimize batch insertion: merge multiple INSERTs into a single statement to reduce network overhead. Support automatic error retry, alarm and notify manual intervention if the retry is still in error, and support a convenient debugger function. Meet the millisecond-level real-time synchronization scenario of high-frequency transactions; at the same time, define field mapping, condition filtering, and calculation rules on the visualization front end to improve the data synchronization efficiency.

[0076] Figure 2 A flowchart showing some embodiments of the data synchronization system of the present disclosure. Among them, a data synchronization system is characterized by including:

[0077] A data layer, including a source business database and a target business database;

[0078] A service layer for performing the following steps:

[0079] Obtain the changed data in the source business database; process the changed data based on a preset rule, and output a changed object adapted to the target business database; execute a structured query language (SQL) statement corresponding to the changed object to the target business database;

[0080] A front-end display layer for displaying the data synchronization status of the source business database and the target business database on the visualization front end.

[0081] Among them, the specific processing steps of each layer of the data synchronization system are the same as Figure 1The data synchronization method shown is not elaborated here.

[0082] The following is an explanation of the data processing in each layer of the system:

[0083] Front-end display layer: Provides a visual rule configuration interface, supports script editing, debugging, and version management. Integrates a real-time monitoring panel to display data synchronization status, throughput, and error logs, and supports data replay after manual repair.

[0084] The service layer mainly processes at least one of the following services:

[0085] Data subscription: Connects to the changed data of the source business database, accepts data of specified schema and table, filters unnecessary data, preprocesses the data, and converts it into a data object in a fixed format. This object includes schema, table, event type (insert, update, delete), and the data before and after the change.

[0086] Rule engine: Mainly based on the QLExpress dynamic script engine parsing tool, supports Groovy / Java syntax, and realizes dynamic loading and hot update of business rules. Responsible for executing the data adaptation script written according to the business logic, and adapting the changed data object of the source database to the changed object of the target database.

[0087] SQL generator: Generates an adapted DML statement according to the target database type (such as MySQL, MongoDB), and supports batch operation optimization.

[0088] SQL execution scheduler: Responsible for executing the SQL statement to the target database, supports transaction rollback, idempotent operation, and automatic isolation of failed SQL.

[0089] Debugger: Responsible for executing the corresponding rule script according to the data change object in debug mode and outputting the corresponding SQL statement.

[0090] Error handling: In case of an error during the execution process, the original data and error stack information will be saved, and the function of replaying and executing specified records after modifying the rule script is supported.

[0091] The data layer includes the system database used by this system, as well as the source business database and target business database for adaptation, etc.

[0092] Figure 3 The flowchart showing some other embodiments of the data synchronization method of the present disclosure.

[0093] As Figure 3As shown in the figure, the Canal data capturer listens to the Binlog of the source database to obtain the changed data of the source business database, and as a producer, pushes the changed data into the queue middleware; the data subscriber, as a consumer, accepts the changed data from the queue middleware according to the configuration metadata, preprocesses the changed data, and converts it into a data object in a fixed format; the rule engine takes the changed data as input according to the rule script of the configuration metadata, executes the script and outputs an adapted change object; the SQL generator generates corresponding SQL statements, including database operation statements such as insert, update, and delete, for the change object according to the configuration metadata; the SQL execution scheduler executes the generated SQL statements into the target business database according to the configuration metadata and ensures eventual consistency; the operation and maintenance management platform can access the debugged changed data into the rule engine to execute and generate SQL statements through the debugger, but it will not reach the SQL executor for actual execution; the error record retry function is that in the scenario of generating error messages, the system will automatically retry the execution. If it still fails after several attempts, an alarm will be sent to notify manual intervention to fix the error, and the operation will re-execute the original changed data. After the execution is successful, its execution status will be marked.

[0094] Based on the above embodiments, the present disclosure adopts an event-driven architecture and a lightweight rule engine, with an end-to-end latency ≤ 50 ms (measured value), meeting the millisecond-level real-time synchronization scenario for high-frequency trading; business personnel can define preset rules such as field mapping, condition filtering, and calculation rules through the visual front-end display layer, and the effective time for the change of the preset rules < 1 minute; the source business database and the target business database support mainstream relational databases.

[0095] Figure 4 The flowchart of some embodiments of the data synchronization device of the present disclosure is shown. It includes:

[0096] An acquisition module 401, configured to acquire the changed data in the source business database;

[0097] A processing module 402, configured to perform rule processing on the changed data based on preset rules and output a change object adapted to the target business database;

[0098] A synchronization module 403, configured to execute the structured query language SQL statement corresponding to the change object into the target business database and display the data synchronization status of the source business database and the target business database on the visual front-end.

[0099] It should be noted that the device embodiments correspond one by one to the above method embodiments. All implementation manners in the above method embodiments are applicable to the device embodiments and can achieve the same technical effects.

[0100] Figure 5Block diagram showing some embodiments of the data synchronization method of the present disclosure.

[0101] The memory 51 is used to store one or more computer-readable instructions. The memory 51 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), flash memory. The memory 51 can store, for example, an operating system, application programs, a boot loader (BootLoader), databases, and other programs, and can also store various application programs and various data, etc.

[0102] The processor 52 is used to run the computer-readable instructions to implement the data synchronization method described in any of the foregoing embodiments. For the specific implementation of each step of the data synchronization method, reference can be made to the foregoing embodiments, and the repeated parts will not be elaborated here.

[0103] The processor 52 can be configured to execute the steps of the data synchronization method in any of the above embodiments. The processor 52 can be embodied as various processing devices, such as a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The central processing unit (CPU) can be of the X86 or ARM architecture, etc.

[0104] The processor 52 and the memory 51 can communicate with each other directly or indirectly. For example, the processor 52 and the memory 51 can communicate through a network. The network can include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 52 and the memory 51 can also communicate with each other through a system bus, and the present disclosure does not limit this.

[0105] It should be noted that Figure 5 The components of the data synchronization device 51 shown are exemplary and not restrictive. According to actual application needs, the data synchronization device 5 may also have other components. The processor 52 can control other components in the data synchronization device 5 to perform the desired functions.

[0106] The data synchronization device 5 can be implemented in software, firmware, and / or hardware, and can be integrated into a device installed with relevant application programs.

[0107] Figure 6 Block diagram showing some embodiments of the electronic device for data synchronization of the present disclosure.

[0108] Figure 6 The electronic device 6 for data synchronization shown can be a computer system with a dedicated hardware structure, and can execute corresponding functions when relevant application programs are installed.

[0109] The electronic device 6 for data synchronization includes, but is not limited to, mobile terminals such as smart phones, laptop computers, personal digital assistants (PDAs), tablet personal computers (TabletPCs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable devices, etc., and fixed terminals such as digital TVs, desktop computers, etc.

[0110] As Figure 6 shown, the central processing unit (CPU) 61 executes various processes according to the programs stored in the read-only memory (ROM) 62 or the programs loaded from the storage section 68 into the random access memory (RAM) 63. In the RAM 63, data required when the CPU 61 executes various processes, etc. is stored as needed. The central processing unit is merely exemplary, and it can also be other types of processors, such as the various processors described above. The ROM 62, RAM 63, and storage section 68 can be various forms of computer-readable storage media. It should be noted that although Figure 6 the ROM 62, RAM 63, and storage section 68 are shown separately, one or more of them can be combined, or located in the same or different memories or storage modules.

[0111] The CPU 61, ROM 62, and RAM 63 are connected to each other via a bus 64. The input / output interface 65 is also connected to the bus 64.

[0112] The following components are connected to the input / output interface 65: an input section 66, such as a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output section 67, including a display, such as a cathode ray tube (CRT), liquid crystal display (LCD), speaker, vibrator, etc.; a storage section 68, including a hard disk, magnetic tape, etc.; and a communication section 69, including a network interface card such as a LAN card, modem, etc. The communication section 69 allows communication processing to be performed via a network such as the Internet. It is easy to understand that although Figure 6 some parts in the electronic device 6 for data synchronization shown communicate via the bus 64, they can also communicate via a network or other means, where the network can include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.

[0113] As needed, the driver 610 is also connected to the input / output interface 65. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the driver 610 as needed, so that a computer program read therefrom is installed in the storage section 68 as needed.

[0114] In the case where the above-described series of processes are implemented by software, a program constituting the software can be installed from a network such as the Internet or a storage medium such as the removable medium 611.

[0115] According to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product that, when run on a computer, causes the computer to implement the data synchronization method described in any one of the foregoing embodiments. The computer program product includes computer instructions carried on a computer-readable medium and includes program code for executing the method shown in the flowchart. In such an embodiment, the computer instructions can be downloaded and installed from the network through the communication section 69, or installed from the storage section 68, or installed from the ROM 62. When the computer program is executed by the CPU 61, the data synchronization method of any embodiment of the present disclosure is executed.

[0116] It should be noted that, in the context of the present disclosure, a computer-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0117] A computer-readable medium can be a computer-readable storage medium, a computer-readable signal medium, or any combination of the two.

[0118] Computer-readable storage media include, but are not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. Computer instructions are stored on the computer-readable storage medium, and when the instructions are executed by a processor, the data synchronization method described in any one of the foregoing embodiments is implemented.

[0119] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0120] The above computer-readable medium may be included in the above electronic device; or may exist separately without being assembled into the electronic device.

[0121] In some embodiments, a computer program is also provided, including: instructions that, when executed by a processor, cause the processor to execute the data synchronization method described in any of the foregoing embodiments. For example, the instructions may be embodied as computer program code.

[0122] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include but are not limited to object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0124] The functions described above can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0125] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and not for limiting the scope of the present disclosure. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A data synchronization method, characterized in that, Including: Obtain the changed data in the source business database; Based on preset rules, perform rule processing on the changed data and output a changed object adapted to the target business database; Execute the Structured Query Language (SQL) statement corresponding to the changed object into the target business database, and display the data synchronization status of the source business database and the target business database on the visual front end.

2. The data synchronization method according to claim 1, wherein The step of obtaining the changed data in the source business database includes: Subscribe to the changed data and preprocess the changed data after obtaining it.

3. The data synchronization method according to claim 1, characterized in that, The step of performing rule processing on the changed data based on preset rules and outputting a changed object adapted to the target business database includes: According to a preset function library, hot-load the preset rules and generate a rule script corresponding to the preset rules; Execute the rule script and output a changed object adapted to the target business database.

4. The data synchronization method according to claim 3, wherein The step of performing rule processing on the changed data based on preset rules and outputting a changed object adapted to the target business database further includes: Update and load the preset rules based on the operation data obtained on the visual front end.

5. The data synchronization method according to claim 1, characterized in that Between the step of outputting a changed object adapted to the target business database and the step of executing the SQL statement corresponding to the changed object into the target business database, the method further includes: Based on an SQL standard extension adapter, identify the type of the target business database; According to the type of the target business database, generate a Data Manipulation Language (DML) adapted to the SQL statement corresponding to the changed object; Merge the SQL statements belonging to the same type of DML language.

6. The data synchronization method according to claims 1-5, characterized in that The method further includes: In case of an execution error, save an error record and repeat the steps of performing rule processing on the changed data based on preset rules, outputting a changed object adapted to the target business database; executing the SQL statement corresponding to the changed object into the target business database, and displaying the data synchronization status of the source business database and the target business database on the visual front end.

7. A data synchronization system, characterized in that, Including: A data layer, including a source business database and a target business database; A service layer, used to perform the following steps: Obtain the changed data in the source business database; Based on preset rules, perform rule processing on the changed data and output a changed object adapted to the target business database; Execute the SQL statement corresponding to the changed object into the target business database; A front-end display layer, used to display the data synchronization status of the source business database and the target business database on the visual front end.

8. A data synchronization device, characterized in that, Including: An acquisition module, used to obtain the changed data in the source business database; A processing module, used to perform rule processing on the changed data based on preset rules and output a changed object adapted to the target business database; A synchronization module, configured to execute a Structured Query Language (SQL) statement corresponding to the changed object to the target business database, and display the data synchronization status of the source business database and the target business database on a visualization front end.

9. A data processing apparatus, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the data processing method according to any one of claims 1 to 6 based on instructions stored in the memory.

10. A computer-readable storage medium, having stored thereon computer instructions which, when executed by a processor, implement the data processing method according to any one of claims 1 to 6.

Images