Data Packet-Based Remote Data Incremental Synchronization Method

Abstract

The invention relates to a remote data increment synchronization method and a device based on a data package. The method comprises a first step of obtaining data manipulation languages (DML) of a database, a second step of leading out the DML into the data package, a third step of sending the data package to a target database in a remote synchronization mode, a fourth step of analyzing the data package into structural query languages (SQL) and changing data, and a fifth step of updating the SQL and the changing data to the target database. According to the remote data increment synchronization method and the device based on the data package, increment records, deleted records and changed records in the database can be extracted in fixed time to form the data package including the increment records. According to network situations and data synchronization demands, the data package is transmitted to a target database system in a java message service (JMS) or a file transfer protocol (FTP) manner by utilizing message-oriented middleware or an FTP client side / server terminal product and being supported by a certain allocation management, and therefore consistency of data of a remote terminal and a near-terminal databases is kept.

Description

technical field [0001] The invention relates to the technical field of databases, in particular to a remote data incremental synchronization method based on data packets. Background technique [0002] Remote data synchronization refers to maintaining data consistency between two database systems across network segments. The source system is generally the business system database deployed at the front end, while the destination system is generally the OLAP data mart or data warehouse system used to support decision analysis. The connection between the front-end business system database and the back-end data center needs to penetrate the firewall. The existing remote data synchronization system has the following problems: [0003] 1. The means of remote data transmission is fixed and single, unable to cope with various complex situations. [0004] Remote data transmission can adopt various methods such as Web service, JMS message middleware and FTP. The method of web servi...

AI Technical Summary

Problems solved by technology

The method of web service can realize the real-time synchronous transmission of data, but only a small amount of data can be transmitted, which is suitable for scenarios where the data changes sporadically, and the quality of data delivery cannot be guaranteed; JMS message middleware is generally suitable for transmitting a small amount of data, but it can guarantee the data It can also achieve the goal in a bad network environment; FTP can transfer large-scale data volume, and can support breakpoint resume

These three methods have their own advantages and disadvantages. The existing remote data transmission methods can only adopt one of them, which cannot cope with various complex situations under different network environments and data synchronization requirements.

[0005] 2. The incremental data collector used in the remote data transmission front end has certain limitations

However, this method requires the dependence of a specific DBMS (Database management system, database management system), and requires the source system to support flashback query, so the scope of application is narrow

Images