Data Packet-Based Remote Data Incremental Synchronization Method
A technology for incremental synchronization and remote data, applied in the database field, it can solve problems such as narrow application scope, inability to guarantee the quality of data delivery, and inability to deal with complex situations, and achieve the effect of ensuring stability
Active Publication Date: 2017-04-05
广东邮电职业技术学院
View PDF2 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
The method of web service can realize the real-time synchronous transmission of data, but only a small amount of data can be transmitted, which is suitable for scenarios where the data changes sporadically, and the quality of data delivery cannot be guaranteed; JMS message middleware is generally suitable for transmitting a small amount of data, but it can guarantee the data It can also achieve the goal in a bad network environment; FTP can transfer large-scale data volume, and can support breakpoint resume
These three methods have their own advantages and disadvantages. The existing remote data transmission methods can only adopt one of them, which cannot cope with various complex situations under different network environments and data synchronization requirements.
[0005] 2. The incremental data collector used in the remote data transmission front end has certain limitations
However, this method requires the dependence of a specific DBMS (Database management system, database management system), and requires the source system to support flashback query, so the scope of application is narrow
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0028] The technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments, so as to understand the essence of the present invention more clearly and intuitively.
[0029] figure 1 It is a flowchart of an incremental data acquisition method based on log capture in an embodiment of the present invention; figure 2 yes figure 1 The flow chart of obtaining the DML of the source database in the illustrated embodiment.
[0030] refer to figure 1 and figure 2 As shown, the embodiment of the present invention first provides a remote data incremental synchronization method based on data packets, including the following steps:
[0031] S1: Get the DML of the source database;
[0032] S2: export the DML as a data package;
[0033] S3: Remotely synchronize the data package to the target database;
[0034] S4: parsing the data packet into an SQL statement and changing data;
[0035] S5: Updating ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to a remote data increment synchronization method and a device based on a data package. The method comprises a first step of obtaining data manipulation languages (DML) of a database, a second step of leading out the DML into the data package, a third step of sending the data package to a target database in a remote synchronization mode, a fourth step of analyzing the data package into structural query languages (SQL) and changing data, and a fifth step of updating the SQL and the changing data to the target database. According to the remote data increment synchronization method and the device based on the data package, increment records, deleted records and changed records in the database can be extracted in fixed time to form the data package including the increment records. According to network situations and data synchronization demands, the data package is transmitted to a target database system in a java message service (JMS) or a file transfer protocol (FTP) manner by utilizing message-oriented middleware or an FTP client side / server terminal product and being supported by a certain allocation management, and therefore consistency of data of a remote terminal and a near-terminal databases is kept.
Description
technical field [0001] The invention relates to the technical field of databases, in particular to a remote data incremental synchronization method based on data packets. Background technique [0002] Remote data synchronization refers to maintaining data consistency between two database systems across network segments. The source system is generally the business system database deployed at the front end, while the destination system is generally the OLAP data mart or data warehouse system used to support decision analysis. The connection between the front-end business system database and the back-end data center needs to penetrate the firewall. The existing remote data synchronization system has the following problems: [0003] 1. The means of remote data transmission is fixed and single, unable to cope with various complex situations. [0004] Remote data transmission can adopt various methods such as Web service, JMS message middleware and FTP. The method of web servi...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08G06F17/30
Inventor 戴浩
Owner 广东邮电职业技术学院