Data synchronization method and apparatus, computer device, and readable medium

By employing a multi-layered data synchronization method, near real-time data synchronization from the OLTP system to the OLAP system is achieved, solving the problem that existing technologies cannot meet the needs of minute-level analysis. It supports full and incremental synchronization, reduces operation and maintenance costs, lowers synchronization latency, and adapts to changes in table structure.

CN110647579BActive Publication Date: 2026-01-23BEIJING BAIDU NETCOM SCI & TECH CO LTD
2 Cites 0 Cited by

Patent Information

Application Number
CN201910758908.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-08-16
Publication Date
2026-01-23
Estimated Expiration
2039-08-16

AI Technical Summary

Technical Problem

Existing data synchronization methods cannot effectively meet the data analysis needs of OLTP systems at the minute or hour level. In particular, in the data synchronization between OLTP systems and OLAP systems, near real-time synchronization cannot be achieved, and DDL statement synchronization cannot be supported, which makes the synchronization process prone to failure.

Method used

A multi-layered data synchronization approach is adopted, including structured object migration, full data migration, and incremental data synchronization. Through database table mapping, syntax parsing and transformation, independent progress management nodes are used for persistent information storage, supporting breakpoint resume and reducing the pressure on the target database through batch writing.

Benefits of technology

It achieves near real-time data synchronization from OLTP to OLAP systems, supports structured, full, and incremental synchronization, reduces operation and maintenance costs, reduces synchronization latency to the second level, meets minute-level analysis needs, and does not cause task failure when table structure changes.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application provides a data synchronization method and device, computer equipment and a readable medium, and relates to the field of big data. The method comprises the following steps: migrating a structure object of a source database of a first system to a target database of a second system; migrating full-amount data corresponding to a preset historical time point in the source database to the target database; and batch synchronizing real-time data corresponding to a time point after the preset historical time point in the source database to the target database. Through the above scheme, the application can realize data migration and data quasi-real-time synchronization from the first system, such as an OLTP system, to the second system, such as an OLAP system. The application can not only support structure synchronization, but also support full-amount synchronization and incremental synchronization, thereby facilitating users to quickly realize the synchronization function.
Need to check novelty before this filing date? Find Prior Art