Method and device for synchronizing service data

A technology of business data and data, applied in database indexing, structured data retrieval, electronic digital data processing, etc., can solve problems such as low latency, and achieve the effects of low latency, no development workload, and great flexibility
CN112416944APending Publication Date: 2021-02-26BEIJING UNISOUND INFORMATION TECH +1
0 Cites 3 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Current Assignee / Owner
BEIJING UNISOUND INFORMATION TECH
Publication Date
2021-02-26

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention relates to a method and device for synchronizing service data. The method comprises the steps of: synchronizing Mysql data serving as the service data to a Kafka cluster through a Maxwell assembly; synchronizing the data in the Kafka cluster to the HBase; and mapping the table stored in the HBase to a Hive library to finish the synchronization of the service data. According to the scheme of the invention, a Maxwell synchronization mode is adopted, no influence is caused to the running service, the delay is lower, more concurrency can be supported, and the flexibility is better.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of data synchronization, in particular to a method and equipment for synchronizing service data. Background technique

[0002] At present, big data is widely used to provide dimension table data support for big data in building various models and real-time analysis scenarios. Dimension table data is various business data stored in a relational database synchronously in real time. When big data is modeled using models such as stars and constellations, the dimension table data will use the business data stored in the relational mysql database table by the business department, and the business data has a change characteristic, so there is a problem in the existing technology The following synchronization methods:

[0003] Master-slave synchronization supported by relational mysql database;

[0004] Full or incremental synchronization using the API interface;

[0005] However, the following problems exist in ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More