Check patentability & draft patents in minutes with Patsnap Eureka AI!

Data synchronization method and system and storage medium

A data synchronization and database technology, applied in the field of big data, can solve problems such as unresponsive data synchronization, time-consuming, incremental synchronization limitations, etc., and achieve the effect of meeting the needs of data synchronization efficiency and real-time performance

Active Publication Date: 2021-08-03
PING AN TECH (SHENZHEN) CO LTD
View PDF8 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, this data synchronization method takes a lot of time for each data relocation, which leads to only one full synchronization per month, and the daily incremental synchronization is greatly limited.
Moreover, when the data synchronization requirements change, this data synchronization method cannot respond to the changed requirements to perform data synchronization in real time

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data synchronization method and system and storage medium
  • Data synchronization method and system and storage medium
  • Data synchronization method and system and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0057] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

[0058] see figure 1 , is a schematic diagram of a network architecture of a data synchronization system provided in an embodiment of the present application. figure 1 The data synchronization system shown includes an application service device 10 , a database server 20 , a database server 30 , and a database server 40 . The database server 20 corresponds to the first database server, the database server 30 corresponds to the second database server, and the database server 40 corresponds to the third database server. In an application scenario, the application service device 10 may be an application server. The database server 20 may be a ClickHouse database server. ClickHouse is an open source columnar storage-oriented database management system DBMS for online analytical processing (OLAP). The ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention provides a data synchronization method and system and a storage medium, and is applied to the technical field of big data, and the system comprises the steps that: application service equipment sends a data synchronization instruction to a first database server, so that the first database server updates a resource pool table according to a target table; the application service equipment sends a resource scheduling instruction to a first database server, so that the first database server divides a plurality of sub-user data sets according to the resource pool table and writes the sub-user data sets into a channel table; and the third database server obtains the target channel table through the application service equipment or the first database server for storage. By the adoption of the invention, the requirements for data synchronization efficiency and real-time performance can be met. The invention relates to a block chain technology, for example, the sending time of a target channel table is written into a block chain.

Description

technical field [0001] The present application relates to the technical field of big data, in particular to a data synchronization method, system and storage medium. Background technique [0002] Common data synchronization methods are data synchronization methods based on Oracle and MongoDB (a database based on distributed file storage). This data synchronization method imports data from the Hive database server to the Oracle database server through Sqoop (a data transfer tool), and then imports the data from the Oracle database server to the MongoDB database server via Kettle (a data migration tool). . After performing a series of operations in MongoDB, and finally through Kettle, the process of exporting data from MongoDB to Oracle. However, this data synchronization method takes a lot of time for each data relocation, which leads to only one full synchronization per month, and the daily incremental synchronization is greatly limited. Moreover, when the data synchroniz...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/27
CPCG06F16/275
Inventor 胡建双
Owner PING AN TECH (SHENZHEN) CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com