Batch data synchronization method and system and electronic equipment

A batch data, batch technology, applied in the field of computer information processing, can solve problems such as poor timeliness of batch data, and achieve the effects of increasing network burden, solving poor timeliness, and quickly and accurately synchronizing

Pending Publication Date: 2021-06-11
北京淇瑀信息科技有限公司
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of this, the present disclosure provides a batch data synchronization method, system, electronic device, and computer-readable medium, which can solve the problem of poor timeliness in batch data synchronization in the prior art, and quickly and accurately perform batch data synchronization. Will not increase network burden

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Batch data synchronization method and system and electronic equipment
  • Batch data synchronization method and system and electronic equipment
  • Batch data synchronization method and system and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040]Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus their repeated descriptions will be omitted.

[0041] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or mor...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a batch data synchronization method and system, electronic equipment and a computer readable medium. The method comprises the steps that a big data distributed stream data stream engine stores to-be-synchronized data to a first preset position of a distributed file system; a heterogeneous data source offline synchronization framework stores historical data of the to-be-synchronized data to a second preset position of the distributed file system; a data warehouse tool stores the to-be-synchronized data in a partitioned manner according to a preset rule; and batch merging is performed on the to-be-synchronized data stored in the partitions within a preset time to realize batch synchronization of the to-be-synchronized data. According to the batch data synchronization method and system, the electronic equipment and the computer readable medium, the problem of poor timeliness of batch data synchronization in the prior art can be solved, batch data synchronization can be quickly and accurately carried out, and the network burden cannot be increased.

Description

technical field [0001] The present disclosure relates to the field of computer information processing, in particular, to a batch data synchronization method, system, electronic equipment and computer readable medium. Background technique [0002] Database operations often become the bottleneck of a system, and the pressure of "reading" in general systems is far greater than that of "writing". The performance of the system can be improved by realizing the separation of reading and writing of the database. By setting the master-slave database to achieve read-write separation, the master database is responsible for "write operations", and the slave database is responsible for "read operations". According to the pressure situation, multiple slave databases can be deployed to increase the speed of "reading", thereby improving the overall system performance. To achieve read-write separation, it is necessary to solve the problem of master-slave database data synchronization. After...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/27G06F16/23G06F16/242G06F16/2455G06F16/17G06F9/445
CPCG06F16/27G06F16/23G06F16/2433G06F16/2455G06F16/1734G06F9/44526
Inventor 闫宇新袁孝锋
Owner 北京淇瑀信息科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap