Data synchronization method and device and storage medium

A data synchronization and data technology, applied in the direction of database update, relational database, database model, etc., can solve the problems of not meeting the project requirements and high threshold of use

Pending Publication Date: 2021-09-10
特赞(上海)信息科技有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

These solutions only describe the data synchronization from a single database table to elast

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data synchronization method and device and storage medium
  • Data synchronization method and device and storage medium
  • Data synchronization method and device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0081] Query incremental update data, as shown below is an implementation of the SQL language, including:

[0082]

[0083]

[0084] Synchronize incremental data to elastic search according to the configured mapping relationship rules. If the data corresponding to the id already exists in elastic search, perform an overwrite update operation. If the data corresponding to id does not exist in elastic search, perform a new operation. Overwrite the time recorded in step 4 with the current execution start time.

[0085] Through the above steps, the continuous update of the big data model can be realized.

[0086] The present invention also provides a data synchronization device for synchronizing the first type of data stored in the relational data model with the second type of data stored in the big data model, such as image 3 shown, including:

[0087] A first acquiring module, configured to acquire a unique first identifier of any one of the first type of data;

[0088...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data synchronization method and device and a storage medium, which are used for synchronizing a first type of data stored in a relational data model and a second type of data stored in a big data model. The method comprises the following steps: obtaining a unique first identifier of any data in the first type of data; obtaining a unique second identifier of any data in the second type of data; establishing a mapping relationship between the first identifier and the second identifier based on a preset conversion rule; and extracting the first data into the big data model based on the mapping relationship to form a big data model after data synchronization. The SQL is a description language of a relational database, and research and development personnel can easily write query SQL meeting business requirements. According to the invention, the mapping relationship between the SQL field and the e l ast i c search field can be configured through a visual interface, then the system automatically completes model conversion, data is synchronized to a big data model in cooperation with a full-amount and incremental strategy, an index is created and provided for research and development personnel to use, and the research and development cost is reduced.

Description

technical field [0001] The present invention relates to the technical field of data synchronization, in particular to a data synchronization method, device and storage medium. Background technique [0002] The era of big data has arrived. Every company generates a large amount of data every moment, and these data are usually stored in the database. Due to the rapid growth and large scale of data, it brings a lot of problems to the storage and use of data. difficulty. [0003] The software R&D personnel of many companies do not understand or only have some understanding of big data related technologies. If they directly use big data technology solutions to carry out R&D work, it will bring many uncertainties, resulting in system instability, project delays, and R&D costs. [0004] Although there are already some big data solutions in the industry, such as elastic search and so on. These solutions only describe the data synchronization from a single database table to elastic...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/27G06F16/23G06F16/28G06F16/242
CPCG06F16/27G06F16/23G06F16/284G06F16/2433
Inventor 范凌王喆
Owner 特赞(上海)信息科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap