Data synchronization method and device and computer readable storage medium
A technology of data synchronization and data storage space, applied in the field of big data processing, can solve the problems of missing data synchronization, affecting the efficiency of program execution, unavailability, etc., to reduce coupling, achieve accuracy and timeliness, and improve timeliness sexual effect
Pending Publication Date: 2020-12-01
CHINA PING AN PROPERTY INSURANCE CO LTD
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0003] The above three methods further include two methods of real-time synchronization and asynchronous data synchronization. Among them, real-time synchronization puts greater pressure on the program and affects the efficiency of program execution. In the case of strong business coupling, there is a risk of failure to write data. Issues such as missing or unavailable data that result in synchronization
In addition, the use of MQ asynchronous double-write method to achieve data synchronization will also face the problem of high degree of business coupling
[0004] At present, in terms of data synchronization, the method of synchronizing data from database tables to ElasticSearch or memory through the timing task JOB, in order to reduce the pressure on the database, data synchronization can only be performed during non-peak business hours, and there are still some problems. The problem of business coupling
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0040] It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
[0041] The invention provides a data synchronization method. refer to figure 1 As shown, it is a schematic flowchart of a data synchronization method provided by an embodiment of the present invention. The method may be performed by a device, and the device may be implemented by software and / or hardware.
[0042] In one embodiment of the present invention, the data synchronization method includes:
[0043] S110: Intercept SQL operations in the data to be synchronized stored in the database.
[0044] Specifically, the SQL interceptor can be used to intercept the SQL operation in the data to be synchronized, and obtain the version value of the data to be synchronized in the database; the Version value, the parameter object and the mapping relationship are encapsulated into the encapsulated data, wherein the version v...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to big data, and discloses a data synchronization method, which comprises the following steps of intercepting SQL (Structured Query Language) operation in to-be-synchronized datastored in a database; analyzing the intercepted SQL operation, and obtaining and packaging a parameter object and a mapping relationship corresponding to the SQL operation so as to obtain packaged data corresponding to the SQL operation; sending the packaged data to a message platform, monitoring the packaged data through the message platform, and sending the monitored message to a data storage space of a subscription message; and receiving and processing the encapsulation data corresponding to the monitored message through the data storage space, and synchronizing the to-be-synchronized datato the data storage space according to the encapsulation data. The invention also relates to a blockchain technology, and the to-be-synchronized data is stored in the blockchain. According to the invention, the data synchronization efficiency and precision can be improved.
Description
technical field [0001] The present invention relates to big data processing, in particular to a data synchronization method, device, electronic equipment and computer-readable storage medium. Background technique [0002] At present, it mainly involves two aspects of synchronization. One is to realize data synchronization through background program coding, and the other is to directly act on the database to realize data synchronization at the database level. Among them, data synchronization is realized through background program coding. The main idea is to update if there is any, and add new if there is no. Other situations are recorded through logs. Data synchronization at the database level is further divided into three ways: 1. Realize synchronization through publish / subscribe, 2. Realize data synchronization through SQL JOB, 3. Realize data synchronization through Service Broker message queuing. [0003] The above three methods further include two methods of real-time s...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G06F16/27
CPCG06F16/27
Inventor 庄志辉
Owner CHINA PING AN PROPERTY INSURANCE CO LTD