A data synchronization method and device

A data synchronization and data technology, applied in the field of data processing, can solve problems such as adverse effects of service nodes, business processing errors, real-time data cannot be pushed in time, and achieve the effect of avoiding data processing problems

Active Publication Date: 2022-04-26
BEIJING HUAWEI DIGITAL TECH
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] This data synchronization method leads to the fact that the real-time data cannot be pushed to the newly launched service node in time. For the aforementioned two kinds of real-time data, if the first type of data cannot be pushed in time, the service node will make mistakes in the existing business processing, because the historical data is usually it's wrong
The failure of the second type of data to be pushed in time will result in the inability of business data to be processed by the service node in time
No matter which of the above situations is caused, it will have an adverse impact on the newly launched service nodes

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data synchronization method and device
  • A data synchronization method and device
  • A data synchronization method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0066] Next, the data synchronization method provided by the embodiment of the present application will be described in detail with reference to the accompanying drawings.

[0067] see figure 2 , figure 2 A flow chart of a data synchronization method is shown, the method includes:

[0068] S201. Obtain a synchronization request of a service node.

[0069] According to its own data processing requirements, the service node needs to obtain the latest data required to realize its functions in the system. Therefore, the service node can send a synchronization request to the MQ BROKER 102 to obtain the required synchronization data from the MQ BROKER 102. At this point, it can be considered that MQ BROKER 102 enters the data recovery state.

[0070] S202. Obtain a snapshot copy generated according to the data set to be synchronized from the key-value database by using the identifier.

[0071] The MQ BROKER 102 needs to specify the set of data to be synchronized that should be...

Embodiment 2

[0100] Based on the data synchronization method provided in the first embodiment, this embodiment provides a data synchronization device. Figure 4 A structural diagram of a data synchronization device provided in this embodiment, the data synchronization device 400 includes a first acquisition unit 401, a second acquisition unit 402, a modification unit 403 and a synchronization unit 404:

[0101] The first acquiring unit 401 is configured to acquire a synchronization request of a service node, where the synchronization request includes an identifier of a data set to be synchronized;

[0102] The second obtaining unit 402 is configured to obtain a snapshot copy generated according to the data set to be synchronized from the key-value database through the identification, and the key information corresponding to the data in the snapshot copy is set with a version number, wherein, The version number of a key information is used to identify the update information of the data corr...

Embodiment 3

[0116] refer to Figure 5 , Figure 5 It is a schematic diagram of the hardware structure of a device for data synchronization provided in the embodiment of the present application. The device 500 includes a memory 501, a receiver 502, and a transmitter 503, and is connected to the memory 501, the receiver 502, and the transmitter 503, respectively. The processor 504 connected to the transmitter 503, the memory 501 is used to store a set of program instructions, and the processor 504 is used to call the program instructions stored in the memory 501 to perform the following operations:

[0117] Triggering the receiver 502 to obtain a synchronization request from the service node, where the synchronization request includes an identifier of the data set to be synchronized;

[0118] The snapshot copy generated according to the data set to be synchronized is obtained from the key-value database through the identification, and the key information corresponding to the data in the sn...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the present application discloses a data synchronization method and a related device. When acquiring a synchronization request from a service node, a snapshot copy of the data set to be synchronized corresponding to the identification is obtained from a key-value database based on the identification in the synchronization request. In the process of data synchronization to the service node based on the snapshot copy, if the updated data for the data set to be synchronized is received, the version number of the target key information corresponding to the updated data is changed incrementally, and then the updated data is used as The data corresponding to the target key information is synchronized to the service node, and the read version number of the target key information is updated to the version number after the change of the target key information, which prevents the newer data corresponding to the key information from being overwritten by the older data , so that the updated data is directly synchronized to the service node during the data synchronization process, without waiting for the completion of the historical data synchronization, which realizes the timely push of the updated data to the service node and avoids data processing problems in the service node.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a data synchronization method and device. Background technique [0002] In a distributed system, when a service node goes online, in order to ensure that the new service goes online without affecting the normal operation of existing services, the distributed system will synchronize the corresponding subscription relationship for the service node according to the subscription relationship of the service node to the data. A synchronized data set generally includes two parts, one part is the historical data of the data set before data synchronization, and the other is the real-time data updated to the data set during the data synchronization process. [0003] In a possible implementation, when the service node goes online, the distributed system uses a message queue (Message Queue, MQ) and a key-value (Key-Value, KV) database to synchronize data of the service node. Among t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): H04L67/1095H04L67/60
CPCH04L67/1095H04L67/60
Inventor 姜戎徐海军张旭东熊怡郭锋
Owner BEIJING HUAWEI DIGITAL TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap