Data synchronization method and device and data synchronization platform

Abstract

The invention provides a data synchronization method and device and a data synchronization platform. The method comprises the steps of obtaining target to-be-synchronized data of a source system; under the condition that multiple pieces of data of the same target primary key in the target to-be-synchronized data exist, accumulating the multiple pieces of data of the same target primary key in the target to-be-synchronized data on the data of the primary key which is the same as the target primary key in the target system in an additional mode to obtain updated data of the primary key in the target system, data synchronization of the target system and the source system is achieved, and the problem that synchronization of multiple pieces of data of the same primary key fails in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of data synchronization technology, and in particular to a data synchronization method, device, computer-readable storage medium, computer program product and data synchronization platform. Background Art

[0002] File-based data synchronization is a process in which the source system organizes the data to be synchronized into files and sends them to the target system. The target system parses the file content and stores the data in the file in some way.

[0003] Traditional file-based data synchronization completely overwrites the original data of the target system, and does not consider how to handle scenarios where there are multiple records with the same primary key data in the same batch but some synchronization fails.

[0004] The existing patents based on file data synchronization methods have the following main deficiencies:

[0005] 1) Often only one synchronization mode is provided, that is, a full coverage mode, which only considers the case where the source system has only one record for the same primary key data, and does not consider how to achieve data synchronization in the scenario where the source system has multiple records and cannot provide merge calculation;

[0006] 2) Regarding the exception handling method for synchronization failure, it simply proposes to resend all or resend after failure, without detailing how to handle the scenario where there are multiple records with the same primary key in the same file (these data will be merged in the target system) but partial synchronization fails. Summary of the invention

[0007] The main purpose of the present application is to provide a data synchronization method, device, computer-readable storage medium, computer program product and data synchronization platform to at least solve the problem of failure in synchronizing multiple data with the same primary key in the prior art.

[0008] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a data synchronization method is provided, including: obtaining target data to be synchronized from a source system; when there are multiple data with the same target primary key in the target data to be synchronized, using an append mode to accumulate the multiple data with the same target primary key in the target data to be synchronized on the data with the same primary key as the target primary key in the target system, to obtain updated data of the primary key in the target system, thereby realizing data synchronization between the target system and the source system.

[0009] Optionally, after achieving data synchronization between the target system and the source system, the method further includes: setting the configuration value of all data of the target system to the current Apollo configuration value of the target system, where the initial value of the Apollo configuration value is 0; when the target data to be synchronized in the source system is corrected, obtaining the modified target data to be synchronized in the source system, and controlling the Apollo configuration value of the target system to be increased by 1; when the Apollo configuration value of the target system is greater than the configuration value of the current data of the target system, merging and calculating multiple data with the same target primary key in the modified target data to be synchronized to obtain one data of the target primary key, and updating the data with the same primary key as the target primary key in the target system to one data of the target primary key, and the current data is the data with the same primary key as the target primary key in the target system.

[0010] Optionally, after achieving data synchronization between the target system and the source system, the method further includes: setting the configuration value of all data of the target system to the current Apollo configuration value of the target system, where the initial value of the Apollo configuration value is 0; obtaining new target data to be synchronized from the source system, and controlling the Apollo configuration value of the target system to remain unchanged; when the Apollo configuration value of the target system is equal to the configuration value of the current data of the target system and there are multiple data with the same target primary key in the new target data to be synchronized, the append mode is used to accumulate multiple data with the same target primary key in the new target data to be synchronized on the data with the same primary key as the target primary key in the target system, to obtain updated data of the primary key in the target system, thereby achieving data synchronization between the target system and the source system.

[0011] Optionally, after obtaining the target data to be synchronized from the source system, the method further includes: when there is only one piece of data with the same primary key in the target data to be synchronized, using an overwrite mode to update the data with the same primary key as the target primary key in the target data to be synchronized, thereby obtaining the updated data of the primary key in the target system and realizing data synchronization between the target system and the source system.

[0012] Optionally, after obtaining the target data to be synchronized from the source system, the method further includes: when there are multiple data with the same target primary key in the target data to be synchronized and the source system supports merge calculation, controlling the source system to merge the multiple data with the same target primary key in the target data to be synchronized to obtain one piece of data with the target primary key, and using an overwrite mode to update the data with the same primary key as the target primary key in the target system to one piece of data with the primary key in the target data to obtain updated data of the primary key in the target system, thereby realizing data synchronization between the target system and the source system.

[0013] Optionally, after achieving data synchronization between the target system and the source system, the method further includes: in the event that data synchronization between the target system and the source system fails, reacquiring the target data to be synchronized of the source system and performing data synchronization between the target system and the source system.

[0014] According to another aspect of the present application, a data synchronization device is provided, including: a first acquisition unit, used to acquire target data to be synchronized from a source system; a first synchronization unit, used to, when there are multiple pieces of data with the same target primary key in the target data to be synchronized, accumulate the multiple pieces of data with the same target primary key in the target data to be synchronized on the data with the same primary key as the target primary key in the target system in an append mode, to obtain updated data of the primary key in the target system, thereby realizing data synchronization between the target system and the source system.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described.

[0016] According to another aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, any one of the methods described above is implemented.

[0017] According to another aspect of the present application, a data synchronization platform is provided, comprising: a source system, a target system, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the described methods.

[0018] By applying the technical solution of the present application, in the above-mentioned data synchronization method, when there are multiple data with the same target primary key in the target data to be synchronized, in this case, the overwrite mode of the prior art is not used to directly update the data with the same primary key as the target primary key in the target system, and it is necessary to merge and calculate the multiple data with the same target primary key. Instead, the append mode is used to accumulate the multiple data with the same target primary key in the target data to be synchronized with the data with the same primary key as the target primary key in the target system, so that data synchronization of multiple data with the same target primary key can be achieved, thereby solving the problem of failure in synchronization of multiple data with the same primary key in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A hardware structure block diagram of a mobile terminal for executing a data synchronization method provided in an embodiment of the present application is shown;

[0020] Figure 2 A schematic diagram of a data synchronization method according to an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram of a process flow of another data synchronization method provided according to an embodiment of the present application is shown;

[0022] Figure 4 A structural block diagram of a data synchronization device provided according to an embodiment of the present application is shown.

[0023] The above drawings include the following reference numerals:

[0024] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0025] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0029] Data synchronization: refers to the operation of maintaining the consistency of a set of data with the same meaning between the source system and the target system.

[0030] Source system: The system that provides data files.

[0031] Target system: The system that receives the data files.

[0032] As introduced in the background technology, synchronization of multiple data with the same primary key fails in the prior art. To solve this technical problem, the embodiments of the present application provide a data synchronization method, device, computer-readable storage medium, computer program product and data synchronization platform.

[0033] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0034] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a data synchronization method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0035] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data synchronization method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] In this embodiment, a data synchronization method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Figure 2 is a flow chart of a data synchronization method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0038] Step S201, obtaining target data to be synchronized from the source system;

[0039] Step S202, when there are multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized, an append mode is adopted to accumulate the multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized on the data with the same primary key as the above-mentioned target primary key in the target system, to obtain the updated data of the above-mentioned primary key in the above-mentioned target system, and realize data synchronization between the above-mentioned target system and the above-mentioned source system.

[0040] In the above data synchronization method, when there are multiple data with the same target primary key in the target data to be synchronized, in this case, the overwrite mode of the prior art is not used to directly update the data with the same primary key as the target primary key in the target system. It is necessary to merge and calculate the multiple data with the same target primary key. The append mode is used to accumulate the multiple data with the same target primary key in the target data to be synchronized on the data with the same primary key as the target primary key in the target system. This can achieve data synchronization of multiple data with the same target primary key, and solves the problem of failure in synchronization of multiple data with the same primary key in the prior art.

[0041] In order to achieve full retransmission, in an optional implementation manner, after achieving data synchronization between the target system and the source system, the method further includes:

[0042] Step S301, setting the configuration value of all data of the target system to the current Apollo configuration value of the target system, where the initial value of the Apollo configuration value is 0;

[0043] Step S302, when the target data to be synchronized of the source system is modified, obtaining the modified target data to be synchronized of the source system, and controlling the Apollo configuration value of the target system to be increased by 1;

[0044] Step S303, when the Apollo configuration value of the target system is greater than the configuration value of the current data of the target system, multiple data with the same target primary key in the modified target data to be synchronized are merged and calculated to obtain one data of the target primary key, and the data with the same primary key as the target primary key in the target system is updated to one data of the target primary key, and the current data is the data with the same primary key as the target primary key in the target system.

[0045] In the above implementation, the target system sets an Apollo configuration value with an initial value of 0. Thereafter, each full synchronization of the same file, that is, synchronization again after the target data to be synchronized is modified, will increase the Apollo configuration value of the target system by 1. The Apollo configuration value of the target system needs to be checked. If the Apollo configuration value of the target system is greater than the configuration value of the current data in the target system, it means that all data of the primary key in the file needs to be merged and recalculated, and the configuration value recorded in each piece of data needs to be updated to the Apollo configuration value.

[0046] In order to achieve normal synchronization, in an optional implementation manner, after achieving data synchronization between the target system and the source system, the method further includes:

[0047] Step S401, setting the configuration value of all data of the target system to the current Apollo configuration value of the target system, where the initial value of the Apollo configuration value is 0;

[0048] Step S402, obtaining new target data to be synchronized from the source system, and controlling the Apollo configuration value of the target system to remain unchanged;

[0049] Step S403, when the Apollo configuration value of the target system is equal to the configuration value of the current data of the target system and there are multiple data with the same target primary key in the new target data to be synchronized, the append mode is adopted to accumulate the multiple data with the same target primary key in the new target data to be synchronized on the data with the same primary key as the target primary key in the target system, to obtain the updated data of the primary key in the target system, and realize data synchronization between the target system and the source system.

[0050] In the above implementation, the new target data to be synchronized of the above source system is obtained, instead of the previous target data to be synchronized being fully resent, and there is no need to add 1 to the Apollo configuration value, that is, it remains unchanged. If the Apollo configuration value of the target system is equal to the configuration value of the current data in the target system, the data value in the file is directly accumulated on the data in the target system.

[0051] In order to achieve synchronization, in an optional implementation manner, after obtaining the target data to be synchronized from the source system, the method further includes:

[0052] Step S501, when there is only one piece of data with the same primary key in the target data to be synchronized, the data with the same primary key as the target primary key in the target system is updated to the data with the primary key in the target data to be synchronized in an overwrite mode, and the updated data of the primary key in the target system is obtained, thereby realizing data synchronization between the target system and the source system.

[0053] In the above implementation, there is only one piece of data with the same primary key in the above target data to be synchronized, and there is no need to perform merge calculation. The overwrite mode can be used to directly update the data with the same primary key as the above target primary key in the above target system.

[0054] In order to achieve synchronization, in an optional implementation manner, after obtaining the target data to be synchronized from the source system, the method further includes:

[0055] Step S601, when there are multiple data with the same above-mentioned primary key in the above-mentioned target data to be synchronized and the above-mentioned source system supports merge calculation, control the above-mentioned source system to merge the multiple data with the same above-mentioned target primary key in the above-mentioned target data to be synchronized, and obtain one piece of data of the above-mentioned target primary key, and use the overwrite mode to update the data of the primary key same as the above-mentioned target primary key in the above-mentioned target data to the one piece of data of the above-mentioned primary key in the above-mentioned target data to be synchronized, obtain the updated data of the above-mentioned primary key in the above-mentioned target system, and realize data synchronization between the above-mentioned target system and the above-mentioned source system.

[0056] In the above implementation, there are multiple data with the same primary key in the above target data to be synchronized, which need to be merged and calculated. The source system supports merge calculation, and merge calculation can be performed on the source system to obtain one piece of data with the above target primary key. The overwrite mode can be used to directly update the data with the same primary key as the above target primary key in the above target system.

[0057] In order to achieve retransmission upon failure, in an optional implementation manner, after achieving data synchronization between the target system and the source system, the method further includes:

[0058] Step S701: when data synchronization between the target system and the source system fails, the target data to be synchronized of the source system is reacquired to synchronize the data between the target system and the source system.

[0059] In the above implementation, some data synchronization may fail due to system reasons during the synchronization process. Therefore, during the synchronization process, the target system needs to record the synchronization status of each data and return it to the source system. The failure of the data synchronization between the target system and the source system indicates that the synchronization is not successful. It is necessary to re-acquire the target data to be synchronized of the source system and perform data synchronization between the target system and the source system until the synchronization is successful.

[0060] It should be noted that if the target system does not have data with the same primary key as the target primary key, it can be created directly during synchronization.

[0061] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the data synchronization method of the present application will be described in detail below in combination with specific embodiments.

[0062] This embodiment relates to a specific data synchronization method, such as Figure 3 As shown, the following steps are included:

[0063] To implement the primary key of the agreed data, the source system may have multiple records with the same primary key, while the target system can only retain one record for the data with the same primary key. Therefore, during the data synchronization process, the target system needs to calculate the situation where there are multiple records and finally merge them into one record.

[0064] (II) The overall processing flow consists of the following three steps:

[0065] 1) Set the Apollo configuration value on the target system: the initial value is 0, and the configuration value is increased by 1 each time the same file is fully synchronized.

[0066] 2) The source system extracts the data production files that need to be synchronized (it is necessary to ensure that the data with the same primary key in this system is in one synchronization file). There are two possible data situations in the file:

[0067] i. The source system is capable of merging data with the same primary key into one piece of data or there is only one piece of data with the same primary key, and this piece of data is marked as overwrite mode;

[0068] ii. The source system cannot merge data with the same primary key, and all data under the primary key is marked as append mode;

[0069] 3) The target system receives the synchronization file for parsing:

[0070] i. When the data is in overwrite mode, query the data in the system based on the primary key. If it does not exist, create it based on the data in the file. If it does exist, update it to the data in the file.

[0071] ii. When the data is in append mode, query the data in this system based on the primary key:

[0072] If it exists, you need to check the configuration value of Apollo. If the configuration value is greater than the configuration value of the current data in the target system, it means that all the data of the primary key in the file needs to be merged and recalculated, and the configuration value recorded in each data is updated to the configuration value of Apollo; if the configuration value is equal to the configuration value of the current data in the target system, the data value in the file is directly accumulated on the data in the target system.

[0073] If it does not exist, create it according to the data in the file;

[0074] During the synchronization process, some data may fail to synchronize due to system reasons. Therefore, during the synchronization process, the target system needs to record the synchronization status of each data and return it to the source system. After the source system receives the synchronization feedback, it can resend in two ways: full resend and failure resend. If the source system chooses full resend, it needs to notify the target system before resending. The target system needs to set the Apollo configuration value plus 1 before performing subsequent synchronization operations.

[0075] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0076] The embodiment of the present application also provides a data synchronization device. It should be noted that the data synchronization device of the embodiment of the present application can be used to execute the data synchronization method provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0077] The following is an introduction to the data synchronization device provided in the embodiment of the present application.

[0078] Figure 4 is a structural block diagram of a data synchronization device according to an embodiment of the present application. Figure 4 As shown, the device comprises:

[0079] A first acquisition unit 10 is used to acquire target data to be synchronized from a source system;

[0080] The first synchronization unit 20 is used for accumulating the multiple data with the same target primary key in the target data to be synchronized to the data with the same primary key as the above target primary key in the target system in an append mode when there are multiple data with the same target primary key in the above target data to be synchronized, so as to obtain the updated data of the above primary key in the above target system and realize data synchronization between the above target system and the above source system.

[0081] In the above-mentioned data synchronization device, when there are multiple data with the same target primary key in the target data to be synchronized, in this case, the overwrite mode of the prior art is not used to directly update the data with the same primary key as the target primary key in the target system. It is necessary to merge and calculate the multiple data with the same target primary key. The append mode is used to accumulate the multiple data with the same target primary key in the target data to be synchronized to the data with the same primary key as the target primary key in the target system. This can achieve data synchronization of multiple data with the same target primary key, and solves the problem of failure in synchronization of multiple data with the same primary key in the prior art.

[0082] In order to achieve full retransmission, in an optional implementation manner, the above-mentioned device further includes:

[0083] A first setting unit is used to set the configuration value of all data of the target system to the current Apollo configuration value of the target system after realizing data synchronization between the target system and the source system, and the initial value of the Apollo configuration value is 0;

[0084] A second acquisition unit is configured to acquire the modified target data to be synchronized of the source system when the target data to be synchronized of the source system is modified, and control the Apollo configuration value of the target system to be increased by 1;

[0085] The second synchronization unit is used to merge and calculate multiple data with the same target primary key in the modified target data to be synchronized when the Apollo configuration value of the target system is greater than the configuration value of the current data of the target system, to obtain one piece of data of the target primary key, and to update the data with the same primary key as the target primary key in the target system to one piece of data of the target primary key, and the current data is the data with the same primary key as the target primary key in the target system.

[0086] In the above implementation, the target system sets an Apollo configuration value with an initial value of 0. Thereafter, each full synchronization of the same file, that is, synchronization again after the target data to be synchronized is modified, will increase the Apollo configuration value of the target system by 1. The Apollo configuration value of the target system needs to be checked. If the Apollo configuration value of the target system is greater than the configuration value of the current data in the target system, it means that all data of the primary key in the file needs to be merged and recalculated, and the configuration value recorded in each piece of data needs to be updated to the Apollo configuration value.

[0087] In order to achieve normal synchronization, in an optional implementation manner, the above device further includes:

[0088] A second setting unit is used to set the configuration value of all data of the target system to the current Apollo configuration value of the target system after realizing data synchronization between the target system and the source system, and the initial value of the Apollo configuration value is 0;

[0089] A third acquisition unit is used to acquire new target data to be synchronized from the source system and control the Apollo configuration value of the target system to remain unchanged;

[0090] The third synchronization unit is used to accumulate the multiple pieces of data with the same target primary key in the new target data to be synchronized on the data with the same primary key as the target primary key in the target system by adopting the append mode, when the Apollo configuration value of the target system is equal to the configuration value of the current data of the target system and there are multiple pieces of data with the same target primary key in the new target data to be synchronized, so as to obtain the updated data of the primary key in the target system and realize data synchronization between the target system and the source system.

[0091] In the above implementation, the new target data to be synchronized of the above source system is obtained, instead of the previous target data to be synchronized being fully resent, and there is no need to add 1 to the Apollo configuration value, that is, it remains unchanged. If the Apollo configuration value of the target system is equal to the configuration value of the current data in the target system, the data value in the file is directly accumulated on the data in the target system.

[0092] In order to achieve synchronization, in an optional implementation manner, the above device further includes:

[0093] The fourth synchronization unit is used to, after acquiring the target data to be synchronized from the source system, update the data of the primary key same as the target primary key in the target data to the data of the primary key in the target data to be synchronized in an overwrite mode, thereby obtaining the updated data of the primary key in the target system and realizing data synchronization between the target system and the source system.

[0094] In the above implementation, there is only one piece of data with the same primary key in the above target data to be synchronized, and there is no need to perform merge calculation. The overwrite mode can be used to directly update the data with the same primary key as the above target primary key in the above target system.

[0095] In order to achieve synchronization, in an optional implementation manner, the above device further includes:

[0096] The fifth synchronization unit is used for, after acquiring the target data to be synchronized from the source system, if there are multiple data with the same target primary key in the target data to be synchronized and the source system supports merge calculation, controlling the source system to merge the multiple data with the same target primary key in the target data to be synchronized, to obtain one piece of data with the target primary key, and using an overwrite mode to update the data with the same primary key as the target primary key in the target data to one piece of data with the primary key in the target data to obtain the updated data of the primary key in the target system, thereby realizing data synchronization between the target system and the source system.

[0097] In the above implementation, there are multiple data with the same primary key in the above target data to be synchronized, which need to be merged and calculated. The source system supports merge calculation, and merge calculation can be performed on the source system to obtain one piece of data with the above target primary key. The overwrite mode can be used to directly update the data with the same primary key as the above target primary key in the above target system.

[0098] The data synchronization device includes a processor and a memory. The first acquisition unit and the first synchronization unit are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0099] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem of failure to synchronize multiple data with the same primary key in the prior art can be solved by adjusting kernel parameters.

[0100] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0101] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the data synchronization method.

[0102] Specifically, the data synchronization method includes:

[0103] Step S201, obtaining target data to be synchronized from the source system;

[0104] Step S202, when there are multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized, an append mode is adopted to accumulate the multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized on the data with the same primary key as the above-mentioned target primary key in the target system, to obtain the updated data of the above-mentioned primary key in the above-mentioned target system, and realize data synchronization between the above-mentioned target system and the above-mentioned source system.

[0105] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the data synchronization method is executed when the program is running.

[0106] Specifically, the data synchronization method includes:

[0107] Step S201, obtaining target data to be synchronized from the source system;

[0108] Step S202, when there are multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized, an append mode is adopted to accumulate the multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized on the data with the same primary key as the above-mentioned target primary key in the target system, to obtain the updated data of the above-mentioned primary key in the above-mentioned target system, and realize data synchronization between the above-mentioned target system and the above-mentioned source system.

[0109] An embodiment of the present invention provides a data synchronization platform, which includes a source system, a target system, a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are implemented:

[0110] Step S201, obtaining target data to be synchronized from the source system;

[0111] Step S202, when there are multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized, an append mode is adopted to accumulate the multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized on the data with the same primary key as the above-mentioned target primary key in the target system, to obtain the updated data of the above-mentioned primary key in the above-mentioned target system, and realize data synchronization between the above-mentioned target system and the above-mentioned source system.

[0112] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0113] Step S201, obtaining target data to be synchronized from the source system;

[0114] Step S202, when there are multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized, an append mode is adopted to accumulate the multiple pieces of data with the same target primary key in the above-mentioned target data to be synchronized on the data with the same primary key as the above-mentioned target primary key in the target system, to obtain the updated data of the above-mentioned primary key in the above-mentioned target system, and realize data synchronization between the above-mentioned target system and the above-mentioned source system.

[0115] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0120] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0121] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0122] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0123] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0124] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0125] 1) In the data synchronization device of the present application, when there are multiple data with the same target primary key in the target data to be synchronized, in this case, the overwrite mode of the prior art is not used to directly update the data with the same primary key as the target primary key in the target system. It is necessary to merge and calculate the multiple data with the same target primary key. The append mode is used to accumulate the multiple data with the same target primary key in the target data to be synchronized on the data with the same primary key as the target primary key in the target system. This can achieve data synchronization of multiple data with the same target primary key, and solves the problem of failure in synchronization of multiple data with the same primary key in the prior art.

[0126] 2) In the data synchronization device of the present application, when there are multiple data with the same target primary key in the target data to be synchronized, in this case, the overwrite mode of the prior art is not used to directly update the data with the same primary key as the target primary key in the target system. It is necessary to merge and calculate the multiple data with the same target primary key. The append mode is used to accumulate the multiple data with the same target primary key in the target data to be synchronized with the data with the same primary key as the target primary key in the target system. This can achieve data synchronization of multiple data with the same target primary key, and solves the problem of failure in synchronization of multiple data with the same primary key in the prior art.

[0127] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data synchronization method, characterized in that: include: Obtain the target data to be synchronized from the source system; When there are multiple pieces of data with the same target primary key in the target data to be synchronized, an append mode is adopted to accumulate the multiple pieces of data with the same target primary key in the target data to be synchronized to the data with the same primary key as the target primary key in the target system to obtain updated data of the primary key in the target system, thereby realizing data synchronization between the target system and the source system.

2. The method according to claim 1, characterized in that: After achieving data synchronization between the target system and the source system, the method further includes: Setting the configuration value of all data of the target system to the current Apollo configuration value of the target system, where the initial value of the Apollo configuration value is 0; When the target data to be synchronized of the source system is modified, the modified target data to be synchronized of the source system is obtained, and the Apollo configuration value of the target system is controlled to be increased by 1; When the Apollo configuration value of the target system is greater than the configuration value of the current data of the target system, multiple data with the same target primary key in the modified target data to be synchronized are merged and calculated to obtain one piece of data of the target primary key, and the data with the same primary key as the target primary key in the target system is updated to one piece of data of the target primary key, and the current data is the data with the same primary key as the target primary key in the target system.

3. The method according to claim 1, characterized in that After achieving data synchronization between the target system and the source system, the method further includes: Setting the configuration value of all data of the target system to the current Apollo configuration value of the target system, where the initial value of the Apollo configuration value is 0; Obtain new target data to be synchronized from the source system, and control the Apollo configuration value of the target system to remain unchanged; When the Apollo configuration value of the target system is equal to the configuration value of the current data of the target system and there are multiple pieces of data with the same target primary key in the new target data to be synchronized, the append mode is used to accumulate the multiple pieces of data with the same target primary key in the new target data to be synchronized on the data with the same primary key as the target primary key in the target system to obtain updated data of the primary key in the target system, thereby realizing data synchronization between the target system and the source system.

4. The method according to any one of claims 1 to 3, characterized in that After obtaining the target data to be synchronized from the source system, the method further includes: When there is only one piece of data with the same primary key in the target data to be synchronized, the overwrite mode is adopted to update the data with the same primary key as the target primary key in the target data to be synchronized with the data of the primary key in the target data to be synchronized, so as to obtain the updated data of the primary key in the target system and realize data synchronization between the target system and the source system.

5. The method according to any one of claims 1 to 3, characterized in that After obtaining the target data to be synchronized from the source system, the method further includes: In the case where there are multiple pieces of data with the same primary key in the target data to be synchronized and the source system supports merge calculation, the source system is controlled to merge the multiple pieces of data with the same target primary key in the target data to be synchronized to obtain one piece of data of the target primary key, and the data with the same primary key as the target primary key in the target data to be synchronized is updated in overwrite mode to one piece of data of the primary key in the target data to be synchronized, to obtain updated data of the primary key in the target system, thereby realizing data synchronization between the target system and the source system.

6. The method according to any one of claims 1 to 3, characterized in that After achieving data synchronization between the target system and the source system, the method further includes: In the case where data synchronization between the target system and the source system fails, the target data to be synchronized of the source system is reacquired to perform data synchronization between the target system and the source system.

7. A data synchronization device, characterized in that: include: A first acquisition unit, used to acquire target data to be synchronized from a source system; The first synchronization unit is used to accumulate the multiple pieces of data with the same target primary key in the target data to be synchronized to the data with the same primary key as the target primary key in the target system in an append mode when there are multiple pieces of data with the same target primary key in the target data to be synchronized, so as to obtain updated data of the primary key in the target system and realize data synchronization between the target system and the source system.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A data synchronization platform, characterized in that: include: A source system, a target system, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of claims 1 to 6.

Images