Data synchronization method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application relate to the field of automatic programming, and disclose a data synchronization method and device, electronic equipment and a storage medium. The method comprises: receiving a synchronization instruction input by a user for data synchronization, and generating a data replication message; sending the data replication message to a data synchronization service, and determining whether the data synchronization service receives the data replication message to obtain a first determination result; when the first determination result is that the data synchronization service receives the data replication message, the data synchronization service calls a data replication service; the data replication message is parsed through the data replication service, and each change data in the data replication message is synchronized to each copy database. The technical scheme of the embodiments of the present application solves the technical problems of poor real-time performance and data inconsistency in the data synchronization method in the prior art, and realizes the technical problems of improving the real-time performance of data synchronization and avoiding data inconsistency.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of automatic programming, and in particular, to a data synchronization method and device, electronic equipment and storage medium. BACKGROUND

[0002] The existing data synchronization method is mainly a database synchronization replication method and a message asynchronous synchronization method. The database synchronization replication method usually copies the data in the entire database at a fixed time, and the amount of data to be copied is large, resulting in poor real-time performance of data synchronization. Further, the message asynchronous synchronization method usually generates a data change message when data changes, and performs data synchronization operation according to the data change message. However, the message asynchronous synchronization method mainly depends on the transmission of messages, and when the message transmission is abnormal, there will be a problem of inconsistent data. SUMMARY

[0003] Embodiments of the present application provide a data synchronization method, device, electronic equipment and storage medium to improve the real-time performance of data synchronization and avoid the problem of inconsistent data.

[0004] In a first aspect, the embodiments of the present application provide a data synchronization method, which comprises:

[0005] receiving a synchronization instruction input by a user for data synchronization to generate a data replication message;

[0006] sending the data replication message to a data synchronization service and determining whether the data synchronization service receives the data replication message to obtain a first determination result;

[0007] when the first determination result is that the data synchronization service receives the data replication message, the data synchronization service calls a data replication service;

[0008] analyzing the data replication message through the data replication service, and synchronizing each changed data in the data replication message to each copy database.

[0009] In a second aspect, the embodiments of the present application further provide a data synchronization device, which comprises:

[0010] a data replication message generation module configured to receive a synchronization instruction input by a user for data synchronization to generate a data replication message;

[0011] a first determination result obtaining module configured to send the data replication message to a data synchronization service and determine whether the data synchronization service receives the data replication message to obtain a first determination result;

[0012] The data replication service calling module is configured to call a data replication service when the first determination result is that the data synchronization service receives the data replication message.

[0013] The replica database synchronization module is configured to parse the data replication message through the data replication service and synchronize each change data in the data replication message to each replica database.

[0014] In a third aspect, an electronic device is provided, and the electronic device includes:

[0015] one or more processors;

[0016] a storage device configured to store one or more programs;

[0017] When the program is executed by the processor, the processor implements the data synchronization method provided by any of the embodiments of the present application.

[0018] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the program is executed by a processor to implement the data synchronization method provided by any of the embodiments of the present application.

[0019] The technical scheme of the embodiment receives a synchronization instruction for data synchronization input by a user to generate a data replication message. The data replication message is sent to a data synchronization service, and it is determined whether the data replication message is received by the data synchronization service to obtain a first determination result. When the first determination result is that the data replication message is received by the data synchronization service, the data synchronization service calls a data replication service. The data replication message is parsed through the data replication service, and each change data in the data replication message is synchronized to each replica database. The change data is synchronized to the replica database, the problem of poor real-time performance of data synchronization and inconsistent data in the prior art is solved, and the technical effects of improving the real-time performance of data synchronization and avoiding inconsistent data are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical scheme of the example embodiments of the present application, the drawings needed in the description of the embodiments are briefly introduced as follows. Obviously, the drawings introduced are only a part of the drawings of the embodiments to be described by the present application, and not all the drawings. Those skilled in the art can obtain other drawings according to these drawings without creating creative labor.

[0021] Figure 1 is a data synchronization method flowchart provided by the embodiment one of the present application;

[0022] Figure 2 is a data synchronization framework structure schematic diagram provided by an embodiment of the present application;

[0023] Figure 3 is a data synchronization method flowchart schematic diagram provided by an embodiment of the present application;

[0024] Figure 4 is a data synchronization method flowchart schematic diagram provided by an embodiment of the present application;

[0025] Figure 5 is a data synchronization method flowchart schematic diagram provided by an embodiment of the present application;

[0026] Figure 6 is a data synchronization method schematic diagram provided by an embodiment of the present application;

[0027] Figure 7 is a message sending method schematic diagram provided by an embodiment of the present application;

[0028] Figure 8 is a message sending method schematic diagram provided by an embodiment of the present application;

[0029] Figure 9 is a message sending method schematic diagram provided by an embodiment of the present application;

[0030] Figure 10 is a message sending method schematic diagram provided by an embodiment of the present application;

[0031] Figure 11 is a data synchronization method schematic diagram provided by an embodiment of the present application;

[0032] Figure 12 is a message state query method schematic diagram provided by an embodiment of the present application;

[0033] Figure 13 is a data synchronization device module schematic diagram provided by an embodiment of the present application;

[0034] Figure 14 is a structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] The present application will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the structures.

[0036] Embodiment one

[0037] Figure 1 is a data synchronization method flowchart provided by an embodiment of the present application. The embodiment can be applied to the case of synchronizing changed data in a copy database in a data synchronization framework. The method can be executed by a data synchronization device. The data synchronization device can be implemented in software and / or hardware. The data synchronization device can be integrated into an electronic device such as a computer or a server.

[0038] It can be understood that the data synchronization framework can be the same or different in different application scenarios. The data synchronization method of the embodiment of the present application can be applied to various data synchronization frameworks. For ease of understanding, the present application is introduced by way of example. Before introducing the data synchronization method of the embodiment, the structure of the example data synchronization framework is introduced. See Figure 2 In the embodiment of the present application, the data synchronization framework structure can include a front end, a distributed message middleware (Kafka message middleware), an event notifier, a background service, a master database (parameter master), and a copy database. The parameter front end can include a parameter maintenance front end, a parameter maintenance unit, and a master database (parameter master). The background service can include a parameter synchronization service (parameter synchronization unit) and a parameter replication service (parameter replication unit).

[0039] For ease of understanding the embodiment of the present application, the data synchronization can be briefly explained based on the above-mentioned data synchronization framework structure. In the embodiment, the front end first sends a data change message to the distributed message middleware. Then, the distributed message middleware sends the received data change message to the parameter synchronization service. When the parameter synchronization service receives the message, the parameter replication service is called to process the data change message, so that the changed data is synchronized to the copy database, thereby realizing data synchronization.

[0040] As shown in Figure 1 , the method of the embodiment includes:

[0041] S110, receiving a synchronization instruction for data synchronization input by a user, and generating a data replication message.

[0042] The synchronization instruction can be a piece of code and can be used to synchronize data. Optionally, when a trigger operation for data synchronization input by a user is received, the synchronization instruction can be generated. The trigger operation can be generated by triggering a physical key or a virtual key.

[0043] The receiving of the synchronization instruction for data synchronization from the user input can be generating the synchronization instruction based on a synchronization command for data synchronization input by the user through an input device.

[0044] The data replication message can be a formatted variable-length information unit, and can be used to deliver information to achieve data synchronization. The data structure of the data replication message can include a message header and message data. The message data can be understood as data attached after the message is sent. The message header can include a message identifier and a message generation time. The message data can include change data and data operation instructions corresponding to the change data. The number of change data can be one, two or more than two. The type of data operation instruction can include addition, modification and deletion. It should be noted that one change data corresponds to one data operation instruction, and different change data can correspond to the same or different data operation instructions.

[0045] The data replication message can be generated by combining the message identifier, the message generation time, the change data and the data operation instructions corresponding to the change data based on the pre-set data structure of the data replication message. The combination connection method can be a pre-set connection symbol, which can be a minus sign (-) or a plus sign (+) and the like.

[0046] In the embodiments of the present application, there are various ways to pre-set the data structure of the data replication message. The specific method can be determined according to actual needs. As an optional embodiment of the present application, for example, a set of change data can be generated by combining each change data, a set of instructions can be generated by combining the data operation instructions corresponding to each change data, and the data replication message can be generated by combining the change data set and the instruction set.

[0047] For example, the message identifier of the data replication message is 1, the time of generating the data replication message is XX year XX month XX day XX hour XX minute XX second, the change data can include data 1 and data 2, the data operation instructions corresponding to the change data can include instruction 1 and instruction 2, and the connection symbol of the information in the data replication message can be a minus sign. The data structure of the data replication message is 1-XX year XX month XX day XX hour XX minute XX second-data 1-data 2-instruction 1-instruction 2.

[0048] As another optional implementation in the embodiments of the present application, one change data and the data operation instruction corresponding to the change data can be taken as a group of information units, and at least one group of information units can be combined to generate a data replication message, that is, the data replication message can be obtained. The data structure of an exemplary data replication message is 1-XX year XX month XX day XX hour XX minute XX second-data 1-instruction 1-data 2-instruction 2.

[0049] Specifically, the data structure of the data replication message is set in advance. When receiving a user input trigger operation for data synchronization, a synchronization instruction can be generated. When detecting that the synchronization instruction generation is completed, the synchronization instruction is received. When receiving the synchronization instruction, the message identifier, the message generation time, the change data, and the data operation instruction corresponding to the change data are combined based on the data structure of the data replication message set in advance, and the data replication message can be obtained, that is, the data replication message is generated.

[0050] S120, the data replication message is sent to the data synchronization service, and it is judged whether the data replication message is received by the data synchronization service, and a first judgment result is obtained.

[0051] Among them, the service can be understood as a program that executes a specified system function, such as a thread or a process, so as to provide a certain type of service in order to support other programs, such as low-level (close to hardware) programs. The data synchronization service can be used to receive the data replication message and call other function services. The data synchronization service can be a pre-set service. Sending the data replication message to the data synchronization service includes: based on the message sending mechanism of the distributed message middleware, sending the data replication message to the data synchronization service.

[0052] Among them, judging whether the data synchronization service receives the data replication message can be understood as judging whether the distributed message middleware sends the data replication message to the data synchronization service, that is, judging whether the data replication message received in the distributed message middleware is consumed. It can be understood that when the data replication message received in the distributed message middleware is sent to the data synchronization service, it means that the data replication message in the distributed message middleware is consumed.

[0053] Among them, the first judgment result can be used to determine whether the data synchronization service receives the data replication message. The first judgment result can be that the data synchronization service receives the data replication message, or that the data synchronization service does not receive the data replication message.

[0054] Specifically, when the data replication message is generated, the data replication message can be received based on the distributed message middleware. When the distributed message middleware receives the data replication message, it can be detected whether the distributed message middleware consumes the data replication message, i.e., whether the distributed message middleware sends the data replication message to the data synchronization service, i.e., whether the data synchronization service receives the data replication message. When the detection of whether the data synchronization service receives the data replication message is completed, a judgment result is obtained, i.e., a first judgment result is obtained.

[0055] S130, when the first judgment result is that the data synchronization service receives the data replication message, the data synchronization service calls the data replication service.

[0056] The data synchronization service receiving the data replication message can be understood as that the data replication message in the distributed message middleware has been sent to the data synchronization service, or that the data replication message in the distributed message middleware has been consumed. The data replication service can be a service provided by a preset data replication interface. The data replication service can be used to process the data replication message. The data synchronization service calling the data replication service can be that the data synchronization service calls the data replication service provided by the preset data replication interface.

[0057] Specifically, when the first judgment result is that the data synchronization service receives the data replication message, a service call request for the data synchronization service to call the data replication service is generated. When it is detected that the generation of the service call request is completed, the service call request is received. According to the service call request, the data synchronization service can call the data replication service provided by the preset data replication interface.

[0058] S140, the data replication message is parsed by the data replication service, and each change data in the data replication message is synchronized to each copy database.

[0059] The number of change data can be one, two or more. The copy database can be used to store data. The copy database can include a database identifier. The number of copy databases can be one, two or more. The advantage of storing data in the copy database is that it can provide a more convenient and fast method for data storage and transmission, and provide more powerful functions for distributed database applications. The correspondence between the change data and the copy database includes at least one of the following: one-to-one, one-to-many, many-to-one and many-to-many.

[0060] The relationship between the change data and the replica database is established in advance, that is, the correspondence between the replica database and the data table is established in advance, and the relationship between the data table and the data is established. It can be understood that one database can include at least one data table, and one data table can include at least one data. The data replication message is parsed by the data replication service, and at least one change data in the data replication message can be determined. According to the relationship between the data table and the data, the data table corresponding to each change data can be determined. According to the correspondence between the replica database and the data table established in advance, the replica database corresponding to each change data can be determined. When it is detected that the determination of the replica database corresponding to each change data is completed, each change data can be synchronized to each replica database.

[0061] Specifically, the correspondence between the replica database and the data table is established in advance, and the relationship between the data table and the data is established. When it is detected that the data synchronization service calls the data replication service successfully, the data replication message is parsed by the data replication service to obtain the change data in the data replication message. According to the relationship between the data table and the data, the data table corresponding to each change data can be determined. According to the correspondence between the replica database and the data table established in advance, the replica database corresponding to each change data can be determined. When it is detected that the determination of the replica database corresponding to each change data is completed, each change data can be synchronized to each replica database.

[0062] For example, the change data includes change data 1 and change data 2. The replica database includes replica database 1 and replica database 2. The change data 1 corresponds to the replica database 1, and the change data 2 corresponds to the replica database 2. The change data is synchronized to the replica data 1, and the change data 2 is synchronized to the replica database 2.

[0063] In order to more conveniently and quickly synchronize the change data to the replica database, each change data in the data replication message can be synchronized to each replica database, which can include: determining the replica database identifier corresponding to each change data in the data replication message; and according to the replica database identifier corresponding to the change data, synchronizing each change data in the data replication message to the replica database corresponding to the change data.

[0064] Specifically, the correspondence between the data and the replica database identifier is established in advance. When the determination of the change data in the data replication message is completed, the replica database identifier corresponding to each change data in the data replication message can be determined according to the correspondence between the data and the replica database identifier established in advance. According to the replica database identifier, the change data is synchronized to the replica database corresponding to the change data.

[0065] The technical scheme of the embodiment receives a synchronization instruction for data synchronization input by a user to generate a data replication message. The data replication message is sent to a data synchronization service, and it is determined whether the data synchronization service receives the data replication message to obtain a first determination result. When the first determination result is that the data synchronization service receives the data replication message, the data synchronization service calls a data replication service. The data replication service analyzes the data replication message, and synchronizes each change data in the data replication message to each copy database, thereby synchronizing the change data to the copy database, solving the problem of poor real-time performance of data synchronization and data inconsistency in the prior art, and achieving the technical effects of improving the real-time performance of data synchronization and avoiding data inconsistency.

[0066] Embodiment Two

[0067] Figure 3 is a data synchronization method flowchart provided by Embodiment Two of the application. On the basis of the foregoing embodiment, optionally, the data replication message is analyzed by the data replication service, including: determining whether the data synchronization service successfully calls the data replication service to obtain a second determination result; and when the second determination result is that the data replication service has been successfully called, the data replication message is analyzed by the data replication service.

[0068] Optionally, the data synchronization method of the embodiment can further include: when the second determination result is that the data replication service has not been successfully called, recording the number of unsuccessful calls of the unsuccessful call; and when it is detected that the number of unsuccessful calls does not reach a preset call number threshold, repeating the operation of calling the data replication service and determining whether the data synchronization service successfully calls the data replication service to obtain the second determination result.

[0069] Optionally, the data synchronization method of the embodiment can further include: when it is detected that the number of unsuccessful calls reaches the preset call number threshold, generating second prompt information and displaying the second prompt information.

[0070] Among them, the same or corresponding technical terms as the above embodiments are not repeated here.

[0071] As shown in Figure 3 , the method of the embodiment can specifically include:

[0072] S210, receiving a synchronization instruction for data synchronization input by a user to generate a data replication message.

[0073] S220, send the data replication message to the data synchronization service, and determine whether the data synchronization service receives the data replication message, to obtain a first determination result, when the first determination result is that the data synchronization service receives the data replication message, then execute S230

[0074] S230, the data synchronization service calls the data replication service.

[0075] S240, determine whether the data synchronization service calling the data replication service is successful, to obtain a second determination result, when the second determination result is that the data replication service has been successfully called, then execute S250; when the second determination result is that the data replication service has not been successfully called, then S260.

[0076] The second determination result can be used to determine whether the data synchronization service successfully calls the data replication service. The second determination result can be that the data synchronization service has successfully called the data replication service, or that the data synchronization service has not successfully called the data replication service.

[0077] Specifically, when the data synchronization service receives the data replication message, the data synchronization service can call the data replication service. Determine whether the data synchronization service calling the data replication service is successful, to obtain a determination result, and take the obtained determination result as the second determination result, that is, obtain the second determination result.

[0078] S250, analyze the data replication message through the data replication service, and synchronize each change data in the data replication message to each copy database.

[0079] Specifically, after obtaining the second determination result, the second determination result can be detected. When it is detected that the second determination result is that the data synchronization service has successfully called the data replication service, that is, when it is detected that the second determination result is that the data replication service has been successfully called, then the data replication message can be analyzed through the data replication service. When the data replication message is analyzed, the change data in the data replication message can be determined. When each change data in the data replication message is determined, the copy database corresponding to each change data can be determined. When the copy database corresponding to each change data is determined, each change data can be synchronized to the copy database corresponding to each change data.

[0080] S260, record the number of unsuccessful calls that have not been successfully called.

[0081] Specifically, after obtaining the second determination result, the second determination result can be detected. When it is detected that the second determination result is that the data synchronization service fails to call the data replication service, that is, when it is detected that the second determination result is that the data replication service fails to be called successfully, the number of times of unsuccessful calling can be recorded, and the number of times of unsuccessful calling is taken as the number of times of unsuccessful calling, that is, the number of times of unsuccessful calling (such as 2 times) is recorded.

[0082] S270, whether the number of times of unsuccessful calling reaches a preset calling number threshold value is detected. When it is detected that the number of times of unsuccessful calling does not reach the preset calling number threshold value, the operation of returning to performing S230 is performed. When it is detected that the number of times of unsuccessful calling reaches the preset calling number threshold value, the operation of performing S280 is performed.

[0083] The calling number threshold value can be set according to experience, and a specific value is not limited herein.

[0084] Specifically, the calling number threshold value is set in advance. When the second determination result is that the data replication service fails to be called successfully, the number of times of unsuccessful calling at the current time is determined. The number of times of unsuccessful calling at the current time is compared with the calling number threshold value set in advance. When the number of times of unsuccessful calling does not reach the calling number threshold value set in advance, the operation of repeatedly calling the data replication service and determining whether the data synchronization service successfully calls the data replication service to obtain the second determination result can be performed.

[0085] S280, second prompt information is generated, and the second prompt information is displayed.

[0086] The second prompt information can be used to prompt that the data replication service has been successfully called. The second prompt information can include information such as the generation time of the message, the receiver of the message, and the sender of the message.

[0087] Specifically, the calling number threshold value is set in advance. When the second determination result is that the data replication service fails to be called successfully, the number of times of unsuccessful calling at the current time is determined. The number of times of unsuccessful calling at the current time is compared with the calling number threshold value set in advance. When the number of times of unsuccessful calling reaches the calling number threshold value set in advance, the prompt information of successfully calling the replication service is generated, and the prompt information of successfully calling the replication service is taken as the second prompt information. When the second prompt information is detected to be generated, the second prompt information can be displayed, so as to determine that the data replication service is successfully called.

[0088] It should be noted that the above label is only an example of the order of step execution of the embodiment, and is not limited. The "first" and "second" in the first prompt information and the second prompt information are only used to distinguish the prompt information corresponding to different information types, and are not limited to the order or content of the prompt information.

[0089] The technical scheme of the embodiment judges whether the data synchronization service successfully calls the data replication service to obtain a second judgment result. When the second judgment result is that the data replication service has been successfully called, the data replication service is used to parse the data replication message and synchronize each change data in the data replication message to each replica database. When the second judgment result is that the data replication service has not been successfully called, the number of times of unsuccessful calling is recorded. When it is detected that the number of times of unsuccessful calling does not reach a preset calling number threshold, the operation of repeatedly calling the data replication service and judging whether the data synchronization service successfully calls the data replication service to obtain a second judgment result is performed. When it is detected that the number of times of unsuccessful calling reaches the preset calling number threshold, a second prompt information is generated and displayed, so that the data replication service can be automatically called when the data replication service is not successfully called, and the technical effect of avoiding data inconsistency is achieved.

[0090] Embodiment three

[0091] Figure 4 is a data synchronization method flowchart provided by the third embodiment of the application. On the basis of the foregoing embodiment, optionally, the operation of judging whether the data synchronization service receives the data replication message to obtain a first judgment result comprises judging whether the data synchronization service receives the data replication message according to the sending state of each data operation task in the data replication message to obtain a first judgment result.

[0092] Optionally, the sending state comprises unsuccessful sending, sending in progress and successful sending. On this basis, optionally, the data synchronization method of the embodiment can further comprise the following operation: when the first judgment result is that the data synchronization service does not receive the data replication message, it is determined that the data operation task with the unsuccessful sending state in the data replication message is determined; the sending state of the data operation task with the unsuccessful sending state is modified from unsuccessful sending to sending in progress, so as to resend the data operation task with the unsuccessful sending state to the data synchronization service.

[0093] Optionally, the data synchronization method of the embodiment can further comprise the following operation: if it is detected that the number of times of re-sending corresponding to the data operation task reaches a preset sending threshold, and the sending state of the data operation task is still unsuccessful sending, it is determined that the data synchronization service does not receive the data replication message.

[0094] Optionally, the data synchronization method of the embodiment can further comprise the following operation: when it is detected that the sending state of the data operation task is unsuccessful sending, the sending state of the data operation task with the unsuccessful sending state is marked as sending failure.

[0095] Optionally, the data synchronization method of the embodiment can further include: when it is detected that the sending state flag of the data operation task is marked as sending failure, and the re-sending number of the data operation task reaches the preset sending threshold, modifying the sending state flag of the data operation task marked as sending failure to abandoned.

[0096] Optionally, the data synchronization method of the embodiment can further include: when the re-sending number of the data operation task does not reach the preset sending number threshold, generating a first prompt information; and sending the first prompt information to the data synchronization service, so that the data synchronization service invokes the data replication service.

[0097] Optionally, the data synchronization method of the embodiment can further include: querying the sending state of each data operation task in the data replication message by a data replication state query service, to obtain a query result of the data replication message.

[0098] Among them, the same or corresponding technical terms as the above embodiments are not repeated here.

[0099] As shown in Figure 4 the method of the embodiment can specifically include:

[0100] S310, receiving a synchronization instruction for data synchronization input by a user, and generating a data replication message.

[0101] S320, sending the data replication message to a data synchronization service, and judging whether the data synchronization service receives the data replication message according to the sending state of each data operation task in the data replication message, to obtain a first judgment result; when the first judgment result is that the data synchronization service does not receive the data replication message, executing S330; and when the first judgment result is that the data synchronization service receives the data replication message, executing S350.

[0102] The data operation task can be a task of changing data in the replica database, and can be used to synchronize the changed data to the replica database. The types of data operation tasks can include deleting data, modifying data, and adding data. The data operation task can include changed data and a data operation instruction corresponding to the changed data. For example, the changed data of the data operation task in the data replication message is unit price = 14, and the data operation instruction is to modify the unit price to 18. The number of data operation tasks in the data replication message can be one, two or more. There can be a hierarchical relationship between the data operation tasks, and the hierarchical relationship of the data operation tasks can be determined according to the execution order of the data operation.

[0103] The sending of the data replication message to the data synchronization service can be the sending of each data operation task in the data replication message to the data synchronization service. The sending state can be understood as the sending state of the message corresponding to the data operation task, and can be used to determine the sending state of the message corresponding to the data operation task. The sending state includes not sent successfully, sending, and sent successfully.

[0104] Specifically, when it is detected that the data replication message is generated, each data operation task in the data replication message can be sent to the data synchronization service. After detecting the sending of each data operation task, the sending state of the message corresponding to each data operation task is detected, and the sending state of the message corresponding to each data operation task is determined. According to the sending state of the message corresponding to each data operation task, it can be judged whether the data synchronization service receives the data replication message. When the sending state of the message corresponding to each data operation task is sent successfully, it can be obtained that the data synchronization service receives the data replication message, and the obtained judgment result that the data synchronization service receives the data replication message can be taken as the first judgment result. When there is a data operation task with a not sent successfully sending state in the data replication message, it can be obtained that the data synchronization service does not receive the data replication message, and the obtained judgment result that the data synchronization service does not receive the data replication message can be taken as the first judgment result.

[0105] S330, determining a data operation task with a not sent successfully sending state in the data replication message.

[0106] The number of data operation tasks with a not sent successfully sending state in the data replication message can be one, two, or more than two.

[0107] Specifically, after obtaining the first judgment result, the first judgment result can be determined. When the first judgment result is that the data synchronization service does not receive the data replication message, the sending state of each data operation task in the data replication message can be detected. According to the sending state of the data operation task, the data operation task with a not sent successfully sending state in the data replication message can be determined.

[0108] For example, the data operation tasks included in the data replication message are task 1, task 2, and task 3. The sending state of task 1 is sent successfully, the sending state of task 2 is not sent successfully, and the sending state of task 3 is not sent successfully. Then it can be determined that the data operation tasks with a not sent successfully sending state in the data replication message are task 2 and task 3.

[0109] S340, modifying the sending state of the data operation task not sent successfully to not sent successfully to sending, so as to resend the data operation task not sent successfully to the data synchronization service.

[0110] In the embodiment of the present application, the modification of the sending state of the unsuccessfully sent data operation task from unsuccessfully sent to in sending can be in various ways. The specific way can be determined according to actual needs. As an optional embodiment of the present application, the sending state of the unsuccessfully sent data operation task can be modified according to the execution order of the data operation task. For example, the unsuccessfully sent data operation tasks are task 1 and task 2, and the execution order of task 1 and task 2 is task 2, task 1. Then, the sending state of task 2 is modified to in sending first, and then the sending state of task 1 is modified to in sending.

[0111] As another optional embodiment of the present application, the sending state of each unsuccessfully sent data operation task can be uniformly modified to in sending. It can be understood that in the present embodiment, when it is determined that the sending state of the data operation task is unsuccessfully sent, the sending state of all unsuccessfully sent data operation tasks in the data replication message is modified to in sending. For example, the unsuccessfully sent data operation tasks are task 3 and task 4. Then, the sending state of task 3 and task 4 is modified to in sending.

[0112] Specifically, the way of modifying the sending state of the unsuccessfully sent data operation task from unsuccessfully sent to in sending is set in advance. After determining the data operation task with the sending state of unsuccessfully sent in the data replication message, the sending state of the unsuccessfully sent data operation task is modified from unsuccessfully sent to in sending according to the way of modifying the sending state of the unsuccessfully sent data operation task from unsuccessfully sent to in sending set in advance. When it is detected that the sending state of the unsuccessfully sent data operation task is modified to in sending, the data operation task that is unsuccessfully sent can be sent to the data synchronization service again.

[0113] In the present embodiment, the way of determining that the data synchronization service has not received the data replication message can be that if it is detected that the retransmission number of the data operation task reaches the preset sending threshold value, and the sending state of the data operation task is still unsuccessfully sent, it is determined that the data synchronization service has not received the data replication message.

[0114] The retransmission number of the data operation task can be understood as the number of times of retransmitting the data operation task. For example, when the sending state of the data operation task is unsuccessfully sent, the data operation task is transmitted again, and at this time, the retransmission number of the data operation task is 1. The sending threshold value can be set according to experience, and the specific value is not limited herein.

[0115] Specifically, a sending threshold is set in advance. When the unsuccessfully sent data operation task is re-sent to the data synchronization service, the number of re-sending of each unsuccessfully sent data operation task can be determined. The maximum value of the number of re-sending corresponding to each unsuccessfully sent data operation task is compared with the sending threshold set in advance. When the maximum value of the number of re-sending of the data operation task is equal to the sending threshold set in advance, and the sending state of the data operation task is still unsuccessfully sent, it can be determined that the data synchronization service does not receive the data replication message.

[0116] For example, the sending threshold is 5. The data operation task includes task 1 and task 2. The number of re-sending of task 1 is 3, and the sending state is unsuccessfully sent. The number of re-sending of task 2 is 5, and the sending state is unsuccessfully sent. The number of re-sending of task 2 is equal to the sending threshold, and the sending state is unsuccessfully sent. Therefore, it can be determined that the data synchronization service does not receive the data replication message.

[0117] In order to facilitate the management of the data operation task, when it is detected that the sending state of the data operation task is unsuccessfully sent, the sending state of the unsuccessfully sent data operation task is marked as sending failure.

[0118] Further, when it is detected that the sending state of the data operation task is marked as sending failure, and the number of re-sending corresponding to the data operation task reaches the preset sending threshold, the sending state of the data operation task marked as sending failure is modified to abandoned.

[0119] The sending state of the unsuccessfully sent data operation task can include sending failure and abandonment. The sending failure can be used to represent that the data operation task fails to send. The abandonment can be used to represent that the data operation task is invalid and needs to be discarded.

[0120] In order to improve the real-time performance of data synchronization, when the number of re-sending corresponding to the data operation task does not reach the preset sending threshold, a first prompt information is generated; the first prompt information is sent to the data synchronization service, so that the data synchronization service calls the data replication service.

[0121] The first prompt information can be used to prompt the data synchronization service to call the data replication service. The generation time of the message can be included in the first prompt information.

[0122] Specifically, a sending threshold is set in advance. When the unsuccessfully sent data operation task is re-sent to the data synchronization service, the number of re-sending of each unsuccessfully sent data operation task can be determined. The number of re-sending of each unsuccessfully sent data operation task is detected to be less than the sending threshold set in advance, and the sending state of the message corresponding to each data operation task is detected.

[0123] Specifically, when it is detected that the sending state of the message corresponding to each data operation task is successfully sent, the prompt information that the sending state of the message corresponding to each data operation task is successfully sent is generated, and the prompt information that the sending state of the message corresponding to each data operation task is successfully sent is generated as the first prompt information, that is, the first prompt information is generated. When it is detected that the generation of the first prompt information is completed, the first prompt information is sent to the data synchronization service. When it is detected that the data synchronization service receives the first prompt information, the data synchronization service can be prompted to call the data replication service in order to synchronize the data.

[0124] S350, the data synchronization service calls the data replication service.

[0125] S360, the data replication message is parsed by the data replication service, and each change data in the data replication message is synchronized to each copy database.

[0126] In order to know the data operation task corresponding to each data operation task in the data replication message in real time, the embodiment of the application further comprises: querying the sending state of each data operation task in the data replication message through the data replication state query service, and obtaining the query result of the data replication message.

[0127] The data replication state query service can be used to query the sending state of the message corresponding to each data operation task in the data replication message. The data replication state query service can be a pre-set service. The data operation task includes a task identifier. The way to query the sending state of each data operation task in the data replication message can be to query based on the task identifier of the data operation task.

[0128] Specifically, the data replication state query service is pre-set. Through the pre-set data replication state query service, the sending state of the message corresponding to the task identifier of each data operation task in the data replication message can be queried according to the task identifier of each data operation task in the data replication message, so as to determine the sending state of the message corresponding to the task identifier of each data operation task, that is, the query result of the data replication message can be obtained.

[0129] It should be noted that the above label is only an example of the order of execution of the steps of the embodiment, and is not limited.

[0130] The technical scheme of the embodiment is that the data replication message is sent to the data synchronization service, and whether the data synchronization service receives the data replication message is determined according to the sending state of each data operation task in the data replication message, to obtain a first determination result. When the first determination result is that the data synchronization service does not receive the data replication message, it is determined that the data operation task with the sending state that is not successfully sent in the data replication message. The sending state of the data operation task that is not successfully sent is modified from not successfully sent to sending, so as to re-send the data operation task that is not successfully sent to the data synchronization service, the automatic sending of each data operation task in the data replication message is realized, thereby facilitating the management of each data operation task in the data replication message, and the technical effect of improving the real-time performance of data synchronization is achieved.

[0131] Embodiment four

[0132] Figure 5 is a data synchronization method flowchart provided by the fourth embodiment of the application, and on the basis of the foregoing embodiments, optionally, the data replication message is parsed by the data replication service, and each change data in the data replication message is synchronized to each replica database, which comprises: the data replication message is parsed by the data replication service to determine each change data in the data replication message and a data operation instruction corresponding to the change data; at least one database execution statement is generated according to the change data and the data operation instruction corresponding to the change data; and each change data in the data replication message is synchronized to a replica database according to the database execution statement.

[0133] Optionally, the data synchronization method of the embodiment can further comprise: determining the total number of database execution statements; when it is detected that the number of executed database execution statements is less than the total number of database execution statements, obtaining a database execution statement that is not executed; and modifying the data of the replica database based on the database execution statement that is not executed.

[0134] Among them, the same or corresponding technical terms as the above embodiments are not repeated here.

[0135] As Figure 5 shown, the method of the embodiment can specifically comprise:

[0136] S410, receiving a synchronization instruction input by a user for data synchronization, and generating a data replication message.

[0137] S420, sending the data replication message to a data synchronization service, and determining whether the data synchronization service receives the data replication message, to obtain a first determination result.

[0138] S430, when the first determination result is that the data synchronization service receives the data replication message, the data synchronization service calls the data replication service.

[0139] S440, the data replication message is parsed by the data replication service to determine each change data in the data replication message and the data operation instruction corresponding to each change data.

[0140] The number of change data in the data replication message can be one, two or more. The type of data operation instruction can include addition, modification and deletion. It should be noted that the data operation instructions of different change data can be the same or different.

[0141] Specifically, when the data synchronization service calls the data replication service is completed, the data replication message can be parsed by the data replication service. When it is detected that the data replication message is parsed, each change data in the data replication message can be obtained, and the data operation instruction corresponding to each change data can be obtained.

[0142] For example, the data replication message includes change data 1, change data 2, data operation instruction 1 and data operation instruction 2. The data operation instruction corresponding to change data 1 is data operation instruction 1, and the data operation instruction corresponding to change data 2 is data operation instruction 2. After the data replication message is parsed by the data replication service, change data 1, change data 2, data operation instruction 1 and data operation instruction 2 can be obtained.

[0143] S450, at least one database execution statement is generated according to each change data and the data operation instruction corresponding to each change data.

[0144] The database execution statement can be understood as an operation statement for operating the database. The number of database execution statements can be one, two or more. One database execution statement can include at least one change data and the data operation instruction corresponding to the change data.

[0145] Specifically, after determining each change data in the data replication message and the data operation instruction corresponding to each change data, the data table in the replica database corresponding to the change data can be determined. When the data table corresponding to the change data is determined, the change data and the data operation instruction corresponding to the change data can be combined to generate a database execution statement based on the syntax rules of the database statement. For example, the change data is data 1, the data operation instruction corresponding to data 1 is insert, and the data table in the replica database corresponding to the change data is table 1. Then, the generated database execution statement is insert into table 1 values (data 1).

[0146] S460, synchronize each change data in the data replication message to the replica database according to the database execution statement.

[0147] Specifically, after obtaining the database execution statement, i.e. after the database execution statement is generated, the number of database execution statements can be determined. If it is detected that the number of database execution statements is greater than one, the execution logic between the database execution statements is determined. According to the execution logic between the database execution statements, the database execution statements are executed. When it is detected that the execution of the database execution statements is completed, each change data can be synchronized to the replica database. If it is detected that the number of database execution statements is equal to one, the database execution statement is executed, thereby synchronizing each change data in the data replication message to the replica database.

[0148] In order to avoid the occurrence of abnormal problems in the execution process of the database execution statement, the total number of database execution statements to be executed can be determined. When it is detected that the number of database execution statements that have been executed in the database execution statements to be executed is less than the total number of database execution statements to be executed, the database execution statement that has not been executed is obtained. The data of the replica database is modified based on the database execution statement that has not been executed.

[0149] Specifically, after obtaining the database execution statement, the total number of database execution statements to be executed can be determined, such as 5. When the total number of database execution statements to be executed is determined, the database execution statements to be executed are executed. When it is detected that the execution of the database execution statements to be executed is completed, the number of database execution statements that have been executed in the database execution statements to be executed can be determined, such as 3. The total number of database execution statements to be executed is compared with the number of database execution statements that have been executed. When the number of database execution statements that have been executed is less than the total number of database execution statements to be executed, the database execution statement that has not been executed can be determined. When the database execution statement that has not been executed is determined, the data of the replica database is modified based on the database execution statement that has not been executed.

[0150] In one embodiment, modifying the data of the replica database based on the database execution statement that has not been executed can include: modifying the data of the replica database based on the database execution statement that has not been executed and the database execution statement corresponding to the database execution statement that has not been executed in the database execution statement that has been executed.

[0151] In another embodiment, modifying the data of the replica database based on the unexecuted database execution statement can include: executing the data operation instruction corresponding to the unexecuted database execution statement to modify the data of the replica database; or modifying the data operation instruction corresponding to the unexecuted database execution statement, and executing the modified data operation instruction to modify the data of the replica database.

[0152] The technical scheme of the embodiment can parse the data replication message through the data replication service, determine each change data in the data replication message and the data operation instruction corresponding to each change data, generate at least one database execution statement according to each change data and the data operation instruction corresponding to each change data, and synchronize each change data in the data replication message to the replica database according to the database execution statement, thereby achieving the technical effect of improving the real-time performance of data synchronization.

[0153] Embodiment five

[0154] The embodiment five of the application provides a preferred example of a data synchronization method, and the specific implementation can be seen from the following embodiments. Wherein, the same or corresponding technical terms as the above embodiments are not repeated here.

[0155] Figure 6 is a schematic diagram of a data synchronization method provided by the embodiment of the application, referring to Figure 2 Based on the user operation, a data replication message, i.e., a parameter replication message, is generated, and after the parameter replication message is generated, the data synchronization can be realized through the pre-set parameter synchronization service and the parameter replication service.

[0156] In the embodiment of the application, there are two methods of data synchronization, and as an optional implementation in the embodiment, the generated parameter replication message can be sent to the parameter replication unit through the event notifier, referring to Figure 6 When it is detected that the parameter replication message is sent to the parameter replication message and the sending fails, the parameter replication message asynchronous retransmission operation can be performed. When it is detected that the parameter replication message is successfully sent to the parameter replication service, each change data is synchronized to the parameter replica corresponding to each change data through the parameter replication unit.

[0157] Wherein, the process of sending the generated parameter replication message to the parameter replication unit and detecting that the parameter replication message is sent to the parameter replication message and the sending fails, and the parameter replication message asynchronous retransmission operation can be performed can be seen from Figure 7 For example Figure 7The parameter replication message can be sent when the parameter replication message is generated or after the parameter replication message is generated. After the parameter replication message is sent, the batch time of each data operation task, the expected batch flow, the association relationship between the record tasks and the parameter replication detail flow can be generated based on the relationship between the data operation tasks in the parameter replication message.

[0158] Specifically, after the parameter replication detail flow is recorded, it can be checked whether there is an unsent, sending and / or sending failed batch. If yes, the batch status is recorded as unsent; if no, the parameter replication message is sent. After the parameter replication message is sent, it is determined whether there is an exception in the process of sending the parameter replication message. If yes, the batch status is recorded as sending failed; if no, the parameter replication service is called.

[0159] Specifically, after the parameter replication service is called, it is determined whether the service calling is abnormal. If yes, referring to Figure 8 , the data operation task in the parameter replication message is sent based on the preset automatic task sending mechanism, and the sending number of the batch corresponding to the data operation task is obtained. It is determined whether the sending number of the batch corresponding to the data operation task reaches the repeated execution number. If no, the batch number of the abnormal batch is determined. When it is determined whether the repeated execution number is reached, if the repeated execution number is reached, it is determined whether the parameter replication service is called successfully each time. If the repeated execution number is not reached, the operation of repeatedly calling the parameter replication service is performed. When it is determined whether the parameter replication service is called successfully each time, if yes, the parameter replication message is parsed by the data replication service, and each changed data is synchronized to the parameter copy corresponding to each changed data; if no, the batch number of the abnormal batch is determined.

[0160] It should be noted that the batch number can be the batch number of the data operation task, and the data operation task corresponding to the batch number can be one or more.

[0161] As another optional implementation in the embodiment of the present application, the parameter replication message can be sent to the parameter synchronization unit based on a distributed message middleware. It is determined whether the parameter replication message is successfully consumed, that is, whether the parameter replication message is successfully sent to the parameter synchronization service. If the parameter replication message is not successfully sent to the parameter synchronization service, the operation of asynchronously re-sending the parameter replication message can be performed. If the parameter replication message is successfully sent to the parameter synchronization service, the parameter replication service in the parameter replication interface (API) is called by the parameter synchronization service in the parameter synchronization unit. The parameter replication message is parsed by the parameter replication service, so that each changed data is synchronized to the parameter copy corresponding to each changed data.

[0162] If the parameter replication message is not successfully sent to the parameter synchronization service, a specific process of performing the operation of asynchronous retransmission of the parameter replication message can be as follows Figure 9 As described above, the correlation between tasks is obtained by pulling the message, updating the batch flow state to synchronization, obtaining all first and second classification combinations under the batch, determining the update time in the parameter type state table, and thus performing batch timing judgment. It is determined whether the current batch already exists, and whether the data operation task in the batch is in a partially successful state. If the current batch already exists, the state is updated to abandoned, and the processing is ended. If the data operation task in the batch is in a partially successful state, the thread is hung (suspended) until the data operation task in the batch is in a fully successful state or times out.

[0163] Specifically, when it is detected that the batch judgment is completed, the addressing transaction code, multi-entity number and other information are mapped by applying the ID to perform configuration center addressing to obtain a target spu (target copy database identifier) address list. After obtaining the target spu address list, concurrent scheduling can be performed based on an automatic task internal call mechanism (see Figure 10 ), and the state of microservice calling corresponding to each copy database is recorded. Then, it is judged whether all threads in the concurrent scheduling process are executed. If all fail, the batch state is updated to abandoned, and the operation is ended. If all succeed, the update time and state related to the parameter type state table are updated to fully successful, and the batch state is updated to fully successful. If partially successful, the update time and state related to the parameter type state table are updated to partially successful, and the batch state is updated to partially successful.

[0164] In the above embodiment, the specific process of synchronizing each change data to the parameter copy corresponding to each change data by the parameter replication unit can be as follows Figure 11 As described above, after obtaining the parsed parameter replication message, the request data can be obtained, and the change data and operation instruction can be obtained by parsing the request data. After parsing the request data, the transaction can be started, and the sql statement (database execution statement) can be executed in ascending order according to the index of the request list in the transaction after the transaction is started.

[0165] Specifically, when executing the sql statement, it is determined whether an exception exists in execution. If an exception occurs in the execution process, an operation type determination is performed. If the operation is an insert operation and the exception is a primary key conflict, the operation is changed to an update operation and the transaction is repeated. If the operation is an update operation and the exception is that a record does not exist, the operation is changed to an insert operation and the transaction is executed. If the operation is a delete operation and the exception is that a record does not exist, the next sql statement is executed. If an exception exists finally, the transaction is rolled back and a failure is returned. If all is successful, the transaction is committed and a success is returned, and it is indicated that the data synchronization operation is executed.

[0166] On the basis of the above embodiment, the embodiment of the present application further provides state query of each data operation task in the parameter replication message, such as Figure 12 As described above, the batch number of each data operation task is determined by querying the batch flow, and the state information corresponding to the batch number (SPU) of each data operation task is queried according to the batch number of each data operation task. The number of batch numbers is counted when the state information corresponding to the batch number of each data operation task is queried. The state of each data operation task can be determined by calling the parameter replication state query service when the number of batch numbers of each data operation task is counted. Whether the parameter replication state query service is abnormal in the calling process can be determined in the process of determining the state of each data operation task by the parameter replication state query service.

[0167] The technical scheme of the embodiment solves the problems of poor real-time performance of data synchronization and data inconsistency in the prior art, and achieves the technical effects of improving the real-time performance of data synchronization and avoiding data inconsistency.

[0168] Embodiment six

[0169] Figure 13 It is a data synchronization device module schematic diagram provided by the embodiment six of the present application, and the present application provides a data synchronization device, which comprises a data replication message generation module 510, a first determination result obtaining module 520, a data replication service calling module 530 and a replica database synchronization module 540.

[0170] The data replication message generation module 510 is configured to receive a synchronization instruction input by a user for data synchronization and generate a data replication message.

[0171] The technical scheme of the embodiment is configured to receive a synchronization instruction input by a user for data synchronization through the data replication message generation module and generate a data replication message. The first judgment result obtaining module is configured to send the data replication message to a data synchronization service and determine whether the data synchronization service receives the data replication message to obtain a first judgment result. The data replication service calling module is configured to call a data replication service by the data synchronization service when the first judgment result indicates that the data synchronization service receives the data replication message. The replica database synchronization module is configured to analyze the data replication message through the data replication service and synchronize each change data in the data replication message to each replica database. The technical scheme realizes synchronization of change data to a replica database, solves the problem of poor real-time performance of data synchronization and data inconsistency in the prior art, and achieves the technical effects of improving the real-time performance of data synchronization and avoiding data inconsistency.

[0172] Optionally, the replica database synchronization module 540 is configured to determine whether the data replication service is successfully called by the data synchronization service to obtain a second judgment result, and analyze the data replication message through the data replication service when the second judgment result indicates that the data replication service is successfully called.

[0173] Optionally, the first judgment result obtaining module 520 is configured to determine whether the data synchronization service receives the data replication message according to a sending state corresponding to each data operation task in the data replication message to obtain a first judgment result.

[0174] Optionally, the sending state comprises not sent successfully, sending and sent successfully; the data synchronization apparatus further comprises a data operation task resending module, configured to: when the first judgment result is that the data synchronization service does not receive the data replication message, determine that the sending state of the data operation task in the data replication message is not sent successfully; modify the sending state of the data operation task not sent successfully from not sent successfully to sending, so as to resend the data operation task not sent successfully to the data synchronization service.

[0175] Optionally, the data synchronization apparatus further comprises an un-received message determining module, configured to: if it is detected that the re-sending number of the data operation task reaches a preset sending threshold, and the sending state of the data operation task is still not sent successfully, determine that the data synchronization service does not receive the data replication message.

[0176] Optionally, the data synchronization apparatus further comprises a sending state marking module, configured to: when it is detected that the sending state of the data operation task is not sent successfully, mark the sending state of the data operation task not sent successfully as sending failure.

[0177] Optionally, the data synchronization apparatus further comprises a sending state marking modification module, configured to: when it is detected that the sending state of the data operation task is marked as sending failure, and the re-sending number of the data operation task reaches the preset sending threshold, modify the sending state of the data operation task sending failure from sending failure to discard.

[0178] Optionally, the data synchronization apparatus further comprises a first prompt message sending module, configured to: when the re-sending number of the data operation task does not reach the preset sending number threshold, generate a first prompt information; send the first prompt information to the data synchronization service, so that the data synchronization service invokes the data replication service.

[0179] Optionally, the data synchronization apparatus further comprises a second judgment result obtaining module, configured to: when the second judgment result is that the data replication service is not invoked successfully, record the number of times of unsuccessful invocation; when it is detected that the number of times of unsuccessful invocation does not reach a preset invocation number threshold, repeatedly perform the operation of invoking the data replication service and judging whether the data synchronization service invokes the data replication service successfully to obtain the second judgment result.

[0180] Optionally, the data synchronization apparatus further comprises a second prompt information display module, configured to: when it is detected that the number of times of unsuccessful invocation reaches the preset invocation number threshold, generate a second prompt information, and display the second prompt information.

[0181] Optionally, the replica database synchronization module 540 is configured to parse the data replication message through the data replication service, determine each change data in the data replication message and a data operation instruction corresponding to the each change data, generate at least one database execution statement according to the each change data and the data operation instruction corresponding to the each change data, and synchronize the each change data in the data replication message to a replica database according to the database execution statement.

[0182] Optionally, the data synchronization apparatus further comprises a database execution statement exception processing module configured to determine a total number of database execution statements, obtain a database execution statement that has not been executed when it is detected that a number of database execution statements that have been executed is less than the total number of database execution statements, and modify data of a replica database based on the database execution statement that has not been executed.

[0183] Optionally, the database execution statement exception processing module comprises a data modification model configured to modify data of a replica database based on a database execution statement that has not been executed and a database execution statement corresponding to the database execution statement that has not been executed in database execution statements that have been executed.

[0184] Optionally, the data modification model is configured to execute a data operation instruction corresponding to the database execution statement that has not been executed to modify data of the replica database, or modify the data operation instruction corresponding to the database execution statement that has not been executed and execute the modified data operation instruction to modify data of the replica database.

[0185] Optionally, the data synchronization apparatus further comprises a data replication message query module configured to query a sending state corresponding to each data operation task in the data replication message through a data replication state query service and obtain a query result of the data replication message.

[0186] Optionally, the replica database synchronization module 540 is configured to determine a replica database identifier corresponding to each change data in the data replication message, and synchronize the each change data in the data replication message to a replica database corresponding to the each change data according to the replica database identifier corresponding to the each change data.

[0187] The apparatus described above can execute the data synchronization method provided by any embodiment of the application and has the corresponding function modules and beneficial effects of executing the data synchronization method.

[0188] It is to be noted that the units and modules included in the data synchronization apparatus are only divided according to function logic, and are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual differentiation, and are not used to limit the protection scope of the embodiments of the present application.

[0189] Embodiment Seven

[0190] Figure 14 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. Figure 14 A block diagram of an exemplary electronic device 12 suitable for implementing any of the embodiments of the present application is shown. Figure 14 The electronic device 12 shown is merely one example and should not be construed as limiting the scope of the embodiments of the present application. The device 12 is typically an electronic device that undertakes processing of configuration information.

[0191] As Figure 14 shown, the electronic device 12 is in the form of a general-purpose computing device. The components of the electronic device 12 can include, but are not limited to, one or more processors or processing units 16, a memory 28, a bus 18 that connects the different components, including the memory 28 and the processing unit 16.

[0192] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures. By way of example, these architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0193] The electronic device 12 typically includes a variety of computer readable media. These media can be any available media that is accessible by the electronic device 12 and includes both volatile and non-volatile media, removable and non-removable media.

[0194] Memory 28 can include computer-readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 can further include other removable / non-removable, volatile / non-volatile computer storage media. By way of example only, storage system 34 can be provided for reading from and writing to a non-removable, non-volatile magnetic media (not shown and typically called a "hard drive"). Figure 14 Although not shown, a magnetic disk drive for reading from and writing to a removable, non-volatile magnetic media (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a removable, non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, etc.), can be provided. In such instances, each drive can be connected to bus 18 by one or more data media interfaces. Storage media 28 can include at least one program product 40 having a set of program modules 42 configured to carry out the functions of embodiments of the application. Program product 40, can be stored in memory 28 by way of example, such as an application program, another program module, and program data, each of which or a combination can include an implementation of a networking environment. Program modules 42 generally carry out the functions and / or methodologies of embodiments of the application described herein.

[0195] The electronic device 12 can also communicate with one or more external devices 14 such as a keyboard, a mouse, a camera, etc. and a display, as well as with one or more devices that enable a user to interact with the electronic device 12 and / or any devices (e.g., a networking card, a modem, etc.) that enable the electronic device 12 to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 22. Still yet, the electronic device 12 can communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and / or the Internet) through a network adapter 20. As depicted, the network adapter 20 communicates with the other components of the electronic device 12 through the bus 18. It should be appreciated that although not shown, other hardware and / or software modules could be used in connection with the electronic device 12. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID devices, tape drives, and data archival storage devices, etc.

[0196] The processor 16 performs various function applications and data processing by running programs stored in the memory 28, such as implementing the data synchronization method provided by the above-mentioned embodiments of the present application, which comprises:

[0197] receiving a synchronization instruction input by a user for data synchronization, generating a data replication message; sending the data replication message to a data synchronization service, and determining whether the data synchronization service receives the data replication message to obtain a first determination result; when the first determination result is that the data synchronization service receives the data replication message, the data synchronization service calls a data replication service; the data replication service analyzes the data replication message, and synchronizes each change data in the data replication message to each copy database.

[0198] Of course, those skilled in the art can understand that the processor can also implement the technical solutions of the data synchronization method provided by any of the embodiments of the present application.

[0199] Embodiment eight

[0200] The embodiment eight of the present application also provides a computer readable storage medium, which stores a computer program, characterized in that, when the program is executed by a processor, such as the data synchronization method provided by the above-mentioned embodiments of the present application, the method comprises:

[0201] receiving a synchronization instruction input by a user for data synchronization, generating a data replication message;

[0202] sending the data replication message to a data synchronization service, and determining whether the data synchronization service receives the data replication message, to obtain a first determination result;

[0203] when the first determination result is that the data synchronization service receives the data replication message, the data synchronization service calls a data replication service;

[0204] analyzing the data replication message through the data replication service, and synchronizing each change data in the data replication message to each copy database.

[0205] The computer storage medium of the embodiment of the application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component.

[0206] The computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, in which a computer readable program code is carried. Such a propagated data signal can take on many forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can be used to carry or store a program for use by or in connection with an instruction execution system, device or component.

[0207] The program code contained on the computer readable medium can be transmitted in any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0208] Computer program code for carrying out operations of embodiments of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0209] It is to be understood that the above description is directed to the preferred embodiments and that those skilled in the art will be able to devise various modifications which, although not specifically described herein, embody the principles of the application and are included within the spirit and scope of the application. Accordingly, while the preferred embodiments have been described above, those skilled in the art will understand that they are not to be limited to the preferred embodiments, but are to include all such embodiments falling within the scope of the application as defined by the appended claims.

Claims

1. A data synchronization method, characterized in that, include: The system receives a synchronization command input from a user for data synchronization and generates a data replication message. The data replication message sequentially includes a message identifier, the time the message was generated, a modified dataset, and a set of instructions. The modified dataset includes at least one modified data item, and the instruction set includes data operation instructions corresponding to each modified data item. The order of the data operation instructions in the instruction set is the same as the order of the modified data items in the modified dataset. The message identifier, the time the data replication message was generated, the modified data items in the modified dataset, and the data operation instructions in the instruction set are sequentially connected by a preset connector. Based on the message sending mechanism of the distributed message middleware, the data replication message is sent to the data synchronization service, and it is determined whether the data synchronization service has received the data replication message to obtain a first determination result. This includes: determining whether the data synchronization service has received the data replication message based on the sending status of each data operation task in the data replication message to obtain a first determination result. When the first determination result is that the data synchronization service has not received the data copy message, it is determined that the data operation task in the data copy message with the sending status is unsuccessful. The sending status of the unsuccessful data operation task is changed from unsuccessful to sending. When the sending status of the unsuccessful data operation task is detected to be changed to sending, the unsuccessful data operation task is resent to the data synchronization service. When the first determination result indicates that the data synchronization service has received the data replication message, the data synchronization service invokes the data replication service to parse the data replication message and synchronize each changed data in the data replication message to each replica database. This includes: parsing the data replication message through the data replication service to determine each changed data in the data replication message and the data operation instructions corresponding to each changed data; generating at least one database execution statement based on each changed data and the data operation instructions corresponding to each changed data; and synchronizing each changed data in the data replication message to the replica database based on the database execution statement.

2. The method according to claim 1, characterized in that, The step of parsing the data replication message through the data replication service includes: Determine whether the data synchronization service successfully calls the data replication service to obtain a second determination result; When the second determination result indicates that the data replication service has been successfully invoked, the data replication message is parsed through the data replication service.

3. The method according to claim 1, characterized in that, The method further includes: If the number of retransmissions corresponding to the data operation task is detected to have reached a preset sending threshold, and the sending status of the data operation task is still "not sent successfully", then it is determined that the data synchronization service has not received the data replication message.

4. The method according to claim 3, characterized in that, The method further includes: If the data operation task is detected to have failed to send, the sending status of the data operation task that failed to send is marked as sending failure.

5. The method according to claim 4, characterized in that, The method further includes: When the sending status of the data operation task is detected as "sending failed" and the number of retransmissions corresponding to the data operation task reaches a preset sending threshold, the sending status of the data operation task that failed to send is changed from "sending failed" to "abandoned".

6. The method according to claim 3, characterized in that, The method further includes: If the number of resends corresponding to the data operation task does not reach the preset sending threshold, a first prompt message is generated. The first prompt message is sent to the data synchronization service so that the data synchronization service can invoke the data replication service.

7. The method according to claim 2, characterized in that, The method further includes: If the second determination result is that the data replication service was not successfully invoked, then the number of unsuccessful invocations is recorded. When it is detected that the number of unsuccessful calls has not reached the preset call count threshold, the operation of repeatedly calling the data replication service is performed, and it is determined whether the data synchronization service successfully calls the data replication service to obtain the second determination result.

8. The method according to claim 7, characterized in that, The method further includes: When the number of unsuccessful calls is detected to reach the preset call count threshold, a second prompt message is generated and displayed.

9. The method according to claim 1, characterized in that, The method includes: Determine the total number of statements to be executed in the database; When the number of completed statements in the database pending execution statements is less than the total number of database pending execution statements, the database execution statements that have not been completed are obtained; Modify data in the replica database based on database execution statements that have not yet been completed.

10. The method according to claim 9, characterized in that, The modification of data in the replica database based on database execution statements that have not yet been completed includes: Based on the database execution statements that have not been completed and the database execution statements that correspond to the database execution statements that have been completed, the data in the replica database is modified.

11. The method according to claim 10, characterized in that, The modification of data in the replica database based on database execution statements that have not yet been completed includes: Execute the data manipulation instructions corresponding to the database execution statements that were not completed, to modify the data in the replica database; or, Modify the data manipulation instructions corresponding to the database execution statements that have not been completed, and execute the modified data manipulation instructions to modify the data in the replica database.

12. The method according to claim 1, characterized in that, The method further includes: The data replication status query service is used to query the sending status of each data operation task in the data replication message, and the query results of the data replication message are obtained.

13. The method according to claim 1, characterized in that, The step of synchronizing each changed data in the data replication message to each replica database includes: Determine the replica database identifier corresponding to each changed data in the data replication message; Based on the replica database identifier corresponding to the changed data, each changed data in the data replication message is synchronized to the replica database corresponding to the changed data.

14. A data synchronization device, characterized in that, include: A data replication message generation module is used to receive synchronization instructions input by the user for data synchronization and generate a data replication message. The data replication message sequentially includes a message identifier, the time the data replication message was generated, a changed dataset, and an instruction set. The changed dataset includes at least one changed data item, and the instruction set includes data operation instructions corresponding to each changed data item. The order of the data operation instructions in the instruction set is the same as the order of the changed data items in the changed dataset. The message identifier, the time the data replication message was generated, the changed data items in the changed dataset, and the data operation instructions in the instruction set are sequentially connected by a preset connector. The first judgment result obtaining module is used to send the data replication message to the data synchronization service based on the message sending mechanism of the distributed message middleware, and to determine whether the data synchronization service has received the data replication message to obtain the first judgment result, including: determining whether the data synchronization service has received the data replication message according to the sending status of each data operation task in the data replication message to obtain the first judgment result; The data operation task resending module is used to determine, when the first judgment result is that the data synchronization service has not received the data replication message, the data operation task whose sending status in the data replication message is unsuccessful; change the sending status of the unsuccessful data operation task from unsuccessful to sending; and when the sending status of the unsuccessful data operation task is detected to be changed to sending, resend the unsuccessful data operation task to the data synchronization service. The data replication service invocation module is used to invoke the data replication service when the first determination result is that the data synchronization service has received the data replication message; The replica database synchronization module parses the data replication message through the data replication service and synchronizes each changed data in the data replication message to each replica database. This includes: parsing the data replication message through the data replication service to determine each changed data in the data replication message and the corresponding data operation instructions; generating at least one database execution statement based on the changed data and the corresponding data operation instructions; and synchronizing each changed data in the data replication message to the replica databases based on the database execution statement.

15. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the program is executed by the processor, the processor implements the data synchronization method as described in any one of claims 1-13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the data synchronization method as described in any one of claims 1-13.

Images