Distributed data synchronization system, method, device, medium and product

Abstract

The embodiment of the invention provides a distributed data synchronization system, method and device, a medium and a product. The method comprises the following steps: sending a to-be-synchronized message to a main service, an observer service and each standby service in a multicast manner through a sending service, and sending a main message serial number corresponding to the to-be-synchronized message to each standby service and the observer service by the main service, sending the message to be synchronized to each standby service, so that each standby service and the observer service determine whether the message is continuous according to the message to be synchronized and the serial number of the main message, if not, acquiring a missing incremental message from the main service, otherwise, updating respective corresponding memory databases through the main service and each standby service on the basis of the message which is confirmed to be continuous, and simultaneously, sending the message to the observer service. And sending the to-be-synchronized message to the remote disaster recovery service through the observer service, so that the remote disaster recovery service updates the corresponding memory database according to the received to-be-synchronized message. The method is used for achieving the effects of improving the data consistency of the databases in the transaction system and improving the reliability and stability of data synchronization of the transaction system.

Description

Technical Field

[0001] The present application relates to the field of securities trading technology, and in particular to a distributed data synchronization system, method, device, medium and product. Background Art

[0002] Currently, users conduct securities transactions through a trading system, and the trading system needs to promptly modify the user's basic data and / or transaction data in a database within the trading system according to the user's request.

[0003] In the prior art, a master database and a standby (or slave) database are deployed in the in-memory database of a trading system. When a user writes data to the trading system, the written data is first cached in the master database, and then the REDO log is synchronized to the standby database through a replication process to synchronize the above data to the standby database. In this way, when the master database fails, service availability is guaranteed through the standby database.

[0004] However, as the number of standby databases continues to increase and the deployment area continues to expand, relying on the primary database to synchronize data with each standby database will result in a large delay. When the primary database fails, the data consistency of each standby database will be reduced. Summary of the invention

[0005] The embodiments of the present application provide a distributed data synchronization system, method, device, medium and product to improve the data consistency of each database in a transaction system and enhance the reliability and stability of data synchronization in the transaction system.

[0006] In a first aspect, an embodiment of the present application provides a distributed data synchronization system, including:

[0007] The sending service is used to obtain the message to be synchronized, and multicast it to the main service, observer service and each backup service, so that the main service and each backup service can update the corresponding memory database according to the message to be synchronized;

[0008] The observer service is used to receive the message to be synchronized, obtain the observer message sequence number corresponding to the message to be synchronized, and receive the master message sequence number sent by the main service. The master message sequence number is determined by the main service according to the received message to be synchronized; determine whether the message is continuous according to the master message sequence number and the observer message sequence number; when it is determined to be discontinuous, generate an observer retransmission request and send it to the main service, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service, otherwise, send the message to be synchronized to the off-site disaster recovery service;

[0009] The off-site disaster recovery service is used to perform off-site persistence processing on the received messages to be synchronized, obtain the corresponding persistent messages, and update the corresponding memory database based on the persistent messages.

[0010] Optionally, the off-site disaster recovery service is also used for:

[0011] After restarting, receive the local stream instance number sent by the observer service and determine the current remote stream instance number; the local stream instance number is generated by the observer service when it is initialized, and the remote stream instance number is synchronized from the observer service to the remote disaster recovery service when the remote disaster recovery service was last started;

[0012] Determine whether the off-site flow instance number is consistent with the local flow instance number; when it is determined that the off-site flow instance number is consistent with the local flow instance number, send an incremental synchronization instruction to the observer service, so that the observer service determines the off-site incremental message according to the incremental synchronization instruction and sends it to the off-site disaster recovery service.

[0013] Optionally, the off-site disaster recovery service is also used for:

[0014] When it is determined that the remote flow instance number is inconsistent with the local flow instance number, the full message corresponding to the remote flow instance number is backed up;

[0015] When it is determined that the full message backup is completed, a full synchronization instruction is sent to the observer service, so that the observer service determines the full message corresponding to the local flow instance number according to the full synchronization instruction and sends it to the off-site disaster recovery service.

[0016] Optionally, the system further comprises:

[0017] The arbitration service is used to receive multiple master election requests sent by the primary service and the backup service at preset master election time intervals, determine the new primary service and multiple new backup services; feedback the primary response information to the new primary service, and feedback the backup response information to the multiple new backup services respectively.

[0018] Optionally, the system further comprises:

[0019] A backup arbitration service is used to receive multiple master election requests sent by the primary service and the backup service;

[0020] The master election request sent by the primary service is sent to the backup arbitration service when the primary service does not receive a response message of the previous master election request sent by the arbitration service within a preset master election time;

[0021] The master election request sent by the standby service is sent to the standby arbitration service when the standby service does not receive a response message of the previous master election request sent by the arbitration service within a preset master election time.

[0022] Optionally, the sending service is also used to multicast the last multicast message to be synchronized to the main service, backup service and observer service at preset intervals, so that the main service, backup service and observer service can determine whether the message is continuous based on the message to be synchronized after receiving the message to be synchronized.

[0023] Optionally, before updating the corresponding memory database according to the message to be synchronized, the primary service is further used to:

[0024] Determining whether the messages are continuous according to the main message sequence number of the received message to be synchronized;

[0025] When discontinuity is determined, a primary retransmission request is sent to the sending service, so that the sending service determines the corresponding incremental message according to the primary retransmission request and sends it to the primary service;

[0026] When continuity is determined, the primary message sequence number is multicasted to each backup service and observer service.

[0027] Optionally, the sending service is also used to:

[0028] When the standby service determines that the received to-be-synchronized messages are discontinuous, receiving a standby retransmission request sent by the standby service, and sending a corresponding incremental message to the standby service according to the standby retransmission request;

[0029] Correspondingly, the sending service is also used to:

[0030] When the observer service determines that the received messages to be synchronized are discontinuous, an observer retransmission request sent by the observer service is received, and a corresponding incremental message is sent to the observer service according to the observer retransmission request.

[0031] In a second aspect, an embodiment of the present application provides a distributed data synchronization method, which is applied to a sending service, including:

[0032] Obtain the message to be synchronized, and send it by multicast to the main service, observer service and each backup service, so that the main service and each backup service can update the corresponding memory database according to the message to be synchronized, and the observer service can determine whether the message is continuous according to the main message sequence number and the observer message sequence number; when it is determined to be discontinuous, generate an observer retransmission request and send it to the main service, so that the main service can determine the incremental message according to the observer retransmission request and send it to the observer service, otherwise, send the message to be synchronized to the off-site disaster recovery service, so that the off-site disaster recovery service can update the corresponding memory database based on the persistent message; wherein, the main message sequence number is sent by the main service, the observer message sequence number is determined by the observer service according to the received message to be synchronized, and the persistent message is obtained by the off-site disaster recovery service after performing off-site persistence processing on the received message to be synchronized.

[0033] Optionally, at preset intervals, the last multicast message to be synchronized is multicasted again to the main service, backup service and observer service, so that the main service, backup service and observer service can determine whether the message is continuous based on the message to be synchronized after receiving the message to be synchronized.

[0034] Optionally, the corresponding incremental message is determined based on the main retransmission request and sent to the main service. The main retransmission request is sent when the main service determines that the message is discontinuous based on the main message sequence number of the received message to be synchronized. When the main service determines that the message is continuous, the main message sequence number is multicast to each backup service and observer service.

[0035] Optionally, when the standby service determines that the received messages to be synchronized are discontinuous, a standby retransmission request sent by the standby service is received, and a corresponding standby increment message is sent to the standby service according to the standby retransmission request;

[0036] Correspondingly, when the observer service determines that the received messages to be synchronized are discontinuous, an observer retransmission request sent by the observer service is received, and a corresponding incremental message is sent to the observer service according to the observer retransmission request.

[0037] In a third aspect, an embodiment of the present application provides a distributed data synchronization method, which is applied to an observer service, including:

[0038] Receive the message to be synchronized, obtain the observer message sequence number corresponding to the message to be synchronized, and receive the master message sequence number sent by the master service, and determine whether the message is continuous according to the master message sequence number and the observer message sequence number; the master message sequence number is determined by the master service according to the received message to be synchronized, and the message to be synchronized is sent to the observer service by multicast after the sending service obtains the message to be synchronized;

[0039] When discontinuity is determined, an observer retransmission request is generated and sent to the main service, so that the main service determines the incremental message based on the observer retransmission request and sends it to the observer service. Otherwise, the message to be synchronized is sent to the off-site disaster recovery service, so that the off-site disaster recovery service updates the corresponding memory database based on the persistent message; the persistent message is obtained by the off-site disaster recovery service after performing off-site persistence processing on the received message to be synchronized.

[0040] Optionally, a re-sent message to be synchronized is received, and whether the message is continuous is determined based on the message to be synchronized, wherein the re-sent message to be synchronized is sent by the sending service at preset time intervals.

[0041] Optionally, a master message sequence number sent by a new master service is received, and a corresponding message to be synchronized is sent to a remote disaster recovery service based on the master message sequence number; wherein the new master service is determined by the arbitration service according to multiple master election requests received at every preset master election time.

[0042] In a fourth aspect, an embodiment of the present application provides a distributed data synchronization method, which is applied to a remote disaster recovery service, including:

[0043] Receive the pending synchronization message sent by the observer service. The pending synchronization message is multicasted to the observer service by the sending service after receiving the pending synchronization message;

[0044] The to-be-synchronized message is processed remotely and persisted to obtain the corresponding persistent message; wherein, the to-be-synchronized service is sent to the remote disaster recovery service by the observer service when the message continuity is determined according to the main message sequence number and the observer message sequence number, the main message sequence number is sent by the main service, and the observer message sequence number is determined by the observer service according to the received to-be-synchronized message;

[0045] Update the corresponding in-memory database based on the persistent message.

[0046] Optionally, after restarting, receive the local stream instance number sent by the observer service, and determine the current remote stream instance number; the local stream instance number is generated by the observer service when it is initialized, and the remote stream instance number is synchronized from the observer service to the remote disaster recovery service when the remote disaster recovery service was last started;

[0047] Determine whether the off-site flow instance number is consistent with the local flow instance number; when it is determined that the off-site flow instance number is consistent with the local flow instance number, send an incremental synchronization instruction to the observer service, so that the observer service determines the off-site incremental message according to the incremental synchronization instruction and sends it to the off-site disaster recovery service.

[0048] Optionally, when it is determined that the remote flow instance number is inconsistent with the local flow instance number, the full message corresponding to the remote flow instance number is backed up;

[0049] When it is determined that the full message backup is completed, a full synchronization instruction is sent to the observer service, so that the observer service determines the full message corresponding to the local flow instance number according to the full synchronization instruction and sends it to the off-site disaster recovery service.

[0050] In a fifth aspect, an embodiment of the present application provides a distributed data synchronization device, which is applied to a sending service, including:

[0051] An acquisition module is used to acquire messages to be synchronized, and multicast them to the main service, observer service, and various backup services, so that the main service and various backup services can update the corresponding memory databases according to the messages to be synchronized, and the observer service can determine whether the messages are continuous according to the main message sequence number and the observer message sequence number; when it is determined to be discontinuous, an observer retransmission request is generated and sent to the main service, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service, otherwise, the messages to be synchronized are sent to the off-site disaster recovery service, so that the off-site disaster recovery service can update the corresponding memory database based on the persistent message; wherein, the main message sequence number is sent by the main service, the observer message sequence number is determined by the observer service according to the received messages to be synchronized, and the persistent message is obtained by the off-site disaster recovery service after performing off-site persistence processing on the received messages to be synchronized.

[0052] Optionally, it also includes:

[0053] The sending module is used to multicast the last multicast message to be synchronized to the main service, backup service and observer service at preset intervals, so that the main service, backup service and observer service can determine whether the message is continuous based on the message to be synchronized after receiving it.

[0054] Optionally, the sending module is also used to determine the corresponding incremental message based on the main retransmission request and send it to the main service. The main retransmission request is sent when the main service determines that the message is discontinuous based on the main message sequence number of the received message to be synchronized. When the main service determines that the message is continuous, the main message sequence number is multicasted to each backup service and observer service.

[0055] Optionally, the acquisition module is further used to receive a standby retransmission request sent by the standby service when the standby service determines that the received message to be synchronized is discontinuous, and send a corresponding standby increment message to the standby service according to the standby retransmission request;

[0056] Correspondingly, when the observer service determines that the received messages to be synchronized are discontinuous, an observer retransmission request sent by the observer service is received, and a corresponding incremental message is sent to the observer service according to the observer retransmission request.

[0057] In a sixth aspect, an embodiment of the present application provides a distributed data synchronization device, applied to an observer service, including:

[0058] The first receiving module is used to receive the message to be synchronized, obtain the observer message sequence number corresponding to the message to be synchronized, and receive the master message sequence number sent by the master service, and determine whether the message is continuous according to the master message sequence number and the observer message sequence number; the master message sequence number is determined by the master service according to the received message to be synchronized, and the message to be synchronized is sent to the observer service by multicast after the sending service obtains the message to be synchronized;

[0059] The first processing module is used to generate an observer retransmission request and send it to the main service when discontinuity is determined, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service; otherwise, the message to be synchronized is sent to the off-site disaster recovery service, so that the off-site disaster recovery service updates the corresponding memory database based on the persistent message; the persistent message is obtained by the off-site disaster recovery service after performing off-site persistence processing on the received message to be synchronized.

[0060] Optionally, the first receiving module is further used to receive the to-be-synchronized message that is multicasted again, and determine whether the message is continuous based on the to-be-synchronized message, wherein the to-be-synchronized message that is multicasted again is the to-be-synchronized message that was multicasted last time and sent by the sending service every preset time.

[0061] Optionally, the first receiving module is also used to receive the main message sequence number sent by the new main service, and send the corresponding message to be synchronized to the off-site disaster recovery service based on the main message sequence number; wherein the new main service is determined by the arbitration service according to multiple main election requests received at every preset main election time.

[0062] In a seventh aspect, an embodiment of the present application provides a distributed data synchronization device, which is applied to a remote disaster recovery service, including:

[0063] The second receiving module is used to receive the to-be-synchronized message sent by the observer service. The to-be-synchronized message is multicasted and sent to the observer service by the sending service after the to-be-synchronized message is obtained.

[0064] The second processing module is used to perform remote persistence processing on the message to be synchronized to obtain the corresponding persistent message; wherein the service to be synchronized is sent to the remote disaster recovery service by the observer service when the message continuity is determined according to the main message sequence number and the observer message sequence number, the main message sequence number is sent by the main service, and the observer message sequence number is determined by the observer service according to the received message to be synchronized;

[0065] The second processing module is also used to update the corresponding memory database based on the persistent message.

[0066] Optionally, the second receiving module is further used to receive the local flow instance number sent by the observer service after restart, and determine the current remote flow instance number; the local flow instance number is generated by the observer service during initialization, and the remote flow instance number is synchronized from the observer service to the remote disaster recovery service when the remote disaster recovery service was last started;

[0067] The second processing module is also used to determine whether the off-site flow instance number is consistent with the local flow instance number; when it is determined that the off-site flow instance number is consistent with the local flow instance number, an incremental synchronization instruction is sent to the observer service, so that the observer service determines the off-site incremental message according to the incremental synchronization instruction and sends it to the off-site disaster recovery service.

[0068] Optionally, the second processing module is further used to back up the full message corresponding to the remote flow instance number when it is determined that the remote flow instance number is inconsistent with the local flow instance number;

[0069] The second processing module is also used to send a full synchronization instruction to the observer service when it is determined that the full message backup is completed, so that the observer service determines the full message corresponding to the local flow instance number according to the full synchronization instruction and sends it to the off-site disaster recovery service.

[0070] In an eighth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor;

[0071] Memory stores computer-executable instructions;

[0072] The processor executes the computer-executable instructions stored in the memory, so that the processor executes various possible implementations of the second to fourth aspects as described above.

[0073] In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement various possible implementation methods in the second to fourth aspects above.

[0074] In a tenth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements various possible implementation methods of the second to fourth aspects above.

[0075] The distributed data synchronization system, method, device, medium and product provided by the embodiment of the present application obtains the message to be synchronized by sending service, and multicasts it to the main service, observer service and each backup service. After the main service and each backup service respectively receive the message to be synchronized, they update their corresponding memory databases according to the message to be synchronized, determine the main message sequence number corresponding to the message to be synchronized through the main service, and multicast it to each backup service and observer service. After the observer service receives the message to be synchronized and the main message sequence number, it determines the observer message sequence number corresponding to the message to be synchronized, determines whether the message is continuous according to the main message sequence number and the observer message sequence number, and generates an observer retransmission request and sends it to the main service when it is determined to be discontinuous, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service. Otherwise, the message to be synchronized is sent to the off-site disaster recovery service, so that the off-site disaster recovery service updates the corresponding memory database according to the received message to be synchronized. The present application improves the data consistency of each database in the transaction system and improves the reliability and stability of data synchronization in the transaction system. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0077] Figure 1 A schematic diagram of the prior art scenario provided for this application;

[0078] Figure 2 Schematic diagram of the structure of the distributed data synchronization system provided for this application Figure 1 ;

[0079] Figure 3 Schematic diagram of the structure of the distributed data synchronization system provided for this application Figure 2 ;

[0080] Figure 4A flow chart of the distributed data synchronization method provided in this application;

[0081] Figure 5 A schematic diagram of the structure of the distributed data synchronization device provided in this application Figure 1 ;

[0082] Figure 6 A schematic diagram of the structure of the distributed data synchronization device provided in this application Figure 2 ;

[0083] Figure 7 A schematic diagram of the structure of the distributed data synchronization device provided in this application Figure 3 ;

[0084] Figure 8 A schematic diagram of the structure of the electronic device provided in this application.

[0085] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0086] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0087] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0088] First, the terms involved in this application are explained:

[0089] Multicast: refers to a one-to-many communication mode between hosts. By adding all receivers that need to receive multicast messages to the same multicast address, one or more multicast sources can send a message to the multicast address, thus sending the same message to multiple receivers.

[0090] In-memory database: refers to a database that stores data in the system memory and can directly perform data operations. The data access rate based on the in-memory database is faster;

[0091] Observer retransmission request: refers to the retransmission request initiated by the observer service;

[0092] Backup retransmission request: refers to the retransmission request initiated by the backup service;

[0093] Primary retransmission request: refers to the retransmission request initiated by the primary service;

[0094] Remote incremental message: refers to the incremental message sent to the remote disaster recovery service.

[0095] Figure 1 The schematic diagram of the prior art scenario provided for this application is as follows: Figure 1 As shown, the specific application scenario of the prior art is: the user writes data to the memory database of the trading system through the application software of the terminal device, and the memory database includes a database corresponding to the main service and databases corresponding to each backup service. By writing data to the database corresponding to the main service, after the main service caches the written data, the data is multicasted to each backup service, so that when each backup service receives the above data, the data is saved in the corresponding database, thereby realizing data synchronization of each backup service database.

[0096] However, when there are many backup services and / or the backup service deployment area is different from the primary service deployment area, it is easy to increase the delay of the primary service to synchronize data with the databases of each backup service, reduce the efficiency of data synchronization, and thus when users read data through application software, it is difficult to provide accurate services to users in a timely manner due to the delay in data synchronization. In addition, when the database of the primary service fails, it is easy to affect the data synchronization process, thereby further reducing the data consistency between the backup service databases.

[0097] The distributed data synchronization method provided by the present application obtains the message to be synchronized by sending the service, and sends it to the main service, observer service and each backup service in a multicast manner. The main service determines whether the main message sequence number of the message to be synchronized is continuous. If it is continuous, the main message sequence number is multicast to each backup service and observer service. The message sequence number corresponding to the obtained message to be synchronized is determined by each backup service and observer service respectively, and whether the message is continuous is determined together with the main message sequence number. When it is determined to be continuous, the obtained message to be synchronized is stored in the corresponding memory database through each backup service, and the main service and each backup service send a master election request to the arbitration service at a preset master election time to avoid failure to timely discover the impact on data synchronization when the main service has a fault. During the continuous switching of the main service, the observer service can send the received message to be synchronized to the off-site disaster recovery service, so that the off-site disaster recovery service will persist the message to be synchronized and store it in the corresponding memory database, thereby realizing data synchronization of off-site disaster recovery services, reducing the delay of cross-domain synchronization, enhancing data consistency, and improving the reliability and stability of data synchronization.

[0098] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0099] Figure 2 Schematic diagram of the structure of the distributed data synchronization system provided for this application Figure 1 ,like Figure 2 As shown, the system includes: a sending service, a main service, multiple backup services, an observer service, and a remote disaster recovery service. The sending service is used to obtain the message to be synchronized, and multicast it to the main service, the observer service, and each backup service, so that the main service and each backup service update the corresponding memory database according to the message to be synchronized; the observer service is used to receive the message to be synchronized, obtain the observer message sequence number corresponding to the message to be synchronized, and receive the main message sequence number sent by the main service, and the main message sequence number is determined by the main service according to the received message to be synchronized; determine whether the message is continuous according to the main message sequence number and the observer message sequence number; when it is determined to be discontinuous, generate an observer retransmission request and send it to the main service, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service, otherwise, send the message to be synchronized to the remote disaster recovery service; the remote disaster recovery service is used to perform remote persistence processing on the received message to be synchronized, obtain the corresponding persistent message, and update the corresponding memory database based on the persistent message. Among them, the memory database of the embodiment of the present application is a distributed memory database, and data synchronization is achieved based on the distributed memory database, which improves the rate of data storage and subsequent data access.

[0100] Optionally, after receiving the message to be sent, the main service updates the memory database corresponding to the main service according to the message to be sent. By determining the message sequence number corresponding to the received message to be sent, the main message sequence number is obtained and multicasted to each backup service and observer service.

[0101] Optionally, when the observer service receives the to-be-synchronized message sent by the sending service for the first time through multicast, it obtains the observer message sequence number by determining the message sequence number corresponding to the received to-be-synchronized message. After the observer service receives the master message sequence number sent by the master service through multicast, it determines the message continuity based on the observer message sequence number and the master message sequence number.

[0102] In a possible embodiment, if the observer service receives a main message sent by the main service multicast with sequence number 10, and the observer message sequence number is sequence number 10, the message continuity is determined, and a sort confirmation message is sent to the main service to notify the main service that the message with sequence number 10 has been successfully synchronized.

[0103] In a possible embodiment, if the observer service has not received the to-be-synchronized message sent by the sending service multicast, and the received main message sent by the main service multicast is number 10, the current observer message sequence number is obtained based on the largest message sequence number in the currently cached synchronization message. When the observer message sequence number is sequence number 9, it is determined that the message is discontinuous, and an observer retransmission request corresponding to sequence number 9 is generated and sent to the main service, so that the main service determines the incremental message (the message corresponding to sequence number 10) that the observer service requests to retransmit based on the observer retransmission request, and retransmits the message to the above-mentioned observer service.

[0104] In a possible embodiment, when the observer service determines that the received messages to be synchronized are continuous, the memory database corresponding to the observer service is updated according to the messages to be synchronized.

[0105] Alternatively, if Figure 2 As shown, the observer service compresses the synchronization messages in batches and transmits them to the off-site disaster recovery service in the off-site disaster recovery computer room via TCP (Transmission Control Protocol), so that the off-site disaster recovery can decompress the received compressed packages, persist them to the off-site disk, and update the corresponding memory database based on the persistent messages in the off-site disk.

[0106] Optionally, the main service, each backup service and the observer service are deployed in the main computer room, and the remote disaster recovery service is deployed in the remote disaster recovery room. The remote disaster recovery room can be deployed in the same area as the main computer room, or in a different area from the main computer room. The embodiments of the present application do not limit this. Among them, the remote disaster recovery room can include one remote disaster recovery service, or multiple remote disaster recovery services. The embodiments of the present application do not limit this. In the embodiments, any remote disaster recovery service is used to illustrate the data synchronization method of the remote disaster recovery room.

[0107] Optionally, when any backup service receives the to-be-synchronized message sent by the first multicast of the sending service, the backup message sequence number is obtained by determining the message sequence number corresponding to the received to-be-synchronized message. After the backup service receives the main message sequence number sent by the main service multicast, the message continuity is determined based on the backup message sequence number and the main message sequence number.

[0108] In a possible embodiment, if the backup service receives a main message sent by the primary service multicast with sequence number 10, and the backup message sequence number is sequence number 10, the message continuity is determined, and a sort confirmation message is sent to the primary service to notify the primary service that the message with sequence number 10 is synchronized successfully.

[0109] In a possible embodiment, if the backup service does not receive the message to be synchronized sent by the sending service multicast, and receives the main message sent by the main service multicast with sequence number 10, the current backup message sequence number is obtained based on the largest message sequence number in the currently cached synchronization message. If the backup message sequence number is sequence number 9, it is determined that the message is discontinuous, and sequence number 9 is sent to the main service, so that the main service determines that the message missing from the backup service is the message corresponding to sequence number 10 based on sequence number 9, and retransmits the message to the above-mentioned backup service.

[0110] The main message sequence number in the present application is the message sequence number obtained by the main service based on the received message, the backup message sequence number is the message sequence number obtained by the backup service based on the received message, and the observer message sequence number is the message sequence number obtained by the observer service based on the received message.

[0111] This embodiment multicasts the message to be synchronized to the main service, each backup service, and the observer service through the sending service, so that after each backup service and the observer service confirm the message continuity with the main service, they update the memory database corresponding to each backup service according to the message to be synchronized, and the observer service transmits the message to be synchronized to the off-site disaster recovery service, so that the off-site disaster recovery service updates the corresponding memory database according to the message to be synchronized, thereby realizing data synchronization of the memory databases corresponding to each backup service and the off-site disaster recovery service, and when there is a message missing, timely ensuring data consistency by retransmitting the message, thereby improving the reliability of data synchronization of the memory database of the off-site disaster recovery service.

[0112] Optionally, the sending service is also used to multicast the last multicast message to be synchronized to the main service, backup service and observer service at preset intervals, so that the main service, backup service and observer service can determine whether the message is continuous based on the message to be synchronized after receiving the message to be synchronized.

[0113] In a possible embodiment, the sending service sends the most recently sent message to be synchronized to the main service, observer service and each backup service at preset intervals, so that the main service, observer service and each backup service can determine the continuity of the message based on the above-mentioned message to be synchronized sent again.

[0114] Exemplarily, for the observer service, the largest message sequence number cached in the memory database corresponding to the observer service is sequence number 9. After a preset time, the to-be-synchronized message sent by the sending service for the second time is received, the message sequence number corresponding to the to-be-synchronized message is determined, and the observer message sequence number is obtained as sequence number 10. When the main message sequence number (sequence number 10) sent by the main service is received, the message continuity is determined.

[0115] Exemplarily, for any backup service, when the largest message sequence number cached in the memory database corresponding to the backup service is sequence number 9, after a preset time, a message to be synchronized sent by the sending service for the second time is received, and the message sequence number corresponding to the message to be synchronized is determined, and the backup message sequence number is obtained as sequence number 10. When the main message sequence number (sequence number 10) sent by the main service is received, it is determined that the messages are continuous.

[0116] This embodiment resends the most recently sent message to be synchronized to each backup service and observer service through the sending service at preset intervals, so that when the backup service or observer service fails to effectively receive the message to be synchronized sent for the first time, the missing data can be supplemented in time, thereby reducing the data synchronization delay caused by initiating retransmission instructions to the main service and improving data synchronization efficiency.

[0117] Optionally, before updating the corresponding memory database according to the message to be synchronized, the main service is also used to: determine whether the message is continuous based on the main message sequence number of the received message to be synchronized; when it is determined to be discontinuous, send a main retransmission request to the sending service, so that the sending service determines the corresponding incremental message according to the main retransmission request and sends it to the main service; when it is determined to be continuous, multicast the main message sequence number to each backup service and observer service.

[0118] In a possible embodiment, after receiving the message to be synchronized sent by the sending service, the main service determines the main message sequence number corresponding to the message to be synchronized, and determines whether the main message sequence number is continuous with the received historical message sequence number. When it is determined to be discontinuous, a main retransmission request is generated and sent to the sending service, so that the sending service determines the missing incremental message according to the main retransmission request, and sends the incremental message to the main service. When it is determined to be continuous, the main message sequence number corresponding to the message to be synchronized is multicasted and sent to each backup service and observer service.

[0119] Exemplarily, the main message sequence number is received through the backup service, and the backup message sequence number of the message to be synchronized sent by the sending service is determined, so as to further determine whether the backup message sequence number and the main message sequence number are consistent. When it is determined to be consistent, it is determined that the received message to be synchronized is continuous, and the message is stored in the corresponding memory database; when it is determined to be inconsistent, it is determined that the received message to be synchronized is discontinuous, and a backup retransmission request is generated by the backup service according to the missing message sequence number and sent to the sending service.

[0120] Exemplarily, the main message sequence number is received through the observer service, and the observer message sequence number of the message to be synchronized sent by the sending service is determined, so as to further determine whether the observer message sequence number and the main message sequence number are consistent. When it is determined to be consistent, it is determined that the received message to be synchronized is continuous, and the message is stored in the corresponding memory database; when it is determined to be inconsistent, it is determined that the received message to be synchronized is discontinuous, and an observer retransmission request is generated by the observer service according to the missing message sequence number and sent to the sending service.

[0121] Optionally, the sending service is also used to: when the standby service determines that the received message to be synchronized is discontinuous, receive a standby retransmission request sent by the standby service, and send a corresponding standby incremental message to the standby service according to the standby retransmission request; accordingly, the sending service is also used to: when the observer service determines that the received message to be synchronized is discontinuous, receive an observer retransmission request sent by the observer service, and send a corresponding incremental message to the observer service according to the observer retransmission request.

[0122] This embodiment ensures the message continuity of the main service through the interaction between the main service and the sending service, and further multicasts the message sequence number to each backup service and observer service through the main service. When the backup service or observer service determines that the received message to be synchronized is discontinuous, it initiates a retransmission request to the sending service to obtain the corresponding incremental message from the sending service, thereby increasing the data synchronization method, improving the reliability and accuracy of data synchronization for distributed databases, and further improving data synchronization efficiency.

[0123] Optionally, the distributed data synchronization system includes an arbitration service and a backup arbitration service.

[0124] Optionally, the arbitration service is used to receive multiple master election requests sent by the primary service and the backup service at preset master election time intervals, determine the new primary service and multiple new backup services; feedback the primary response information to the new primary service, and feedback the backup response information to the multiple new backup services respectively; the observer service is also used to receive the primary message sequence number sent by the new primary service, and send the corresponding to-be-synchronized message to the remote disaster recovery service based on the primary message sequence number. The primary response information is used to feedback the information of successful master election to the new primary service, and the backup response information is used to feedback the information of failed master election to each new backup service.

[0125] Exemplarily, each backup service and active service sends a master election request to the arbitration service at a preset master election time, so that the arbitration service determines the first master election request received and determines the service that sends the first master election request as the new master service.

[0126] In one possible embodiment, Figure 3 Schematic diagram of the structure of the distributed data synchronization system provided for this application Figure 2 ,like Figure 3 As shown, the main service and each backup service send a master election request to the arbitration service respectively. If the first master election request received by the arbitration service is sent by the original backup service, the original backup service is determined as the new main service, and the main response information is fed back to the new main service. At the same time, the original main service is determined as the new backup service, and the backup response information is fed back to the new backup service and the remaining backup services. In this embodiment, the arbitration service switches the main service according to the data transmission rate of the current main service and each backup service at every preset master election time to ensure a faster transmission rate of the main service, reduce the time for the arbitration service to judge whether the main service and each backup service are abnormal, and avoid the possibility of service switching errors due to misjudgment, improve the stability of data synchronization, and further enhance data consistency.

[0127] Optionally, the standby arbitration service is used to receive multiple master election requests sent by the main service and the standby service, and determine a new main service and multiple new standby services; wherein the master election request sent by the main service is sent to the standby arbitration service when the main service has not received a response message of the previous master election request sent by the arbitration service within a preset master election time; the standby service is sent to the standby arbitration service when the standby service has not received a response message of the previous master election request sent by the arbitration service within a preset master election time.

[0128] In a possible embodiment, Figure 3As shown, after the preset master selection time, after the new standby service, the new master service and each standby service respectively send a master selection request to the arbitration service, if the response information fed back by the arbitration service is not received within the preset master selection time, the master selection request is sent to the standby arbitration service, so that the standby arbitration service determines the next new master service and multiple new standby services, and executes the corresponding master service switching. In this embodiment, when the standby service or the master service does not receive the response information fed back normally by the arbitration service, it is determined that the arbitration service is abnormal. At this time, the recipient of the master selection request is switched to the standby arbitration service, thereby improving the reliability of data synchronization.

[0129] Exemplarily, the data transmission mode in the process of the primary service and each backup service and the arbitration service selecting a primary response is unicast transmission. The transmission mode when the primary service transmits the primary message sequence number to each backup service and the observer service is multicast transmission.

[0130] Alternatively, if Figure 3 As shown, the observer service does not participate in the master service election and response process of the arbitration service and / or the backup arbitration service. During the master service and each backup service's election and response process with the arbitration service, the observer service determines whether the message is continuous based on the sequence number of the main message sent by the new main service multicast. When it is determined that the message is continuous, it will synchronously transmit the message to be synchronized sent by the sending service multicast to the off-site disaster recovery service.

[0131] Optionally, the off-site disaster recovery service is also used to: after restart, receive the local flow instance number sent by the observer service, and determine the current off-site flow instance number; the local flow instance number is generated by the observer service during initialization, and the off-site flow instance number is synchronized to the off-site disaster recovery service by the observer service when the off-site disaster recovery service was last started; determine whether the off-site flow instance number is consistent with the local flow instance number; when it is determined that the off-site flow instance number is consistent with the local flow instance number, send an incremental synchronization instruction to the observer service, so that the observer service determines the off-site incremental message according to the incremental synchronization instruction and sends it to the off-site disaster recovery service.

[0132] Exemplarily, the observer service of the main computer room is initialized when the next trading day arrives, and after initialization, the local flow instance number corresponding to the next trading day is persisted to the local disk. After the remote disaster recovery service of the remote disaster recovery room is started, the current instance number is synchronized from the observer service, and the synchronized instance number is persisted as the remote flow instance number to the corresponding remote disk.

[0133] In a possible embodiment, when the off-site disaster recovery service determines that the off-site flow instance number is consistent with the local flow instance number, it determines the target historical message sequence number based on the message sequence number corresponding to the previous persistent message, thereby obtaining the incremental synchronization instruction corresponding to the target historical message sequence number, and sends it to the observer service, so that the observer service determines the off-site incremental message corresponding to the above-mentioned target historical message sequence number according to the incremental synchronization instruction, and sends it to the off-site disaster recovery service.

[0134] Optionally, the off-site disaster recovery service is also used to: when it is determined that the off-site flow instance number is inconsistent with the local flow instance number, back up the full message corresponding to the off-site flow instance number; when it is determined that the full message backup is completed, send a full synchronization instruction to the observer service, so that the observer service determines the full message corresponding to the local flow instance number according to the full synchronization instruction, and sends it to the off-site disaster recovery service.

[0135] Exemplarily, when the off-site disaster recovery service determines that the off-site flow instance number is inconsistent with the local flow instance number, it backs up the full message under the off-site flow instance number to the backup file, and after the backup is completed, sends a full synchronization instruction to the observer service.

[0136] Optionally, the off-site disaster recovery service initializes the backup file once every preset backup time.

[0137] In the process of periodic election switching of the main service in this embodiment, the observer service is used to retransmit data to the restarted off-site disaster recovery service, so that the switching of the main service in the main computer room does not affect the data synchronization of the off-site disaster recovery service, reducing the latency of cross-domain synchronization of data, and when the off-site disaster recovery service is restored after an abnormality, the off-site disaster recovery service adaptively communicates with the observer service to complete data synchronization, further improving the efficiency of cross-domain synchronization, enhancing data consistency across memory databases, and reducing the operation and maintenance costs of the transaction system.

[0138] In one possible embodiment, Figure 4 A flow chart of the distributed data synchronization method provided in this application is as follows: Figure 4 As shown, in this embodiment Figure 3 Based on the embodiment, a distributed data synchronization method when an arbitration service and / or a backup arbitration service switches to a primary service is described in detail, and the method includes:

[0139] S401. The remote disaster recovery service loads the persistent remote flow instance number when restarting.

[0140] More specifically, when the off-site disaster recovery service is restarted due to a failure, the off-site stream instance number persisted to the off-site disk is loaded.

[0141] S402. The remote disaster recovery service sends the remote flow instance number to the observer service.

[0142] More specifically, the off-site disaster recovery service of the off-site disaster recovery computer room sends the off-site flow instance number to the observer service of the main computer room.

[0143] S403: The observer service sends the local stream instance number to the remote disaster recovery service.

[0144] More specifically, after receiving the remote flow instance number sent by the remote disaster recovery service, the observer service sends the local flow instance number to the remote disaster recovery service. The local flow instance number is used to indicate the current trading day; the remote flow instance number is used to indicate the trading day before the remote disaster recovery service has an abnormality, which can be the current trading day or the previous trading day.

[0145] S404: The remote disaster recovery service determines whether the remote flow instance number is consistent with the local flow instance number.

[0146] More specifically, the remote disaster recovery service determines whether the remote flow instance number is consistent with the local flow instance number. For example, if the remote disaster recovery service determines that the remote flow instance number is inconsistent with the local flow instance number, it is determined that the remote disaster recovery service is synchronizing the message of the previous trading day when the exception occurs, otherwise, it is determined that the remote disaster recovery service is synchronizing the message of the current trading day when the exception occurs.

[0147] S405. When the remote disaster recovery service determines that the data are consistent, the remote disaster recovery service sends an incremental synchronization instruction to the observer service.

[0148] More specifically, the off-site disaster recovery service determines the last persistently processed message, obtains the message sequence number corresponding to the message, that is, the target historical message sequence number, generates an incremental synchronization instruction based on the target historical message sequence number, and sends it to the observer service.

[0149] S406. The observer service determines the remote incremental message according to the incremental synchronization instruction.

[0150] More specifically, the observer service determines the messages that failed to be synchronized to the off-site disaster recovery service based on the incremental synchronization instruction, thereby obtaining off-site incremental messages.

[0151] S407. The observer service sends an off-site incremental message to the off-site disaster recovery service.

[0152] More specifically, the observer service compresses the remote incremental messages in batches and sends the compressed packets to the remote disaster recovery service via TCP transmission.

[0153] S408. The off-site disaster recovery service updates the corresponding memory database according to the off-site incremental message.

[0154] More specifically, the off-site disaster recovery service updates the memory database corresponding to the off-site disaster recovery service according to the off-site incremental message.

[0155] The distributed data synchronization method provided in the embodiment of the present application retransmits data to the restarted off-site disaster recovery service through the observer service during the periodic switching of the main service in the host room, thereby realizing regional isolation between the host room and the off-site disaster recovery room. The switching of the main service in the host room does not affect the data synchronization of the off-site disaster recovery service, thereby reducing the latency of cross-domain synchronization. The observer service adaptively determines the consistency of the off-site stream instance number and the local stream instance number to trigger the incremental synchronization or full synchronization process, thereby effectively improving the efficiency of cross-domain synchronization of data, improving the data consistency of the in-memory database of the trading system, and reducing the operation and maintenance cost of the trading system.

[0156] Figure 5 A schematic diagram of the structure of the distributed data synchronization device provided in this application Figure 1 , used for sending services, such as Figure 5 As shown, the distributed data synchronization device 50 provided in this embodiment includes:

[0157] The acquisition module 501 is used to acquire the message to be synchronized, and multicast it to the main service, the observer service and each backup service, so that the main service and each backup service can update the corresponding memory database according to the message to be synchronized, and the observer service can determine whether the message is continuous according to the main message sequence number and the observer message sequence number; when it is determined to be discontinuous, an observer retransmission request is generated and sent to the main service, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service, otherwise, the message to be synchronized is sent to the off-site disaster recovery service, so that the off-site disaster recovery service updates the corresponding memory database based on the persistent message; wherein, the main message sequence number is sent by the main service, the observer message sequence number is determined by the observer service according to the received message to be synchronized, and the persistent message is obtained by the off-site disaster recovery service after performing off-site persistence processing on the received message to be synchronized.

[0158] Optionally, it also includes:

[0159] The sending module 502 is used to multicast the synchronization message sent last time to the main service, backup service and observer service at preset intervals, so that the main service, backup service and observer service can determine whether the message is continuous based on the synchronization message after receiving the synchronization message.

[0160] Optionally, the sending module 502 is also used to determine the corresponding incremental message based on the main retransmission request and send it to the main service. The main retransmission request is sent when the main service determines that the message is discontinuous based on the main message sequence number of the received message to be synchronized. When the main service determines that the message is continuous, the main message sequence number is multicasted to each backup service and observer service.

[0161] Optionally, the acquisition module 501 is further configured to receive a standby retransmission request sent by the standby service when the standby service determines that the received message to be synchronized is discontinuous, and send a corresponding standby increment message to the standby service according to the standby retransmission request;

[0162] Correspondingly, when the observer service determines that the received messages to be synchronized are discontinuous, an observer retransmission request sent by the observer service is received, and a corresponding incremental message is sent to the observer service according to the observer retransmission request.

[0163] The distributed data synchronization device provided in this embodiment is applied to sending services and can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0164] Figure 6 A schematic diagram of the structure of the distributed data synchronization device provided in this application Figure 2 , applied to observer services, such as Figure 6 As shown, the distributed data synchronization device 60 provided in this embodiment includes:

[0165] The first receiving module 601 is used to receive the message to be synchronized, obtain the observer message sequence number corresponding to the message to be synchronized, and receive the master message sequence number sent by the master service, and determine whether the message is continuous according to the master message sequence number and the observer message sequence number; the master message sequence number is determined by the master service according to the received message to be synchronized, and the message to be synchronized is sent by the sending service to the observer service by multicast after obtaining the message to be synchronized;

[0166] The first processing module 602 is used to generate an observer retransmission request and send it to the main service when discontinuity is determined, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service; otherwise, the message to be synchronized is sent to the off-site disaster recovery service, so that the off-site disaster recovery service updates the corresponding memory database based on the persistent message; the persistent message is obtained by the off-site disaster recovery service after performing off-site persistence processing on the received message to be synchronized.

[0167] Optionally, the first receiving module 601 is also used to receive the to-be-synchronized message that is multicasted again, and determine whether the message is continuous based on the to-be-synchronized message, wherein the to-be-synchronized message that is multicasted again is the to-be-synchronized message that was multicasted last time and sent by the sending service every preset time.

[0168] Optionally, the first receiving module 601 is also used to receive the main message sequence number sent by the new main service, and send the corresponding message to be synchronized to the off-site disaster recovery service based on the main message sequence number; wherein the new main service is determined by the arbitration service according to multiple main election requests received at every preset main election time.

[0169] The distributed data synchronization device provided in this embodiment is applied to the observer service and can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar and will not be described in detail in this embodiment.

[0170] Figure 7 A schematic diagram of the structure of the distributed data synchronization device provided in this application Figure 3 , applied to off-site disaster recovery services, such as Figure 7 As shown, the distributed data synchronization device 70 provided in this embodiment includes:

[0171] The second receiving module 701 is used to receive the to-be-synchronized message sent by the observer service. The to-be-synchronized message is multicasted and sent to the observer service by the sending service after obtaining the to-be-synchronized message.

[0172] The second processing module 702 is used to perform remote persistence processing on the message to be synchronized to obtain a corresponding persistent message; wherein the service to be synchronized is sent to the remote disaster recovery service by the observer service when the message continuity is determined according to the main message sequence number and the observer message sequence number, the main message sequence number is sent by the main service, and the observer message sequence number is determined by the observer service according to the received message to be synchronized;

[0173] The second processing module 702 is further configured to update the corresponding memory database based on the persistent message.

[0174] Optionally, the second receiving module 701 is further used to receive the local flow instance number sent by the observer service after restart, and determine the current remote flow instance number; the local flow instance number is generated by the observer service when it is initialized, and the remote flow instance number is synchronized from the observer service to the remote disaster recovery service when the remote disaster recovery service was last started;

[0175] The second processing module 702 is also used to determine whether the off-site flow instance number is consistent with the local flow instance number; when it is determined that the off-site flow instance number is consistent with the local flow instance number, an incremental synchronization instruction is sent to the observer service, so that the observer service determines the off-site incremental message according to the incremental synchronization instruction and sends it to the off-site disaster recovery service.

[0176] Optionally, the second processing module 702 is further configured to back up the full message corresponding to the remote flow instance number when it is determined that the remote flow instance number is inconsistent with the local flow instance number;

[0177] The second processing module 702 is also used to send a full synchronization instruction to the observer service when it is determined that the full message backup is completed, so that the observer service determines the full message corresponding to the local flow instance number according to the full synchronization instruction and sends it to the off-site disaster recovery service.

[0178] The distributed data synchronization device provided in this embodiment is applied to off-site disaster recovery services and can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0179] Figure 8 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 8 As shown, the electronic device 80 provided in this embodiment includes: at least one processor 801 and a memory 802. Optionally, the device 80 also includes a communication component 803. The processor 801, the memory 802 and the communication component 803 are connected via a bus 804.

[0180] In a specific implementation process, at least one processor 801 executes the computer-executable instructions stored in the memory 802, so that at least one processor 801 executes the above method.

[0181] The specific implementation process of the processor 801 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0182] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0183] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0184] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0185] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0186] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0187] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0188] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0189] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0190] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0191] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0192] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0193] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0194] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A distributed data synchronization system, characterized in that: include: A sending service is used to obtain the message to be synchronized, and multicast it to the main service, the observer service and each backup service, so that the main service and each backup service update the corresponding memory database according to the message to be synchronized; An observer service, configured to receive the message to be synchronized, obtain an observer message sequence number corresponding to the message to be synchronized, and receive a master message sequence number sent by the master service, wherein the master message sequence number is determined by the master service according to the received message to be synchronized; Determine whether the message is continuous according to the master message sequence number and the observer message sequence number; when it is determined to be discontinuous, generate an observer retransmission request and send it to the master service, so that the master service determines the incremental message according to the observer retransmission request and sends it to the observer service, otherwise, send the message to be synchronized to the off-site disaster recovery service; The off-site disaster recovery service is used to perform off-site persistence processing on the received messages to be synchronized, obtain corresponding persistent messages, and update the corresponding memory database based on the persistent messages.

2. The system according to claim 1, characterized in that The off-site disaster recovery service is also used for: After restarting, receiving the local stream instance number sent by the observer service, and determining the current remote stream instance number; the local stream instance number is generated by the observer service when it is initialized, and the remote stream instance number is synchronized by the observer service to the remote disaster recovery service when the remote disaster recovery service was last started; Determine whether the off-site flow instance number is consistent with the local flow instance number; when it is determined that the off-site flow instance number is consistent with the local flow instance number, send an incremental synchronization instruction to the observer service, so that the observer service determines the off-site incremental message according to the incremental synchronization instruction and sends it to the off-site disaster recovery service.

3. The system according to claim 2, characterized in that The off-site disaster recovery service is also used for: When it is determined that the remote flow instance number is inconsistent with the local flow instance number, backing up the full message corresponding to the remote flow instance number; When it is determined that the full message backup is completed, a full synchronization instruction is sent to the observer service, so that the observer service determines the full message corresponding to the local flow instance number according to the full synchronization instruction, and sends it to the off-site disaster recovery service.

4. The system according to any one of claims 1 to 3, characterized in that: The system further comprises: The arbitration service is used to receive multiple master election requests sent by the primary service and the backup service at preset master election time intervals, determine a new primary service and multiple new backup services; feedback primary response information to the new primary service, and feedback backup response information to the multiple new backup services respectively; The observer service is also used to receive the master message sequence number sent by the new master service, and send the corresponding message to be synchronized to the off-site disaster recovery service based on the master message sequence number.

5. The system according to claim 4, characterized in that The system further comprises: A backup arbitration service, used to receive multiple master election requests sent by the primary service and the backup service, and determine a new primary service and multiple new backup services; The master election request sent by the primary service is sent to the backup arbitration service by the primary service when the primary service does not receive a response message of the previous master election request sent by the arbitration service within a preset master election time; The master election request sent by the standby service is sent to the standby arbitration service when the standby service does not receive response information of the previous master election request sent by the arbitration service within a preset master election time.

6. The system according to claim 1, characterized in that The sending service is also used to multicast the last multicast message to be synchronized to the main service, the backup service and the observer service at preset intervals, so that the main service, the backup service and the observer service can determine whether the message is continuous based on the message to be synchronized after receiving it.

7. The system according to claim 6, characterized in that Before the main service updates the corresponding memory database according to the message to be synchronized, the main service is further used to: Determining whether the messages are continuous according to the received main message sequence number of the message to be synchronized; When discontinuity is determined, sending a primary retransmission request to the sending service, so that the sending service determines a corresponding incremental message according to the primary retransmission request and sends it to the primary service; When continuity is determined, the primary message sequence number is multicasted and sent to each backup service and observer service.

8. The system according to claim 7, characterized in that The sending service is also used to: When the standby service determines that the received messages to be synchronized are discontinuous, receiving a standby retransmission request sent by the standby service, and sending a corresponding incremental message to the standby service according to the standby retransmission request; Accordingly, the sending service is also used for: When the observer service determines that the received messages to be synchronized are discontinuous, an observer retransmission request sent by the observer service is received, and a corresponding incremental message is sent to the observer service according to the observer retransmission request.

9. A distributed data synchronization method, characterized in that: Applicable to delivery services, including: Obtain the message to be synchronized, and send it by multicast to the main service, observer service and each backup service, so that the main service and each backup service update the corresponding memory database according to the message to be synchronized, and the observer service determines whether the message is continuous according to the main message sequence number and the observer message sequence number; when it is determined to be discontinuous, generate an observer retransmission request and send it to the main service, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service, otherwise, send the message to be synchronized to the off-site disaster recovery service, so that the off-site disaster recovery service updates the corresponding memory database based on the persistent message; wherein the main message sequence number is sent by the main service, the observer message sequence number is determined by the observer service according to the received message to be synchronized, and the persistent message is obtained by the off-site disaster recovery service after performing off-site persistence processing on the received message to be synchronized.

10. A distributed data synchronization method, characterized in that: Applies to observer services, including: Receive a message to be synchronized, obtain the observer message sequence number corresponding to the message to be synchronized, and receive the master message sequence number sent by the master service, and determine whether the message is continuous according to the master message sequence number and the observer message sequence number; the master message sequence number is determined by the master service according to the received message to be synchronized, and the message to be synchronized is multicasted and sent to the observer service by the sending service after obtaining the message to be synchronized; When discontinuity is determined, an observer retransmission request is generated and sent to the main service, so that the main service determines the incremental message according to the observer retransmission request and sends it to the observer service; otherwise, the message to be synchronized is sent to the off-site disaster recovery service, so that the off-site disaster recovery service updates the corresponding memory database based on the persistent message; the persistent message is obtained by the off-site disaster recovery service after performing off-site persistence processing on the received message to be synchronized.

11. A distributed data synchronization method, characterized in that: Applied to off-site disaster recovery services, including: Receiving a to-be-synchronized message sent by an observer service, wherein the to-be-synchronized message is multicasted and sent to the observer service by a sending service after the sending service obtains the to-be-synchronized message; The message to be synchronized is processed remotely and persistently to obtain a corresponding persistent message; wherein the service to be synchronized is sent to the remote disaster recovery service by the observer service when the message continuity is determined according to the main message sequence number and the observer message sequence number, the main message sequence number is sent by the main service, and the observer message sequence number is determined by the observer service according to the received message to be synchronized; The corresponding in-memory database is updated based on the persistent message.

12. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 9 to 11.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 9 to 11 when executed by a processor.

14. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 9 to 11 when being executed by a processor.

Images