Data synchronization method, database system, and device

Abstract

The present disclosure provides a data synchronization method, a database system and an apparatus. The system comprises a master, at least one first backup and at least one second backup. The master is configured to update data on the master by executing events. The first backup is configured to synchronize the data from the master in real time by executing the same events as the master. The method comprises: obtaining, by the second backup, an identifier of a first event at a second time, the second time being later than a first time at which the master executes the first event, and the second time being separated from the first time by a preset time length; and executing, by the second backup, a second event according to the identifier of the first event, the second event comprising the first event and events executed by the master before the first event. The present disclosure can use delayed synchronization in a second backup, that is, after the master updates data, the second backup does not synchronize the data immediately, but synchronizes the data after a certain time. In this way, the number of restarts of the second backup can be reduced, and the load of the second backup is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of financial technology, and in particular to a data synchronization method, database system, and device. Background Technology

[0002] In the field of financial technology (Fintech), financial data typically needs to be stored in a database system. Database systems are usually distributed systems, including a primary server and at least one backup server, with failover between them. The primary server stores data and provides data services to users, while the backup server synchronizes data from the primary server, serving as a backup.

[0003] During the process of the standby machine synchronizing data from the master machine, how to reduce the load on the standby machine is an urgent problem to be solved. Summary of the Invention

[0004] This disclosure provides a data synchronization method, database system, and device to reduce the load on the standby machine.

[0005] In a first aspect, this disclosure provides a data synchronization method applied to a database system, the database system including a host, at least one first standby machine, and at least one second standby machine, the host being used to update data on the host by executing events, and the first standby machine being used to synchronize data from the host in real time by executing the same events as the host, the method comprising:

[0006] The second standby machine acquires the identifier of the first event at a second time, which is later than the first time when the main machine executes the first event, and the second time and the first time are separated by a preset time interval;

[0007] The second standby machine executes a second event based on the identifier of the first event, the second event including the first event and an event executed by the master machine prior to the first event.

[0008] Optionally, the second standby unit acquires the identifier of the first event at a second time, including:

[0009] The second standby machine obtains the identifier of the first event from a preset list at a second time. The preset list includes the identifier of at least one event and the time when the host machine executes the event.

[0010] Optionally, the method further includes:

[0011] The second standby machine determines the newly added log file on the second standby machine;

[0012] The second standby machine writes the identifier of the event in the newly added log file and the time when the host machine executed the event into the preset list.

[0013] Optionally, the second standby machine writes the identifier of the event in the newly added log file and the time when the host executed the event into the preset list, including:

[0014] The second standby machine writes the identifier of the first event in the newly added log file and the time when the host machine executed the first event into the preset list.

[0015] Optionally, after the second standby machine executes the second event based on the identifier of the first event, it further includes:

[0016] The second standby machine removes the target information from the preset list. The target information includes the identifier of the third event and the time when the host machine executes the third event. The third event includes the first event and events in the preset list that precede the first event.

[0017] Optionally, the method further includes:

[0018] The host executes a time write event according to a preset time period, and the time write event is used to write the current time into the target data table of the host.

[0019] The host generates a first log file, which includes the identifier of the time write event and the time when the host executes the time write event;

[0020] The second standby machine generates a second log file based on the first log file. The second log file includes the identifier of the time write event and the time when the host executes the time write event.

[0021] Optionally, the database system further includes a control device, and the database system has a corresponding configuration file, which includes a flag for the standby machine. The method further includes:

[0022] If a backup machine marked as the target exists in the configuration file, the control device will identify the backup machine marked as the target as the second backup machine.

[0023] Optionally, the configuration file may also include a delay synchronization switch, and the method may further include:

[0024] The control device receives an instruction to open the delay synchronization switch;

[0025] In response to the activation command, if a backup machine marked with the target mark exists in the configuration file, the control device sets the delay synchronization switch to the on state. When the delay synchronization switch is on, the second backup machine is used to delay and synchronize data from the host machine.

[0026] If the backup machine marked as the target does not exist in the configuration file, the control device keeps the delay synchronization switch in the off state. When the delay synchronization switch is off, the second backup machine is used to synchronize data from the host machine in real time.

[0027] Secondly, this disclosure provides a database system, comprising: a host, at least one first standby machine, and at least one second standby machine, wherein the host is configured to update data on the host by executing events, the first standby machine is configured to synchronize data from the host in real time by executing the same events as the host, and the second standby machine is configured to:

[0028] The identifier of the first event is obtained at a second time, which is later than the first time when the host executes the first event, and the second time is spaced apart by a preset time.

[0029] The second event is executed based on the identifier of the first event, the second event including the first event and an event executed by the host prior to the first event.

[0030] Optionally, the second standby unit is also used for:

[0031] The identifier of the first event is obtained from a preset list at a second time. The preset list includes the identifier of at least one event and the time when the host executes the event.

[0032] Optionally, the second standby unit is also used for:

[0033] Identify the newly added log file on the second standby machine;

[0034] The event identifier and the time when the host executed the event are written into the preset list from the newly added log file.

[0035] Optionally, the second standby unit is also used for:

[0036] The identifier of the first event in the newly added log file and the time when the host executed the first event are written into the preset list.

[0037] Optionally, the second standby unit is also used for:

[0038] After the second standby machine executes the second event according to the identifier of the first event, the target information is deleted from the preset list. The target information includes the identifier of the third event and the time when the host executes the third event. The third event includes the first event and the events in the preset list that precede the first event.

[0039] Optionally, the host is further configured to:

[0040] A time write event is executed according to a preset time period, and the time write event is used to write the current time to the target data table of the host.

[0041] A first log file is generated, which includes the identifier of the time write event and the time when the host executes the time write event;

[0042] The second standby machine generates a second log file based on the first log file. The second log file includes the identifier of the time write event and the time when the host executes the time write event.

[0043] Optionally, the database system further includes a control device, and the database system has a corresponding configuration file, which includes a flag for the standby machine. The control device is used for:

[0044] If a standby machine marked as the target exists in the configuration file, then the standby machine marked as the target is determined as the second standby machine.

[0045] Optionally, the configuration file also includes a delay synchronization switch, and the control device is further used for:

[0046] Receive an instruction to open the delay synchronization switch;

[0047] In response to the enable command, if a backup machine marked with the target mark exists in the configuration file, the delay synchronization switch is set to the enabled state. When the delay synchronization switch is enabled, the second backup machine is used to delay and synchronize data from the host machine.

[0048] If the backup machine marked as the target does not exist in the configuration file, the delay synchronization switch is kept in the off state. When the delay synchronization switch is off, the second backup machine is used to synchronize data from the host machine in real time.

[0049] Thirdly, this disclosure provides an electronic device, including: at least one processor and a memory;

[0050] The memory stores computer-executed instructions;

[0051] The at least one processor executes computer execution instructions stored in the memory, causing the electronic device to implement the method described in the first aspect above.

[0052] Fourthly, this disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, cause a computing device to implement the method described in the first aspect above.

[0053] Fifthly, this disclosure provides a computer program for implementing the method described in the first aspect above.

[0054] The data synchronization method, database system, and device disclosed herein are applied to a database system. The system includes a host, at least one first standby machine, and at least one second standby machine. The host is used to update data on the host by executing events, and the first standby machine is used to synchronize data from the host in real time by executing the same events as the host. The method includes: the second standby machine acquiring an identifier of a first event at a second time, the second time being later than the first time the host executes the first event, and a preset time interval between the second and first times; the second standby machine executing a second event according to the identifier of the first event, the second event including the first event and an event executed by the host before the first event. This disclosure allows for delayed synchronization on a second standby machine, meaning that after the host updates data, the second standby machine does not immediately synchronize the data, but synchronizes it after a certain period of time. This reduces the number of restarts of the second standby machine and lowers its load. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram of data synchronization between the primary and backup machines in existing technology;

[0057] Figure 2 This is a schematic diagram of the structure of a database system provided in an embodiment of this disclosure;

[0058] Figure 3 This is a flowchart illustrating the specific steps of the data synchronization method provided in this embodiment.

[0059] Figure 4 This is a detailed flowchart of the data synchronization method provided in the embodiments of the present invention;

[0060] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0062] The embodiments disclosed herein can be used in database systems. Database systems typically include one primary machine and at least one backup machine. Figure 1 This is a schematic diagram illustrating data synchronization between the primary and backup machines in existing technology. (Refer to...) Figure 1 As shown, the database system includes a primary server D1, a standby server D2, and a standby server D3. Each standby server D2 and D3 is used to synchronize data from the primary server in real time. That is, whenever the primary server updates a piece of data, the standby server needs to immediately update this data to ensure that the data between the primary and standby servers is always consistent.

[0063] It is understandable that the above Figure 1 The example provided shows two backup machines, but in practical applications, the number of backup machines is unlimited.

[0064] In the real-time synchronization scenario described above, the standby server needs to be restarted after each data update for the updated data to take effect. This results in a high load on the standby server.

[0065] Furthermore, if the primary server updates data by deleting data, the standby server immediately deletes the corresponding data. This results in the deleted data not existing in either the primary or standby server, making it impossible to recover the deleted data.

[0066] Furthermore, existing data backup solutions only focus on backup, neglecting data recovery. This can lead to untimely recovery when the backup volume is large, or backup files becoming corrupted or missing, resulting in backup unavailability. The conventional solution to this problem is to constantly scan and frequently restart to prevent backup unavailability. However, this constant scanning and frequent restarting can lead to high load and excessive resource consumption.

[0067] To address the aforementioned issues, embodiments of this disclosure can utilize delayed synchronization on a standby machine. That is, after the primary machine updates data, the standby machine does not immediately synchronize the data, but rather synchronizes it after a certain period of time. This reduces the number of restarts required by the standby machine and lowers its load.

[0068] Furthermore, if the primary machine deletes a piece of data, the backup machine can recover the deleted data since the backup machine has not yet deleted it.

[0069] The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present disclosure will now be described with reference to the accompanying drawings.

[0070] The embodiments disclosed herein can be applied to a database system, which includes a host and at least one first standby machine and at least one second standby machine. Figure 2 This is a schematic diagram of the structure of a database system provided in an embodiment of this disclosure. (Refer to...) Figure 2 As shown, the database system 100 includes a host 101, two first standby machines 102 and a second standby machine 103.

[0071] Each standby machine 102 is used to synchronize data from the master machine in real time. That is, whenever the master machine updates a piece of data, in order to ensure that the data between the master machine and the standby machine is always the same, the standby machine needs to update the data immediately.

[0072] The second standby machine 103 is used to synchronize data from the primary machine with a delay. In other words, after the primary machine updates a piece of data, the second standby machine 103 can delay synchronizing that data for a period of time.

[0073] Understandably, the aforementioned host is used to update the data on the host by executing events, the first standby is used to synchronize data from the host in real time by executing the same events as the host, and the second standby is also used to synchronize data from the host with a delay by executing the same events as the host.

[0074] The aforementioned second backup machine can be from Figure 1 In a database system, a standby machine is selected from multiple existing standby machines, with the remaining standby machines serving as the first standby. In this scenario, the number of standby machines for real-time data synchronization decreases, thereby reducing the data availability of the database system and resulting in poor system robustness. For example, refer to... Figure 1 As shown, the number of backup machines is 2. If... Figure 1 If one of the backup machines, D2, is designated as the second backup machine, then... Figure 1Only one standby machine, D3, remains. Therefore, if both the primary machine D1 and the standby machine D2 fail, the database system will be unable to provide data prior to the delayed synchronization of the second standby machine D3, as neither primary nor standby machine D2 contains real-time data. This reduces the data availability and robustness of the database system.

[0075] To avoid the aforementioned problems, the second backup machine can be a newly added backup machine. For example, Figure 2 Database system 100 in the middle is for Figure 1 The database system in the middle was obtained by adding a second standby machine, and Figure 1 Host D1 in the middle is as Figure 2 Host 101 in the middle, Figure 1 The backup units D2 and D3 are both Figure 2 The first backup unit, 102, Figure 2 The second standby machine 103 is newly added. This ensures that the number of standby machines for real-time data synchronization is not reduced, thus maintaining database data availability and system robustness.

[0076] Furthermore, during the primary / standby switchover, the second standby machine does not participate in the primary election; that is, it will not be switched to the primary. This is because if the second standby machine were switched to the primary, to ensure the availability of the database system, the delayed data on the second standby machine would need to be replenished to guarantee that the database system can provide all the data. However, replenishing this data takes time, thus increasing the primary / standby switchover time. Since the database system cannot provide services during the primary / standby switchover process, it will affect the normal operation of the database system.

[0077] In the data synchronization process of the aforementioned database system, the first standby machine sends a synchronization success message to the master machine after real-time data synchronization is complete, while the second standby machine sends a synchronization success message to the master machine after a delayed data synchronization. After receiving success messages from all the first standby machines, the master machine can reply with a success message to the client to confirm successful real-time synchronization.

[0078] based on Figure 2 The database system in China Figure 3 This is a flowchart illustrating the specific steps of the data synchronization method provided in this embodiment. (Refer to...) Figure 3 As shown, the method includes:

[0079] S201: The second standby machine obtains the identifier of the first event at a second time, which is later than the first time when the main machine executes the first event, and the interval between the second time and the first time is a preset duration.

[0080] The identifier for the first event can be composed of the host's UUID (universally unique identifier) ​​and an event identifier. The preset duration can be fixed; ideally, there should be a fixed delay between the data from the second standby machine and the data from the host machine. For example, there could be a 1-hour data delay between the second standby machine and the data.

[0081] The aforementioned second time can be the current time. That is, the second standby machine checks every preset interval whether an event occurred before the preset interval, and uses this as the first event. This first event is the event executed by the primary machine to update data. For example, it checks every hour whether an event last executed one hour ago exists, and uses this as the first event.

[0082] Ideally, if the second standby machine can obtain the time of each event executed by the first standby machine, then the second standby machine can accurately determine the first event executed by the master machine at the first moment, thereby ensuring that the second standby machine and the master machine always maintain a fixed preset duration.

[0083] The second standby machine can obtain the timestamps of the events executed by the primary host from the local second log files. However, this requires the second standby machine to scan all second log files at preset intervals. Since scanning a second log file requires opening it and establishing a connection, this process is time-consuming. To save time, the second standby machine can add the event identifiers and the timestamps of events from newly created log files to a preset list during the first scan. This way, the second log files are only scanned once immediately after creation and not repeatedly. Subsequent scans only require retrieving the event identifiers and timestamps from the preset list, reducing the number of scans and saving time.

[0084] The aforementioned preset list may include multiple records. Each record includes the identifier of each event and the time when the host executes the event. In other words, the event identifier and the time when the host executes the event are stored correspondingly.

[0085] As explained above, when the second standby machine obtains the identifier of the first event at the second time, it can retrieve it from a preset list. Specifically, these records can be sorted in the preset list according to the time when the primary machine executes the event. Thus, the second standby machine can search backwards from the latest time in this order to find the identifier of the first event whose duration between the current time and the second time is greater than a preset duration, and use this as the identifier of the first event.

[0086] The process of identifying the first event is illustrated below with a specific example.

[0087] For example, the preset list includes the following 5 records: {(10:10, EID1), (10:18, EID2), (10:20, EID3), (10:31, EID4), (10:40, EID5)}. Assume the second time is 11:00 and the preset duration is 30 minutes.

[0088] The record corresponding to the latest time is (10:40, EID5). The duration between the time of the event corresponding to the host execution event identifier EID5 (10:40) and the second time (11:00) is 20 minutes. Since this duration of 20 minutes is less than the preset duration of 30 minutes, EID5 is not the identifier of the first event.

[0089] The next later time corresponds to the record (10:31, EID4). This record includes a host execution event identifier EID4. The duration between the time 10:31 and the second time 11:00 is 29 minutes. Since this duration of 29 minutes is less than the preset duration of 30 minutes, EID4 is not the identifier of the first event.

[0090] The next later time corresponds to the record (10:20, EID3). This record contains a record where the duration between the time 10:20 and the second time 11:00 for the host execution event identifier EID3 is 40 minutes. Since this duration of 40 minutes is greater than the preset duration of 30 minutes, EID3 is the identifier for the first event. After finding the identifier for the first event, the search can be stopped.

[0091] As can be seen from the above process of obtaining the identifier of the first event, the size of the preset list affects the efficiency of the acquisition process. A large preset list not only occupies the storage space of the second standby machine but also reduces the efficiency of obtaining the identifier of the first event. Therefore, to improve this efficiency, this embodiment of the invention needs to ensure that the number of records included in the preset list is as small as possible. To reduce the number of records, records that will no longer be used can be deleted from the preset list. Specifically, after the second standby machine executes the second event, the second standby machine can delete target information from the preset list. The target information includes the identifier of the third event and the time when the host machine executes the third event. The third event includes the first event and events in the preset list preceding the first event.

[0092] It's understandable that after each S201 and S202 sequence on the second standby machine, the second event for that current execution has already been executed, so it won't be executed again subsequently. Therefore, the first event and the time preceding it in the preset list are no longer needed. In this case, this part of the information can be deleted from the preset list to reduce the number of records in the preset list.

[0093] In the first scenario, the identifier of each event in the second log file and the time when the host executed the event are recorded in a preset list. In this case, the events executed by the host before the first event are the same as the events in the preset list that occurred before the first event.

[0094] In the second scenario, the identifiers of some events and the time the host executed the event are recorded in a preset list in the second log file, while the identifiers of the remaining events and the time the host executed the event are not recorded in the preset list. In this case, the events executed by the host before the first event are different from the events in the preset list that are before the first time. Specifically, the host executes more events before the first event than the events in the preset list that are before the first time.

[0095] The second scenario described above is designed to reduce costs. Since the second standby machine writes the time of each event into a preset list, the data volume in the preset list becomes large, resulting in a significant workload and high costs for the second standby machine. To reduce costs, considering that the second standby machine synchronizes data with a delay, we can tolerate a further delay beyond the preset duration. In other words, the actual delay can be longer than the preset duration.

[0096] Based on the above considerations, the second standby machine can write the identifier of the first event in the newly added log file and the time when the primary machine executed the first event into a preset list. The newly added log file is then called the newly added second log file. In this way, the preset list only stores the identifier of the first event in each second log file, and the number of records in the preset list corresponds to the number of second log files, thus reducing the number of records in the preset list.

[0097] For example, the system is configured to set the size of the second log file to 100MB. When the size reaches 100MB, a new second log file will be generated. The number of events in a second log file can reach tens of thousands. Therefore, adding the identifier of the first event to the preset list is equivalent to reducing the number of events in the preset list from tens of thousands to single digits, which means reducing the number of restarts of the second standby machine from tens of thousands to single digits.

[0098] Based on experience, the execution time of an event in the second log file is approximately 24 seconds. This means that if the primary server executes the first event in the second log file after a preset time interval, the second standby server immediately performs data synchronization, resulting in a time delay between the second standby server and the primary server equal to the preset time interval. However, if the second standby server performs data synchronization after the primary server has completed executing the last event in the second log file, the time delay between the second standby server and the primary server will be the preset time interval plus 24 seconds—an additional 24-second delay. Compared to the preset delay of one hour, this extra 24-second delay has a relatively small impact on the database system.

[0099] If you wait for all events in a second log file to be synchronized together, the estimated recovery time for ealy-log synchronization is about 24 seconds, which has almost no impact on the overall recovery time.

[0100] It is understandable that when the aforementioned preset list includes the identifier of the first event in each second log file, the identifier of the first event determined at preset intervals is also the identifier of the first event in one of the second log files. When events are executed frequently, this identifier of the first event can be considered the identifier of the last event in the previous second log file. Therefore, executing a second event means executing events in one or more second log files, equivalent to executing events in log files preceding the second log file containing the first event. Since scanning the second log files always starts from the beginning, obtaining the identifier of the first event in a second log file incurs less overhead than obtaining the identifier of the last event in each second log file, thus improving the efficiency of obtaining the identifier of the first event.

[0101] The event identifiers and timestamps in the second log file are obtained from the first log file. Specifically, the host executes time write events according to a preset time period. These time write events are used to write the current time to the host's target data table. Then, the host generates a first log file, which includes the identifiers of the time write events and the times when the host executes the time write events. Finally, the second standby machine generates a second log file based on the first log file. This second log file also includes the identifiers of the time write events and the times when the host executes the time write events.

[0102] The preset time period can be seconds, so that the host writes the current time to the target data table every second. For example, when the target data table includes the write time, the host's IP address, and the host's port, the target data table can contain two records: {(2021-06-18 16:10:52, IP1, POT1), (2021-06-18 16:10:53, IP2, POT1)}.

[0103] Since every time the host writes data to any table on the host, a log entry is generated in the host's first log file to record the data write event, a time write event will be generated in the first log file every second when the host writes a time. For example, for the two records in the target data table mentioned above, two time write events E1 and E2 will be generated in the first log file, corresponding to the times 2021-06-18 16:10:52 and 2021-06-18 16:10:53, respectively.

[0104] After the primary machine generates the first log file, the secondary machine can generate a second log file. The content of the second log file is identical to that of the first log file; it can be understood as a copy of the first log file. In practical applications, the first log file can be called a binlog, and the second log file can be called a relay-log; both are binary files. The size of the first log file is preset. When the size of a first log file reaches the preset size, a new first log file is generated, and the update records are written to this newly generated log file. Of course, the second log file can be the same size as the first log file.

[0105] In this embodiment of the invention, the second standby machine can obtain the time of the host execution event from the second log file without having to obtain it from the host, thus improving the efficiency of the second standby machine in obtaining the time.

[0106] Furthermore, since the host writes time to the target data table according to a preset time period, it can be ensured that the second log file includes a large number of events executed according to the preset time period. When the preset time period is short, it can be ensured that the duration before the current time is close to the preset duration, thereby ensuring that the delay between the second standby machine and the host machine is maintained as long as possible.

[0107] S202: The second standby machine executes the second event based on the identifier of the first event. The second event includes the first event and the event executed by the master machine before the first event.

[0108] Specifically, the statement execution thread of the second standby machine can execute statements based on the identifier of the first event to execute the first event and the events preceding the first event in the second log file. The execution statements can be as follows:

[0109] START SLAVE["connection_name"][SQL_THREAD]UNTIL

[0110] MASTER_GTID_POS=<GTID position>

[0111] slave

[0112] The connection_name mentioned above is the name of the second standby machine, and the GTID position is used to indicate the identifier of the first event.

[0113] As can be seen, the embodiments of the present invention can execute the first event and the events preceding the first event based solely on the identifier of the first event, without needing to know the identifier of each event preceding the first event.

[0114] The aforementioned second standby machine can be automatically identified by the control equipment in the database system. This database system has a corresponding configuration file, which includes a tag for each standby machine. If a standby machine with the target tag exists in the configuration file, the control equipment will identify that standby machine as the second standby machine. In this way, staff can configure the standby machine simply by modifying its tag in the configuration file, allowing for flexible modification of the standby machine.

[0115] For example, Figure 2 There are three standby machines, and the configuration file can set a flag for each of these three standby machines. Among them, the flag for standby machine 102 is 0, and the flag for standby machine 103 can be the target flag 1. Thus, it can be determined from the configuration file that standby machine 103 is the second standby machine and standby machine 102 is the first standby machine.

[0116] As can be seen, the standby machine is identified by a tag, not by its IP (Internet Protocol) address. This means that if the second standby machine itself remains unchanged, but its IP address changes, no configuration file modification is required.

[0117] The aforementioned control devices are used to manage and schedule the primary and backup machines in a database system; for example, ZooKeeper.

[0118] In this embodiment of the invention, in order to further improve the flexibility of delayed synchronization data, the configuration file also includes a delayed synchronization switch. The control device receives an enable command for the delayed synchronization switch; in response to the enable command, the control needs to determine whether there is a second standby machine in the database system that supports delayed synchronization data.

[0119] If a standby machine marked as the target exists in the configuration file, it is determined that a second standby machine exists in the database system. At this time, the control device sets the delay synchronization switch to the on state. With this delay synchronization switch on, the second standby machine is used to delay and synchronize data from the primary machine. That is, the second standby machine can execute steps S201 to S202 and their sub-steps to ensure a data delay exists between the second standby machine and the primary machine.

[0120] If no standby machine marked as the target is found in the configuration file, it is determined that no second standby machine exists in the database system. In this case, the control device keeps the delay synchronization switch off. With the delay synchronization switch off, the second standby machine is used to synchronize data from the primary machine in real time, ensuring real-time data synchronization between the second standby machine and the primary machine.

[0121] Figure 4 This is a detailed flowchart of the data synchronization method provided in an embodiment of the present invention. (Refer to...) Figure 4 As shown, the detailed flowchart of this data synchronization method may include S301 to S306.

[0122] S301: The host writes the current time to the target data table according to the preset time period.

[0123] S302: The primary machine generates the first log file, and the standby machine generates the second log file.

[0124] S303: The second standby unit checks at preset time intervals whether there are any newly added log files. If so, it executes S304; otherwise, it proceeds to S305.

[0125] The preset time interval here can be set flexibly, and is usually greater than or equal to the preset time period. For example, if the preset time period is 1 second, the preset time interval can be 3 seconds.

[0126] S304: The second standby machine will write the identifier of the first event in the newly added log file and the time when the master machine executed the event into the preset list.

[0127] S305: The second standby unit checks whether the first event exists in the preset list. If it exists, proceed to S306; otherwise, proceed to S303.

[0128] S306: The second standby machine executes the second event according to the identifier of the first event, and continues to execute S303 after S306.

[0129] As can be seen from the above description of the embodiments of the present invention, the technology used in the embodiments of the present invention is the basic technology that can be provided by the host and standby machines in the database system, so it can be used in different versions of the database system. That is, the above method of the embodiments of the present invention can be compatible with different versions of the database system.

[0130] For example, the database system described above can be a MySQL database system. This embodiment of the invention uses the following technologies from the MySQL database:

[0131] 1) The first log file, binlog, and the second log file, relay-log, record the transactions executed by the host. Each version of the MySQL database has these log files.

[0132] 2) The synchronization process from the first log file of the host to the second log file of the standby host is supported by different versions of MySQL database.

[0133] 3) The statement that calls the identifier of the first event and stops execution when it reaches the first event is supported in different versions of the MySQL database system.

[0134] 4) Monitoring and scanning the transaction timestamps in the relay log to see if they meet the set delay time does not depend on the MySQL version.

[0135] Corresponding to the data synchronization method in the above embodiments, Figure 2 This is a schematic diagram of the structure of a database system provided in an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 2 The aforementioned database system 100 includes: a host 101, at least one first standby machine 102, and at least one second standby machine 103. The host 101 is used to update data on the host 101 by executing events. The first standby machine 102 is used to synchronize data from the host 101 in real time by executing the same events as the host 101. The second standby machine 103 is used for:

[0136] The identifier of the first event is obtained at a second time, which is later than the first time when the host 101 executes the first event, and the second time is spaced apart by a preset time.

[0137] The second event is executed based on the identifier of the first event, the second event including the first event and an event executed by the host 101 prior to the first event.

[0138] Optionally, the second standby unit 103 is further configured to:

[0139] The identifier of the first event is obtained from a preset list at a second time. The preset list includes the identifier of at least one event and the time when the host 101 executes the event.

[0140] Optionally, the second standby unit 103 is further configured to:

[0141] Identify the newly added log file on the second standby machine 103;

[0142] The identifier of the event in the newly added log file and the time when the host 101 executed the event are written into the preset list.

[0143] Optionally, the second standby unit 103 is further configured to:

[0144] The identifier of the first event in the newly added log file and the time when the host 101 executes the first event are written into the preset list.

[0145] Optionally, the second standby unit 103 is further configured to:

[0146] After the second standby machine executes the second event according to the identifier of the first event, the target information is deleted from the preset list. The target information includes the identifier of the third event and the time when the host 101 executes the third event. The third event includes the first event and the events in the preset list that precede the first event.

[0147] Optionally, the host 101 is further configured to:

[0148] A time write event is executed according to a preset time period, and the time write event is used to write the current time into the target data table of the host 101.

[0149] A first log file is generated, which includes the identifier of the time write event and the time when the host 101 executes the time write event;

[0150] The second standby machine 103 generates a second log file based on the first log file. The second log file includes the identifier of the time write event and the time when the host 101 executes the time write event.

[0151] Optionally, the database system further includes a control device, and the database system has a corresponding configuration file, which includes a flag for the standby machine. The control device is used for:

[0152] If a standby machine marked as the target exists in the configuration file, then the standby machine marked as the target is determined as the second standby machine 103.

[0153] Optionally, the configuration file also includes a delay synchronization switch, and the control device is further used for:

[0154] Receive an instruction to open the delay synchronization switch;

[0155] In response to the enable command, if a standby machine marked with the target mark exists in the configuration file, the delay synchronization switch is set to the enable state. When the delay synchronization switch is enabled, the second standby machine 103 is used to delay and synchronize data from the host machine 101.

[0156] If the backup machine marked as the target does not exist in the configuration file, the delay synchronization switch is kept in the off state. When the delay synchronization switch is off, the second backup machine 103 is used to synchronize data from the host machine 101 in real time.

[0157] The data processing apparatus provided in this embodiment can be used to perform the above-described... Figure 3 The technical solutions of the method embodiments shown are similar in implementation principle and technical effect, and will not be described again here.

[0158] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present disclosure. The electronic device 600 includes a memory 602 and at least one processor 601.

[0159] The memory 602 stores computer-executed instructions. At least one processor 601 executes the computer-executed instructions stored in the memory 602, causing the electronic device 601 to perform the steps executed by the second standby device, the host device, or the control device in the aforementioned data synchronization method embodiment.

[0160] In addition, the electronic device may also include a receiver 603 and a transmitter 604, wherein the receiver 603 is used to receive information from other devices or equipment and forward it to the processor 601, and the transmitter 604 is used to send information to other devices or equipment.

[0161] This disclosure also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, the electronic device performs the steps executed by the second backup device, the host device, or the control device in the aforementioned data synchronization method embodiment.

[0162] This disclosure also provides a computer program for implementing the steps performed by the second standby machine, the host machine, or the control device in the aforementioned data synchronization method embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A data synchronization method, characterized in that, This is applied to a database system, which includes a host, at least one first standby machine, at least one second standby machine, a control device, and a configuration file. The host is used to update data on the host by executing events, and the first standby machine is used to synchronize data from the host in real time by executing the same events as the host. The configuration file includes a flag for the standby machine. The control device is used to determine the standby machine marked with the target mark as the second standby machine if there is a standby machine marked with the target mark in the configuration file; wherein, during the primary-standby switchover, the second standby machine will not be switched to the primary machine; The method includes: The second standby machine retrieves the identifier of the first event from a preset list at a second time; the preset list includes the identifier of at least one event and the time when the host executes the event, the second time being later than the first time when the host executes the first event, and the second time being spaced apart by a preset duration; the second standby machine executes the second event according to the identifier of the first event, the second event including the first event and events executed by the host before the first event; The second standby machine determines the newly added log file on the second standby machine; The second standby machine writes the identifier of the first event in the newly added log file and the time when the host machine executed the first event into the preset list.

2. The data synchronization method according to claim 1, characterized in that, After the second standby machine executes the second event based on the identifier of the first event, it also includes: The second standby machine removes the target information from the preset list. The target information includes the identifier of the third event and the time when the host machine executes the third event. The third event includes the first event and events in the preset list that precede the first event.

3. The data synchronization method according to claim 1, characterized in that, The method further includes: The host executes a time write event according to a preset time period, and the time write event is used to write the current time into the target data table of the host. The host generates a first log file, which includes the identifier of the time write event and the time when the host executes the time write event; The second standby machine generates a second log file based on the first log file. The second log file includes the identifier of the time write event and the time when the host executes the time write event.

4. The data synchronization method according to claim 1, characterized in that, The configuration file also includes a delay synchronization switch, and the method further includes: The control device receives an instruction to open the delay synchronization switch; In response to the activation command, if a backup machine marked with the target mark exists in the configuration file, the control device sets the delay synchronization switch to the on state. When the delay synchronization switch is on, the second backup machine is used to delay and synchronize data from the host machine. If the backup machine marked as the target does not exist in the configuration file, the control device keeps the delay synchronization switch in the off state. When the delay synchronization switch is off, the second backup machine is used to synchronize data from the host machine in real time.

5. A database system, characterized in that, include: The system comprises a primary machine, at least one first standby machine, and at least one second standby machine. The primary machine is configured to update data on the primary machine by executing events. The first standby machine is configured to synchronize data from the primary machine in real time by executing the same events as the primary machine. The second standby machine is configured to: The identifier of the first event is obtained from a preset list at a second time; the preset list includes the identifier of at least one event and the time when the host executes the event, the second time is later than the first time when the host executes the first event, and the second time is spaced apart by a preset time. Execute a second event based on the identifier of the first event, the second event including the first event and an event executed by the host prior to the first event; Identify the newly added log file on the second standby machine; The identifier of the first event in the newly added log file and the time when the host executed the first event are written into the preset list.

6. An electronic device, characterized in that, The electronic device includes: at least one processor and a memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the electronic device to implement the data synchronization method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, cause a computing device to implement the data synchronization method as described in any one of claims 1 to 4.

Images