Data recovery method, apparatus and device

Abstract

The application discloses a data recovery method, device and equipment, and aims to solve the problem that the primary and backup databases are inconsistent due to abnormal scenarios such as power failure during recovery of a backup machine in the prior art. The application records the interrupted recovery process by replaying an offset record file and an offset record cache, records the historical offset amount during replay interruption, enables the backup machine to start replaying the log file from the historical offset amount during continuous replay, avoids repeated replay of part of the log file, improves efficiency and saves time, and writes real-time log files of the host into a log cache while the backup machine is recovering the log file, parses the real-time log files in the log cache after the backup machine recovers the log file, recovers the real-time log files, and ensures that the data in the primary and backup databases in the dual-machine server is consistent.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data recovery method, apparatus, and device. Background Technology

[0002] In security scenarios, when one data server goes offline and then comes back online, the database content on the primary and backup servers becomes inconsistent. It's necessary to synchronize the primary database content to the backup server. The backup server can only come online after the primary and backup databases are completely consistent, ensuring it can provide services even after the primary server goes offline, thus guaranteeing data consistency. This requires the primary server to synchronize differentiated data to the backup server as quickly as possible. Currently, a common technical solution is to use a single thread to parse logs, a multi-threaded process to recover the logs, and then insert the log data into the database after the backup server comes online.

[0003] However, if the standby server encounters an abnormal scenario such as a power outage during recovery, the recovery process may be stopped halfway through. Restarting the server and continuing the recovery from scratch will not only consume a lot of time, but may also lead to inconsistencies between the primary and standby databases. Summary of the Invention

[0004] The purpose of this application is to provide a data recovery method, apparatus, and device to solve the problem of inconsistency between primary and backup databases caused by abnormal scenarios such as power outages during backup recovery in related technologies.

[0005] In a first aspect, this application provides a data recovery method applied to a host in a dual-machine server, the method comprising:

[0006] During the log replay process, for each replay thread, the historical offset of the replay thread to be processed and recorded is determined based on the offset record cache and offset record file.

[0007] The starting point for reading the parsing thread is determined based on the historical offset of the records processed by each playback thread.

[0008] The log file is parsed from the starting point of the read operation, and the parsed log file is inserted into the replay queue corresponding to the replay thread.

[0009] After confirming that the parsing of the log file is complete, the real-time log file in the log cache is parsed and the parsed real-time log file is inserted into the replay queue corresponding to the replay thread. The replay queue is used to send the files in the replay queue to the standby machine of the dual-machine server.

[0010] If it is determined that all replay queues are empty, a re-back-to-work command is sent to the standby machine.

[0011] In one possible implementation, before inserting the real-time log files in the log cache into their respective replay queues, the method further includes:

[0012] The maximum number of records and the medium number of records are set in the replay queue, where the medium number of records is a preset percentage of the maximum number of records.

[0013] In one possible implementation, setting the maximum number of records and the medium number of records in the replay queue further includes:

[0014] If it is determined that the replay queue is full, after a preset time interval, and after confirming that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the maximum number of records and the medium number of records in the replay queue are updated to obtain the new maximum number of records and the new medium number of records in the replay queue.

[0015] If it is determined that the new maximum number of records after the update is less than the current number of records in the queue, after a preset time interval, once it is determined that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the new maximum number of records and the new medium number of records in the replay queue are updated again until the replay queue is empty.

[0016] In one possible implementation, updating the maximum number of records and the medium number of records in the replay queue, or updating the new maximum number of records and the new medium number of records in the replay queue, includes:

[0017] Determine the current maximum number of records in the replay queue and the current number of records in the queue. Add the difference between the current maximum number of records and the current number of records in the queue to the current number of records in the queue using a preset ratio to obtain the new maximum number of records in the replay queue.

[0018] In one possible implementation, determining the historical offset of the record processed by each playback thread based on the offset record cache and the offset record file includes:

[0019] By comparing the first offset in the offset record cache and the second offset in the offset record file, the larger value among the offsets corresponding to the records processed by the same playback thread is taken as the historical offset of the record processed by the current playback thread. The offset record cache is used to record the first offset corresponding to each record processed by the playback thread, and the offset record file is used to periodically record the second offset corresponding to each record processed by the playback thread.

[0020] In one possible implementation, determining the read starting point of the parsing thread based on the historical offset of the playback thread processing record corresponding to each of the playback threads includes:

[0021] Based on the highest bit of the offset record file, determine whether the playback thread corresponding to the offset record file has completed the previous playback task;

[0022] If it is determined that any playback thread has not completed the previous playback task, the minimum value among the historical offsets of the playback thread processing records corresponding to all playback threads that have not completed the previous playback task is used as the starting point for the parsing thread to read.

[0023] In one possible implementation, the method further includes:

[0024] If it is determined that the offset recorded in the offset log file is 0, and it is determined that there is an old log file in the dual-machine server, a new log file is written directly after the old log file to obtain a real-time log file;

[0025] If it is determined that the offset recorded in the offset record file is not 0, and it is determined that there is an old log file in the dual-machine server, a new log file is written at the offset position in the old log file to obtain a real-time log file;

[0026] The real-time log file is stored in the log cache.

[0027] In one possible implementation, before parsing the log files of the host of the dual-machine server from the read start point, the method further includes:

[0028] Determine whether the log files are contiguous;

[0029] If it is determined that the log file is continuous, the log file is parsed from the starting point of the read operation, and the parsed log file is inserted into the replay queue corresponding to the replay thread.

[0030] Secondly, this application provides a data recovery device applied to the host in a dual-server system, the device comprising:

[0031] The historical offset determination module is configured to, during the log replay process of the log file, determine the historical offset of the replay thread corresponding to the replay thread for each replay thread based on the offset record cache and the offset record file.

[0032] The read start point determination module is configured to determine the read start point of the parsing thread based on the historical offset of the records processed by the playback thread corresponding to each playback thread.

[0033] The parsing module is configured to parse the log file from the starting point of the read and insert the parsed log file into the replay queue corresponding to the replay thread.

[0034] The real-time log file parsing module is configured to parse the real-time log file in the log cache after determining that the parsing of the log file is completed, and insert the parsed real-time log file into the replay queue corresponding to the replay thread. The replay queue is used to send the files in the replay queue to the standby machine of the dual-machine server.

[0035] The online operation command sending module is configured to send a re-online operation command to the standby machine if it is determined that all replay queues are empty.

[0036] In one possible implementation, before inserting the real-time log files in the log cache into their respective replay queues, the parsing module is further configured to:

[0037] The maximum number of records and the medium number of records are set in the replay queue, where the medium number of records is a preset percentage of the maximum number of records.

[0038] In one possible implementation, when setting the maximum and medium number of records in the replay queue, the parsing module is further configured to:

[0039] If it is determined that the replay queue is full, after a preset time interval, and after confirming that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the maximum number of records and the medium number of records in the replay queue are updated to obtain the new maximum number of records and the new medium number of records in the replay queue.

[0040] If it is determined that the new maximum number of records after the update is less than the current number of records in the queue, after a preset time interval, once it is determined that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the new maximum number of records and the new medium number of records in the replay queue are updated again until the replay queue is empty.

[0041] In one possible implementation, the parsing module is configured to either update the maximum and medium number of records in the replay queue, or update the new maximum and new medium number of records in the replay queue.

[0042] Determine the current maximum number of records in the replay queue and the current number of records in the queue. Add the difference between the current maximum number of records and the current number of records in the queue to the current number of records in the queue using a preset ratio to obtain the new maximum number of records in the replay queue.

[0043] In one possible implementation, the process of determining the historical offset of each playback thread's processed records based on the offset record cache and offset record file is performed, wherein the historical offset determination module is configured to:

[0044] By comparing the first offset in the offset record cache and the second offset in the offset record file, the larger value among the offsets corresponding to the records processed by the same playback thread is taken as the historical offset of the record processed by the current playback thread. The offset record cache is used to record the first offset corresponding to each record processed by the playback thread, and the offset record file is used to periodically record the second offset corresponding to each record processed by the playback thread.

[0045] In one possible implementation, the step of determining the read starting point of the parsing thread based on the historical offset of the playback thread processing record corresponding to each of the playback threads is performed, wherein the read starting point determination module is configured to:

[0046] Based on the highest bit of the offset record file, determine whether the playback thread corresponding to the offset record file has completed the previous playback task;

[0047] If it is determined that any playback thread has not completed the previous playback task, the minimum value among the historical offsets of the playback thread processing records corresponding to all playback threads that have not completed the previous playback task is used as the starting point for the parsing thread to read.

[0048] In one possible implementation, the device further includes:

[0049] If it is determined that the offset recorded in the offset log file is 0, and it is determined that there is an old log file in the dual-machine server, a new log file is written directly after the old log file to obtain a real-time log file;

[0050] If it is determined that the offset recorded in the offset record file is not 0, and it is determined that there is an old log file in the dual-machine server, a new log file is written at the offset position in the old log file to obtain a real-time log file;

[0051] The real-time log file is stored in the log cache.

[0052] In one possible implementation, before parsing the log files of the host of the dual-machine server from the read start point, the parsing module is further configured to:

[0053] Determine whether the log files are contiguous;

[0054] If it is determined that the log file is continuous, the log file is parsed from the starting point of the read operation, and the parsed log file is inserted into the replay queue corresponding to the replay thread.

[0055] Thirdly, this application provides an electronic device, comprising:

[0056] Memory is used to store the processor's executable instructions;

[0057] A processor for executing the executable instructions to implement the data recovery method as described in any one of the first aspects above.

[0058] Fourthly, this application provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the data recovery method as described in any one of the first aspects above.

[0059] Fifthly, this application provides a computer program product, including a computer program:

[0060] When the computer program is executed by a processor, it implements the data recovery method as described in any one of the first aspects above.

[0061] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:

[0062] In this embodiment, the interruption recovery process is recorded through a playback offset record file and a playback offset record cache. This allows the historical playback offset to be recorded when playback is interrupted, so that when the standby machine resumes playback, it can start replaying the log file from the historical offset, avoiding repeated playback of parts of the log file, improving efficiency and saving time. Furthermore, while the standby machine is recovering the log file, the real-time log file of the primary machine is written to the log cache. After the standby machine finishes recovering the log file, it parses the real-time log file in the log cache and recovers it, ensuring data consistency between the primary and standby databases in the dual-machine server.

[0063] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0064] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1This is a schematic diagram of the overall process of the data recovery method provided in the embodiments of this application;

[0066] Figure 2 A schematic diagram of the offset record cache provided in an embodiment of this application;

[0067] Figure 3 A schematic diagram of the offset recording file provided in the embodiments of this application;

[0068] Figure 4 A flowchart illustrating step 102 provided in an embodiment of this application;

[0069] Figure 5 A flowchart illustrating the process of acquiring real-time log files provided in this application embodiment;

[0070] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0071] Figure 7 This is a schematic diagram of the data recovery device provided in an embodiment of this application. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0073] Furthermore, in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0074] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0075] 1. Database consistency:

[0076] Database consistency means that the result of a transaction must transform the database from one consistent state to another. Ensuring database consistency means that when a transaction completes, all data must be in a consistent state. In relational databases, all rules must be applied to transaction modifications to maintain the integrity of all data.

[0077] 2. Multi-threaded recovery:

[0078] Using multiple threads to replay log records increases concurrency and speeds up recovery.

[0079] In security scenarios, related technologies require synchronizing the contents of the primary database to the standby server. The standby server can only come online after the primary and standby databases are completely consistent, and it must be able to provide services even after the primary server goes offline, ensuring data consistency. This necessitates the primary server synchronizing differentiated data to the standby server as quickly as possible. Currently, a common technical solution involves parsing logs using a single thread, recovering the logs using multiple threads, and then inserting the log data into the database after the standby server comes online.

[0080] However, if the standby server encounters an abnormal scenario such as a power outage during recovery, the recovery process may be stopped halfway through. Restarting the server and continuing the recovery from scratch will not only consume a lot of time, but may also lead to inconsistencies between the primary and standby databases.

[0081] In view of this, this application provides a data recovery method, apparatus and device to solve the problem of inconsistency between primary and backup databases caused by abnormal scenarios such as power outages during backup machine recovery in related technologies.

[0082] The inventive concept of this application can be summarized as follows: During the log replay process of the log file of the host of the dual-machine server, for each replay thread, the historical offset of the record processed by the replay thread is determined according to the offset record cache and the offset record file. Then, according to the historical offset of the record processed by the replay thread, the reading starting point of the parsing thread is determined. The log file of the host of the dual-machine server is then parsed from the reading starting point. The parsed log file is inserted into the replay queue corresponding to the replay thread. After the parsing of the log file is completed, the real-time log file in the log cache is parsed, and the parsed real-time log file is inserted into the replay queue corresponding to the replay thread. If it is determined that all replay queues are empty, a re-online running instruction is sent to the standby machine. This application embodiment records the interruption recovery process through a playback offset record file and a playback offset record cache. It can record the playback historical offset when playback is interrupted, so that when the standby machine continues playback, it can start playing back the log file from the historical offset, avoiding repeated playback of parts of the log file, improving efficiency and saving time. Furthermore, while the standby machine is recovering the log file, the real-time log file of the primary machine is written to the log cache. After the standby machine finishes recovering the log file, it parses the real-time log file in the log cache and recovers it, ensuring data consistency between the primary and standby databases in the dual-machine server.

[0083] After introducing the main inventive concepts of the embodiments of this application, the following is a brief description of the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0084] To facilitate understanding of the data recovery method provided in the embodiments of this application, further explanation will be given below in conjunction with the accompanying drawings.

[0085] In one possible implementation, this application provides a data recovery method applied to the host in a dual-machine server, the overall flowchart of which is shown below. Figure 1 As shown, it includes the following:

[0086] In step 101, during the log playback process of the log file, for each playback thread, the historical offset of the playback thread to be processed is determined based on the offset record cache and the offset record file.

[0087] In step 102, the starting point for reading the parsing thread is determined based on the historical offset of the records processed by the playback thread corresponding to each playback thread.

[0088] In step 103, the log file is parsed from the starting point of reading, and the parsed log file is inserted into the replay queue corresponding to the replay thread.

[0089] In step 104, after confirming that the parsing of the log file is complete, the real-time log file in the log cache is parsed and the parsed real-time log file is inserted into the replay queue corresponding to the replay thread. The replay queue is used to send the files in the replay queue to the standby machine of the dual-machine server.

[0090] It should be noted that the primary and backup servers in a dual-server setup can switch roles. For example, if the backup server encounters an abnormal scenario such as a power outage during recovery, the primary server executes steps 101 to 104 above to complete the online process on the dual-server. If the original primary server encounters an abnormal scenario such as a power outage, the original backup server will then execute steps 101 to 104 above to complete data synchronization. In this case, the primary and backup servers have switched roles.

[0091] In step 105, if it is determined that all replay queues are empty, a re-run command is sent to the standby machine.

[0092] In one possible implementation, step 101, determining the historical offset of the record processed by each playback thread based on the offset record cache and the offset record file, can be implemented as follows:

[0093] By comparing the first offset in the offset record cache and the second offset in the offset record file, the smaller value among the offsets of the records processed by the same playback thread is taken as the historical offset of the record processed by the current playback thread. The offset record cache is used to record the first offset corresponding to each record processed by the playback thread, and the offset record file is used to periodically record the second offset corresponding to each record processed by the playback thread.

[0094] In one possible implementation, to record the state of the replay thread, it is necessary to record the offset of each replay thread's processed record. This ensures that if replay is interrupted, it can resume processing from the previously unfinished record upon replay, avoiding inconsistencies between the primary and backup databases caused by multiple log replays. In this embodiment, the offset file contains two formats: an offset record file and an offset record cache, the details of which are as follows:

[0095] (1) The offset record cache is used to record the offset value of each playback thread's processing record in real time, i.e. the first offset. Since this file is written frequently, this application only writes the offset record cache to the memory file.

[0096] The offset record cache format is as follows: Figure 2As shown, it includes the following: uint64 offsets and uint32 checksums. Each line is the size of a page, and there are a total of 64 lines, which can record the offset addresses of 64 threads.

[0097] (2) The offset record file is a redundant file, containing two files: recovery_file.idx1 and recovery_file.idx2. These files are used to periodically record the offset of each playback thread's processed record (the period is configurable), i.e., the second offset. These two files are written synchronously, ensuring that the contents of the files are completely consistent. This guarantees that in the event of a file write failure, at least one file will be complete. Failure is only considered successful if both files fail to write, even if only one file is successfully written.

[0098] The offset log file format is as follows: Figure 3 As shown, it includes the following:

[0099] UID: Uint64 type, incremented starting from 1, incremented by 1 each time a file is written. In the above recovery_file.idx1 and recovery_file.idx2 files, if the verification is correct, the file with the larger UID shall prevail; otherwise, the file with the correct verification shall prevail.

[0100] Size: The total number of offsets, currently 64 by default, which is the offset for 64 threads.

[0101] Offset1, offset2...offset64: The offset for each playback thread. The highest bit indicates whether the record at that offset has been played back. The remaining 63 bits represent the offset.

[0102] edianType: Endianness type, 0 indicates little endianness, 1 indicates big endianness.

[0103] ChkSum: The checksum of all the above content.

[0104] For example, if the first offset of the record processed by a replay thread is 15 and the second offset is 20, this embodiment of the application uses the larger value of the offsets of the records processed by the same replay thread as the historical offset of the current replay thread's records, that is, the second offset is used as the historical offset, which avoids the replay thread's omission of some log files, resulting in data inconsistency between the primary and backup servers.

[0105] In one possible implementation, after determining the historical offset, in step 102, the starting point for the parsing thread is determined based on the historical offset of the records processed by each playback thread, as illustrated in the flowchart below. Figure 4As shown, it includes the following:

[0106] In step 401, based on the highest bit of the offset record file, it is determined whether the playback thread of the corresponding offset record file has completed the previous playback task.

[0107] In step 402, if it is determined that any playback thread has not completed the previous playback task, the minimum value among the historical offsets of the playback thread processing records corresponding to all playback threads that have not completed the previous playback task is used as the starting point for the parsing thread to read.

[0108] For example, if 200 files need to be recovered, this embodiment of the application distributes these files to 10 playback threads for playback. Each playback thread needs to recover 20 files, numbered 1 to 200. When the first playback thread completes its previous playback task, it indicates that the files it needs to recover have been backed up, so this embodiment of the application does not play back those files. If the first and second playback threads have not completed their previous playback tasks, the corresponding historical offsets are 12 and 25, respectively. The historical offset is the file number at the time of playback interruption, indicating that files 13 to 20 have not completed their playback tasks. To avoid missing some files during data recovery, this embodiment of the application uses 12 as the starting point for the parsing thread, ensuring data consistency in the primary and backup databases of the dual-machine server.

[0109] In one possible implementation, to enable the standby machine to come online as quickly as possible, this application dynamically adjusts the length of the replay queue during the standby machine's recovery phase of the real-time log files. This prevents the standby machine from failing to come online due to an excessively large volume of real-time log files. Specifically, before inserting the real-time log files in the log cache into their respective replay queues, this application sets a maximum number of records and a medium number of records for the replay queue, where the medium number of records is a preset percentage of the maximum number of records. Specifically, this can be implemented as follows:

[0110] If it is determined that the replay queue is full, after a preset time interval, once it is confirmed that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, update the maximum number of records and the medium number of records in the replay queue to obtain the new maximum number of records and the new medium number of records in the replay queue.

[0111] If it is determined that the new maximum number of records after the update is less than the current number of records in the queue, after a preset time interval, once it is confirmed that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the new maximum number of records and the new medium number of records in the replay queue are updated again until the replay queue is empty.

[0112] For example, you can set the maximum number of records in the queue and the medium number of records. The medium number of records = the maximum number of records * 2 / 3. For example, if the maximum number of records in the queue is 200, then the medium number of records is 133.

[0113] Once it is determined that all log files in the replay queue have been processed, the parsed log cached log file is inserted into the queue. If the replay queue is found to be full, the insertion will fail. After a preset time interval, such as 10 milliseconds, the maximum and medium number of records in the queue will be adjusted. The specific adjustment method is as follows:

[0114] Determine the current maximum number of records in the replay queue and the current number of records in the queue. Add the preset ratio of the difference between the current maximum number of records and the current number of records in the queue to the current number of records in the queue to obtain the new maximum number of records in the replay queue. This can be implemented using the following formula:

[0115] New maximum number of records = (original maximum number of records – current number of records in the queue) / 2 + current number of records in the queue.

[0116] The number of new medium records = the number of new maximum records * 2 / 3.

[0117] When insertion is successful, it means that the replay queue is not full. At this time, the original maximum number of records is 200, and the current number of records in the queue is 195. According to the above formula, the new maximum number of records is 198 and the new medium number of records is 132.

[0118] If the maximum number of new records after the update is determined to be less than the current number of records in the queue (e.g., the current queue has 195 records, the maximum number of new records is 198, and the number of new medium records is 132), but the replay queue has not yet finished processing the log file, and a file is received from the parsing thread, causing the current queue to have 200 records, resulting in the maximum number of new records after the update being less than the current number of records in the queue, this embodiment will, at preset time intervals, after confirming that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, update the maximum number of new records and the number of new medium records in the replay queue again according to the above formula, until the replay queue is empty. This process continues, continuously decreasing the maximum number of records in the replay queue, thus speeding up the processing of log files and reducing the standby machine's uptime, thereby accelerating the standby machine's uptime.

[0119] In one possible implementation, the process of acquiring real-time log files is as follows: Figure 5 As shown, it includes the following:

[0120] In step 501, if it is determined that the offset recorded in the offset record file is 0, and it is determined that there is an old log file in the dual-machine server, the new log file is written directly after the old log file to obtain the real-time log file.

[0121] In step 502, if it is determined that the offset recorded in the offset record file is not 0, and it is determined that there is an old log file in the dual-machine server, the new log file is written at the offset position in the old log file to obtain the real-time log file.

[0122] In step 503, the real-time log file is saved in the log cache.

[0123] The above steps write the real-time log files sequentially, ensuring the continuity of the real-time log files and avoiding errors.

[0124] In one possible implementation, before parsing the log file of the host of the dual-machine server from the starting point of reading, this embodiment of the application determines whether the log file is continuous. If the log file is determined to be continuous, it is parsed from the starting point of reading, and the parsed log file is inserted into the playback queue corresponding to the playback thread. Through the above steps of verifying file continuity, the integrity of the log file is ensured, and incomplete log files are excluded, reducing the occupation of server space and the waste of time.

[0125] In summary, this application embodiment records the interruption recovery process through a replay offset record file and a replay offset record cache. It records the historical replay offset when replay is interrupted, allowing the standby server to resume replaying from the historical offset, avoiding repeated replaying of parts of the log file, improving efficiency and saving time. Furthermore, while the standby server is recovering the log file, the master server's real-time log file is written to the log cache. After the standby server finishes recovering the log file, it parses the real-time log file in the log cache and recovers it, ensuring data consistency between the master and standby databases in the dual-machine server. By dynamically adjusting and reducing the length of the replay queue, the standby server can quickly consume log files, accelerating the standby server's online speed.

[0126] Based on the same inventive concept, embodiments of this application also provide a data recovery device, such as... Figure 6 As shown, the device 600, used in a dual-machine server, includes:

[0127] The historical offset determination module 601 is configured to, during the log replay process of the log file, determine the historical offset of the replay thread processing record corresponding to the replay thread for each replay thread based on the offset record cache and the offset record file.

[0128] The read start point determination module 602 is configured to determine the read start point of the parsing thread based on the historical offset of the playback thread processing record corresponding to each playback thread.

[0129] The parsing module 603 is configured to parse the log file from the reading start point and insert the parsed log file into the replay queue corresponding to the replay thread.

[0130] The real-time log file parsing module 604 is configured to parse the real-time log file in the log cache after determining that the parsing of the log file is completed, and insert the parsed real-time log file into the replay queue corresponding to the replay thread. The replay queue is used to send the files in the replay queue to the standby machine of the dual-machine server.

[0131] The online operation instruction sending module 605 is configured to send a re-online operation instruction to the standby machine if it is determined that all replay queues are empty.

[0132] In one possible implementation, before inserting the real-time log files in the log cache into their respective replay queues, the parsing module is further configured to:

[0133] The maximum number of records and the medium number of records are set in the replay queue, where the medium number of records is a preset percentage of the maximum number of records.

[0134] In one possible implementation, when setting the maximum and medium number of records in the replay queue, the parsing module is further configured to:

[0135] If it is determined that the replay queue is full, after a preset time interval, and after confirming that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the maximum number of records and the medium number of records in the replay queue are updated to obtain the new maximum number of records and the new medium number of records in the replay queue.

[0136] If it is determined that the new maximum number of records after the update is less than the current number of records in the queue, after a preset time interval, once it is determined that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the new maximum number of records and the new medium number of records in the replay queue are updated again until the replay queue is empty.

[0137] In one possible implementation, the parsing module is configured to either update the maximum and medium number of records in the replay queue, or update the new maximum and new medium number of records in the replay queue.

[0138] Determine the current maximum number of records in the replay queue and the current number of records in the queue. Add the difference between the current maximum number of records and the current number of records in the queue to the current number of records in the queue using a preset ratio to obtain the new maximum number of records in the replay queue.

[0139] In one possible implementation, the process of determining the historical offset of each playback thread's processed records based on the offset record cache and offset record file is performed, wherein the historical offset determination module is configured to:

[0140] By comparing the first offset in the offset record cache and the second offset in the offset record file, the larger value among the offsets corresponding to the records processed by the same playback thread is taken as the historical offset of the record processed by the current playback thread. The offset record cache is used to record the first offset corresponding to each record processed by the playback thread, and the offset record file is used to periodically record the second offset corresponding to each record processed by the playback thread.

[0141] In one possible implementation, the step of determining the read starting point of the parsing thread based on the historical offset of the playback thread processing record corresponding to each of the playback threads is performed, wherein the read starting point determination module is configured to:

[0142] Based on the highest bit of the offset record file, determine whether the playback thread corresponding to the offset record file has completed the previous playback task;

[0143] If it is determined that any playback thread has not completed the previous playback task, the minimum value among the historical offsets of the playback thread processing records corresponding to all playback threads that have not completed the previous playback task is used as the starting point for the parsing thread to read.

[0144] In one possible implementation, the device further includes:

[0145] If it is determined that the offset recorded in the offset log file is 0, and it is determined that there is an old log file in the dual-machine server, a new log file is written directly after the old log file to obtain a real-time log file;

[0146] If it is determined that the offset recorded in the offset record file is not 0, and it is determined that there is an old log file in the dual-machine server, a new log file is written at the offset position in the old log file to obtain a real-time log file;

[0147] The real-time log file is stored in the log cache.

[0148] In one possible implementation, before parsing the log files of the host of the dual-machine server from the read start point, the parsing module is further configured to:

[0149] Determine whether the log files are contiguous;

[0150] If it is determined that the log file is continuous, the log file is parsed from the starting point of the read operation, and the parsed log file is inserted into the replay queue corresponding to the replay thread.

[0151] The following reference Figure 7 To describe an electronic device 130 according to this embodiment of the present application. Figure 7 The electronic device 130 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0152] like Figure 7 As shown, the electronic device 130 is presented in the form of a general-purpose electronic device. The components of the electronic device 130 may include, but are not limited to: at least one processor 131, at least one memory 132, and a bus 133 connecting different system components (including memory 132 and processor 131).

[0153] Bus 133 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or local bus using any of the various bus structures.

[0154] The memory 132 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0155] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0156] Electronic device 130 can also communicate with one or more external devices 134 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with electronic device 130, and / or with any device that enables electronic device 130 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 135. Furthermore, electronic device 130 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 136. As shown, network adapter 136 communicates with other modules used in electronic device 130 via bus 133. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0157] In an exemplary embodiment, this application also provides a computer-readable storage medium including instructions, such as a memory 132 including instructions, which can be executed by a processor 131 of an electronic device 130 to complete the data recovery method described above. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0158] In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by a processor 131, implements the data recovery method provided in this application.

[0159] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0160] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0161] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0162] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0163] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A data recovery method, characterized by, The method, applied to a host in a dual-machine server, includes: During the log replay process, for each replay thread, the historical offset of the replay thread to be processed and recorded is determined based on the offset record cache and offset record file. The starting point for reading the parsing thread is determined based on the historical offset of the records processed by each playback thread. The log file is parsed from the starting point of the read operation, and the parsed log file is inserted into the replay queue corresponding to the replay thread. After confirming that the parsing of the log file is complete, the real-time log file in the log cache is parsed and the parsed real-time log file is inserted into the replay queue corresponding to the replay thread. The replay queue is used to send the files in the replay queue to the standby machine of the dual-machine server. If it is determined that all replay queues are empty, a re-back-to-work command is sent to the standby machine; Before inserting the real-time log files in the log cache into their respective replay queues, the method further includes: If it is determined that the replay queue is full, after a preset time interval, and after confirming that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the maximum number of records and the medium number of records in the replay queue are updated to obtain the new maximum number of records and the new medium number of records in the replay queue; the medium number of records is a preset percentage of the maximum number of records. If it is determined that the new maximum number of records after the update is less than the current number of records in the queue, after a preset time interval, once it is determined that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the new maximum number of records and the new medium number of records in the replay queue are updated again until the replay queue is empty.

2. The method of claim 1, wherein, Updating the maximum and medium number of records in the replay queue, or updating the new maximum and medium number of records in the replay queue, includes: Determine the current maximum number of records in the replay queue and the current number of records in the queue. Add the difference between the current maximum number of records and the current number of records in the queue to the current number of records in the queue using a preset ratio to obtain the new maximum number of records in the replay queue.

3. The method according to claim 1, characterized in that, The step of determining the historical offset of each playback thread based on the offset record cache and offset record file includes: By comparing the first offset in the offset record cache and the second offset in the offset record file, the larger value among the offsets corresponding to the records processed by the same playback thread is taken as the historical offset of the record processed by the current playback thread. The offset record cache is used to record the first offset corresponding to each record processed by the playback thread, and the offset record file is used to periodically record the second offset corresponding to each record processed by the playback thread.

4. The method according to claim 1, characterized in that, The step of determining the reading starting point of the parsing thread based on the historical offset of the records processed by each playback thread includes: Based on the highest bit of the offset record file, determine whether the playback thread corresponding to the offset record file has completed the previous playback task; If it is determined that any playback thread has not completed the previous playback task, the minimum value among the historical offsets of the playback thread processing records corresponding to all playback threads that have not completed the previous playback task is used as the starting point for the parsing thread to read.

5. The method according to claim 1, characterized in that, The method further includes: If it is determined that the offset recorded in the offset log file is 0, and it is determined that there is an old log file in the dual-machine server, a new log file is written directly after the old log file to obtain a real-time log file; If it is determined that the offset recorded in the offset record file is not 0, and it is determined that there is an old log file in the dual-machine server, a new log file is written at the offset position in the old log file to obtain a real-time log file; The real-time log file is stored in the log cache.

6. The method according to claim 1, characterized in that, Before parsing the log files of the host of the dual-machine server from the read start point, the method further includes: Determine whether the log files are contiguous; If it is determined that the log file is continuous, the log file is parsed from the starting point of the read operation, and the parsed log file is inserted into the replay queue corresponding to the replay thread.

7. A data recovery device, characterized in that, The device is used as a host in a dual-machine server and includes: The historical offset determination module is configured to, during the log replay process of the log file, determine the historical offset of the replay thread corresponding to the replay thread for each replay thread based on the offset record cache and the offset record file. The read start point determination module is configured to determine the read start point of the parsing thread based on the historical offset of the records processed by the playback thread corresponding to each playback thread. The parsing module is configured to parse the log file from the starting point of the read and insert the parsed log file into the replay queue corresponding to the replay thread. The real-time log file parsing module is configured to parse the real-time log file in the log cache after determining that the parsing of the log file is completed, and insert the parsed real-time log file into the replay queue corresponding to the replay thread. The replay queue is used to send the files in the replay queue to the standby machine of the dual-machine server. The online operation instruction sending module is configured to send a re-online operation instruction to the standby machine if it is determined that all replay queues are empty. Before inserting the real-time log files in the log cache into their respective replay queues, the parsing module is further configured to: If it is determined that the replay queue is full, after a preset time interval, and after confirming that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the maximum number of records and the medium number of records in the replay queue are updated to obtain the new maximum number of records and the new medium number of records in the replay queue; the medium number of records is a preset percentage of the maximum number of records. If it is determined that the new maximum number of records after the update is less than the current number of records in the queue, after a preset time interval, once it is determined that the real-time log file in the log cache has been successfully inserted into the corresponding replay queue, the new maximum number of records and the new medium number of records in the replay queue are updated again until the replay queue is empty.

8. An electronic device, characterized in that, include: Memory is used to store the processor's executable instructions; A processor for executing the executable instructions to implement the data recovery method as described in any one of claims 1-6.

Images