Data recovery methods, devices, electronic equipment and storage media

By recording and querying the checkpoint thread status in the write-ahead log, and using different recovery methods to recover the memory data of the distributed caching system, the problems of low data recovery efficiency and consistency are solved, and fast and reliable data recovery is achieved.

CN118819949BActive Publication Date: 2025-10-31CHINA MOBILE GRP FUJIAN CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410274606.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-11
Publication Date
2025-10-31
Estimated Expiration
2044-03-11

AI Technical Summary

Technical Problem

Existing technologies suffer from low data recovery efficiency and transaction consistency risks when distributed caching systems crash or restart, resulting in long business interruptions and poor user experience.

Method used

By recording modification operations of distributed cache nodes in the write-ahead log and using checkpoint threads to persist data to local storage, the checkpoints in the write-ahead log are queried to identify incomplete threads, and different recovery methods are used to recover in-memory data, including using page snapshots and incremental data.

Benefits of technology

It improves the efficiency of data recovery, ensures transactional consistency in data recovery, reduces business downtime, and enhances user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118819949B_ABST
    Figure CN118819949B_ABST
Patent Text Reader

Abstract

This application proposes a data recovery method, apparatus, electronic device, and storage medium. The method includes: querying the Write-Ahead Log (WAL) to obtain the checkpoints (CPs) associated with the currently started target node; determining, based on the timestamps of the associated CPs, whether there are any incomplete CP threads associated with the target node in the WAL; determining the target recovery method for the target node's memory data based on the presence of incomplete CP threads associated with the target node in the WAL; and recovering the target node's memory data based on the target recovery method and the records associated with the target node in the WAL. This improves the efficiency of data recovery, ensures transactional consistency in data recovery, and enhances the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data recovery method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the booming development of internet businesses, the volume of business data has shown explosive growth. Therefore, in the event of unexpected situations such as system crashes or restarts, how to quickly recover data and reduce business interruption time is the key to improving business continuity and user experience. Summary of the Invention

[0003] This application aims to at least partially address one of the technical problems in the related art.

[0004] Therefore, the first objective of this application is to propose a data recovery method to achieve high-speed recovery of persistent data, thereby improving the efficiency of data recovery and transaction consistency.

[0005] The second objective of this application is to provide an apparatus.

[0006] The third objective of this application is to propose an electronic device.

[0007] The fourth objective of this application is to provide a computer-readable storage medium.

[0008] The fifth objective of this application is to provide a computer program product.

[0009] To achieve the above objectives, a data recovery method is proposed in the first aspect of this application, comprising:

[0010] Query the Write-Ahead Log (WAL) to obtain the checkpoint (CP) associated with the currently started target node in the WAL.

[0011] Based on the timestamp of the associated CP, determine whether there is an incomplete CP thread associated with the target node in the WAL;

[0012] Based on whether there are any incomplete CP threads associated with the target node in the WAL, determine the target recovery method for the memory data of the target node;

[0013] Based on the target recovery method and the records associated with the target node in the WAL, the memory data of the target node is recovered.

[0014] To achieve the above objectives, a second aspect of this application provides a data recovery apparatus, comprising:

[0015] The acquisition module is used to query the Write-Ahead Log (WAL) and obtain the checkpoint (CP) associated with the currently started target node in the WAL.

[0016] The first determining module is used to determine, based on the timestamp of the associated CP, whether there is an incomplete CP thread associated with the target node in the WAL;

[0017] The second determining module is used to determine the target recovery method for the memory data of the target node based on whether there are any unfinished CP threads associated with the target node in the WAL.

[0018] The recovery module is used to recover the memory data of the target node according to the target recovery method and the records associated with the target node in the WAL.

[0019] To achieve the above objectives, a third aspect of this application provides an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the data recovery method as described in the first aspect embodiment.

[0020] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the data recovery method as described in the first aspect embodiment.

[0021] To achieve the above objectives, a fifth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the data recovery method described in the first aspect embodiment.

[0022] The data recovery method, apparatus, electronic device, and storage medium provided in this application determine different data recovery methods by identifying whether the write-ahead log contains incomplete checkpoint threads associated with the target node for data recovery, thereby improving the efficiency of data recovery, ensuring transactional consistency of data recovery, and enhancing the user experience.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0025] Figure 1This is a flowchart illustrating a data recovery method provided in an embodiment of this application.

[0026] Figure 2 A flowchart illustrating another data recovery method provided in an embodiment of this application;

[0027] Figure 3 A WAL diagram containing an incomplete CP thread is provided for this application;

[0028] Figure 4 This is a schematic diagram of the structure of a data recovery device provided in an embodiment of this application. Detailed Implementation

[0029] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0030] The data recovery method and apparatus of this application are described below with reference to the accompanying drawings.

[0031] It should be noted that the data recovery method in this application embodiment is executed by a data recovery device, which can be implemented by software and / or hardware and can be configured in an electronic device, including but not limited to a terminal, server, etc.

[0032] Figure 1 This is a schematic flowchart of a data recovery method provided in an embodiment of this application.

[0033] Existing data recovery solutions utilize distributed caches (such as Ignite clusters) as the primary data storage to provide high-performance read and write services. However, when the distributed cache fails, it's necessary to retrieve all data from the database, load it into the corresponding target cache within the distributed cache, merge the caches that need to be merged using atomic operations, and then create a cache index after merging. This data recovery solution is heavily reliant on disk input / output (IO) operations and requires frequent disk scans, resulting in significant time consumption and poor performance. Furthermore, batch asynchronous persistence of data to third-party data sources may pose risks to transaction consistency.

[0034] To address these issues, this application utilizes write-ahead logging to record data modification operations within each cache node of the distributed cache, and persists the data to local storage (such as disk) via checkpoint threads. This allows for the reading of persistent backup files from the local disk to recover node data in the event of a distributed cache restart or crash, improving data recovery efficiency. Furthermore, based on the completion status of checkpoint threads associated with each node in the write-ahead log, different methods can be determined to recover the node's memory data, significantly reducing data recovery time and ensuring transactional consistency during distributed data recovery.

[0035] like Figure 1 As shown, this data recovery method includes the following steps:

[0036] Step 101: Query the Write-Ahead Log (WAL) and obtain the checkpoint (CP) associated with the currently started target node in the WAL.

[0037] The Write-Ahead Log (WAL) is used to record all modifications, deletions, and other operations performed on in-memory data on each node.

[0038] The target node currently being started refers to any node in a distributed caching system (such as an Ignite cluster) used to store and process cached data.

[0039] Checkpointing (CP) is used to mark the points where data modification operations recorded in WAL are synchronized to the persistent file (Partition File), which can ensure the consistency of data between memory and persistent files.

[0040] It should be noted that data recovery is based on data that has been persisted to local storage beforehand. Therefore, the process of persisting data locally can include: when a modification operation is received on the memory data of any node, the modification operation can be recorded in WAL, and the dirty data in memory can be synchronized to the persistent file associated with any node through the CP thread.

[0041] The modification operation can be any of the following: deletion, addition, copying, etc., and this disclosure does not limit it.

[0042] Dirty data refers to data that has been modified in memory but has not been written to local storage.

[0043] In this embodiment, modifications to memory data within any node are not immediately synchronized to the persistent file. Instead, the modifications are first recorded in the Write-Ahead Log (WAL), ensuring the performance of data writing under the persistence mechanism. Furthermore, the CP thread synchronizes dirty data in memory to the persistent file associated with that node, allowing for targeted reading of the persistent file during data recovery. This improves the efficiency of data recovery and ensures data consistency and accuracy.

[0044] It should be noted that since the data stored on each node in the distributed cache is different, and the files that the data within each node is persisted to in local storage may also be different, each checkpoint in the write-ahead log should include information such as the identifier of the node to which the data belongs when it is generated, so that the corresponding log record can be quickly identified when restoring node data.

[0045] In this embodiment of the disclosure, the target node to be started can be determined by means of cluster manager or monitoring node status, and then the target node's metadata information (such as node identifier, status information, version number, etc.) can be queried in WAL to obtain the CP associated with the target node to be started.

[0046] It is understandable that during the process of recording data modification operations within a node in the write-ahead log, the CP thread can be used multiple times to synchronize the data to the persistent file. Therefore, the WAL may include one or more checkpoints associated with the currently started target node.

[0047] Step 102: Based on the timestamp of the associated CP, determine whether there is an incomplete CP thread associated with the target node in the WAL.

[0048] The timestamp of a CP can include the start and end timestamps of the CP.

[0049] Additionally, the start timestamp refers to the time when the CP start file was generated, and the end timestamp refers to the time when the CP end file was generated. In the write-ahead log, the records located between the start and end timestamps represent the CP threads associated with the target node. These CP threads are responsible for persisting the data in memory to local storage.

[0050] Optionally, if the start timestamp of any CP in the associated CP is not included, it can be determined that there is an incomplete CP thread associated with the target node in the WAL.

[0051] Alternatively, if each CP contained in the associated CP has a corresponding start timestamp and end timestamp, it can be determined that there are no incomplete CP threads associated with the target node in the WAL.

[0052] Step 103: Determine the target recovery method for the target node's memory data based on whether there are any incomplete CP threads associated with the target node in the WAL.

[0053] It's important to note that when there are incomplete CP threads associated with the target node in the WAL, the original in-memory data on the target node may not be fully persisted, resulting in inconsistencies in the page storage on local storage. Conversely, when there are no incomplete CP threads associated with the target node in the WAL, the page storage data should be consistent. Furthermore, depending on whether the page storage data is consistent, the records required for data recovery will also differ. Therefore, the target recovery method for the target node's in-memory data can be determined based on the presence or absence of incomplete CP threads associated with the target node in the WAL.

[0054] In this embodiment of the disclosure, if there are no incomplete CP threads associated with the target node in the WAL, the page storage data is consistent. During data recovery, only the operation information of the memory data needs to be recovered. Therefore, it can be determined that the target recovery method is to recover the memory data of the target node based on the logical record associated with the target node in the WAL.

[0055] It should be noted that logical records can include data records, transaction records, and checkpoint records. Data records document the data content within a node, such as data values ​​and locations. Transaction records document transaction operations within a node, such as clicks and modifications to data. Checkpoint records document the state of a node, allowing for recovery to the most recent consistent data state in the event of a crash.

[0056] Alternatively, if there are incomplete CP threads associated with the target node in the WAL, data recovery can be performed in two stages. First, using the logical records associated with the target node in the WAL, memory data recovery of the target node is performed based on records in the WAL before the start timestamp of the incomplete CP thread. Then, using physical records such as complete page snapshots and incremental data associated with the target node in the WAL, memory data recovery of the target node continues based on records in the WAL after the start timestamp of the incomplete CP thread.

[0057] Step 104: Recover the memory data of the target node according to the target recovery method and the records associated with the target node in the WAL.

[0058] In this embodiment of the disclosure, logical or physical records that conform to the target recovery method can be applied sequentially according to the transaction commit order based on the records associated with the target node in the WAL, and the memory data of the target node can be updated step by step until the latest record in the WAL is applied to obtain the recovered memory data of the target node.

[0059] In this embodiment, the Write-Ahead Log (WAL) is first queried to obtain the checkpoint threads (CPs) associated with the currently initiated target node. Then, based on the timestamps of the associated CPs, it is determined whether there are any incomplete CP threads associated with the target node in the WAL. Next, based on the presence of these incomplete CP threads in the WAL, the target recovery method for the target node's memory data is determined. Finally, based on the target recovery method and the records associated with the target node in the WAL, the target node's memory data is recovered. Therefore, by determining whether the Write-Ahead Log contains incomplete checkpoint threads associated with the target node undergoing data recovery, different data recovery methods are determined to recover the target node's memory data, improving data recovery efficiency, ensuring transactional consistency during data recovery, and enhancing the user experience.

[0060] This embodiment provides another data recovery method. Figure 2 This is a schematic flowchart of a data recovery method provided in an embodiment of this application.

[0061] like Figure 2 As shown, the data recovery method may include the following steps:

[0062] Step 201: Query the Write-Ahead Log (WAL) and obtain the checkpoint (CP) associated with the currently started target node in the WAL.

[0063] Step 202: Based on the timestamp of the associated CP, determine whether there is an incomplete CP thread associated with the target node in the WAL.

[0064] For a detailed description of steps 201 and 202 above, please refer to other embodiments of this disclosure, which will not be repeated here.

[0065] Step 203: If there are incomplete CP threads associated with the target node in the WAL, determine that the target recovery method is to perform memory data recovery on the target node based on the page snapshots and incremental data located after the first timestamp and before the second timestamp in the WAL, and the logical records located after the second timestamp.

[0066] The first timestamp is the start timestamp of the first CP thread associated with the target node, and the second timestamp is the start timestamp of the incomplete CP thread associated with the target node.

[0067] A page snapshot (PS) is a snapshot or copy of memory at a specific point in time. It contains all the data and interface content in memory and can be used to avoid repeatedly applying incremental data.

[0068] Incremental data (Delta) refers to changes in memory that occur after a page snapshot, which can include data that has been added, modified, or deleted.

[0069] Step 204: Recover the memory data of the target node according to the target recovery method and the records associated with the target node in the WAL.

[0070] The following is combined Figure 3 Further explanation is provided on how to recover the memory data of the target node. Figure 3 This is a WAL diagram containing incomplete CP threads. For example... Figure 3 As shown, the WAL contains checkpoints CP1 and CP2 associated with the target node. Data stored on pages prior to CP1 remains consistent. CP1 has a corresponding start timestamp (i.e., the first timestamp, corresponding to...). Figure 3 The CP1 start file and end timestamp (i.e., corresponding to the start file) in the CP1 file. Figure 3 If the CP1 thread ends (in the provided text), then the CP1 thread is complete. CP2 has a corresponding start timestamp (i.e., the second timestamp, corresponding to...). Figure 3 The CP2 thread has a start file (begin file) but no corresponding end timestamp, so the CP2 thread is in an incomplete state.

[0071] Therefore, when restoring the memory data of the target node, the data can be restored sequentially based on the physical records in the WAL that are located after the first timestamp and before the second timestamp, namely page snapshot PS1 and incremental data Delta1, from the page data stored before CP1. At this time, logical records such as data records, transaction records, and checkpoint records are ignored in the CP1 thread in the WAL.

[0072] Then, logical records located after the second timestamp can be used to process the recovered page data according to the transaction order until the WAL processing is completed, thus restoring the target node's memory data. At this point, physical records (i.e., ...) are ignored in the CP2 thread within the WAL. Figure 3 (Page snapshot PS2).

[0073] It is understandable that, such as Figure 3As shown, when the cache crashes, the CP2 thread has started but not yet completed. This means that the page content corresponding to records after CP2 may have been partially written to the page storage, but the write operation was not completed due to interruption of the page write process. Therefore, to recover potentially corrupted page storage, page change records before CP2 (i.e., page snapshot PS1 and incremental data Delta1) can be used to update the same page at the same location in the page storage to ensure the consistency of the recovered page storage data.

[0074] It should be noted that after a transaction operation based on WAL replay is completed, a CP2 end timestamp can be added to the WAL. Furthermore, if any transaction in the WAL only has a start record but no corresponding end record, this transaction does not need to be executed when restoring in-memory data.

[0075] In this embodiment, by restoring the memory data of the target node based on the page snapshots and incremental data located after the first timestamp and before the second timestamp in the WAL, and the logical records located after the second timestamp, when there are incomplete CP threads associated with the target node in the WAL, the reliability of data recovery and transaction consistency are further improved.

[0076] To achieve the above embodiments, this application also proposes a data recovery device.

[0077] Figure 4 This is a schematic diagram of a data recovery device provided in an embodiment of this application.

[0078] like Figure 4 As shown, the data recovery device 400 includes:

[0079] The module 401 is used to query the Write-Ahead Log (WAL) and obtain the checkpoint (CP) associated with the currently started target node in the WAL.

[0080] The first determining module 402 is used to determine whether there is an incomplete CP thread associated with the target node in the WAL based on the timestamp of the associated CP;

[0081] The second determining module 403 is used to determine the target recovery method of the target node's memory data based on whether there are any incomplete CP threads associated with the target node in the WAL.

[0082] Recovery module 404 is used to recover the memory data of the target node according to the target recovery method and the records associated with the target node in the WAL.

[0083] In some embodiments, the first determining module 402 is specifically used for:

[0084] If the associated CP includes the start timestamp of any CP but does not include the end timestamp of any CP, it is determined that there is an incomplete CP thread associated with the target node in the WAL.

[0085] In some embodiments, the first determining module 402 is specifically used for:

[0086] If each CP included in the associated CP has a corresponding start timestamp and end timestamp, it is determined that there are no incomplete CP threads associated with the target node in the WAL.

[0087] In some embodiments, the second determining module 403 is specifically used for:

[0088] If there are no incomplete CP threads associated with the target node in the WAL, the target recovery method is determined to be memory data recovery of the target node based on the logical records associated with the target node in the WAL.

[0089] In some embodiments, the logical record includes the following: data record, transaction record, and checkpoint record.

[0090] In some embodiments, the second determining module 403 is specifically used for:

[0091] In the case that there are incomplete CP threads associated with the target node in the WAL, the target recovery method is determined as follows: based on the page snapshots and incremental data in the WAL that are after the first timestamp and before the second timestamp, and the logical records that are after the second timestamp, the memory data of the target node is recovered. Here, the first timestamp is the start timestamp of the first CP thread associated with the target node, and the second timestamp is the start timestamp of the incomplete CP thread associated with the target node.

[0092] In some embodiments, the data recovery device 400 is further configured to:

[0093] The synchronization module is used to record the modification operation in WAL when it receives a modification operation on the memory data of any node, and to synchronize the dirty data in memory to the persistent file associated with any node through the CP thread.

[0094] It should be noted that the foregoing explanation of the data recovery method embodiment also applies to the data recovery device of this embodiment, and will not be repeated here.

[0095] In this embodiment, the Write-Ahead Log (WAL) is first queried to obtain the checkpoint threads (CPs) associated with the currently started target node. Then, based on the timestamps of the associated CPs, it is determined whether there are any incomplete CP threads associated with the target node in the WAL. Next, based on the presence of these incomplete CP threads in the WAL, the target recovery method for the target node's memory data is determined. Finally, based on the target recovery method and the records associated with the target node in the WAL, the target node's memory data is recovered. Therefore, by determining whether the Write-Ahead Log contains incomplete checkpoint threads associated with the target node undergoing data recovery, different data recovery methods are determined to recover the target node's memory data, improving data recovery efficiency, ensuring transactional consistency during data recovery, and enhancing the user experience.

[0096] To implement the above embodiments, this application also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0097] To implement the above embodiments, this application also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0098] To implement the above embodiments, this application also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0099] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0100] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0101] This application is intended to provide an implementation scheme for users to selectively prevent the use or access to their personal information data. Specifically, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.

[0102] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0103] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0104] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0105] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0106] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0107] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0108] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0109] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A data recovery method, characterized in that, include: Query the Write-Ahead Log (WAL) to obtain the checkpoint (CP) associated with the currently started target node in the WAL. Based on the timestamp of the associated CP, determine whether there is an incomplete CP thread associated with the target node in the WAL; Based on whether there are any incomplete CP threads associated with the target node in the WAL, determine the target recovery method for the memory data of the target node; Based on the target recovery method and the records associated with the target node in the WAL, the memory data of the target node is recovered; The step of determining the target recovery method for the target node's memory data based on whether there are any incomplete CP threads associated with the target node in the WAL includes: If there are incomplete CP threads associated with the target node in the WAL, the target recovery method is determined as follows: based on the page snapshots and incremental data located after the first timestamp and before the second timestamp in the WAL, and the logical records located after the second timestamp, the memory data of the target node is recovered, wherein the first timestamp is the start timestamp of the first CP thread associated with the target node, and the second timestamp is the start timestamp of the incomplete CP thread associated with the target node.

2. The method as described in claim 1, characterized in that, The step of determining whether there are any incomplete CP threads associated with the target node in the WAL based on the timestamp of the associated CP includes: If the associated CP includes the start timestamp of any CP but does not include the end timestamp of any CP, it is determined that there is an incomplete CP thread associated with the target node in the WAL.

3. The method as described in claim 1, characterized in that, The step of determining whether there are any incomplete CP threads associated with the target node in the WAL based on the timestamp of the associated CP includes: If each CP included in the associated CP has a corresponding start timestamp and end timestamp, it is determined that there is no incomplete CP thread associated with the target node in the WAL.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the target recovery method for the target node's memory data based on whether there are any incomplete CP threads associated with the target node in the WAL includes: If there are no incomplete CP threads associated with the target node in the WAL, the target recovery method is determined to be memory data recovery of the target node based on the logical records associated with the target node in the WAL.

5. The method as described in claim 4, characterized in that, The logical records include the following: data records, transaction records, and checkpoint records.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: Upon receiving a modification operation on the memory data of any node, the modification operation is recorded in the WAL, and the dirty data in memory is synchronized to the persistent file associated with any node via the CP thread.

7. A data recovery device, characterized in that, include: The acquisition module is used to query the Write-Ahead Log (WAL) and obtain the checkpoint (CP) associated with the currently started target node in the WAL. The first determining module is used to determine, based on the timestamp of the associated CP, whether there is an incomplete CP thread associated with the target node in the WAL; The second determining module is used to determine the target recovery method for the memory data of the target node based on whether there are any unfinished CP threads associated with the target node in the WAL. The recovery module is used to recover the memory data of the target node according to the target recovery method and the records associated with the target node in the WAL; The second determining module is specifically used for; If there are incomplete CP threads associated with the target node in the WAL, the target recovery method is determined as follows: based on the page snapshots and incremental data located after the first timestamp and before the second timestamp in the WAL, and the logical records located after the second timestamp, the memory data of the target node is recovered, wherein the first timestamp is the start timestamp of the first CP thread associated with the target node, and the second timestamp is the start timestamp of the incomplete CP thread associated with the target node.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the data recovery method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data recovery method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the data recovery method according to any one of claims 1-6.

Citation Information

Patent Citations

  • Data migration method and device for distributed data storage cluster

    CN110807013A

  • Data recovery method and device and storage medium

    CN115858252A