Data recovery method, device, system and electronic equipment for distributed database

By obtaining state record information before power outage in a distributed database and obtaining data from other nodes, the problem of data loss when the storage node is powered off is solved, and efficient data recovery is achieved, avoiding the problems of increased hardware costs and low performance.

CN118503018BActive Publication Date: 2024-12-06HANGZHOU HIKVISION SYST TECH CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202410961196.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-12-06
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

In distributed databases, how to recover metadata lost when the storage node is powered off without increasing hardware costs. In the prior art, the direct write-down method leads to low performance, and the use of powered off protection hardware increases costs.

Method used

By obtaining the state record information of the nearest time before the power outage when the storage node is powered off, determining the data recovery time range, and obtaining relevant data from other storage nodes for recovery, avoiding duplicate data recovery work.

Benefits of technology

It realizes the recovery of data lost when the storage node is powered off without increasing hardware costs, reducing the workload of the data recovery process, avoiding duplicate data recovery due to numerous tasks, and not affecting read and write performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118503018B_ABST
    Figure CN118503018B_ABST
Patent Text Reader

Abstract

The present application discloses a data recovery method, device, system and electronic device for a distributed database, which relates to the field of data recovery technology and is used to recover the data lost by the storage node during power failure as much as possible after the storage node of the distributed database is powered off and powered on again. The method includes: in response to the first storage node being powered on again after power failure, obtaining a preset number of state record information of the first storage node at the most recent time before the power failure, the state record information is used to record the progress status of the task performed by the first storage node at the time when the state record information is generated; based on the recording time of the preset number of state record information, determining the data recovery time range; obtaining data related to the task within the data recovery time range from other storage nodes in the storage node group; and performing data recovery based on the data related to the task within the data recovery time range.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data recovery technology, and in particular to a data recovery method, device, system and electronic equipment for a distributed database. Background Art

[0002] In application scenarios that require uninterrupted data storage, such as real-time video detection scenarios, video files need to be stored uninterruptedly. Distributed databases are currently often used to store data metadata. The distributed database includes multiple storage nodes, and different metadata are stored in different storage nodes. The distributed database can share the storage load.

[0003] When the amount of metadata stored in a storage node reaches a certain amount, the storage node will write the metadata stored in it to the non-volatile memory. In this case, some metadata may be lost when the storage node is powered off. There are currently two ways to avoid this situation: one is that the distributed database uses direct write to disk, does not cache metadata, and directly writes metadata to the non-volatile memory, but this will increase the metadata writing time and lead to poor performance of the distributed database; the other is to use power-off protection hardware (such as uninterruptible power supply (UPS) power supply, etc.), but this will increase the hardware cost of the distributed database.

[0004] Therefore, how to recover the metadata lost by storage nodes in a distributed database during a power outage without increasing the hardware cost of the distributed database is a problem that needs to be solved. Summary of the invention

[0005] The present application provides a distributed database data recovery method, device, system and electronic device, which are used to restore the data lost by the storage node during power failure as much as possible after the storage node of the distributed database is powered off and then powered on again.

[0006] In order to achieve the above technical objectives, this application adopts the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a data recovery method for a distributed database, the distributed database including at least one storage node group, the method being applied to a first storage node in the storage node group, the method comprising:

[0008] In response to the first storage node being powered on again after being powered off, obtaining a preset number of state record information of the first storage node that is most recently before the power off moment, the state record information being used to record the progress state of the task executed by the first storage node at the moment when the state record information is generated;

[0009] Determine the data recovery time range based on the recording time of a preset number of status record information;

[0010] Obtain task-related data within the data recovery time range from other storage nodes in the storage node group;

[0011] Data recovery is performed based on task-related data within the data recovery time range.

[0012] The technical solution provided by the present application brings at least the following beneficial effects: the storage nodes in the distributed database will continuously generate status record information. After the first storage node is powered off and powered on again, by obtaining the state record information of the first storage node at the latest preset number of times before the power off, it can be known what task the first storage node was performing before the power off and the progress status of the executed task. Then, the first storage node can know when the first storage node started to lose data based on the recording time of the preset number of state record information, and thus determine the data recovery time range, that is, the data within the time period that the first storage node needs to recover, which greatly reduces the workload that needs to be processed in the subsequent data recovery process. Since the first storage node is powered off, other storage nodes may not be powered off either. The first storage node can obtain task-related data within the data recovery time range from other storage nodes in the storage node group, thereby recovering as much data as possible that was lost by the first storage node during the power off, accurately determining the time range of the data that needs to be recovered in the massive data, and avoiding a large amount of repeated data recovery work caused by the re-execution of all tasks due to the numerous tasks. Since the data recovery process is carried out in the storage node, there is no need to set power-off protection hardware in the distributed database, and the present application will not increase the hardware cost of the distributed database. This application does not directly write data into non-volatile memory, so it will not affect the read and write performance of the distributed database.

[0013] In one possible implementation, a data recovery time range is determined based on the recording times of a preset number of status record information, including: taking the earliest recording time among the recording times of the preset number of status record information as the starting time of the data recovery time range; taking the latest time during the term of office of the leading node of the first storage node when power is off as the ending time of the data recovery time range.

[0014] In one possible implementation, the latest moment in the term of the leading node of the first storage node when the power is off is used as the end time of the data recovery time range, including: when the first storage node acts as the leading node when the power is off, the moment when the power is off at the first storage node is used as the latest moment in the term of the first storage node; the latest moment in the term of the leading node of the first storage node before the power is off is used as the end time of the data recovery time range.

[0015] In a possible implementation, acquiring data related to the task within the data recovery time range from other storage nodes in the storage node group includes: acquiring data related to the task within the data recovery time range from all other storage nodes in the storage node group.

[0016] In one possible implementation, the latest moment in the term of the leading node of the first storage node when power is cut off is used as the end moment of the data recovery time range, including: when there is a second storage node as the leading node when power is cut off, if the term of the second storage node has not ended at the current moment, the current moment is used as the latest moment in the term of the second storage node; or, if the term of the second storage node has ended before the current moment, the end moment of the term of the second storage node is used as the latest moment in the term of the second storage node; wherein the second storage node is another storage node in the storage node group except the first storage node; the latest moment in the term of the second node is used as the end moment of the data recovery time range.

[0017] In a possible implementation, acquiring data related to the task within the data recovery time range from other storage nodes in the storage node group includes: acquiring data related to the task within the data recovery time range from the second storage node.

[0018] In a possible implementation, the status record information includes at least one of the following: a user request received by the first storage node; a request execution progress of the user request received by the first storage node; and a task execution progress of a background task executed by the first storage node.

[0019] In one possible implementation, in response to the first storage node being powered on again after a power outage, a preset number of status record information of the first storage node at the time most recently before the power outage is obtained, including: in response to the first storage node being powered on again after a power outage, a preset number of status record information of the first storage node at the time most recently before the power outage is obtained from a non-volatile memory.

[0020] In a possible implementation, the method further includes: generating a new state record information at every preset time interval, deleting the state record information with the earliest recording time, so as to save a preset number of state record information.

[0021] In a possible implementation, the method also includes: after the first storage node is powered on and serves as a leading node, after recovering the data related to the task within the data recovery time range, receiving a data recovery request message sent by other storage nodes of the storage node group, the data recovery request message is used to request the first data, the first data is the data within a target time range stored by the first storage node, and the target time range is the time range of the data that other storage nodes need to recover; sending a data recovery response message to the other storage nodes, the data recovery response message including the first data.

[0022] In a possible implementation, the method also includes: when the leading node in the storage node group when the first storage node is powered on is different from the leading node in the storage node group when the first storage node is powered on, receiving a data synchronization task sent by the third storage node that serves as the leading node when the first storage node is powered on, the data synchronization task is used to instruct the first storage node to synchronously store the second data, the second data being data within all terms after the power-off term, and the power-off term being the term of the leading node in the storage node group when the first storage node is powered on; sending a data synchronization request message to the third storage node, the data synchronization request message being used to request the second data; receiving and storing a data synchronization response message sent by the third storage node, the data synchronization response message including the second data.

[0023] In a second aspect, the present application provides a data recovery device for a distributed database, the distributed database comprising at least one storage node group, the storage node group comprising a first storage node, the device comprising: a transmission module and a processing module;

[0024] The transmission module is used for, in response to the first storage node being powered on again after being powered off, obtaining a preset number of state record information of the first storage node that is most recently before the power-off moment, the state record information being used for recording the progress state of the task executed by the first storage node at the moment when the state record information is generated;

[0025] The processing module is used to determine the data recovery time range based on the recording time of the preset number of status record information;

[0026] The processing module is also used to obtain task-related data within the data recovery time range from other storage nodes in the storage node group;

[0027] The processing module is also used to perform data recovery based on task-related data within a data recovery time range.

[0028] In one possible implementation, the processing module is specifically used to: use the earliest recording time among the recording times of a preset number of status record information as the starting time of the data recovery time range; use the latest time during the term of office of the leading node of the first storage node when the power is off as the ending time of the data recovery time range.

[0029] In one possible implementation, the processing module is specifically used to: when the first storage node acts as a leader node during a power outage, use the time when the power outage occurs in the first storage node as the latest time during the term of the first storage node; use the latest time during the term of the leader node before the power outage of the first storage node as the end time of the data recovery time range.

[0030] In a possible implementation, the processing module is specifically configured to obtain task-related data within a data recovery time range from all other storage nodes in the storage node group.

[0031] In one possible implementation, the processing module is specifically used for: when the first storage node is powered off and there is a second storage node as a leader node, if the term of the second storage node has not ended at the current moment, the current moment is used as the latest moment of the second storage node during the term; or, if the term of the second storage node has ended before the current moment, the end moment of the term of the second storage node is used as the latest moment of the second storage node during the term; wherein the second storage node is another storage node in the storage node group except the first storage node; and the latest moment of the second node during the term is used as the end moment of the data recovery time range.

[0032] In a possible implementation manner, the processing module is specifically configured to: obtain task-related data within a data recovery time range from the second storage node.

[0033] In a possible implementation, the status record information includes at least one of the following: a user request received by the first storage node; a request execution progress of the user request received by the first storage node; and a task execution progress of a background task executed by the first storage node.

[0034] In a possible implementation, the transmission module is specifically configured to obtain, in response to the first storage node being powered on again after being powered off, a preset number of state record information of the first storage node most recently before the power-off moment from the non-volatile memory.

[0035] In a possible implementation, the processing module is further configured to generate a new state record information at every preset time interval, and delete the state record information with the earliest recording time, so as to save a preset number of state record information.

[0036] In one possible implementation, the transmission module is also used for: after the first storage node is powered on and acts as a leading node, after recovering the data related to the task within the data recovery time range, receiving a data recovery request message sent by other storage nodes in the storage node group, the data recovery request message is used to request the first data, the first data being the data within a target time range stored by the first storage node, and the target time range being the time range of the data that other storage nodes need to recover; sending a data recovery response message to other storage nodes, the data recovery response message including the first data.

[0037] In one possible implementation, the transmission module is also used for: when the leading node in the storage node group when the first storage node is powered on is different from the leading node in the storage node group when the first storage node is powered on, receiving a data synchronization task sent by the third storage node that serves as the leading node when the first storage node is powered on, the data synchronization task is used to instruct the first storage node to synchronously store the second data, the second data being data within all terms after the power-off term, the power-off term being the term of the leading node in the storage node group when the first storage node is powered off; sending a data synchronization request message to the third storage node, the data synchronization request message being used to request the second data; receiving a data synchronization response message sent by the third storage node, the data synchronization response message including the second data; the data recovery device of the distributed database also includes a storage module, the storage module being used to store the data synchronization response message sent by the third storage node.

[0038] In a third aspect, the present application provides a data recovery system for a distributed database, the distributed database comprising at least one storage node group, the storage node group comprising a first storage node and at least one other storage node; the system comprising: a first storage node and other storage nodes;

[0039] The first storage node is used to, in response to the first storage node being powered on again after being powered off, obtain a preset number of state record information of the first storage node that is most recent before the power-off moment, the state record information being used to record the progress state of the task executed by the first storage node at the moment when the state record information is generated;

[0040] The first storage node is further used to determine a data recovery time range based on the recording time of a preset number of status record information;

[0041] The first storage node is further used to obtain task-related data within a data recovery time range from other storage nodes in the storage node group;

[0042] The other storage nodes are used to provide the first storage node with task-related data within a data recovery time range;

[0043] The first storage node is further used to perform data recovery based on task-related data within a data recovery time range.

[0044] In a fourth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any distributed database data recovery method provided in the first aspect.

[0045] In a fifth aspect, the present application provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement any of the distributed database data recovery methods provided in the first aspect above.

[0046] In a sixth aspect, the present application provides a computer program product, including computer instructions, which, when executed by a processor, implement any distributed database data recovery method provided in the first aspect above.

[0047] For the specific description of the second to sixth aspects and their various implementations in the present application, reference may be made to the detailed description in the first aspect and its various implementations; and for the beneficial effects of the second to sixth aspects and their various implementations, reference may be made to the beneficial effects analysis in the first aspect and its various implementations, which will not be repeated here.

[0048] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of the structure of a distributed database data recovery system applicable to a distributed database data recovery method provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of the hardware structure of a computing device provided in an embodiment of the present application;

[0051] Figure 3 A flowchart of a distributed database data recovery method provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of an application scenario of a distributed database data recovery method provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of the application logic of a distributed database data recovery method provided in an embodiment of the present application Figure 1 ;

[0054] Figure 6A schematic diagram of the application logic of a distributed database data recovery method provided in an embodiment of the present application Figure 2 ;

[0055] Figure 7 A logical schematic diagram of a data recovery method for a distributed database provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of the structure of a data recovery device for a distributed database provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. The terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, "multiple" means two or more.

[0059] In application scenarios that require uninterrupted data storage, such as real-time video detection scenarios, video files need to be stored uninterruptedly. Distributed databases are currently often used to store data metadata. The distributed database includes multiple storage nodes, and different metadata are stored in different storage nodes. The distributed database can share the storage load.

[0060] When the amount of metadata stored in a storage node reaches a certain amount, the storage node will write the metadata stored in it to the non-volatile memory. In this case, some metadata may be lost when the storage node is powered off. There are currently two ways to avoid this situation: one is that the distributed database uses direct write to disk, does not cache metadata, and directly writes metadata to the non-volatile memory, but this will increase the metadata writing time and lead to poor performance of the distributed database; the other is to use power-off protection hardware (such as uninterruptible power supply (UPS) power supply, etc.), but this will increase the hardware cost of the distributed database.

[0061] In this regard, the embodiment of the present application provides a data recovery method for a distributed database, in which the storage nodes in the distributed database continuously generate status record information. Then, after the first storage node is powered off and then powered on again, by obtaining the state record information of the first storage node at the latest preset number of times before the power outage, it is possible to know what task the first storage node was executing before the power outage and the progress status of the executed task. Furthermore, the first storage node can know when the first storage node started to lose data based on the recording time of the preset number of state record information, and thus determine the data recovery time range, that is, the data within the time period that the first storage node needs to recover, which greatly reduces the workload that needs to be processed in the subsequent data recovery process. Since the first storage node is powered off, other storage nodes may not be powered off either, so the first storage node can obtain task-related data within the data recovery time range from other storage nodes in the storage node group, thereby recovering as much data lost by the first storage node during the power outage as possible, accurately determining the time range of the data that needs to be recovered in the massive data, and avoiding a large amount of repeated data recovery work caused by the re-execution of all tasks due to the numerous tasks. Since the data recovery process is performed in the storage node, there is no need to set up power failure protection hardware in the distributed database, and this application will not increase the hardware cost of the distributed database. This application does not directly write data into the non-volatile memory, so it will not affect the read and write performance of the distributed database.

[0062] Please refer to Figure 1 , which shows a schematic diagram of the structure of a distributed database data recovery system to which the distributed database data recovery method provided by the present application is applicable. Figure 1 As shown, the data recovery system 1 of a distributed database may include: a plurality of storage nodes 10 .

[0063] The plurality of storage nodes 10 are divided into a plurality of storage node groups according to user needs. For example, the data recovery system 1 of the distributed database includes 3 storage nodes (storage node A, storage node B and storage node C), and the user needs 100 storage node groups with 3 copies (each copy of data is synchronously stored in three storage nodes), then each storage node group of the 100 storage node groups finally divided includes storage node A, storage node B and storage node C.

[0064] Optionally, the storage node groups included in the data recovery system 1 for a distributed database may be the same or different. For example, the data recovery system 1 for a distributed database includes three storage nodes (storage node A, storage node B, and storage node C), and the user requires 10 storage node groups with two copies (each copy of data is synchronously stored in two storage nodes). Then, the 10 storage node groups finally divided may include: storage node group ① (storage node A and storage node B), storage node group ② (storage node B and storage node C), storage node group ③ (storage node A and storage node B), storage node group ④ (storage node B and storage node C), storage node group ⑤ (storage node A and storage node B), storage node group ⑥ (storage node A and storage node C), storage node group ⑦ (storage node A and storage node C), storage node group ⑧ (storage node B and storage node C), storage node group ⑨ (storage node A and storage node B), storage node group ⑩ (storage node B and storage node C).

[0065] A communication connection is established between each storage node 10 in each storage node group. It should be understood that the connection method can be a wireless connection, such as a Bluetooth connection, a wireless fidelity (Wi-Fi) connection, etc.; or, the connection method can also be a wired connection, such as an optical fiber connection, etc., without limitation. Exemplarily, a storage node group includes two storage nodes, and the two storage nodes can be connected to the Internet through a router, thereby realizing a communication connection between the two storage nodes.

[0066] In some embodiments, the storage node 10 is used to store metadata of data. For example, if the data is a video file, the metadata of the video data may include the duration of the video file, identification information of the video file, the time when the video file was generated, the amount of storage resources occupied by the video file, and the storage location of the video file.

[0067] Each storage node group includes a leader node, and the other storage nodes in the storage node group are non-leader nodes (follow nodes). The leader node is connected to the user device and can receive user requests from the user device, such as data access requests. The leader node then queries the metadata corresponding to the data that the user wants to access as indicated by the data access request, and determines the storage location of the data that the user wants to access from the metadata. The leader node then sends the storage location of the data that the user wants to access to the user device, so that the user device can read the data that the user wants to access based on the storage location of the data that the user wants to access. The follower node is connected to the leader node to receive and store the metadata sent by the leader node, thereby synchronizing the metadata stored by the leader node and the follower node, and realizing backup storage of the metadata.

[0068] When the leader node loses power, a follower node in the storage node group that is not powered off will become the new leader node, interacting with the user device uninterruptedly to ensure the user experience.

[0069] In some embodiments, a storage node group in the data recovery system 1 of a distributed database includes a first storage node and other storage nodes. The first storage node is used to obtain a preset number of state record information of the first storage node before the power-off moment in response to the first storage node being powered on again after power failure, and the state record information is used to record the progress status of the task performed by the first storage node at the time when the state record information is generated; the first storage node is also used to determine the data recovery time range based on the recording time of the preset number of state record information; the first storage node is also used to obtain task-related data within the data recovery time range from other storage nodes in the storage node group; other storage nodes are used to provide the first storage node with task-related data within the data recovery time range; the first storage node is also used to perform data recovery based on the task-related data within the data recovery time range.

[0070] In some embodiments, the storage node 10 includes a business module and a snapshot module. The business module is used to process user requests and background tasks. When the business module receives a user request, the request execution progress of the user request changes, or the task execution progress of the background task changes, the business module will send information to the snapshot module to trigger the snapshot module to update the status record information. The snapshot module can also be called a status collection module.

[0071] In some embodiments, the storage node 10 may be a single server or a server cluster, or the storage node 10 may be a terminal device, such as a personal computer (PC), a notebook computer, a mobile device, a tablet computer, a laptop computer, etc. The embodiments of the present application do not limit the specific form of the storage node 10.

[0072] The hardware structure of the storage node 10 includes: Figure 2 The computing device shown in FIG. Figure 2 Taking the computing device shown as an example, the hardware structure of the storage node 10 is introduced.

[0073] like Figure 2 As shown, the computing device may include a processor 201, a memory 202, a communication interface 203, and a bus 204. The processor 201, the memory 202, and the communication interface 203 may be connected via a bus 204.

[0074] The processor 201 is the control center of the computing device, and can be a processor or a general term for multiple processing elements. For example, the processor 201 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0075] As an embodiment, the processor 201 may include one or more CPUs, such as Figure 2 CPU 0 and CPU 1 are shown in .

[0076] The memory 202 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0077] In a possible implementation, the memory 202 may exist independently of the processor 201, and the memory 202 may be connected to the processor 201 via a bus 204 to store instructions or program codes. When the processor 201 calls and executes the instructions or program codes stored in the memory 202, the model deployment method provided in the embodiment of the present application can be implemented.

[0078] In another possible implementation, the memory 202 may also be integrated with the processor 201 .

[0079] The communication interface 203 is used for the computing device to connect to other devices through a communication network, which may be Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 203 may include a receiving unit for receiving data and a sending unit for sending data.

[0080] The bus 204 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0081] It should be pointed out that Figure 2 The structure shown in the figure does not constitute a limitation on the computing device, except Figure 2 In addition to the components shown, the computing device may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0082] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0083] The data recovery method for a distributed database provided in an embodiment of the present application may be executed by a first storage node, and the first storage node may be any storage node in any storage node group in the data recovery system for the distributed database described above.

[0084] like Figure 3 As shown, an embodiment of the present application provides a data recovery method for a distributed database, the method comprising the following steps:

[0085] S101 . In response to a first storage node being powered on again after being powered off, obtaining a preset number of state record information of the first storage node most recently before the power off moment.

[0086] The status record information is used to record the progress status of the task executed by the first storage node at the time when the status record information is generated.

[0087] In some embodiments, the status record information includes at least one of the following: a user request received by the first storage node; a request execution progress of the user request received by the first storage node; a task execution progress of a background task executed by the first storage node.

[0088] The background task performed by the first storage node may be a task of compressing data, a task of synchronizing data stored in other storage nodes, and the like.

[0089] In some embodiments, the leader node has a term corresponding to it, and when the leader node changes, the term changes accordingly. For example, when storage node A is the leader node, the identification information of the term is 1. When storage node A is powered off, storage node B in the same storage node group becomes the leader node, and the identification information of the term is 2. If the term is known, it is known which storage node is the leader node and in what time period.

[0090] Specifically, when the term is represented by a number, if the term changes, the number of the new term is the number of the previous term plus 1. For example, storage node A is the leader node with a term of 1. After storage node A is powered off, storage node B becomes the leader node, and the term of storage node B is 2.

[0091] In actual applications, the first storage node records the received user request as a commit value, which includes the term of the first storage node when receiving the user request and the order in which the user request was received during the term. The first storage node records the execution progress of the received user request as an apply value, which includes the term of the first storage node when receiving the user request and the order in which the user requests have been executed during the term. The task execution progress of the background task executed by the first storage node recorded by the first storage node may include: the term of the background task executed by the first storage node, the table corresponding to the executed background task, the table number corresponding to the executed background task, the keyword / word in the table corresponding to the executed background task, index, etc.

[0092] For example, storage node A is the leader node with a term of office 1. The committee value corresponding to the first user request received during term 1 is (term 1, order 1), the committee value corresponding to the second user request received is (term 1, order 2), the committee value corresponding to the third user request received is (term 1, order 3), and the committee value corresponding to the third user request received is (term 1, order 4). At this time, storage node A has completed execution of the third user request, but has not completed execution of the fourth user request. The apply value corresponding to the execution progress of the user request is (term 1, order 3).

[0093] Optionally, the committee value can also be expressed as a numeric value containing a decimal point. The number before the decimal point represents the term, and the number after the decimal point represents the order of user requests received during the term. For example, a committee value of 1.1 represents the first user request received during term 1.

[0094] Similarly, the apply value can also be expressed as a numerical value containing a decimal point. The number before the decimal point represents the term, and the number after the decimal point represents the order of user requests that have been executed within the term. For example, an apply value of 1.3 means that the third user request received within term 1 has been executed.

[0095] In some embodiments, the first storage node generates a new state record information at a preset time interval, and deletes the state record information with the earliest recording time, so as to save a preset number of state record information.

[0096] For example, Figure 4 As shown, the first storage node collects the progress status of the task currently executed by the first storage node in real time, and saves the status record information once every preset time. Each time the status record information is saved is called a status snapshot. Figure 4 In the above example, time T1 is the last-snapshot of the status saved, time T2 is the last-snapshot of the status saved, and time T3 is the time point at which the snapshot will be saved in the future. The time interval between T1 and T2 is the preset time length, and the time interval between T2 and T3 is the preset time length. After the snapshot is saved at time T3, the snapshot saved at time T1 will be deleted.

[0097] The preset duration can be changed according to user needs and can be determined based on the latest disk write time of the first storage node asynchronously. For example, if the latest disk write time is 30 seconds, the preset duration can be set to be greater than 30 seconds. This can expand the data recovery time range and ensure that the final recovered data is more complete.

[0098] In another example, the preset duration is 2 minutes, the preset number is 2, the first storage node generates a status record information at 2:00:00, generates another status record information at 2:02:00, and when the status information is generated again at 2:04:00, the status record information generated at 2:00:00 will be deleted. Between 2:02:00-2:04:00, the first storage node stores the status record information generated at 2:00:00 and 2:02:00, and between 2:04:00-2:06:00, the first storage node stores the status record information generated at 2:02:00 and 2:04:00.

[0099] By setting the preset duration and the preset number, it can be assumed that the status record information with the earliest recording time among the preset number of latest status record information has been written to the disk and will not be lost even if the power is off. Therefore, when performing subsequent data recovery, only the data after the status record information with the earliest recording time among the preset number of latest status record information is considered for recovery.

[0100] For example, Figure 5 As shown, the snapshot module conducts regular inspections, and the default inspection is once every 2 minutes (users can configure it according to user needs). During the inspection, the snapshot module reads and traverses the status record information corresponding to all services executed by all business modules, analyzes each status record information, and records whether all status record information has been traversed. If all status record information has been traversed, the snapshot module continues to inspect, read and traverse the status record information corresponding to all services executed by all business modules. If all status record information has not been traversed, determine whether temporary status record information (value_tmp) is generated within the inspection interval. If not, record and analyze the next status record information. If value_tmp is generated, assign the status record information (value) to the last recorded status record information (last_value), assign value_tmp to value, check the timestamp of value_tmp, and determine whether the timestamp of value_tmp has been through an inspection cycle compared with the current time. If the timestamp of value_tmp has been through an inspection cycle compared with the current time, delete the value_tmp. If the timestamp of value_tmp has not been through an inspection cycle compared with the current time, check the next status record information until all status record information corresponding to all services are traversed. After the inspection cycle ends, perform the next round of inspection.

[0101] This application proposes the concept of snapshot based on the characteristics of distributed databases, and uses the concept of snapshot = data recovery to perform data recovery work during power outages.

[0102] In some embodiments, the first storage node will also generate temporary status record information within a preset duration. The temporary status record information is the status record information generated by the first storage node when the progress status of the task executed by the first storage node changes. Only one temporary status record information is generated within the preset duration, that is, each time the progress status of the task executed by the first storage node changes, the first storage node generates a temporary status record information and deletes the temporary status record information generated last time. When the preset duration ends, the first storage node saves the latest temporary status record information generated within the preset duration as the status record information. For example, a status record information is generated at time T1, and the status record information is written to the disk. After time T1, when the progress status of the task executed by the first storage node changes for the first time, the first storage node generates temporary status record information 1. When the progress status changes for the second time, the first storage node generates temporary status record information 2. At this time, the first storage node deletes temporary status record information 1. If the progress status does not change again before T2, then at time T2, the first storage node writes temporary status record information 2 to the disk. The time interval between time T1 and time T2 is the preset duration.

[0103] For example, Figure 6 As shown, the business module updates the status of the business it executes, and updates the latest business status to the snapshot module. The snapshot module first checks whether it has saved the status of the business before. If it has not saved the status of the business, it returns an error (status missing). If the status of the business has been saved, it sets the latest status to the temporary status record information and returns success. The business module receives the return value sent by the snapshot module. If the return value is an error, the business module first reads the stored status record information, calls the snapshot module, and sends the stored status record information to the snapshot module; if the return value is successful, the snapshot module directly saves the latest status record information. The snapshot module saves the current latest status record information, assigns the previous status record information to last_value, and assigns the latest status record information to value and value_tmp. The business module updates the latest status record information.

[0104] In some embodiments, step S101 may be specifically implemented as: in response to the first storage node being powered on again after being powered off, obtaining a preset number of state record information of the first storage node most recently before the power off moment from a non-volatile memory.

[0105] Among them, non-volatile memory is a memory that will not lose data when power is off, and can specifically be a hard disk, a solid-state drive, etc.

[0106] For example, based on Figure 4After time T2, the first storage node is powered off. If at the current time (T-now) (time T-now is after time T2), the first storage node is online again, and the preset number is 2, then the two most recent state record information before the power off time is the state record information recorded at time T1 and the state record information recorded at time T2. The first storage node obtains the state record information recorded at time T1 and the state record information recorded at time T2 from the non-volatile memory.

[0107] In this way, by obtaining a preset number of status record information of the first storage node most recently before the power outage from the non-volatile memory, the first storage node can determine the time range of the data that needs to be restored in the subsequent process, and then obtain the data that needs to be restored from other storage nodes in the storage node group.

[0108] S102: Determine a data recovery time range based on the recording times of a preset number of status record information.

[0109] In some embodiments, step S102 can be specifically implemented as follows: the first storage node uses the earliest recording time among the recording times of a preset number of status record information as the starting time of the data recovery time range; the first storage node uses the latest time during the term of office of the leader node of the first storage node when the power is off as the end time of the data recovery time range.

[0110] As a possible implementation method, the first storage node uses the latest moment of the term of the leading node when the first storage node is powered off as the end time of the data recovery time range. It can be specifically implemented as follows: when the first storage node acts as the leading node when the power is off, the first storage node uses the moment when the power is off at the first storage node as the latest moment of the term of the first storage node; the first storage node uses the latest moment of the term of the leading node before the power is off as the end time of the data recovery time range.

[0111] Exemplarily, when power is cut off, storage node A is the leader node in the storage node group with a term of 1, the power off time is 2:07:00, the preset duration is 2 minutes, and the time nodes of the two most recent saved status record information before the power off are 2:04:00 and 2:06:00. When storage node A is powered off, storage node B becomes the leader node in the storage node group with a term of 2. Storage node A is powered on again at 3:01:00, then storage node A can determine that the starting time of the data recovery time range is 2:04:00, and the end time of the data recovery time range is 2:07:00.

[0112] In this way, the first storage node is the leading node when power is cut off. By taking the latest time during the term of the first storage node as the end time of the data recovery range, the first storage node can determine the data that needs to be recovered during the term when power is cut off, and thus perform data recovery operations in subsequent processes.

[0113] Based on this, step S103 can be specifically implemented as follows: the first storage node obtains task-related data within the data recovery time range from all other storage nodes in the storage node group.

[0114] Exemplarily, storage node A is the leading node when power is cut off, and the determined data recovery time range is 2:04:00-2:07:00. The storage node group also includes storage node B and storage node C. After storage node A is powered on again, storage node A obtains data within 2:04:00-2:07:00 from storage node B and storage node C.

[0115] In this way, the first storage node is the leader node when power is cut off. By obtaining task-related data within the data recovery time range from all other storage nodes in the storage node group, the data recovered by the first storage node is the most complete data, which makes it convenient for the first storage node to synchronize data to other storage nodes later.

[0116] As another possible implementation, the first storage node uses the latest moment of the term of the leader node of the first storage node when the power is cut off as the end time of the data recovery time range. It can be specifically implemented as follows: when there is a second storage node as the leader node when the first storage node is powered off, if the term of the second storage node has not ended at the current moment, the first storage node uses the current moment as the latest moment of the second storage node within its term; or, if the term of the second storage node has ended before the current moment, the first storage node uses the end time of the second storage node's term as the latest moment of the second storage node within its term; wherein the second storage node is another storage node in the storage node group except the first storage node; the first storage node uses the latest moment of the second node within its term as the end time of the data recovery time range.

[0117] In an example, the storage node group includes storage node A, storage node B and storage node C, and the preset duration is 2 minutes. At 2:07:00, storage node A is powered off. The time nodes of the two most recent saved status record information before the power outage are 2:04:00 and 2:06:00. At this time, storage node B is the leader node. At 2:35:00, storage node A is powered on again. At this time, storage node B is still the leader node. Then storage node A can determine that the starting time of the data recovery time range is 2:04:00, and the end time of the data recovery time range is 2:35:00.

[0118] In another example, the storage node group includes storage node A, storage node B and storage node C. The preset duration is 2 minutes. At 2:07:00, storage node A is powered off. The two most recent time nodes for saving status record information before the power outage are 2:04:00 and 2:06:00. At this time, storage node B is the leader node. At 2:35:00, storage node A is powered on again. At this time, the leader node is storage node C. Storage node B ends its term at 2:28:00, and storage node C becomes the leader node. Then storage node A can determine that the starting time of the data recovery time range is 2:04:00, and the end time of the data recovery time range is 2:28:00.

[0119] In this way, the first storage node is not the leader node when it is powered off. By determining the data recovery time range, the task-related data within the data recovery time range can be obtained from the leader node when the first storage node in the storage node group is powered off. Finally, the first storage node can restore the data of the term when the power was off, and the synchronization of the data stored in each storage node in the storage node group can be achieved.

[0120] Based on this, step S103 can be specifically implemented as follows: the first storage node obtains task-related data within the data recovery time range from the second storage node.

[0121] Exemplarily, when storage node A is powered off, storage node B is the leader node. After storage node A is powered on again, the data recovery time range is determined to be 2:04:00-2:28:00, and storage node A obtains data within 2:04:00-2:28:00 from storage node B.

[0122] In this way, the first storage node is not the leader node when power is cut off. By obtaining task-related data within the data recovery time range from the leader node when the first storage node in the storage node group is powered off, the first storage node recovers the data of the term when power is cut off, thereby realizing the synchronization of data stored in each storage node in the storage node group.

[0123] S103. Obtain task-related data within the data recovery time range from other storage nodes in the storage node group.

[0124] In some embodiments, the first storage node that is powered on again after a power outage actively initiates a data recovery request to other storage nodes, and the data recovery request includes a data recovery time range.

[0125] For example, the storage node group includes storage node A, storage node B and storage node C. Storage node A is the leader node when it is powered off. After storage node A is powered on again to determine the data recovery range, it actively sends a data recovery request to storage node B and storage node C to request storage node B and storage node C to send the data within the data recovery range to storage node A.

[0126] For another example, a storage node group includes storage node A, storage node B, and storage node C. When storage node A is powered off, storage node B is the leader node. After storage node A is powered on again to determine the data recovery range, it actively sends a data recovery request to storage node B to request storage node B to send the data within the data recovery range to storage node A.

[0127] In this way, the first storage node actively initiates a data recovery request after power failure and power on again, and the first storage node calculates the time range of the data to be recovered by itself, and the processing logic is orderly, and the workload of other storage nodes will not be increased.

[0128] In some embodiments, if the storage node that receives the data recovery request has a data recovery task in progress, it will process data recovery requests sent by other storage nodes only after the storage node completes its own data recovery.

[0129] Optionally, if the storage node does not receive data corresponding to the issued data recovery request, the storage node continuously issues data recovery requests at a preset frequency, so as to process the data recovery requests issued by the storage node in a timely manner when other storage nodes are idle.

[0130] S104. Perform data recovery based on task-related data within the data recovery time range.

[0131] In some embodiments, after the first storage node receives task-related data within the data recovery time range sent by other storage nodes, it can determine the speed of data recovery and the amount of data processed each time according to its own business volume and its own hardware performance.

[0132] For example, the first storage node currently executes a large amount of business, and the hardware performance of the first storage node is not that strong, and there are not many available processing resources. In this case, the first storage node can reduce the speed of data recovery and reduce the amount of data recovered each time. When there are more available processing resources, the speed of data recovery can be increased and the amount of data recovered each time can be increased.

[0133] In this way, the first storage node determines the data recovery speed and the amount of data to be recovered each time by itself, which can not only ensure the normal execution of the business of the first storage node itself, but also gradually recover the data lost during power failure.

[0134] In some embodiments, when the first storage node acts as a leading node after being powered on, after recovering data related to a task within a data recovery time range, the first storage node receives a data recovery request message sent by other storage nodes in the storage node group, where the data recovery request message is used to request first data, where the first data is data within a target time range stored by the first storage node, and the target time range is a time range for data that other storage nodes need to recover; the first storage node sends a data recovery response message to other storage nodes, where the data recovery response message includes the first data.

[0135] It should be understood that if the first storage node is powered off and other storage nodes are not powered off, the leader node in the storage node group will change. If all storage nodes are powered off at the same time and the first storage node is powered on the earliest, the leader node in the storage node group will still be the first storage node.

[0136] Exemplarily, the storage node group includes storage node A, storage node B and storage node C. After storage node A is powered on, it is the leading node, and its data recovery operation is to obtain data within the data recovery time range from storage node B and storage node C. Even if storage node A, storage node B and storage node C are powered off at the same time, these three storage nodes will not necessarily lose data, and even if data is lost, it is not necessarily all data. After the recovery is completed, the data in storage node A is the most complete and has the largest range. After storage node A completes the data recovery operation, it starts to process the data recovery request messages sent by storage node B and storage node C respectively to obtain the data recovered by them. For example, assuming that storage node A, storage node B, and storage node C are all powered off at 02:15:23, storage node A is powered on again at 02:26:31, storage node B is powered on again at 02:34:53, and storage node C is powered on again at 02:42:22, and storage node B determines that the time range of the data that needs to be restored is 02:12:23-02:34:53, then after completing its own data recovery process, storage node A sends a data recovery response message including the data within 02:12:23-02:34:53 to storage node B. Storage node C determines that the time range of the data that needs to be restored is 02:13:13-02:42:22, then after completing its own data recovery process, storage node A sends a data recovery response message including the data within 02:13:13-02:42:22 to storage node C. Thus, storage node B and storage node C receive the data recovery response message and store the data in the data recovery response message, that is, synchronize the data in storage node A. At this time, the data stored in storage node A, storage node B and storage node C are synchronized and as much lost data as possible is recovered.

[0137] In this way, after the first storage node is powered off and then powered on again, after completing the data recovery operation, by sending data synchronization tasks to other storage nodes, the data stored in each storage node in the storage node group is synchronized, and the data lost due to power failure is restored as much as possible.

[0138] In some embodiments, after receiving the data recovery request message sent by other storage nodes, the leader node decides whether to respond to the data recovery request message sent by other storage nodes according to the task it performs. For example, if the leader node receives the data recovery request message sent by other storage nodes and the leader node is in the process of data recovery, the leader node does not respond to the data recovery request message sent by other storage nodes.

[0139] In some embodiments, if the leader node does not respond to the data recovery request message sent by the storage node, the storage node sends a data recovery request message to the leader node at a preset interval until the leader node responds to the data recovery request message and sends a data recovery response message to the storage node.

[0140] In some embodiments, when the leading node in the storage node group when the first storage node is powered on is different from the leading node in the storage node group when the first storage node is powered on, the first storage node receives a data synchronization task sent by the third storage node that serves as the leading node when the first storage node is powered on, and the data synchronization task is used to instruct the first storage node to synchronously store the second data, where the second data is data within all terms after the power-off term, and the power-off term is the term of the leading node in the storage node group when the first storage node is powered on; the first storage node sends a data synchronization request message to the third storage node, and the data synchronization request message is used to request the second data; the first storage node receives and stores the data synchronization response message sent by the third storage node, and the data synchronization response message includes the second data.

[0141] Exemplarily, the storage node group includes storage node A, storage node B and storage node C. When storage node A is powered off, the leading node is storage node B. When storage node A is powered on, the leading node is storage node C. Based on the aforementioned data recovery process, storage node A only recovers the data within the term of storage node B, and there is no data within the term after the term of storage node B. Then, after storage node A is powered on, storage node C sends a data synchronization task to storage node A, so that storage node A can recover the data within the term after the term of storage node B.

[0142] In this way, the first storage node not only recovers the data during the term of the leader node when the power is off, but also recovers the data during all terms after the term of the leader node when the power is off by receiving data synchronization tasks sent by the leader node in the new term. The first storage node recovers more data and keeps it synchronized with the data stored by the leader node.

[0143] In some embodiments, after the leader node initiates a data synchronization task to other storage nodes, the other storage nodes may decide whether to respond to the data synchronization task initiated by the leader node based on the tasks they are performing. For example, if the leader node initiates a data synchronization task and other storage nodes are in the process of data recovery, the other storage nodes will not respond to the data synchronization task.

[0144] In some embodiments, if the storage node does not respond to the data synchronization task sent by the leader node, the leader node sends the data synchronization task to the storage node at a preset interval until the storage node responds to the data synchronization task and sends a data synchronization request message to the leader node.

[0145] In some embodiments, when the first storage node generates status record information for the first time, the first storage node determines the first generated status record information as the status record information generated at the last preset time interval, that is, when the first storage node generates status record information for the first time, the first generated status record information is used as the last value.

[0146] In some embodiments, the above-mentioned data recovery process may pre-set a power failure recovery function, and the user may choose whether to use the power failure recovery function according to their own needs. When the user chooses to turn on the power failure recovery function, all storage nodes in the data recovery system of the distributed database restart to turn on the power failure recovery function.

[0147] Figure 3The technical solution shown at least brings the following beneficial effects: the storage nodes in the distributed database will continuously generate status record information. After the first storage node is powered off and powered on again, by obtaining the state record information of the first storage node at the latest preset number of times before the power off, it can be known what task the first storage node was performing before the power off and the progress status of the task being performed. Then, the first storage node can know when the first storage node started to lose data based on the recording time of the preset number of state record information, and thus determine the data recovery time range, that is, the data within the time period that the first storage node needs to recover, which greatly reduces the workload that needs to be processed in the subsequent data recovery process. Since the first storage node is powered off, other storage nodes may not be powered off either. The first storage node can obtain task-related data within the data recovery time range from other storage nodes in the storage node group, thereby recovering as much data as possible that was lost by the first storage node during the power off, accurately determining the time range of the data that needs to be recovered in the massive data, and avoiding a large amount of repeated data recovery work caused by the re-execution of all tasks due to the numerous tasks. Since the data recovery process is carried out in the storage node, there is no need to set power-off protection hardware in the distributed database, and the present application will not increase the hardware cost of the distributed database. This application does not directly write data into non-volatile memory, so it will not affect the read and write performance of the distributed database.

[0148] In summary, the solution provided by this application is described below from an overall perspective:

[0149] like Figure 7 As shown in the figure, trigger a status record information update to ensure that the status record information is written to the disk; analyze the status of the leader node's term recorded in the snapshot to determine whether data synchronization is required. If synchronization is not required, delete it directly. The scenarios where data recovery is not required are as follows:

[0150] a) The storage node is a follow node, but the last_value contains only one term of data;

[0151] b) The storage node is a follower node, but the leader node information is not saved in last_value and value;

[0152] c) The storage node is a leader node, but the term information does not contain any follower node information;

[0153] If data synchronization is required:

[0154] Determine whether the storage node is the leader node when the power is off. If not, directly initiate a data synchronization request to the leader node when the storage node is powered off;

[0155] If the storage node is the leader node when it is powered off, and the storage node is still the leader node when it is powered on, notify other storage nodes to initiate data synchronization requests to this leader node;

[0156] If the storage node is the leader node when it is powered off, and the storage node is not the leader node when it is powered on, a data synchronization request is initiated to other storage nodes.

[0157] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy to realize that the technical goals in this field are combined with the units and algorithm steps of each example described in the embodiments disclosed in this article, and the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical goals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0158] like Figure 8 As shown, the embodiment of the present application also provides a distributed database data recovery device, which is used to execute the distributed database data recovery method shown in the above method embodiment. The distributed database data recovery device 300 includes: a transmission module 301, a processing module 302 and a storage module 303.

[0159] Among them, the transmission module 301 is used to obtain a preset number of status record information of the first storage node most recently before the power-off moment in response to the first storage node being powered on again after a power-off moment, and the status record information is used to record the progress status of the task performed by the first storage node at the moment when the status record information is generated; the processing module 302 is used to determine the data recovery time range based on the recording time of the preset number of status record information; the processing module 302 is also used to obtain task-related data within the data recovery time range from other storage nodes in the storage node group; the processing module 302 is also used to perform data recovery based on the task-related data within the data recovery time range.

[0160] In one possible implementation, the processing module 302 is specifically used to: use the earliest recording time among the recording times of a preset number of status record information as the starting time of the data recovery time range; use the latest time during the term of office of the leading node of the first storage node when the power is off as the ending time of the data recovery time range.

[0161] In another possible implementation, the processing module 302 is specifically used for: when the first storage node acts as a leader node during a power outage, taking the time when the power outage occurs on the first storage node as the latest time during the term of the first storage node; taking the latest time during the term of the leader node before the power outage of the first storage node as the end time of the data recovery time range.

[0162] In another possible implementation, the processing module 302 is specifically configured to obtain task-related data within a data recovery time range from all other storage nodes in the storage node group.

[0163] In another possible implementation, the processing module 302 is specifically used for: when the first storage node is powered off and there is a second storage node as a leader node, if the term of the second storage node has not ended at the current moment, the current moment is used as the latest moment of the second storage node during the term; or, if the term of the second storage node has ended before the current moment, the end moment of the term of the second storage node is used as the latest moment of the second storage node during the term; wherein the second storage node is another storage node in the storage node group except the first storage node; the latest moment of the second node during the term is used as the end moment of the data recovery time range.

[0164] In another possible implementation, the processing module 302 is specifically configured to obtain task-related data within a data recovery time range from the second storage node.

[0165] In another possible implementation, the status record information includes at least one of the following: a user request received by the first storage node; a request execution progress of the user request received by the first storage node; and a task execution progress of a background task executed by the first storage node.

[0166] In another possible implementation, the transmission module 301 is specifically configured to obtain a preset number of state record information of the first storage node most recently before the power failure from the non-volatile memory in response to the first storage node being powered on again after power failure.

[0167] In another possible implementation, the processing module 302 is further configured to generate a new state record information at a preset time interval, and delete the state record information with the earliest recording time, so as to store a preset number of state record information.

[0168] In another possible implementation, the transmission module 301 is also used for: after the first storage node is powered on and acts as a leading node, after recovering the data related to the task within the data recovery time range, receiving a data recovery request message sent by other storage nodes in the storage node group, the data recovery request message is used to request the first data, the first data being the data within a target time range stored by the first storage node, and the target time range being the time range of the data that other storage nodes need to recover; sending a data recovery response message to other storage nodes, the data recovery response message including the first data.

[0169] In another possible implementation, the transmission module 301 is also used for: when the leading node in the storage node group when the first storage node is powered on is different from the leading node in the storage node group when the first storage node is powered on, receiving a data synchronization task sent by the third storage node that serves as the leading node when the first storage node is powered on, the data synchronization task is used to instruct the first storage node to synchronously store the second data, the second data being data within all terms after the power-off term, and the power-off term being the term of the leading node in the storage node group when the first storage node is powered on; sending a data synchronization request message to the third storage node, the data synchronization request message being used to request the second data; receiving a data synchronization response message sent by the third storage node, the data synchronization response message including the second data; the storage module 303 is used to store the data synchronization response message sent by the third storage node.

[0170] It should be noted that Figure 8 The division of modules in the example is schematic and is only a logical function division. There may be other division methods in actual implementation. For example, two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or software function modules.

[0171] Another embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the distributed database data recovery method shown in the above embodiment.

[0172] In actual implementation, the transmission module 301, the processing module 302 and the storage module 303 can be implemented by the processor of the electronic device calling the computer program code in the memory. The specific execution process can refer to the description of the data recovery method of the above distributed database, which will not be repeated here.

[0173] Another embodiment of the present application further provides a computer-readable storage medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of the distributed database data recovery method shown in the above embodiment are implemented.

[0174] In another embodiment of the present application, a computer program product is provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the steps of the distributed database data recovery method shown in the above embodiment are implemented.

[0175] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer execution instruction is loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instruction can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by the computer or a data storage device such as a server, data center, etc. that contains one or more servers that can be integrated with the medium. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), etc.

[0176] The above is only a specific implementation of the present application. Those skilled in the art may conceive of changes or substitutions based on the specific implementation provided by the present application, which should all be included in the protection scope of the present application.

Claims

1. A data recovery method for a distributed database, characterized in that: The distributed database includes at least one storage node group, the method is applied to a first storage node in the storage node group, and the method includes: In response to the first storage node being powered on again after being powered off, obtaining a preset number of state record information of the first storage node that is most recently before the power off moment, the state record information being used to record the progress state of the task executed by the first storage node at the moment when the state record information is generated; Determining a data recovery time range based on the recording time of the preset number of status record information; Acquire data related to the task within the data recovery time range from other storage nodes in the storage node group; Performing data recovery based on the data related to the task within the data recovery time range; In the case where the first storage node serves as a leading node after being powered on, after recovering the data related to the task within the data recovery time range, the first storage node receives a data recovery request message sent by other storage nodes of the storage node group, wherein the data recovery request message is used to request first data, and the first data is data stored by the first storage node within a target time range, and the target time range is a time range for data that needs to be recovered by the other storage nodes; sends a data recovery response message to the other storage nodes, and the data recovery response message includes the first data; or In a case where the leading node in the storage node group when the first storage node is powered on is different from the leading node in the storage node group when the first storage node is powered on, a data synchronization task is received from a third storage node that serves as the leading node when the first storage node is powered on, wherein the data synchronization task is used to instruct the first storage node to synchronously store second data, wherein the second data is data within all terms after a power-off term, and the power-off term is the term of the leading node in the storage node group when the first storage node is powered on; a data synchronization request message is sent to the third storage node, wherein the data synchronization request message is used to request the second data; and a data synchronization response message sent by the third storage node is received and stored, wherein the data synchronization response message includes the second data.

2. The method according to claim 1, characterized in that The determining of the data recovery time range based on the recording time of the preset number of status record information includes: The earliest recording time among the recording times of the preset number of status recording information is used as the starting time of the data recovery time range; The latest time during the term of office of the leader node of the first storage node when the power is off is used as the end time of the data recovery time range.

3. The method according to claim 2, characterized in that The taking the latest time of the leader node of the first storage node during the term of office as the end time of the data recovery time range when the first storage node is powered off includes: In the case where the first storage node serves as the leader node when the power is cut off, the time when the first storage node is cut off is used as the latest time during the term of office of the first storage node; The latest time during the term of office of the leader node of the first storage node before power failure is used as the end time of the data recovery time range.

4. The method according to claim 3, characterized in that The acquiring the task-related data within the data recovery time range from other storage nodes of the storage node group includes: Acquire data related to the task within the data recovery time range from all other storage nodes in the storage node group.

5. The method according to claim 2, characterized in that: The taking the latest time of the leader node of the first storage node during the term of office as the end time of the data recovery time range when the first storage node is powered off includes: In the case where there is a second storage node as the leader node when the first storage node is powered off, if the term of the second storage node has not ended at the current moment, the current moment is used as the latest moment in the term of the second storage node; or, if the term of the second storage node has ended before the current moment, the term end moment of the second storage node is used as the latest moment in the term of the second storage node; wherein the second storage node is another storage node in the storage node group except the first storage node; The latest time of the second storage node during its term of office is used as the end time of the data recovery time range.

6. The method according to claim 5, characterized in that The acquiring the task-related data within the data recovery time range from other storage nodes of the storage node group includes: Acquire data related to the task within the data recovery time range from the second storage node.

7. The method according to claim 1, characterized in that The status record information includes at least one of the following: A user request received by the first storage node; a request execution progress of a user request received by the first storage node; The task execution progress of the background task executed by the first storage node.

8. The method according to claim 1, characterized in that: In response to the first storage node being powered on again after being powered off, obtaining a preset number of state record information of the first storage node that is closest to the power off time, including: In response to the first storage node being powered on again after being powered off, a preset number of state record information of the first storage node that is most recently before the power off moment is obtained from a non-volatile memory.

9. The method according to claim 1, characterized in that: The method further comprises: A new state record information is generated at every preset time interval, and the state record information with the earliest recording time is deleted to save the preset number of state record information.

10. A data recovery device for a distributed database, characterized in that: The distributed database includes at least one storage node group, and the storage node group includes a first storage node; the device includes a transmission module and a processing module; The transmission module is used for, in response to the first storage node being powered on again after being powered off, obtaining a preset number of state record information of the first storage node at the most recent time before the power-off moment, the state record information being used for recording the progress state of the task executed by the first storage node at the moment when the state record information is generated; The processing module is used to determine the data recovery time range based on the recording time of the preset number of status record information; The processing module is further used to obtain data related to the task within the data recovery time range from other storage nodes of the storage node group; The processing module is further used to perform data recovery based on the data related to the task within the data recovery time range; The transmission module is also used for, when the first storage node acts as a leading node after being powered on, receiving a data recovery request message sent by other storage nodes of the storage node group after completing the recovery of the data related to the task within the data recovery time range, the data recovery request message being used to request first data, the first data being data stored by the first storage node within a target time range, the target time range being a time range for data that the other storage nodes need to recover; sending a data recovery response message to the other storage nodes, the data recovery response message including the first data; or, The transmission module is also used for receiving a data synchronization task sent by a third storage node that serves as a leader node when the first storage node is powered on, when the leader node in the storage node group when the first storage node is powered on is different from the leader node in the storage node group when the first storage node is powered on, the data synchronization task is used to instruct the first storage node to synchronously store second data, the second data being data within all terms after a power-off term, and the power-off term being the term of the leader node in the storage node group when the first storage node is powered off; A data synchronization request message is sent to the third storage node, wherein the data synchronization request message is used to request the second data; a data synchronization response message sent by the third storage node is received, wherein the data synchronization response message includes the second data; and the processing module is further used to store the data synchronization response message sent by the third storage node.

11. A data recovery system for a distributed database, characterized in that: The distributed database includes at least one storage node group, the storage node group includes a first storage node and at least one other storage node; the system includes the first storage node and the other storage nodes; The first storage node is used to, in response to the first storage node being powered on again after being powered off, obtain a preset number of state record information of the first storage node that is most recently before the power-off moment, the state record information being used to record the progress state of the task executed by the first storage node at the moment when the state record information is generated; The first storage node is further used to determine a data recovery time range based on the recording time of the preset number of status record information; The first storage node is further used to obtain data related to the task within the data recovery time range from other storage nodes in the storage node group; The other storage nodes are used to provide the first storage node with data related to the task within the data recovery time range; The first storage node is further used to perform data recovery based on the data related to the task within the data recovery time range; The other storage nodes are also used to, when the first storage node acts as a leading node after being powered on, send a data recovery request message to the first storage node, the data recovery request message being used to request first data, the first data being data within a target time range stored by the first storage node, the target time range being a time range for data that the other storage nodes need to recover; the first storage node is also used to, after recovering the data related to the task within the data recovery time range, receive a data recovery request message sent by other storage nodes of the storage node group; the first storage node is also used to send a data recovery response message to the other storage nodes, the data recovery response message including the first data; or, The first storage node is also used to receive a data synchronization task sent by a third storage node that serves as a leader node when the first storage node is powered on, when the leader node in the storage node group when the first storage node is powered on is different from the leader node in the storage node group when the first storage node is powered on. The data synchronization task is used to instruct the first storage node to synchronously store second data, where the second data is data in all terms after a power-off term, and the power-off term is the term of the leader node in the storage node group when the first storage node is powered off. The first storage node is also used to send a data synchronization request message to the third storage node, where the data synchronization request message is used to request the second data. The first storage node is also used to receive and store a data synchronization response message sent by the third storage node, where the data synchronization response message includes the second data.

12. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

13. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

14. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Citation Information

Patent Citations

  • Method and equipment for recovering database data by using logs

    CN103699548A

  • Fault processing method of k8s cluster, computing device and cluster

    CN117851099A

  • Database flash-back method based on in-situ update storage

    CN118069431A

  • Recovery Dataset Management For Security Threat Monitoring

    US20210216628A1

  • Selective utilization of processor cores while rebuilding data previously stored on a failed data storage drive

    US20220100621A1