Data recovery method and system, computing device cluster and storage medium

Abstract

The embodiment of the invention discloses a data recovery method and system, a computing device cluster and a storage medium, relates to the technical field of storage and is used for improving data recovery efficiency. The method is applied to a database system comprising a first computing node and a first storage node, under the condition that a fault node exists in the database system, the first computing node sends a first message to the first storage node, and the first message is used for indicating a target log needing to be recovered and corresponding to the fault node; the first computing node is a computing node with fault recovery capability in the database system; the first storage node is a storage node for managing the target log in the database system; the first storage node responds to the first message and sends a recovery set to the first computing node, the recovery set is used for indicating a target log and a target data block, and the target data block comprises a data block actually needing to be recovered in the target log; the first computing node receives the recovery set from the first storage node, and performs log playback based on the recovery set.

Description

Technical Field

[0001] This application relates to the field of storage technologies, and in particular, to a data recovery method, system, computing device cluster, and storage medium. Background Art

[0002] A Real Application Cluster (RAC) is a distributed database based on a shared-disk architecture. As Figure 1 shown, it is a schematic diagram of the architecture of a RAC. Among them, a RAC consists of a computing cluster and a storage cluster. The computing cluster includes more than two computing nodes, such as Figure 1 the computing node 1, computing node 2, and computing node 3 in Figure 1 ; the storage cluster includes several storage nodes, such as

[0003] the storage node 1, storage node 2, and storage node 3 in

[0004] In a RAC, the resources of several storage nodes form a shared storage. A database instance runs in each computing node, and the computing node obtains any data in the shared storage based on the database instance. Currently, in order to maintain the atomicity and persistence of the database, when processing data, a redo log recording the processing operation is generated and persistently stored. When a database instance or a computing node in the RAC fails, resulting in the loss of data modified in memory before the failure, the database instance in other available computing nodes can replay the redo log that recorded the data modification operation and stored it on the disk before the failure to recover the lost data. Among them, in order to reduce the computing power and time cost consumed by the computing node in replaying the log, the computing node obtains and parses the redo log of the faulty node from the storage node to obtain the data block recorded in the log, and refers to the block written records (BWR) of the data block actually written to the disk to screen out the target data block that actually needs to be recovered from the data block recorded in the log; the computing node obtains the redo log of the faulty node from the storage node again, and performs log replay on the log corresponding to the target data block to complete the fault repair. However, in the case of a long log, the computing node obtaining the redo log from the storage node will generate a large communication overhead. Moreover, since the database system needs to suspend services during fault repair, the above method will result in a long recovery time objective (RTO) for the system. For databases applied in time-sensitive business scenarios such as finance and the Internet, the business requirements cannot be met. Summary of the invention

[0005] The embodiments of the present application provide a data recovery method, system, computing device cluster and storage medium, in which the storage node determines the data block information that actually needs to be recovered based on the log, thereby reducing the computing power overhead of the computing node during fault recovery, making full use of the computing resources of the storage system, and ultimately reducing the RTO duration of the system to meet the business needs of latency-sensitive databases.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a data recovery method is provided, which is applied to a database system, wherein the database system includes a first computing node and a first storage node; when there is a faulty node in the database system, the first computing node sends a first message to the first storage node, the first message being used to indicate a target log that needs to be recovered corresponding to the faulty node; the first computing node is a computing node with fault recovery capability in the database system; the first storage node is a storage node that manages the target log in the database system; the first storage node sends a recovery set to the first computing node in response to the first message, the recovery set being used to indicate the target log and the target data block, the target data block including the data block that actually needs to be recovered in the target log; the first computing node receives the recovery set from the first storage node, and performs log playback based on the recovery set.

[0008] Currently, the database system includes computing nodes that run database instances and storage nodes that store data. When a database instance fails or a computing node fails, resulting in data loss in memory, other computing nodes running available database instances need to perform fault recovery by replaying the redo logs of the failed node. The computing node used to recover the failure reads the log twice to obtain the data blocks that actually need to be recovered, thereby accelerating the log playback process. However, in this fault recovery method, the communication overhead generated by the transmission process of the computing node reading the log twice from the storage node is large, and when there are many logs, the transmission time is long, which makes the RTO time of the database system unable to operate normally longer and the recovery efficiency is poor.

[0009] Based on this, how to improve data recovery efficiency to shorten the RTO time of the database system is a technical problem that needs to be solved urgently.

[0010] In this regard, the present application provides a data recovery method through the first aspect, which is used to recover the faulty first computing node to instruct the first storage node to determine the target data block that actually needs to be recovered, and the first storage node generates a recovery set and feeds it back to the first computing node, wherein the recovery set is used to indicate the target data block and the target log of the faulty node; the first computing node replays the log corresponding to the target data block based on the recovery set to achieve data recovery. This method delegates the process of first reading the redo log to determine the target data block in the prior art to the storage node for execution, so that the log playback process can be executed by transmitting the redo log and the target data block information once between the computing node and the storage node, reducing the communication overhead between the computing node and the storage node, and making full use of the computing resources of the storage node, reducing the computing power overhead of the computing node, and improving the overall recovery efficiency of the database system; this method effectively shortens the RTO duration, which helps to meet the business needs of latency-sensitive databases.

[0011] In a possible implementation, the recovery set includes a target log and marking information corresponding to a data block recorded in the target log; the marking information corresponding to the data block is used to indicate whether the data block is a data block that has been written to the disk. A data block that has not been written to the disk is a target data block, and a data block that has been written to the disk is a data block that does not actually need to be recovered.

[0012] This possible implementation provides a specific implementation of a recovery set, wherein the first computing node determines whether the data block is a target data block by referring to the tag information, and replays the log of the target data block in the target log, while other logs are not replayed, thereby accelerating log playback, shortening the RTO time of the database system, and improving data recovery efficiency.

[0013] In one possible implementation, the first message includes the storage address of the target log; before sending the recovery set to the first computing node, the method also includes: the first storage node obtains the target log according to the storage address of the target log; the first storage node parses the target log to obtain the data block recorded in the target log; the first storage node determines the target data block based on the log sequence number LSN of the data block recorded in the target log and the log sequence number LSN of the data block on the disk.

[0014] This possible implementation provides a specific implementation for the first storage node to determine the target data block. The first storage node locates the target log in the disk through the first message sent by the first computing node, and determines whether the data block is the target data block by comparing the LSN of the data block in the target log with the LSN of the data block in the disk. In the prior art, when the computing node executes this process, it is implemented by comparing whether the data block in the log exists in the block write record, while in the data recovery method provided in this application, the storage node implements it by comparing the LSNs; this helps to make full use of the computing power of the storage node, save the computing power resources of the computing node, and improve the overall efficiency of data recovery.

[0015] In a possible implementation, based on the data block recorded in the target log and the log sequence number LSN of the data block in the disk, determining the target data block includes: for each data block recorded in the target log, the first storage node obtains the maximum LSN of the data block; comparing the maximum LSN of the data block with the LSN of the data block in the disk; when the LSN of the data block in the disk is less than the maximum LSN of the data block, the first storage node determines that the data block is the target data block; the method further includes: the first storage node determines the marking information of the data block as the first marking information, and the first marking information is used to indicate that the data block has not been written to the disk.

[0016] This possible implementation provides a specific implementation for the first storage node to determine the target data block by comparing the LSNs. Among them, each log in the target log has a corresponding LSN. The modification of the same data block may involve multiple logs, so it is necessary to determine the maximum LSN. The log corresponding to this maximum LSN represents the latest modification of the data block recorded in the log; in this way, by comparing with the LSN in the disk, it can be determined whether the data block in the disk is the updated data block. If so, the log corresponding to this data block does not need to be replayed; if not, the log corresponding to this data block needs to be replayed to update the data block in the disk; moreover, by indicating the target data block through the first marking information, it helps the subsequent first computing node to refer to the first marking information and replay the corresponding log, shortening the log replay duration and improving the data recovery efficiency.

[0017] In a possible implementation, the method further includes: when the LSN of the data block in the disk is greater than or equal to the maximum LSN of the data block, the first storage node determines the marking information of the data block as the second marking information, and the second marking information is used to indicate that the data block has been written to the disk.

[0018] This possible implementation provides a specific implementation for a first storage node to mark data blocks in a target log. Among them, the first storage node also marks data blocks that do not require log replay, which is used for the first computing node to skip corresponding logs when identifying the second marking information, accelerating log replay and improving data recovery efficiency.

[0019] In a possible implementation, the method further includes: the first computing node obtains metadata information of the original data blocks managed by the faulty node; the first computing node allocates the original data blocks to at least one available second computing node in the database system for management; the first computing node stores the correspondence between the original data blocks and the at least one second computing node.

[0020] This possible implementation provides a specific implementation of the data recovery process. The first computing node reallocates the original data blocks managed by the faulty node to other available computing nodes for management, which helps to obtain the operation permissions for the corresponding data blocks during subsequent log replay and complete data recovery.

[0021] In a possible implementation, when the target data block is an original data block, the first computing node performs log replay based on the recovery set, including: for any target data block in the target data blocks, the first computing node obtains operation permissions from the computing node managing the target data block based on the correspondence, and the operation permissions are used to indicate whether modification of the target data block is allowed; in the case where the operation permissions indicate that modification of the target data block is allowed, the first computing node replays the log of the target data block in the target log.

[0022] This possible implementation provides a specific implementation of the data recovery process. When the first computing node performs log replay, it needs to obtain the operation permissions for the corresponding data blocks. When the operation permissions indicate that modification of the data block is allowed, based on the modification operations recorded in the log, the modification is performed to obtain the modified data block and complete data recovery.

[0023] In a possible implementation, when the first computing node replays the log of the target data block in the target log, the method further includes: the first computing node obtains metadata information of the target data block from the computing node managing the target data block, and the metadata information includes whether there is a pre-image of the data block. In the case where the first computing node determines that there is a pre-image of the target data block, the first computing node obtains the LSN of the pre-image of the target data block; compares the LSN of the pre-image of the target data block and the LSN of the target data block in the target log; in the case where the LSN of the pre-image of the target data block is less than the LSN in the target log, replays the log of the target data block.

[0024] This possible implementation provides a specific implementation for accelerating the data recovery process. When the first computing node performs log replay, it can also refer to the metadata information of the data block. When it is determined that there is a pre-image of the data block, the log replay can be performed based on the pre-image, thereby reducing the logs that need to be replayed and shortening the log recovery duration.

[0025] In a possible implementation, the failed node includes the third computing node in the database system, or the database instance in the third computing node, and the third computing node is the same as or different from the first computing node.

[0026] This possible implementation provides a specific implementation of the failed node. Among them, when multiple database instances are running on a computing node, when a certain database instance fails, the data recovery can be performed by other available database instances in the computing node. Of course, the data recovery can also be performed by other available computing nodes in the database system.

[0027] In a second aspect, a data recovery method is provided, including: the first storage node receives a first message sent by the first computing node, and the first message is used to indicate the target log to be recovered corresponding to the failed node in the database system; the first computing node is a computing node with fault recovery ability in the database system, and the first storage node is a storage node that manages the target log in the database system; the first storage node responds to the first message and sends a recovery set to the first computing node for performing log replay, and the recovery set is used to indicate the target log and the target data block, and the target data block includes the data blocks that actually need to be recovered in the target log.

[0028] In a third aspect, a data recovery method is provided, including: the first computing node sends a first message to the first storage node, and the first message is used to indicate the target log to be recovered corresponding to the failed node in the database system; the first computing node is a computing node with fault recovery ability in the database system, and the first storage node is a storage node that manages the target log in the database system; the first computing node receives the recovery set sent by the first storage node, and the recovery set is used to indicate the target log and the target data block, and the target data block includes the data blocks that actually need to be recovered in the target log; the first computing node performs log replay based on the recovery set.

[0029] Fourth aspect, a database system is provided, which includes a first computing node and a first storage node. The first computing node is configured to send a first message to the first storage node, and the first message is used to indicate the target log to be recovered corresponding to the failed node; the first computing node is a computing node with the ability of fault recovery in the database system; the first storage node is a storage node in the database system that manages the target log, and the failed node is the node to which the failed database instance in the database system belongs or the failed computing node; the first storage node is configured to, in response to the first message, send a recovery set to the first computing node, and the recovery set is used to indicate the target log and the target data blocks, and the target data blocks include the data blocks that actually need to be recovered in the target log; the first computing node is further configured to receive the recovery set from the first storage node and perform log replay based on the recovery set.

[0030] Fifth aspect, a computing device is provided, which is used to execute the actions of the first storage node in any one of the methods provided in the first aspect or the second aspect. It includes multiple functional units, and the actions executed by each functional unit are implemented by hardware or by hardware executing corresponding software. For example, the computing device may include: a receiving unit and a sending unit. The receiving unit is configured to receive the first message sent by the first computing node, and the first message is used to indicate the target log to be recovered corresponding to the failed node in the database system; the first computing node is a computing node with the ability of fault recovery in the database system, and the computing device is a storage node in the database system that manages the target log. The sending unit is configured to, in response to the first message, send a recovery set to the first computing node for performing log replay, and the recovery set is used to indicate the target log and the target data blocks, and the target data blocks include the data blocks that actually need to be recovered in the target log.

[0031] In a possible implementation manner, the computing device further includes a processing unit, which is configured to: obtain the target log according to the storage address of the target log; parse the target log to obtain the data blocks recorded in the target log; and determine the target data blocks based on the log sequence number LSN of the data blocks recorded in the target log and the LSN of the data blocks in the disk.

[0032] In a possible implementation manner, the processing unit is specifically configured to: for each data block recorded in the target log, obtain the maximum LSN of the data block; compare the maximum LSN of the data block with the LSN of the data block in the disk; and when the LSN of the data block in the disk is less than the maximum LSN of the data block, determine the data block as the target data block.

[0033] Sixth aspect, a computing device is provided for performing the actions of the first computing node in any of the methods provided by the first aspect or the third aspect, which includes a plurality of functional units, and the actions performed by each functional unit are implemented by hardware or by hardware executing corresponding software. For example, the computing device may include: a sending unit, a receiving unit, and a processing unit. The sending unit is configured to send a first message to a first storage node, and the first message is used to indicate a target log to be recovered corresponding to a faulty node in the database system; the computing device is a computing node in the database system with the ability to perform fault recovery, and the first storage node is a storage node in the database system that manages the target log. The receiving unit is configured to receive a recovery set sent by the first storage node, and the recovery set is used to indicate the target log and target data blocks, and the target data blocks include the data blocks that actually need to be recovered in the target log. The processing unit is configured to perform log replay based on the recovery set.

[0034] Seventh aspect, a computing device cluster is provided. The computing device cluster includes at least one computing device, and each computing device includes a processor and a memory; the memory in the at least one computing device is configured to store computer program instructions; the processor in the at least one computing device calls the computer program instructions stored in the memory to execute any of the methods provided by the first aspect or the second aspect or the third aspect.

[0035] Eighth aspect, a chip is provided, which includes: a processor and an interface circuit; the interface circuit is configured to receive code instructions and transmit them to the processor; the processor is configured to run the code instructions to execute any of the methods provided by the first aspect or the second aspect or the third aspect.

[0036] Ninth aspect, a computer-readable storage medium is provided, which includes computer program instructions, and when the computer program instructions run on a computing device, the computing device is caused to execute any of the methods provided by the first aspect or the second aspect or the third aspect.

[0037] Tenth aspect, a computer program product is provided, which includes computer program instructions, and when the computer program instructions run on a computing device, the computing device is caused to execute any of the methods provided by the first aspect or the second aspect or the third aspect.

[0038] For the technical effects brought by any implementation manner in the second aspect to the tenth aspect, reference can be made to the technical effects brought by the corresponding implementation manner in the first aspect, which will not be elaborated here. Description of the Drawings

[0039] Figure 1 A schematic diagram of the architecture of a RAC provided for the embodiments of the present application;

[0040] Figure 2A schematic diagram of the hardware structure of a computing device provided by an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a scenario for recording and recycling redo logs provided by an embodiment of the present application;

[0042] Figure 4 A flowchart of a data recovery method provided by an embodiment of the present application;

[0043] Figure 5 A flowchart of a data recovery method provided by an embodiment of the present application;

[0044] Figure 6 A schematic diagram of a scenario for determining a target data block provided by an embodiment of the present application;

[0045] Figure 7 A schematic diagram of a scenario for database failure repair provided by an embodiment of the present application;

[0046] Figure 8 A flowchart of a data recovery method provided by an embodiment of the present application;

[0047] Figure 9 A schematic diagram of the composition of a database system provided by an embodiment of the present application;

[0048] Figure 10 A schematic diagram of the composition of a computing device provided by an embodiment of the present application;

[0049] Figure 11 A schematic diagram of the composition of a computing device cluster provided by an embodiment of the present application;

[0050] Figure 12 A schematic diagram of the connection mode between computing devices in a computing device cluster provided by an embodiment of the present application. Detailed implementation manners

[0051] Next, the technical solutions in the embodiments of the present application will be described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0052] Unless otherwise defined, all scientific and technological terms used herein have the same meaning as those known to those of ordinary skill in the art. In the embodiments of the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B, which may represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are a kind of "or" relationship. "At least one of the following (individuals)" or its similar expressions refers to any combination of these items, including any combination of single items (individuals) or plural items (individuals). For example, at least one of a, b or c (individuals) may represent: a, b, c, a and b, a and c, b and c or a, b and c, wherein a, b and c may be single or multiple. In addition, in the embodiments of the present application, the words "first", "second" and the like do not limit the quantity and order.

[0053] In addition, in the embodiments of the present application, directional terms such as "upper" and "lower" are defined relative to the orientation of the components in the drawings. It should be understood that these directional terms are relative concepts. They are used for relative description and clarification, and they may change accordingly according to changes in the orientation of the components in the drawings.

[0054] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0056] As described in the background art, in RAC, the computing cluster can access the data on all disks in the storage cluster through a database instance. Among them, the cache occupied during the operation of the database instance includes a local buffer cache and a global cache. Among them, the global cache is proposed based on the cache fusion technology. The data in the global cache can be transmitted to different computing nodes through a network (fast inter-node messaging). To maintain the cache consistency of the data blocks in the global cache, a global cache service is also provided in RAC to track and manage the status and location of the data blocks. Specifically, the computing node that manages the data block executes the recording of the metadata information of the data block, and this computing node can also be called the master node of the data block. When any computing node in the computing cluster accesses the target data, it needs to apply for the operation permission from the master node of the target data. When the operation permission is obtained, the target data is obtained.

[0057] Among them, through a certain resource allocation algorithm, the data blocks can be scattered to each computing node in the computing cluster for management.

[0058] For example, in Figure 1 , computing node 1 can obtain any piece of data in the shared storage and read it into the global cache. When other computing nodes in the system request to obtain this data, they can obtain the data copy in the global cache by copying. Compared with the traditional shared disk architecture database, where nodes can only obtain the same data by accessing the shared disk, data transmission through the global cache omits the process of computing nodes writing data to the disk. For example, in the above example, other computing nodes do not need to wait for computing node 1 to write the processed data to the disk again. Based on this, the data transmission efficiency between computing nodes is greatly improved.

[0059] Among them, in terms of hardware implementation, the above computing nodes or storage nodes can include but are not limited to the computing devices as shown in Figure 2 . As shown in Figure 2 , it is a schematic diagram of the hardware structure of a computing device provided by an embodiment of the present application. The computing device 20 can be used to implement the functions of the above various nodes.

[0060] Figure 2The computing device 20 shown may include: a processor 201, a memory 202, a disk 203, a network card 204, and a bus 205. The processor 201 and the memory 202 form a control unit of the computing device 20, which is used to implement the computing service of the computing device 20. The processor 201, the memory 202, the disk 203, and the network card 204 may be connected via a bus 205.

[0061] The processor 201 includes one or more CPUs, and may also be other general-purpose processors, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0062] As an example, in Figure 2 In the figure, the processor 201 may include CPU 0 and CPU 1. A CPU has one or more CPU cores. The figure is only used as an example, and in actual applications, more or fewer CPUs and CPU cores may be included.

[0063] Memory 202 refers to an internal memory that directly exchanges data with the processor. It can read and write data at any time and at a high speed, and serves as a temporary data storage for the operating system or other running programs. Memory includes at least two types of memory, for example, memory can be either random access memory or read only memory (ROM).

[0064] For example, the random access memory is a dynamic random access memory (DRAM) or a storage class memory (SCM). DRAM is a semiconductor memory, and like most random access memories (RAM), it is a volatile memory device. SCM is a composite storage technology that combines the characteristics of traditional storage devices and memory. Storage class memory can provide faster read and write speeds than hard disks, but has a slower access speed than DRAM and is cheaper than DRAM. However, DRAM and SCM are only exemplary descriptions in this embodiment, and the memory may also include other random access memories, such as static random access memory (SRAM).

[0065] For example, the read-only memory can be a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), etc.

[0066] In addition, the memory 202 can also be a dual in-line memory module or a dual in-line memory module (DIMM), that is, a module composed of dynamic random access memory (DRAM), and can also be a solid state disk (SSD). In practical applications, multiple memories 202 and different types of memories 202 can be configured in the computing device 20. The number and type of the memory 202 are not limited in this embodiment. In addition, the memory 202 can be configured to have a power retention function. The power retention function means that when the system loses power and then powers on again, the data stored in the memory 202 will not be lost. The memory with the power retention function is called a non-volatile memory.

[0067] The disk 203 is used to store data, including floppy disks and hard disks. The hard disk specifically includes a solid state drive and a mechanical hard disk (such as a shingled magnetic recording hard disk).

[0068] The network card 204 is used for the computing device 20 to connect with other devices through a communication network. As shown in Figure 1 , each node communicates with other nodes in the network through the network card. The network card 204 has a unique address in the network. The communication network can be an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc. The network card 204 can also be called a communication interface, including a receiving unit for receiving data and a sending unit for sending data.

[0069] Among them, the network card 204 includes a processor and a memory, which are used to implement the sending and receiving of data. For example, data is usually transmitted in parallel through the bus 205 in the computing device 20, while it is usually transmitted in serial form in the network. The processor and memory in the network card need to perform serial-parallel conversion on the data to be sent or received. When sending data, the processor 201 copies the data in the memory 202 to the memory in the network card, and the processor in the network card processes the data and then sends it. When receiving data, the data is first stored in the memory of the network card, and then the processor 201 copies the data in the memory of the network card to the memory 202 for the processor 201 to read.

[0070] The bus 205 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 it is only represented by a thick line in the figure, but it does not mean that there is only one bus or one type of bus.

[0071] It should be noted that Figure 2 the structure shown in the figure does not constitute a limitation on the computing device 20. Except Figure 2 for the components shown, the computing device 20 may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0072] Combined with each of the nodes described above Figure 1 in the computing device for implementing the corresponding functions of the computing node, it may include a processor, a memory, and a network card. Among them, the processor is used to process or generate read / write requests for data and temporarily store the relevant data in the memory. When the amount of data in the memory reaches a certain threshold, the processor sends the data in the memory to the storage node through the network card for persistent storage. In the computing device for implementing the corresponding functions of the storage node, it includes a processor, a memory, and a hard disk. Among them, the processor and the memory are used to implement simple computing functions to assist in reading and writing data in the hard disk. In the embodiments of the present application, each storage node may include one or more hard disks for persistent data storage.

[0073] It can be understood that, in terms of function, since Figure 1 the main function of the computing node in the figure is to perform computing services, and persistent storage can be achieved by using a remote memory (such as the hard disk in the storage node) when storing data. Therefore, it has fewer local memories such as hard disks than the storage node, thus achieving cost and space savings. However, this does not mean that the computing node cannot have local memories. In actual implementation, the computing node can also be built-in with a small number of hard disks or externally connected with a small number of hard disks.

[0074] Among them, the above-mentioned computing node can specifically be a computing device such as a server or a desktop computer. The storage node can be a computing device such as a server running a database for managing a large amount of data.

[0075] Currently, database transactions follow the following four characteristics, namely:

[0076] Atomicity, which can also be called indivisibility, specifically means that the operations in a transaction need to be all committed successfully, or all rolled back in case of failure.

[0077] Consistency means that before and after the execution of a transaction, the database must be in a correct state and satisfy integrity.

[0078] Isolation means that when multiple transactions are executed concurrently, the execution of one transaction does not affect the execution of other transactions. The modifications made by a transaction are not visible to other transactions until the transaction is committed successfully.

[0079] Durability means that once a transaction is committed successfully, the data modifications involved will be permanently saved to the database. Even if the database system crashes, fails, or goes down, the data should not be lost.

[0080] Based on the first letter of each characteristic in English, the above four characteristics can also be called ACID characteristics.

[0081] During the execution of a transaction in a database system, data on the disk needs to be modified. To meet the durability of the transaction, the modified data needs to be persisted to the disk. This disk writing process is on the critical input / output (I / O) path of the database and is a random access to the disk, which will affect the performance of foreground transactions. Therefore, many databases usually adopt the write ahead log (WAL) mechanism to record operations on the database by sequentially writing redo logs to the disk, and the persistent modification of data is done asynchronously. For example, during the execution of a database transaction, each data modification generates a redo log to record the new value after the modification, and each redo log will be stored in the disk log file before the data is written to the disk, as shown in Figure 1 the log file shown.

[0082] Among them, redo logs are usually sequentially written to the log file in an append-write manner, as shown in Figure 3As shown in the figure, it is a schematic diagram of a scenario for recording and recycling redo logs. A checkpoint thread runs in each computing node, which is used to record the dirty page information table and the transaction information table. The dirty page information table includes the dirty page identifier and the earliest log sequence number (LSN) corresponding to the dirty page indicated by the dirty page identifier. The transaction information table corresponds to the uncommitted transaction identifier and the latest LSN corresponding to the transaction indicated by the uncommitted transaction identifier. Among them, LSN is the logical sequence number of the log, which gradually increases as the log is written. The checkpoint thread executes according to a cycle to refresh relevant information. For example, when a dirty page is written to the disk, the dirty page information table is updated, and when a transaction is committed, the transaction information table is updated. In this way, Figure 3 in it, when a transaction is committed and the dirty page is flushed to the disk, the reclaimable / erasable position (checkpoint position) of the log moves clockwise, so that the redo log corresponding to the dirty page can be reclaimed, and the storage area can be written with a new redo log again. After a new redo log is written, the write position of the log will move backward. When the write position catches up with the checkpoint position, the foreground transaction cannot continue. It is necessary to wait for the background to flush some dirty pages to disk and then the checkpoint position advances before the foreground transaction can continue to write the redo log. In Figure 3 it, the shaded part between the checkpoint and the write position refers to the dirty pages recorded in the redo log that have not been written to the disk; while the blank part between the checkpoint and the write position refers to the reclaimed redo log, and this part of the storage area can be written with a new redo log.

[0083] When the system crashes, the redo log can be replayed based on the LSN corresponding to the checkpoint closest to the crash point, that is, Figure 3 the redo log in the shaded part in it, to restore the data that has not been written to the disk.

[0084] In addition, in order to ensure the atomicity of transactions, the database system also records rollback logs (undo logs) during the execution of transactions. During the execution of a transaction, when modifying data on the disk, each data modification generates a redo log to record the new value after the modification, and at the same time generates an undo log record to record the state before the modification. In this way, when a failure occurs during the execution of a transaction, the lost data is restored through the redo log to maintain the durability and consistency of the database; and then the incomplete transactions executed before the failure are rolled back through the undo log to ensure the atomicity of the database.

[0085] When different database systems weigh performance and functionality, they will adopt different strategies to selectively record redo logs and undo logs, adapting to different application scenarios while ensuring the ACID properties of the database. Usually, the persistence strategies of the database are described from two dimensions: force and steal. As shown in Table 1 below, it is a schematic table of a database management strategy combination. Among them, the force strategy requires all data to be forcibly flushed to disk when a transaction commits, while no-force does not. The steal strategy allows data to be written to the database in advance when a transaction has not committed, while no-steal does not allow it. Based on this, the four combinations in Table 1 are obtained. For the steal / force combination, the steal indicates that data is allowed to be written to the database in advance when the transaction has not committed. At this time, in order to maintain the atomicity of the transaction, the undo log before the data is written needs to be recorded and stored so that the data can be restored based on the undo log when the transaction rolls back; force means that the data needs to be forcibly flushed to disk when the transaction commits, that is, it maintains consistency regardless of whether the transaction commits successfully or not, and there is no need to replay the data through the redo log. For the steal / no-force combination, as described above, the undo log needs to be recorded; and no-force means that the data does not need to be forcibly flushed to disk when the transaction commits. At this time, for the data that has been committed by the transaction, in order to avoid data loss, the redo log needs to be recorded, and the data can be restored by replaying the redo log after the data is lost. Usually, the steal / no-force combination can achieve the best performance. For the no-steal / force combination, data is not allowed to be written to the database in advance when the transaction has not committed, and all data needs to be written when the transaction commits. It can be seen that the no-steal / force combination will bring the slowest performance. Similarly, for the no-steal / no-force combination, data is not allowed to be written in advance when the transaction has not committed, and all data does not need to be written when the transaction commits. Therefore, the redo log needs to be recorded to maintain consistency, and there is no need to record the undo log to maintain the atomicity of the transaction. In common database products in the current industry, such as Oracle, MySQL, etc., they all adopt the steal / no-force strategy to ensure the performance of the database.

[0086] Table 1. Schematic Table of a Database Management Strategy Combination

[0087] steal no-steal force no redo / undo no redo / no undo no-force redo / undo redo / no undo

[0088] In the embodiment of this application, in the application scenario of the shared cache with RAC as an example, the steal / no-force strategy is adopted to ensure the performance of the database. When the database fails, the redo log and undo log are referred to for fault repair.

[0089] In a RAC, the data in the shared storage can be scattered to each computing node in the computing cluster for management according to a certain resource allocation algorithm. The disk includes multiple data pages (pages), usually with sizes of 8K, 16K, etc. Each data page includes multiple data blocks (blocks). In other words, a data page can also be understood as being composed of multiple data blocks. Depending on the different storage granularities / different storage memory sizes, in this application, the data page can be alternatively described as a data block without limitation. Exemplarily, in Figure 1 it is assumed that each data file in each storage node is 100 data blocks, and three computing nodes can respectively manage 100 data blocks. Among them, each computing node is the master node of the 100 data blocks it manages. The metadata information of these 100 data blocks is recorded in this master node. The metadata information includes the role of each data block, the lock mode information, and whether there is a pre-image (past image, pi block).

[0090] The role of a data block includes global (G) or local (L). Local means that the data block is first read into memory. For example, when computing node 1 reads data block 1 from storage node 1 into the shared cache, the role of data block 1 at this time is L. Global means that there are multiple copies of the data block in the cluster. Combining the above example, when computing node 2 copies data block 1 from computing node 1 to obtain a copy of data block 1, the role of data block 1 changes from L to G at this time.

[0091] The lock mode information refers to the holding permissions of the computing node for the data block. It includes three types: read-only (share), exclusive, and null. This lock mode information is used to mark the usage status of the computing node for the data block. For example, when computing node 1 reads data block 1, it needs to first obtain a read-only lock (which can also be called a shared lock), and then read data block 1 from the disk. At this time, the lock mode information of data block 1 is read-only. At the same moment, read-only locks can be allocated to multiple computing nodes. For example, while computing node 1 has obtained a read-only lock, when computing node 3 requests to read data block 1, it can obtain a read-only lock and copy it through computing node 1. The exclusive lock is unique in the cluster. For example, when computing nodes 1 and 3 are reading data block 1, if computing node 2 requests to modify data block 1, then computing nodes 1 and 3 need to release their read-only locks, and computing node 2 obtains the exclusive lock, copies data block 1 for modification. At this time, the lock mode information of data block 1 is exclusive. Also, any read / write request needs to wait until computing node 2 finishes modifying data block 1 and releases the exclusive lock before it can obtain the exclusive lock or read-only lock again. The lock mode information being null means that no node is performing read or modification operations on data block 1.

[0092] The pre-image refers to the dirty block that has been modified by other nodes before the data block is modified by the current node in the same transaction. For example, in the above example, during the period when computing node 2 is modifying data block 1, if computing node 4 requests to modify data block 1, after computing node 2 modifies it, it sends a copy of data block 1 to computing node 4. The data block modified by computing node 2 but not yet stored on the disk is a dirty block, which can be marked as a pre-image data block, and after waiting for computing node 4 to store the modified data block 1 on the disk, it is then deleted.

[0093] The role, lock mode information, and pre-image of the above data block, as metadata information of the data block, can be used to indicate the status and location of the data block. For example, based on the role, it can be known whether there are replicas of the data block; based on the lock mode information, the latest read / write status of the data block can be known; and based on the pre-image, the dirty block and the node where the dirty block is located can be known.

[0094] When any computing node or database instance on a computing node in a computing cluster fails, it will be sensed by other computing nodes. Available computing nodes in the database system can perform fault repair, which specifically includes two processes. First, the metadata information of the data blocks managed by the failed node needs to be restored, and the data blocks are redistributed to the available computing nodes for management. This process can be called the remaster process. Further, the redo log of the failed node is obtained, the data blocks that need to be restored are determined and replayed, and the data that was not stored in the disk in time due to the failure is restored. This process can be called the replay process. Among them, the replay process includes two recovery phases (two pass recovery). The first log reading phase (first-pass log read) is used for the computing node to read the redo log and parse the corresponding data block information as the recovery set (recovery set). Combined with the block write record of the data block actually written to the disk, the data block information that is the same as that in the BWR is marked in the recovery set to determine the target recovery set. The second-pass log read phase is used by the computing node to obtain the redo log again, and selectively replay the redo log in combination with the above target recovery set to obtain the recovered data. Currently, both processes are executed by an available computing node in the system, which can be any available computing node in the system, such as the computing node that discovers the faulty node, or the computing node pre-designated in the system for fault repair, etc.

[0095] However, this fault repair method has low recovery efficiency. When the log is long, the RTO time of the system is long. For databases used in latency-sensitive business scenarios such as finance and the Internet, it will not be able to meet business needs. In this regard, the present application provides a database repair method, in which the storage node determines the redo log corresponding to the data block that needs to be recovered, and the computing node plays it back to complete the fault repair, thereby reducing the computing power overhead of the computing node during the fault recovery period, making full use of the computing resources of the storage system, and ultimately reducing the system RTO time to meet the business needs of latency-sensitive databases. This method is applied to Figure 1 The database system shown in FIG. 1 has a faulty node in the data block system, and the database system includes at least a first computing node and a first storage node. Figure 4 As shown, the method includes the following steps S401-S403.

[0096] S401: A first computing node sends a first message to a first storage node. Correspondingly, the first storage node receives the first message.

[0097] Among them, the first message is used to indicate the target log that needs to be restored corresponding to the faulty node. Specifically, the storage address of the target log is included in the first message. For example, the faulty node includes the checkpoint position and write position recorded by the checkpoint thread closest to the crash time point of the faulty node. Combining the above description about Figure 3 it can be known that the target log is the redo log in the shaded part.

[0098] Among them, the faulty node refers to the computing node where the faulty database instance is located, or the faulty computing node.

[0099] Among them, the first computing node is a computing node in the database system with the ability of fault recovery, and the first storage node is the storage node in the database system that manages the target log.

[0100] In a possible implementation, the first computing node is the computing node that detects the faulty node in the database system.

[0101] Among them, the faulty node can be determined in the following two ways. One way is to detect the status of the peer node by periodically sending ping packets between computing nodes. If a response is received within a preset time, it is determined that the peer node is running normally; otherwise, it is determined that the peer node has failed, and the computing node that sends the ping packet is used as the first computing node to repair the faulty node.

[0102] In another possible implementation, since each node needs to periodically run the checkpoint thread to flush dirty pages and update the control file stored on the disk at the same time, and the control file records the checkpoint position and write position of the redo log, and the control file is also information shared by each node, therefore, the running status of the computing node can be determined by judging whether the control file is updated. In the case of no update, it is determined that the computing node has failed. The control file records information such as the positions of the data files and log files managed by the computing node, the current LSN, and the above-mentioned checkpoint position and write position.

[0103] It should be noted that when the faulty node is the computing node where the faulty database instance is located, the first computing node can be the faulty node. That is to say, the faulty node can recover data through other available database instances.

[0104] Among them, the failure types of the faulty node include the following four types: transaction failure, system failure, media failure, etc. Transaction failure means that the transaction fails to be committed successfully. For example, the deadlock phenomenon of concurrent transactions competing for resources resulting in mutual waiting, arithmetic overflow, etc. Transaction failure occurs in the database instance and can usually be recovered from the failure by other database instances in the computing node running the database instance, or by other computing nodes in the database system. System failure means that the operating system or software in the computing node cannot run normally, causing the computing node to crash. System failure is usually recovered from the failure by other computing nodes in the database system. Among them, system failure includes process failure, which means that the process running in the database system fails. System failure can also be called soft failure. Media failure can be called hard failure, which specifically means that the hardware in the system fails. For example, the disk in the storage node cannot perform read and write operations due to interference or damage, etc. In the embodiment of the present application, failure repair is performed for transaction failure and system failure.

[0105] In the present application, the database management system usually realizes data access or update operations in units of database transactions (or simply referred to as transactions). A transaction is the basic unit for the database to perform concurrent control and failure recovery. A database transaction usually contains a sequence of read / write operations on the database.

[0106] Exemplarily, in Figure 1 when the computing node 1 fails, the computing node 2 serves as the first computing node to perform failure recovery; or, there are faulty database instances and available database instances in the computing node 1, and failure recovery is performed based on the available database instances, and the computing node 1 can serve as the first computing node to perform failure recovery. In Figure 1 the storage node 1 stores the log of the computing node 1, and the above step S401 specifically sends the storage address of the log of the computing node 1 from the computing node 1 or the computing node 2 to the storage node 1.

[0107] Before the above step S401, the method further includes: the first computing node obtains the storage address of the target log of the faulty node and generates a first message.

[0108] In a possible implementation manner, after the first computing node monitors that the faulty node fails, it reads the control file of the faulty node in the shared storage and obtains the storage address of the target log corresponding to the faulty node.

[0109] In another possible implementation, the first computing node can be any available computing node in the database system. The first computing node receives an indication message sent by other available nodes in the database system, reads the control file of the faulty node in the shared storage based on the indication message, and obtains the storage address of the target log; alternatively, the storage address of the target log is included in the indication message. The indication message is used to instruct the first computing node to recover the data of the faulty node.

[0110] S402. In response to the first message, the first storage node sends a recovery set to the first computing node, where the recovery set is used to indicate the target log and the target data blocks that actually need to be recovered in the target log.

[0111] The recovery set is used to indicate the target log and the target data blocks, and the target data blocks include the data blocks that actually need to be recovered in the target log.

[0112] Optionally, the recovery set includes the target log and the marking information corresponding to the data blocks recorded in the target log. The marking information corresponding to the data blocks recorded in the target log is used to indicate whether the data blocks are the data blocks that have been written to the disk. Among them, the data blocks that have not been written to the disk are the target data blocks, and the data blocks that have been written to the disk are the data blocks that actually do not need to be recovered.

[0113] It can be understood that, combined with the above Figure 3 description, the target log of the faulty node is updated following the checkpoint thread. Since the checkpoint thread runs periodically, during the period without updates, the dirty blocks in the memory will continue to be written to the data file. Although the redo logs corresponding to these data blocks can be recycled, since the checkpoint thread is not running, they are not updated to recyclable logs. Therefore, through this step, the data blocks corresponding to the actually recyclable redo logs can be trimmed to obtain the target data blocks that have not actually been written to the disk. For the first computing node, only replaying the redo logs corresponding to the target data blocks will help save computing power resources and time costs.

[0114] In a possible implementation, before executing the above step S402, the method further includes steps S501 - S503 as Figure 5 shown.

[0115] S501. The first storage node obtains the target log.

[0116] In a possible implementation, the storage address of the target log is included in the first message in the above step S401, and the first storage node obtains the target log based on the storage address of the target log in the received first message.

[0117] S502. The first storage node parses the target log to obtain the data blocks recorded in the target log.

[0118] In a possible implementation, the first storage node parses the target log to obtain the identifiers used to indicate the data blocks.

[0119] S503. The first storage node determines the target data block based on the LSN of the data block recorded in the target log and the LSN of the data block in the disk.

[0120] It can be understood that each log has a unique LSN, and the LSN increases gradually as the logs increase. The LSN of the data block in the disk is used to identify the latest LSN when the data block is stored in the disk. By comparison, the size relationship between the LSN of the log and the LSN of the data block in the disk can be known, and whether the data block in the disk needs to be updated can be determined accordingly.

[0121] This step S503 specifically includes: the first storage node determines the data blocks recorded in the target log and the maximum LSN of each data block, compares the maximum LSN with the LSN of the corresponding data block in the disk. When the maximum LSN is greater than the LSN of the corresponding data block in the disk, mark the data block; or, when the maximum LSN is less than or equal to the LSN of the corresponding data block in the disk, mark the data block.

[0122] As Figure 6 shown, it is a schematic diagram of a scenario for determining the target data block provided by an embodiment of the present application.

[0123] In Figure 6 , the first storage node determines that the target log includes Log 1 to Log 8. Among them, Log 1 to Log 5 record the modification operations of Data Block 1, and Log 6 to Log 8 record the modification operations of Data Block 2. And the maximum LSN of Data Block 1 is LSN_max 1, and the maximum LSN of Data Block 2 is LSN_max 2. The first storage node locates Data Block 1 and Data Block 2 in the shared storage and obtains the LSN 1' of Data Block 1 and the LSN 2' of Data Block 2 in the disk respectively. When LSN_max 1 ≤ LSN 1', mark Data Block 1; similarly, when LSN_max 2 ≤ LSN 2', mark Data Block 2.

[0124] It can be understood that in Figure 6 , when LSN_max 1 ≤ LSN 1', it means that the version of Data Block 1 in the disk is the latest version. After marking Data Block 1, correspondingly, the unmarked Data Block 2 is the target data block. Similarly, when LSN_max 2 ≤ LSN 2', the marked Data Block 2 does not need to be restored, and the unmarked Data Block 1 is the target data block.

[0125] In the above manner, it helps to determine whether a data block is a target data block based on whether the data block is marked.

[0126] It should be noted that in the above example, the unmarked data block is the target data block. The first storage node can also perform marking when LSN_max1 > LSN 1'. At this time, the marked data block is the target data block. That is to say, the first storage node can mark or not mark the target data block through different judgment methods to distinguish the data blocks that actually need to be restored and the data blocks that actually do not need to be restored.

[0127] Alternatively, the first storage node can also mark both the target data block and the data blocks that actually do not need to be restored. For example, the first storage node indicates the target data block through the first marking information and indicates the data blocks that actually do not need to be restored through the second marking information. For example, in Figure 6 when LSN_max 1 ≤ LSN 1', the first storage node determines that the marking information of data block 1 is the second marking information and determines that the marking information of data block 2 is the first marking information.

[0128] In one example, the first storage node uses 0 and 1 to mark and distinguish data blocks. It should be noted that the first storage node can also use other methods for marking, which is not limited herein.

[0129] S403. The first computing node performs log replay based on the recovery set.

[0130] Optionally, the first computing node replays the logs corresponding to the target data blocks according to the target logs and marking information in the recovery set to restore the target data blocks.

[0131] Among them, after the above step S401, the first computing node executes the remaster process, which specifically includes the following steps S21 - S22.

[0132] S21. The first computing node obtains the metadata information of the original data blocks managed by the faulty node.

[0133] Specifically, the first computing node obtains the metadata information of all data blocks managed by the faulty node in the shared storage.

[0134] It can be understood that the relationship between the data in the shared storage and the master node that manages the data is visible to any computing node, and the metadata information of each data block is recorded in the data file where the data block is located. In this way, the first computing node can obtain the required information in the shared storage.

[0135] Among them, the metadata information includes the lock mode information, role, and whether there is a pre-image described above.

[0136] S22. The first computing node distributes the data blocks managed by the faulty node to at least one second computing node in the database system for management, and stores the corresponding relationship between the data blocks and the at least one second computing node.

[0137] Optionally, the first computing node evenly disperses the data blocks managed by the faulty node to at least one available second computing node in the system for management. For example, if the faulty node manages 100 data blocks and there are 5 available second computing nodes, then each second computing node manages 20 data blocks. Here, the at least one second computing node may or may not include the first computing node, and there is no restriction on this.

[0138] It should be noted that the first computing node can also randomly disperse the data blocks to the available computing nodes, or allocate them based on a preset allocation strategy. For example, the first computing node allocates with reference to the computing power of the available computing nodes, and the computing node with stronger computing power manages a larger number of data blocks.

[0139] Based on steps S21 - S22, when the first computing node executes the above step S403, it specifically includes: the first computing node replays the target log in sequence. When any one of the target data blocks is the above - mentioned original data block, based on the stored corresponding relationship, it obtains the operation permission from the second computing node that manages the original data block. In the case where the operation permission indicates that modification of the original data block is allowed, the first computing node replays the corresponding log.

[0140] Optionally, after the first computing node obtains the data blocks to be repaired, it can also refer to the metadata information of the data blocks to shorten the repair time of the data blocks to be repaired.

[0141] In a possible implementation, the first computing node determines the lock - mode information of the data block to be repaired. When the lock - mode information of the data block to be repaired is read - only, it means that the data block was in a read state before the system crashed, that is, the repair of this data block at the faulty node is excluded. Then, the redo log of this data block was repaired by the faulty node before it became read - only. At this time, the repaired data block is copied and read by other nodes. Then, the first computing node can obtain the repaired data block from other nodes without log replay.

[0142] Generally, this situation can be excluded by comparing the second LSN of the data block stored on the disk with the first LSN in the log at the first storage node.

[0143] When the lock mode of the repaired data block is exclusive, it indicates that the data block was in a modified state before the system crashed, and the first computing node needs to replay the log corresponding to this data block for repair.

[0144] In a possible implementation, the first computing node determines the role of the data block to be repaired. When the role is global, it indicates that there is a copy of the data block to be repaired in the system; when the role is local, it indicates that there is no copy of the data block to be repaired in the system. When the role is global, the first computing node can further determine whether there is a pre-image for the data block. If there is, it can obtain the pre-image from other nodes, obtain the LSN in the pre-image, compare the size of the LSN with the value of the LSN in the target log. If it is greater than the LSN in the target log, the first computing node can refrain from replaying that segment of the log.

[0145] It can be understood that through the above metadata information, the computing node can reduce the process of replaying the redo log, thereby improving the efficiency of restoring data.

[0146] It should be noted that the above log replay obtains the restored data. Further, the undo log can also be referred to for transaction rollback, thereby improving the fault repair.

[0147] Through the above method, the storage node determines the data block information to be restored based on the log. Compared with the process in the prior art where the computing node reads and trims the log, the computing power overhead of the computing node is significantly reduced, and the computing resources of the storage node are fully utilized; in addition, after the computing node trims the log, it needs to obtain the log from the storage node again for replay. However, with the repair method provided in the above embodiment, the storage node reads and trims the log from the disk and then transfers it to the computing node, saving the communication overhead between storage and computing, and ultimately reducing the RTO time of the inoperable services during system recovery.

[0148] Combined with Figure 1 , such as Figure 7 shown, is a schematic diagram of a database fault repair scenario provided by an embodiment of the present application.

[0149] During the normal operation of the database cluster, the fault repair thread on the computing node side (recovery thread) and the log reading / parsing thread on the storage node side (redo log read thread) are both in a dormant state. When a fault occurs and is detected, the fault repair thread in the computing node instance is awakened, and the log reading / parsing thread in the storage node is awakened through a message, informing the storage node of the storage address of the target log to be parsed. The log reading / parsing thread of the storage node reads and parses the target log, generates a recovery set, and then sends the recovery set to the computing node to execute the log replay process.

[0150] It can be understood that combined with the above Figure 4In the described data recovery method, the remaster process of steps S21 - S22 is executed after starting a fault repair thread on the computing node side, and the log parsing thread of the first storage node can execute concurrently with the remaster process. Specifically, as Figure 8 shown, it is a flowchart of a data recovery method provided by this application, which is applied to a distributed storage system including a first computing node and a first storage node, and specifically includes the following steps S801 - S808.

[0151] S801. Whether the first computing node detects a fault.

[0152] If yes, proceed to step S802; if no, continue to wait.

[0153] Among them, the relevant description of the first computing node and the relevant description of determining the fault can refer to the above text. The first computing node can periodically execute this step S801.

[0154] S802. The first computing node starts a fault repair thread.

[0155] Among them, the fault repair thread is used to repair the data loss in the memory due to the fault.

[0156] S803. The first storage node obtains the first message sent by the computing node and wakes up the log reading / parsing thread.

[0157] At this time, on the computing side, execute step S804a. The first computing node restores the metadata information of the data block. On the storage side, execute steps S804b and S804c.

[0158] S804b. The first storage node obtains the target log and the data block recorded in the target log.

[0159] S804c. The first storage node determines the target data block, generates and sends a recovery set.

[0160] It can be understood that the above steps S804a, S804b, and S804c respectively correspond to the remaster process and the first step in recovery during the data recovery process. In this way, the computing node and the storage node execute this process synchronously, greatly shortening the time consumed for data recovery. Moreover, the storage node participating in the data recovery process helps to reduce the computing power burden of the computing node, thereby improving the overall resource utilization rate.

[0161] S805. The first computing node reads the target log in the recovery set.

[0162] S806. The first computing node determines whether the data block recorded in the target log is the target data block.

[0163] If so, perform step S807, and the first computing node replays the target log corresponding to the data block.

[0164] If not, perform step S808, and the first computing node skips the log without replaying it.

[0165] It can be understood that when the first computing node replays the log, it can refer to the target data block indicated by the recovery set, so as to determine whether to replay the log or skip the log, thereby accelerating the log replay process and reducing the data recovery time.

[0166] Based on Figure 8 the method shown, by having the computing side and the storage side participate in the data recovery process simultaneously, it helps to reduce the RTO time for the system to recover non - acceptable services to meet the service requirements of latency - sensitive databases.

[0167] It should be noted that Figure 8 the steps in Figure 4 can all refer to the above description of

[0168] The above mainly introduces the solution of the embodiment of the present application from the perspective of the method. It can be understood that in order to implement the above functions, the storage node includes at least one of the corresponding hardware structures and software modules for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving the hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0169] The embodiment of the present application can divide the functional units of the storage node according to the above - mentioned method examples. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above - integrated unit can be implemented in the form of hardware or in the form of a software functional unit. It should be noted that the division of units in the embodiment of the present application is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0170] The present application also provides a database system, as Figure 9 shown, the database system 90 includes a first computing node 901 and a first storage node 902, and this system specifically includes:

[0171] The first computing node 901 is used to send a first message to the first storage node 902. The first message is used to indicate the target log that needs to be recovered corresponding to the faulty node. The first computing node 901 is a computing node with fault recovery capabilities in the database system. The first storage node 902 is a storage node in the database system that manages the target log. The faulty node is the node to which the faulty database instance in the database system belongs or the faulty computing node.

[0172] The first storage node 902 is used to send a recovery set to the first computing node 901 in response to the first message. The recovery set is used to indicate the target log and the target data blocks. The target data blocks include the data blocks that actually need to be recovered in the target log.

[0173] The first computing node 901 is further used to receive the recovery set from the first storage node 902 and perform log replay based on the recovery set.

[0174] In one example, the recovery set includes the target log and the marking information corresponding to the data blocks recorded in the target log. The marking information corresponding to the data block is used to indicate whether the data block is a data block that has been written to the disk. The data block that has not been written to the disk is the target data block, and the data block that has been written to the disk is the data block that actually does not need to be recovered.

[0175] In one example, the first message includes the storage address of the target log. The first storage node 902 is further used to: obtain the target log according to the storage address of the target log; parse the target log to obtain the data blocks recorded in the target log; and determine the target data blocks based on the log sequence number LSN of the data blocks recorded in the target log and the log sequence number LSN of the data blocks in the disk.

[0176] In one example, the first storage node 902 is specifically used to: for each data block recorded in the target log, obtain the maximum LSN of the data block; compare the maximum LSN of the data block with the LSN of the data block in the disk; when the LSN of the data block in the disk is less than the maximum LSN of the data block, determine the data block as the target data block. The first storage node 902 is further used to determine the marking information of the data block as the first marking information, and the first marking information is used to indicate that the data block has not been written to the disk.

[0177] In one example, the first storage node 902 is further used to determine the marking information of the data block as the second marking information, and the second marking information is used to indicate that the data block has been written to the disk.

[0178] In one example, the database system further includes at least one second computing node 903; the first computing node 901 is further configured to obtain metadata information of the original data blocks managed by the faulty node; allocate the original data blocks to at least one available second computing node 903 in the database system for management; and store the correspondence between the original data blocks and the at least one second computing node 903.

[0179] In one example, the first computing node 901 is specifically configured to: for any target data block in the target data blocks, obtain an operation permission from the computing node managing the target data block based on the correspondence, where the operation permission is used to indicate whether modification of the target data block is allowed; and replay the log of the target data block in the target log when the operation permission indicates that modification of the target data block is allowed.

[0180] In one example, the first computing node 901 is further configured to: obtain metadata information of the target data block from the computing node managing the target data block, where the metadata information includes whether there is a pre-image of the data block, and when it is determined that there is a pre-image of the target data block, obtain the LSN of the pre-image of the target data block; compare the LSN of the pre-image of the target data block and the LSN of the target data block in the target log; and replay the log of the target data block when the LSN of the pre-image of the target data block is less than the LSN of the target data block in the target log.

[0181] In one example, the faulty node includes a third computing node 904 in the database system 90, or a database instance in the third computing node 904, and the third computing node is the same as or different from the first computing node.

[0182] For the explanations and descriptions of the beneficial effects of the relevant content in the above-provided database system 90, reference can be made to the corresponding embodiments above, and details are not elaborated here.

[0183] The first computing node 901, the first storage node 902, the at least one second computing node 903, and the third computing node 904 can all be implemented by software or by hardware. Exemplarily, the implementation manner of the first computing node 901 is introduced next. Similarly, the implementation manners of the first storage node 902, the at least one second computing node 903, and the third computing node 904 can refer to the implementation manner of the first computing node 901.

[0184] As an example of a software functional unit, the first computing node 901 may include code running on a computing instance. Herein, the computing instance may be at least one of computing devices such as a physical host (computing device), a virtual machine, a container, etc. Further, the above-mentioned computing devices may be one or more. For example, the first computing node 901 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers for running the application may be distributed in the same region or in different regions. The multiple hosts / virtual machines / containers for running the code may be distributed in the same availability zone (AZ) or in different AZs, and each AZ includes one data center or multiple geographically proximate data centers. Usually, one region may include multiple AZs.

[0185] Similarly, the multiple hosts / virtual machines / containers for running the code may be distributed in the same VPC or in multiple VPCs. Usually, one VPC is set within one region. For cross-region communication between two VPCs within the same region and between VPCs in different regions, a communication gateway needs to be set in each VPC, and the interconnection between VPCs is realized through the communication gateway.

[0186] As an example of a hardware functional unit, the first computing node 901 may include at least one computing device, such as a server, etc. Alternatively, the first computing node 901 may also be a device implemented using ASIC or PLD, etc. Herein, the above-mentioned PLD may be implemented by CPLD, FPGA, GAL or any combination thereof.

[0187] The multiple computing devices included in the first computing node 901 may be distributed in the same region or in different regions. The multiple computing devices included in the first network processing unit may be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first network processing unit may be distributed in the same VPC or in multiple VPCs. Herein, the multiple computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0188] This application also provides a computing device 100. As Figure 10As shown, computing device 100 includes: bus 102, processor 104, memory 106, and communication interface 108. The processor 104, memory 106, and communication interface 108 communicate with each other via bus 102. The computing device 100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

[0189] The bus 102 can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 only one line is used in the figure, but it does not mean that there is only one bus or one type of bus. The bus 104 can include a path for transmitting information between various components of the computing device 100 (for example, the memory 106, the processor 104, and the communication interface 108).

[0190] The processor 104 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0191] The memory 106 can include volatile memory, such as random access memory (RAM). The processor 104 can also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0192] The memory 106 stores executable program code, and each processing unit in the processor 104 executes the executable program code to respectively implement the functions of the foregoing first computing node and first storage node, thereby implementing the data recovery method. That is, the memory 106 stores instructions for executing the data recovery method.

[0193] Alternatively, executable code is stored in the memory 106, and the processor 104 executes the executable code to implement the functions of the foregoing first computing node and first storage node respectively, thereby implementing the data recovery method. That is to say, instructions for executing the data recovery method are stored on the memory 106.

[0194] The communication interface 103 uses a transceiver module such as, but not limited to, a network interface card or a transceiver. The transceiver module includes a receiving unit and a transmitting unit, and is used to implement communication between the computing device 100 and other devices or communication networks.

[0195] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0196] As Figure 11 shown, the computing device cluster includes at least one computing device 100. Instructions for executing the data recovery method may be stored in the memories 106 of one or more of the computing devices 100 in the computing device cluster.

[0197] In some possible implementation manners, partial instructions for executing the data recovery method may also be stored separately in the memories 106 of one or more of the computing devices 100 in the computing device cluster. In other words, a combination of one or more computing devices 100 may jointly execute the instructions for executing the data recovery method.

[0198] It should be noted that the memories 106 in different computing devices 100 in the computing device cluster may store different instructions, respectively for executing partial functions of the first computing node. That is to say, the instructions stored in the memories 106 of different computing devices 100 may implement the functions of one or more modules in the first computing node and the first storage node.

[0199] In some possible implementation manners, one or more computing devices in the computing device cluster may be connected through a network. Wherein, the network may be a wide area network or a local area network, etc. Figure 12 shows a possible implementation manner. As Figure 12 shown, two computing devices 100A and 100B are connected through a network. Specifically, they are connected to the network through the communication interfaces in each computing device. In this type of possible implementation manners, instructions for executing the functions of the first computing node are stored in the memory 106 of the computing device 100A. At the same time, instructions for executing the functions of the first storage node are stored in the memory 106 of the computing device 100B.

[0200] Figure 12 The connection mode between the computing device clusters shown can be such that considering that the data recovery method provided in this application requires a large amount of data storage, it is therefore considered to hand over the functions implemented by the first storage node to the computing device 100B for execution.

[0201] It should be understood that Figure 12 the functions of the computing device 100A shown in can also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B can also be completed by multiple computing devices 100.

[0202] The embodiments of this application also provide another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to Figure 11 and Figure 12 the connection mode of the described computing device cluster. The difference is that the same instructions for executing the data recovery method can be stored in the memory 106 of one or more of the computing devices 100 in this computing device cluster.

[0203] In some possible implementation manners, partial instructions for executing the data recovery method can also be separately stored in the memory 106 of one or more of the computing devices 100 in this computing device cluster. In other words, a combination of one or more computing devices 100 can jointly execute the instructions for executing the data recovery method.

[0204] It should be noted that the memories 106 in different computing devices 100 in the computing device cluster can store different instructions for executing partial functions of the database system. That is, the instructions stored in the memories 106 of different computing devices 100 can implement the functions of one or more devices in the first computing node and the first storage node.

[0205] The embodiments of this application also provide a computer program product containing instructions. The computer program product can be software or a program product containing instructions that can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, it causes at least one computing device to execute the data recovery method.

[0206] The embodiments of this application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive), etc. The computer-readable storage medium includes instructions that direct the computing device to execute the data recovery method.

[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data recovery method, characterized in that, the method is applied to a database system, and the database system includes a first computing node and a first storage node; in the case where there is a faulty node in the database system, the first computing node sends a first message to the first storage node, and the first message is used to indicate a target log to be recovered corresponding to the faulty node; the first computing node is a computing node with fault recovery ability in the database system; the first storage node is a storage node in the database system that manages the target log; the first storage node responds to the first message and sends a recovery set to the first computing node, and the recovery set is used to indicate the target log and target data blocks, and the target data blocks include data blocks that actually need to be recovered in the target log; the first computing node receives the recovery set from the first storage node and performs log replay based on the recovery set.

2. The method according to claim 1, characterized in that, the recovery set includes the target log and marking information corresponding to the data blocks recorded in the target log; the marking information corresponding to the data block is used to indicate whether the data block is a data block that has been written to the disk, the data block that has not been written to the disk is the target data block, and the data block that has been written to the disk is a data block that actually does not need to be recovered.

3. The method according to claim 1 or 2, characterized in that, the first message includes the storage address of the target log; before sending the recovery set to the first computing node, the method further includes: the first storage node obtains the target log according to the storage address of the target log; the first storage node parses the target log to obtain the data blocks recorded in the target log; the first storage node determines the target data blocks based on the log sequence number LSN of the data blocks recorded in the target log and the log sequence number LSN of the data blocks in the disk.

4. The method according to claim 3, characterized in that, determining the target data blocks based on the data blocks recorded in the target log and the log sequence number LSN of the data blocks in the disk includes: for each data block recorded in the target log, the first storage node obtains the maximum LSN of the data block; comparing the maximum LSN of the data block with the LSN of the data block in the disk; when the LSN of the data block in the disk is less than the maximum LSN of the data block, the first storage node determines the data block as the target data block; the method further includes: the first storage node determines the marking information of the data block as first marking information, and the first marking information is used to indicate that the data block has not been written to the disk.

5. The method according to claim 4, characterized in that, the method further includes: When the LSN of the data block in the disk is greater than or equal to the maximum LSN of the data block, the first storage node determines the marking information of the data block as second marking information, and the second marking information is used to indicate that the data block has been written to the disk.

6. The method according to claim 1 or 2, wherein, the method further includes: the first computing node obtains the metadata information of the original data block managed by the faulty node; the first computing node allocates the original data block to at least one available second computing node in the database system for management; the first computing node stores the correspondence between the original data block and the at least one second computing node.

7. The method according to claim 6, wherein, when the target data block is the original data block, the first computing node performs log replay based on the recovery set, including: for any target data block in the target data block, the first computing node obtains an operation permission from the computing node managing the target data block based on the correspondence, and the operation permission is used to indicate whether modification of the target data block is allowed; when the operation permission indicates that modification of the target data block is allowed, the first computing node replays the log of the target data block in the target log.

8. The method according to claim 7, wherein, when the first computing node replays the log of the target data block in the target log, the method further includes: the first computing node obtains the metadata information of the target data block from the computing node managing the target data block, and the metadata information includes whether there is a previous image piblock for the data block; when the first computing node determines that there is a previous image for the target data block, the first computing node obtains the LSN of the previous image of the target data block; comparing the LSN of the previous image of the target data block and the LSN of the target data block in the target log; when the LSN of the previous image of the target data block is less than the LSN in the target log, replaying the log of the target data block.

9. The method according to any one of claims 1-8, wherein, the faulty node includes a third computing node in the database system, or a database instance in the third computing node, and the third computing node is the same as or different from the first computing node.

10. A data recovery method, wherein, it includes: a first storage node receives a first message sent by a first computing node, and the first message is used to indicate a target log to be recovered corresponding to a faulty node in a database system; the first computing node is a computing node with fault recovery ability in the database system, and the first storage node is a storage node in the database system that manages the target log; In response to the first message, the first storage node sends a recovery set to the first computing node for performing log replay. The recovery set is used to indicate the target log and target data blocks, and the target data blocks include the data blocks that actually need to be recovered in the target log.

11. The method according to claim 10, wherein, the recovery set includes the target log and marker information corresponding to the data blocks recorded in the target log; the marker information corresponding to the data block is used to indicate whether the data block is a data block that has been written to the disk. The data block that has not been written to the disk is the target data block, and the data block that has been written to the disk is the data block that actually does not need to be recovered.

12. The method according to claim 10 or 11, wherein, the first message includes the storage address of the target log; before sending the recovery set to the first computing node, the method further includes: the first storage node obtains the target log according to the storage address of the target log; the first storage node parses the target log to obtain the data blocks recorded in the target log; the first storage node determines the target data blocks based on the log sequence number (LSN) of the data blocks recorded in the target log and the LSN of the data blocks in the disk.

13. The method according to claim 12, wherein, determining the target data blocks based on the data blocks recorded in the target log and the LSN of the data blocks in the disk includes: for each data block recorded in the target log, the first storage node obtains the maximum LSN of the data block; comparing the maximum LSN of the data block with the LSN of the data block in the disk; when the LSN of the data block in the disk is less than the maximum LSN of the data block, the first storage node determines the data block as the target data block; the method further includes: the first storage node determines the marker information of the data block as first marker information, and the first marker information is used to indicate that the data block has not been written to the disk.

14. The method according to claim 13, wherein, the method further includes: when the LSN of the data block in the disk is greater than or equal to the maximum LSN of the data block, the first storage node determines the marker information of the data block as second marker information, and the second marker information is used to indicate that the data block has been written to the disk.

15. A data recovery method, wherein, includes: a first computing node sends a first message to a first storage node, and the first message is used to indicate the target log that needs to be recovered corresponding to a faulty node in a database system; the first computing node is a computing node with fault recovery capability in the database system, and the first storage node is a storage node that manages the target log in the database system; the first computing node receives the recovery set sent by the first storage node, and the recovery set is used to indicate the target log and target data blocks, and the target data blocks include the data blocks that actually need to be recovered in the target log; The first computing node performs log replay based on the recovery set.

16. The method according to claim 15, wherein, the recovery set includes the target log and the marker information corresponding to the data blocks recorded in the target log; the marker information corresponding to the data blocks is used to indicate whether the data blocks are the data blocks written to the disk, the data blocks not written to the disk are the target data blocks, and the data blocks written to the disk are the data blocks that actually do not need to be recovered.

17. The method according to claim 15 or 16, wherein, the method further includes: the first computing node obtains the metadata information of the original data blocks managed by the faulty node; the first computing node allocates the original data blocks to at least one available second computing node in the database system for management; the first computing node stores the correspondence between the original data blocks and the at least one second computing node.

18. The method according to claim 17, wherein, when the target data block is the original data block, the first computing node performing log replay based on the recovery set includes: for any target data block in the target data blocks, the first computing node obtains an operation permission from the computing node managing the target data block based on the correspondence, and the operation permission is used to indicate whether modification of the target data block is allowed; in the case where the operation permission indicates that modification of the target data block is allowed, the first computing node replays the log of the target data block in the target log.

19. A database system, wherein, it includes a first computing node and a first storage node, the first computing node is configured to send a first message to the first storage node, and the first message is used to indicate the target log that needs to be recovered corresponding to the faulty node; the first computing node is a computing node with fault recovery ability in the database system; the first storage node is a storage node in the database system that manages the target log, and the faulty node is the node to which the faulty database instance in the database system belongs or the faulty computing node; the first storage node is configured to, in response to the first message, send a recovery set to the first computing node, and the recovery set is used to indicate the target log and the target data blocks, and the target data blocks include the data blocks that actually need to be recovered in the target log; the first computing node is further configured to receive the recovery set from the first storage node and perform log replay based on the recovery set.

20. The database system according to claim 19, wherein, the first storage node is specifically configured to: obtain the target log according to the storage address of the target log; parse the target log to obtain the data blocks recorded in the target log; determine the target data blocks based on the log sequence number LSN of the data blocks recorded in the target log and the log sequence number LSN of the data blocks in the disk.

21. The database system according to claim 20, wherein, The first storage node is specifically configured to: For each data block of the target log record, the first storage node obtains the maximum LSN of the data block; Compare the maximum LSN of the data block with the LSN of the data block in the disk; When the LSN of the data block in the disk is less than the maximum LSN of the data block, the first storage node determines the data block as the target data block.

22. A computing device, characterized in that, it includes: a receiving unit and a sending unit; The receiving unit is configured to receive a first message sent by a first computing node, where the first message is used to indicate a target log to be recovered corresponding to a faulty node in a database system; the first computing node is a computing node with fault recovery capability in the database system, and the computing device is a storage node in the database system that manages the target log; The sending unit is configured to, in response to the first message, send a recovery set to the first computing node for performing log replay, where the recovery set is used to indicate the target log and target data blocks, and the target data blocks include data blocks that actually need to be recovered in the target log.

23. The computing device according to claim 22, characterized in that, The computing device further includes a processing unit, configured to: Obtain the target log according to the storage address of the target log; Parse the target log to obtain the data blocks recorded in the target log; Determine the target data blocks based on the log sequence number LSN of the data blocks recorded in the target log and the log sequence number LSN of the data blocks in the disk.

24. The computing device according to claim 23, characterized in that, The processing unit is specifically configured to: For each data block of the target log record, obtain the maximum LSN of the data block; Compare the maximum LSN of the data block with the LSN of the data block in the disk; When the LSN of the data block in the disk is less than the maximum LSN of the data block, determine the data block as the target data block.

25. A computing device, characterized in that, it includes: a sending unit, a receiving unit, and a processing unit; The sending unit is configured to send a first message to a first storage node, where the first message is used to indicate a target log to be recovered corresponding to a faulty node in a database system; the computing device is a computing node with fault recovery capability in the database system, and the first storage node is a storage node in the database system that manages the target log; The receiving unit is configured to receive the recovery set sent by the first storage node, where the recovery set is used to indicate the target log and target data blocks, and the target data blocks include data blocks that actually need to be recovered in the target log; The processing unit is configured to perform log replay based on the recovery set.

26. A computing device, characterized in that, comprising a memory and a processor; the memory is used for storing program codes; the processor is used for calling the program codes to execute the method according to any one of claims 10 - 14 or any one of claims 15 - 18.

27. A cluster of computing devices, characterized in that the cluster of computing devices includes at least one computing device, and each computing device includes a processor and a memory; the memory in the at least one computing device is used for storing computer program instructions; the processor in the at least one computing device calls the computer program instructions stored in the memory to execute the method according to any one of claims 1 - 9, any one of claims 10 - 14 or any one of claims 15 - 18.

28. A computer-readable storage medium, characterized in that it includes computer program instructions, and when the computer program instructions are executed by a computing device, the computing device executes the method according to any one of claims 1 - 9, any one of claims 10 - 14 or any one of claims 15 - 18.

29. A computer program product containing instructions, characterized in that it includes computer program instructions, and when the computer program instructions are run by a computing device, the computing device is caused to execute the method according to any one of claims 1 - 9, any one of claims 10 - 14 or any one of claims 15 - 18.

Images