Data recovery method and related device

Abstract

The invention discloses a data recovery method and a related device, the method can be applied to a server running a database system, and the method comprises the steps that in response to modification of a first data page, a first log is generated, modification of the first data page is recorded in the first log, and a first identifier indicates the version of the first data page; under the condition that the database system fails, the version of a second data page is determined, the second data page is the data page of the data to be recovered, and the page identifier of the second data page is the same as the page identifier of the first data page; and under the condition that the version of the first log is greater than that of the second data page and the version indicated by the first identifier is the same as that of the second data page, performing data recovery on the second data page by using the first log. Through the first identifier, data recovery on the second data page by using a log with a wrong version is avoided, and the accuracy of data recovery is ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data recovery method and related devices. Background Art

[0002] Currently, databases have been widely used. During the operation of the database, the database may be unexpectedly closed due to sudden reasons such as power outages and crashes. At this time, the database needs to be restored.

[0003] Specifically, when an operation is performed on a data page of a database, the operation is recorded to form a log and stored in a log file. The log file and the data page (data file) are stored together on the disk. The modification of the data page according to the operation instruction is executed in the memory. After the memory processes the data page according to the operation instruction, a dirty page is generated. The dirty page is a data page that has been modified but not yet stored on the disk. The operation of storing the log file on the disk precedes the operation of storing the dirty page on the disk in time. In the process of restoring the database according to the log file, the starting point of the log recovery is first determined according to the checkpoint, and then each log in the log file is read one by one from the starting point. According to each log, the data page on the disk is retrieved for redo (Redo), and the redone data page is stored back to the disk. When all logs are redone and the redone data page is stored back to the disk, the database recovery operation is completed.

[0004] As application scenarios become increasingly complex, data pages and log files may undergo multiple processing and conversions. If a data page is continuously modified and a redo log is lost during the modification process, the data in the disk may be redone using the redo log during data recovery, and the redoed data page may be incorrect. Summary of the invention

[0005] In a first aspect, an embodiment of the present application provides a data recovery method, which is applied to a server running a database system, and the method includes:

[0006] In response to modifying the first data page, the first log is generated, the first log includes a first identifier, the first identifier indicates a version of the first data page, and the first log records the modification to the first data page; in the event of a failure of the database system, a failure recovery phase or a data recovery phase is entered, and a version of a second data page is determined, the second data page is a data page of data to be recovered, and a page identifier of the second data page is the same as a page identifier of the first data page; when the version of the first log is greater than the version of the second data page, and the version indicated by the first identifier is the same as the version of the second data page, the first log is used to perform data recovery on the second data page.

[0007] In this embodiment, the version of the data page is used to indicate the logical order of data changes that occur in the data page. In terms of timing, the smaller the version of the data page, the earlier the data change occurs in the data page; the larger the version of the data page, the later the data change occurs in the data page. Furthermore, the version of the data page can be identified in a variety of ways, which are not limited in the embodiments of the present application. For example, the log sequence number (LSN) of the data page indicates the version of the data page. For another example, the modification timestamp of the data page indicates the version of the data page.

[0008] Similarly, the version of a log is used to indicate the logical order in which data changes occur in the data page corresponding to the log. In terms of time sequence, the smaller the version of the log, the earlier the time when the log is generated, or the earlier the time when the data page corresponding to the log changes; the larger the version of the log, the later the time when the log is generated, or the later the time when the data page corresponding to the log changes. Further, the version of the log can be identified in a variety of ways, which is not limited by the embodiments of the present application. For example, the log sequence number (LSN) of the log indicates the version of the log. For another example, the modification timestamp of the log indicates the version of the log.

[0009] In an embodiment of the present application, after the first data page is modified, the server generates a first log, in which the modification to the first data page is recorded. The first log includes a first identifier, and the first identifier indicates the version of the first data page. When the second data page needs to recover data, the first log can be used to recover data on the second data page only when the version of the first log is greater than the version of the second data page and the version indicated by the first identifier is the same as the version of the second data page. Through the first identifier, the use of a log with an incorrect version to recover data on the second data page is avoided, thereby ensuring the accuracy of data recovery.

[0010] In a possible implementation, the first identifier is a log sequence number of the first data page. It is understandable that the first identifier may also be a modification timestamp of the first data page, and the version indicated by the first identifier may be determined according to the modification timestamp. Alternatively, the first identifier may also be a version number of the first data page.

[0011] In a possible implementation, the first log is a redo log.

[0012] In a possible implementation, when the version indicated by the first identifier is different from the version of the second data page, an error message is generated, and the error message indicates that the second data page cannot be restored. The error message facilitates the discovery and location of the log error, thereby improving the data security of the database.

[0013] In a possible implementation, the error information is further used to indicate any one or more of the following information: a log loss occurs before the first log, a parallel recovery logic error occurs, or a data page error occurs.

[0014] In one possible implementation, data recovery of the second data page is performed using the first log, including: redoing the second data page using the first log to obtain a third data page; and writing the third data page to the disk to replace the second data page in the disk.

[0015] Optionally, after the third data page is generated, it is possible to continue to check whether there are other logs after the recovery starting point in the log file. If there are, it is checked whether the other logs can be used to perform data recovery processing on the third data page in the log file. If there are other logs that can perform data recovery processing on the third data page, data recovery processing is performed on the third data page; if there are no other logs, the third data page is written to the disk.

[0016] In the second aspect, an embodiment of the present application proposes a server including: a receiving module, a processing module and a sending module, wherein the receiving module and the sending module are used to transmit data, and the processing module is used to process the data. The server includes: a processing module, which is used to generate the first log in response to modifying the first data page, the first log includes a first identifier, the first identifier indicates the version of the first data page, and the first log records the modification of the first data page; the processing module is also used to determine the version of the second data page in the event of a failure of the database system, the second data page is the data page of the data to be recovered, and the page identifier of the second data page is the same as the page identifier of the first data page; the processing module is also used to use the first log to perform data recovery on the second data page when the version of the first log is greater than the version of the second data page and the version indicated by the first identifier is the same as the version of the second data page.

[0017] In a possible implementation, the first identifier is a log sequence number of the first data page.

[0018] In a possible implementation, the first log is a redo log.

[0019] In a possible implementation, the processing module is further configured to generate an error message when the version indicated by the first identifier is different from the version of the second data page, wherein the error message indicates that data recovery of the second data page is impossible.

[0020] In one possible implementation,

[0021] The error message is also used to indicate any one or more of the following information:

[0022] A log loss occurs before the first log, a parallel recovery logic error occurs, or a data page error occurs.

[0023] In one possible implementation,

[0024] The processing module is further configured to perform redo processing on the second data page using the first log to obtain a third data page;

[0025] The processing module is further configured to write the third data page to the disk to replace the second data page in the disk.

[0026] In one possible implementation,

[0027] A receiving module, used to obtain at least one log from a recovery starting point in a log file, wherein the log file includes one or more logs;

[0028] The processing module is further used to detect the log sequence number of the log in the log file and the log sequence number of the second data page;

[0029] The processing module is further configured to determine that the second data page does not need to be restored when the log sequence numbers of all logs in the log file are greater than or equal to the log sequence number of the second data page;

[0030] The processing module is further used to detect whether the version of the first data page before modification is the same as the version of the second data page when the first log exists in the log file and the log sequence number of the second data page is smaller than the log sequence number of the first log.

[0031] In one possible implementation,

[0032] The processing module is further used to determine the recovery starting point in the log file according to a checkpoint, wherein the checkpoint indicates the storage location of the last refreshed dirty page, and the dirty page is a data page that has been modified and not refreshed to the disk.

[0033] In one possible implementation,

[0034] The receiving module is further used to obtain the log sequence number of the second data page;

[0035] The processing module is further used to determine the version of the second data page according to the log sequence number of the second data page.

[0036] In a third aspect, the present application proposes a server comprising a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the server executes a method as described in any one of the first aspects.

[0037] A fourth aspect of the present application provides a database system, comprising a server as described in the second aspect or any implementation of the second aspect, or a server as described in the third aspect or any implementation of the third aspect.

[0038] In a fifth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect.

[0039] In a sixth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A possible network architecture diagram provided for an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the structure of a server 201 provided in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of the architecture of a storage system involved in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of a data recovery scenario;

[0044] Figure 5 This is a schematic diagram of the structure of a data page in MySQL;

[0045] Figure 6 A schematic diagram of a data recovery process;

[0046] Figure 7 A schematic diagram of an embodiment of a data recovery method proposed in an embodiment of the present application;

[0047] Figure 8 A schematic diagram of a redo log structure proposed in an embodiment of the present application;

[0048] Fig. 9 This is a schematic diagram of an application scenario in an embodiment of the present application;

[0049] Fig.10 This is a schematic diagram of an application scenario in an embodiment of the present application;

[0050] Fig.11 A schematic diagram of an application scenario proposed in an embodiment of the present application;

[0051] Fig.12 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0053] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0054] To facilitate understanding, the technical terms involved in the embodiments of the present application are first introduced below.

[0055] Metadata: A type of data used to describe data. Metadata is mainly information that describes the attributes of data and is used to assist in data retrieval.

[0056] Baseline data: a version of data based on a certain point in time.

[0057] Incremental data: A new data version generated based on the baseline data, that is, a new data version generated after adding, deleting, and modifying the baseline data.

[0058] Checkpoint: A mechanism similar to bookmarks. After a checkpoint is made, all previous logs are deleted from the system and permanently stored in the storage disk.

[0059] Structured Query Language (SQL): A database query and programming language used to access data and query, update, and manage relational database systems. In simple terms, SQL is a standard language for clients to operate on data in a database.

[0060] A transaction is a set of continuous operations, specifically a program execution unit that accesses and possibly updates various data items in a database. For example, if A transfers 1,000 yuan to B, the amount in A's account will be reduced by 1,000 in the database, and the amount in B's account will be increased by 1,000.

[0061] Just like the special effect of transactions - atomicity, atomicity means that a transaction is an indivisible unit of work, or a transaction is a program execution unit. The operations included in the transaction are either all done (i.e. all executed successfully) or none of them are done (i.e. all executed failed). For example, in the above example, either the transfer is successful, the amount in account A decreases by 1000, and the amount in account B increases by 1000. Or the transfer fails, and the original amount in accounts A and B remains unchanged.

[0062] It can be seen that transactions have another characteristic - consistency. Consistency means that after the transaction is executed, the database (or transaction processing system) changes from one consistent state to another consistent state. Transactions also have other characteristics. Please refer to the introduction of related technologies, which will not be explained here one by one.

[0063] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application. The application scenario shows a client 100 and a transaction processing system 200, wherein the transaction processing system 200 includes one or more servers 201 ( Figure 1 One is shown, but this application is not limited to this).

[0064] The client 100 is deployed on the user side and can be a virtual machine or a physical machine. The physical machine can be a desktop computer, a desktop computer, a laptop computer, a server, etc. The client 100 can also be software deployed on the user side device (such as a database client), which is not limited in this application. The client 100 can communicate with the server 201 through the network 150, wherein the network 150 generally represents any telecommunication or computer network, including, for example, an intranet, a wide area network (WAN), a local area network (LAN), a personal area network (PAN), or the Internet. Alternatively, the client 100 can also communicate through a fiber optic switch ( Figure 1 However, the fiber switch 110 is only an optional device, and the fiber switch 110 can also be replaced by an Ethernet switch, an Infiniband switch, a Remote Direct Memory Access over Converged Ethernet (RDMA over converged ethernet, RoCE) switch, etc.

[0065] Server 201 can be an independent physical server, a desktop computer, a laptop computer, etc., or it can be a server cluster composed of multiple physical servers or a computing node or storage node in a distributed system. The server provides various cloud services, such as cloud database, cloud computing, cloud storage and other basic cloud computing services.

[0066] The server involved in this solution may be a hardware server or may be embedded in a virtualized environment. For example, the server involved in this solution may be a virtual machine executed on a hardware server including one or more other virtual machines.

[0067] The server 201 can provide services such as data access services for the client 100. Specifically, the server 201 can run a database such as PostgreSQL, MySQL, Oracle Database, etc., for data storage and data management, and can conveniently perform operations such as adding, deleting, viewing, and modifying data.

[0068] The user sends a request to the server 201 through the client 100 to access the data in the server 201. The request may be a transaction-level request (referred to as a transaction). The server 201 involved in the transaction is the participant of the transaction (referred to as a transaction participant). The transaction participant is used to process the transaction and perform a set of continuous operations on the database.

[0069] For example, in the above-mentioned "transfer", the transaction may have two transaction participants, namely server 201A and server 201B, wherein the database of server 201A stores data items such as the amount of A, and the database of server 201B stores data items such as the amount of B. Of course, the two amounts (A and B) can also be stored in one server 201 (such as server 201A), and the transaction participant can also be only one node, namely server 201A. It should be noted that the above-mentioned "transfer" is only an example. In fact, the transaction may include more operations, which are not limited in this application.

[0070] Therefore, in order to ensure the consistency of transactions, etc., at present, the write ahead log (WAL) method is usually used to process transactions. That is, before each transaction participant modifies the data item of the database stored in the hard disk, it first records the modification operation of the data item in a log, and stores the log in a non-volatile memory (such as a hard disk, which will be introduced below). In this way, even if the transaction participant loses power or restarts due to a failure, these logs can still be used to restore to the progress of transaction processing before the restart, thereby ensuring the consistency of the transaction.

[0071] Figure 2A hardware structure diagram of a server 201 (transaction participant) provided in an embodiment of the present application, see Figure 2 As shown, at the hardware level, the server 201 includes at least a processor 212, a memory 213, a network card 214, and a hard disk 215. Among them, the processor 212, the memory 213, the hard disk 215, and the network card 214 are connected through a bus 216. The bus 216 includes, but is not limited to: peripheral component interconnect express (PCIe), extended industry standard architecture (EISA) bus, unified bus (Ubus or UB), computer express link (compute express link, CXL), cache coherent interconnect for accelerators (CCIX), generation Z (GENZ), open coherent accelerator processor interface (OpenCAPI), NVIDIA link (NVlink), and interconnect bus supporting multiple protocols, which is not limited in this application.

[0072] The processor 212 may be a central processing unit (CPU) for processing requests from the client 100, which will be described in detail below and will not be repeated here. Figure 2Only one CPU 212 is shown. In actual applications, there are often multiple CPUs 212, wherein one CPU 212 has one or more CPU cores. This embodiment does not limit the number of CPUs and the number of CPU cores. In addition, the processor 212 may also be a graphics processing unit (GPU), a data processing unit (DPU), an application specific integrated circuit (ASIC), a system on chip (SOC), a field programmable gate array (FPGA), an embedded neural network processor (NPU) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, artificial intelligence chips, chips on chips, etc.

[0073] Memory 213 refers to an internal memory that directly exchanges data with processor 212. It can read and write data at any time and at a very fast speed, and serves as a temporary data storage for an operating system or other running programs. Memory 213 includes multiple types of memory, such as random access memory (RAM) and read-only memory (ROM). Random access memory includes, but is not limited to, dynamic random access memory (DRAM), double data rate synchronous dynamic random access memory (DDR), static random access memory (SRAM), etc. Read-only memory includes, but is not limited to, programmable read only memory (PROM), erasable programmable read only memory (EPROM), etc. In actual applications, multiple memories 213 and different types of memories 213 can be configured in server 201. This embodiment does not limit the number and type of memory 213.

[0074] The network card 214 is used to communicate with external devices, such as receiving a request (such as a transaction request) sent by the client 100, and sending the processing result of the transaction (execution success or execution failure) to the client 100. It is also used to communicate with components inside the server 201, such as sending the transaction request received from the client 100 to the processor 212, etc.

[0075] The hard disk 215 may also be referred to as an auxiliary memory. The hard disk 215 may be a non-volatile memory, such as a read-only memory (ROM), a hard disk drive (HDD) or a solid state drive (SSD). The difference between the hard disk 215 and the memory 213 is that the data in the memory 213 is usually volatile when power is off, while the hard disk 215 can be used to store data persistently, including the data of the database run by the server 201.

[0076] In the present application, the hard disk 215 includes an area for storing redo logs (redo logs will be introduced below) (redo logs are referred to as log areas) and an area for storing database data (data areas). The database management system accesses data in units of pages, that is, the data area stores pages one by one, and the sizes of pages managed by different databases may be different, such as the default page size of MySQL (my structured query language) is 16KB, and the pages of Oracle open gauss are usually 8KB / 16KB, etc., which are not limited in the present application. It should be noted that the page in the present application can also be understood as a data unit, which is a storage space unit for database management data, and does not specifically refer to the page recorded in the relevant technology, such as the page in the SSD (usually 512B, or 4KB). A page in the data area is referred to as a data page as follows.

[0077] The data page is also the smallest unit for the database to perform I / O operations, such as reading one or more data pages in the data area into the memory 213. Correspondingly, the memory 213 also includes an area for storing data pages, such as a data page pool (pagepool). When the processor 212 responds to the request of the client 100, the data page requested to be accessed by the request can be first obtained from the page pool of the memory 213 (referred to as the target data page). If the memory 213 does not hit the target data page, the processor 212 can read the target data page from the data area of ​​the hard disk 215 into the page pool of the memory 213.

[0078] For ease of understanding, the following describes in detail the read or write operation of the server 201 in conjunction with a possible application scenario. Figure 3 , Figure 3 A schematic diagram of the architecture of a storage system involved in an embodiment of the present application. Figure 3 The illustrated storage system includes a server 201 including computing nodes and storage nodes. The computing nodes include a master node and one or more slave nodes. It should be noted that the server 201 in the embodiment of the present application may also have other structures, and the embodiment of the present application does not limit this.

[0079] Specifically, the storage system includes a client on the user side and computing nodes and storage nodes located in the cloud, which together constitute the server 201. Database as a service (DBaaS) is a cloud computing service model that provides users with a certain form of access to a database. Users can access the database deployed in the cloud through a client (such as a web application) to configure and manage the database.

[0080] Among them, the computing node includes a master node, or a master node and one or more slave nodes, the master node is used to process the read request and write request of the client, the slave node only processes the read request of the client, the storage node is used to store data, the computing node and the master node are connected through a network, and the master node and the slave node share a storage node. When adding computing nodes in this way, there is no need to adjust the storage resources, and the separation of computing and storage is achieved. Among them, the master node can be any one of a physical machine, a virtual machine or a container, and the slave node can be any one of a physical machine, a virtual machine or a container. When the master node and the slave node are virtual machines or containers, the master node and the slave node can be deployed in the same physical machine or in different physical machines, and the embodiments of the present application do not make specific restrictions.

[0081] Combine the following Figure 3 The relevant concepts involved in the embodiments of the present application are introduced.

[0082] A data page is the basic unit for storing data in a database. In the same database, the size of each page is fixed, for example, 16 kilobytes (kb). Each page has a unique page identifier (page identity, page id), for example, the page identifier number increases from 0. In the storage system, data pages exist in two locations: the disk of the storage node and the memory of the computing node. For example Figure 3The buffer pool is a memory area in the computing node that is used to store data pages read from the storage node's disk.

[0083] In the embodiment of the present application, the data page in the disk of the storage node is called the base page. When the data in a data page in the memory of the computing node is inconsistent with the data in the data page with the same page identifier in the disk, the data page with the page identifier in the memory is called a dirty page. That is, a dirty page is a data page obtained after being modified in the memory of the computing node. For example, Figure 3 After the data page with page ID 5 in the disk of the storage node is read into the memory by the master node, the master node deletes a row of data in the data page to obtain a modified data page. The modified data page is a dirty page in the master node, and the page ID is still 5.

[0084] The redo log records the state of the data page after the write request is executed. The storage node can obtain the data page that has been updated in the computing node but not written to the disk based on the base page and the redo log. In the storage system, the redo log exists in two places: one is the redo log buffer in the memory of the computing node, which is volatile. When the computing node is powered off, the redo log in the memory will be deleted; the other is the redo log file in the disk of the storage node. The redo log in the disk is persistent.

[0085] When a base page on the disk is read by the master node into the master node's memory and modified by the master node, the master node will generate a corresponding redo log, which is first stored in the master node's redo log buffer, and then sent to the storage node according to preset rules (for example, every one second), and stored in the disk by the storage node; the storage node can obtain the modified data of the base page based on a base page and the redo log corresponding to the base page.

[0086] The redo log includes at least a page identifier, a data identifier, the result of executing the write request and the log sequence number (LSN). The page identifier indicates which data page is modified in the redo log record, the data identifier indicates which data in the data page indicated by the page identifier is modified, and the value of the LSN is used to indicate the length of the redo log. The value of the LSN increases with the length of the redo log written into the redo log buffer. The larger the value of the LSN, the later the corresponding log is written. Therefore, the LSN can also indicate the order of writing the redo log. For example, before executing the first write request to modify the data in the data page with the page identifier of 10, the maximum value of the LSN recorded in the redo log buffer is 100, indicating that a redo log of 100 bytes has been generated before. If the length of the redo log generated after the first write request is executed is 100 bytes, after the redo log is written into the redo log buffer, the maximum LSN recorded in the redo log buffer becomes 200, and the LSN recorded in the redo log generated by executing the first write request is 200. Afterwards, the master node executes a second write request to modify the data in the data page with page identifier 15. The length of the redo log generated after executing the second write request is 150 bytes. After the redo log is written to the redo log buffer, the maximum LSN recorded in the redo log buffer becomes 350. The LSN recorded in the redo log generated by executing the second write request is 350.

[0087] In addition, the LSN is also recorded in the data page. The LSN recorded in the data page is the LSN recorded in the log generated when the data page was last modified, which is used to indicate the version information of the data page. For example, after the first write request is executed to modify the data in the data page with page ID 10, the LSN recorded in the generated redo log is 200, then the LSN recorded in the data page after the modification is also 200.

[0088] Assume that the request is a transaction, which is a set of modification operations on the target data page (such as modification operation 1, modification operation 2, and modification operation 3 in sequence). Figure 4 , the process of processor 212 processing the transaction may include: (1) obtaining the target data page from the page pool of memory 213 (2) sequentially executing a set of modification operations required by the transaction on the target data page: first, executing modification operation 1 on the target data page (such as data page V0) to obtain data page V1 → executing modification operation 2 on data page V1 to obtain data page V2 → executing modification operation 3 on data page V2 to obtain data page V3.

[0089] The transaction processing adopts the log-first (WAL) method to record the execution stage of the transaction, that is, the modification operation performed on the data page in the memory 213 is recorded in the log. Before writing the modified data page in the memory 213 to the hard disk 215, the log is first written to the log area of ​​the hard disk 215. The log here is used to redo the modification operation recorded in the log after the system crashes, so as to restore the data page in the memory 213 before the system crashes. Therefore, these logs used to perform data recovery can also be called "redo logs". The redo logs in the log area are stored in chronological order.

[0090] If you continue to see Figure 4 In the above transaction processing, the processor 212 can generate three redo logs, namely redo log1, which is used to record modification operation 1. redo log2, which is used to record modification operation 2. redo log3, which is used to record modification operation 3. The redo logs are sequentially stored in the log area of ​​the hard disk 215. Before the transaction is committed, all the redo logs of the transaction must be written to the hard disk 215. It is worth noting that only one transaction is used as an example here. In fact, the processor 212 can process multiple transactions in parallel, that is, the log area may store the redo logs of multiple transactions. The redo logs of multiple transactions may be stored interspersed, for example, Figure 4 In the log area shown, redo logs of other transactions may be stored between redo log1 and redo log2. In addition, in addition to recording the modification operation, the redo log may also record other information, such as the identifier of the data page on which the modification operation is performed (such as the key value of the data page, referred to as the page identifier), the identifier of the transaction, the modification time, etc., which is not limited in the embodiment of the present application. It should be understood that whether the data page is in the memory 213 or in the hard disk 215, the identifier of the data page is unchanged.

[0091] The processor 212 can periodically store data pages in the memory 213 (page pool) to the hard disk 215 for persistent storage to reduce the amount of data that needs to be restored when the system crashes, such as Figure 4At time T1, processor 212 performs a "checkpoint" operation: write data page V0 in memory 213 to the data area of ​​hard disk 215, and at the same time, mark the position after redolog0 in log area as recovery point (such as writing a checkpoint mark at this position). The redo log before the recovery point is invalid and can be recycled. After that, if a system crash occurs, the redo log after the recovery point is used for recovery. It is worth noting that server 201 may also process other transactions at the same time. When executing a checkpoint, processor 212 can store all data pages in the page pool of memory 213 at the current moment (such as executing the checkpoint moment T1), not just the data page of a certain transaction, so that the entire data state of the page pool at that moment can be saved. After that, the redo logs of each transaction generated are sequentially stored in the log area of ​​hard disk 215. If a system crash occurs later, the redo log after the recovery point is used for recovery. It should be noted that recovery here refers to the recovery of data pages that were not stored in memory 213 before the system crash.

[0092] Taking MySQL as an example, the page header and page footer structure of MySQL data page are as follows Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of a data page in MySQL. The first 38 bytes of the data page are the page header, and the last 8 bytes of the data page are the page footer. In the first 38 bytes of the page header, the 16th to 24th bytes (8 bytes) record the log sequence number (LSN) of the redo log (redolog) that modifies this data page.

[0093] The data recovery process is described below. Taking the case where data recovery is triggered after a server 201 (e.g., a database) fails and restarts, as an example, the starting point of data recovery is found based on the checkpoint record, and then the redo log is read in sequence. Check the relationship between the LSN of the redo log and the LSN of the data page. If the LSN of the data page is greater than the LSN of the redo log, the modification operation corresponding to the redo log was not written to the disk at the time of the failure, that is, the data page generated by the modification operation corresponding to the redo log was not written to the disk, and the version of the data page on the disk is inconsistent with the version of the redo log. It is necessary to perform data recovery processing (also called data redo processing, or redo processing) based on the data page on the disk and the redo log to generate data pages that are not written to the disk. In other words, data recovery is performed based on the data page on the disk and the redo log to generate data pages corresponding to the modification operation. Check multiple redo logs in sequence until the LSN of the data page is equal to the LSN of the redo log, and it is considered that the current data page is already the latest version, and no data recovery is required. As application scenarios become increasingly complex, data pages and log files may undergo multiple processing and conversions. If a data page undergoes continuous modification and a redo log is lost during the modification process, then during the data recovery process, the redo log is used to redo the data on the disk, and the redoed data page may be wrong.

[0094] For easier understanding, see Figure 6 , Figure 6 The figure is a schematic diagram of a data recovery process. The data page on the disk is data page V0. The redo logs that should be stored in the log area theoretically include: redo log1, redo log2, and redo log3. When redo log2 is lost in the redo log of the log area, the process of using data page V0 and redo log for data recovery is as follows: First, redo data based on redo log1 and data page V0 to obtain data page V1; then, redo data based on redo log3 (redo log2 is lost at this time) and data page V1 to obtain data page VX. Since redo log2 is lost, the data page VX obtained by redoing the data is not the data page V3 that actually needs to be recovered, and the data page obtained by redoing the data is wrong.

[0095] Based on the above problems, an embodiment of the present application proposes a data recovery method. After the first data page is modified, a first log is generated, and the modification to the first data page is recorded in the first log. The first log includes a first identifier, and the first identifier indicates the version of the first data page. When the second data page needs to recover data, the first log can be used to recover data on the second data page only when the version of the first log is greater than the version of the second data page and the version indicated by the first identifier is the same as the version of the second data page. Through the first identifier, the use of a log with an incorrect version to recover data on the second data page is avoided, thereby ensuring the accuracy of data recovery.

[0096] The following is an introduction to the embodiments of the present application in conjunction with the accompanying drawings. Figure 7 , Figure 7 The following is a flow chart of a data recovery method according to an embodiment of the present application. The following is a data recovery method according to an embodiment of the present application, comprising:

[0097] S1. Obtain a first identifier of a first log and determine a version of a first data page before modification, where the first identifier indicates the version of the first data page.

[0098] In step S1, the database system (hereinafter referred to as the database) enters a data recovery state or a fault recovery phase. For example, after the database is restarted due to a fault, it triggers the entry into a data recovery state or a fault recovery state. According to the checkpoint record, the starting point of data recovery is found from the log file. The starting point of data recovery is also called the recovery starting point. The log file includes one or more logs. The checkpoint indicates the storage location of the last refreshed dirty page. The dirty page is a data page that has been modified and not refreshed to the disk. The dirty page can also be understood as a data page that has modified data and is not stored in the disk. Then, at least one log is obtained from the recovery starting point in the log file. The specific acquisition method is to obtain the logs in sequence according to the version order of the logs, for example: the logs are obtained in sequence from the old version to the new version of the log version. Taking the first log as an example, the first log is a log in the log file, and the first log is the log after the starting point of data recovery in the log file.

[0099] The first log includes a first identifier, and the first identifier indicates a version of the first data page.

[0100] Optionally, the first identifier is the log sequence number of the first data page. Exemplarily, the version of the first data page is: LSN#1. After the first data page is modified, for example, the first data page undergoes data modification in response to modification operation 1, and a first log is generated. The version of the first log is consistent with the version of the modified first data page, which is LSN#2, and LSN#2 is greater than LSN#1. The first log also includes a first identifier, which indicates the version of the first data page, for example, the first identifier is LSN#1. In other words, in addition to the version of the first log (LSN#2), the first log also includes a first identifier (LSN#1).

[0101] In a possible implementation, the first log is a redo log, and the first identifier is carried in the first field of the redo log. The first field may also be referred to as a base page LSN field of the baseline version. Figure 8 As shown, Figure 8 A schematic diagram of a redo log structure proposed in an embodiment of the present application. The redo log includes: a single record flag field, a type field, a space identifier (space ID), a data page number (page number), a baseline version of the data page log sequence number field (base page LSN) and redo log data (redo log data). Among them, the space ID and the page number indicate the data page corresponding to the log, and the log records the modifications to the data page. When modifying a data page, the version of the data page before modifying the data page is recorded in the log corresponding to the data page. Specifically, the version of the data page is recorded in the first identifier of the log. For example, if the version of the data page is indicated by the LSN of the data page, the LSN of the data page is recorded in the base page LSN of the log. The LSN of the data page before modification can be read from the page header of the data page, such as the aforementioned Figure 5 This is just a hint and will not be elaborated here.

[0102] For ease of explanation, the data page of the data to be recovered is referred to as the second data page. The page identifier of the second data page is the same as the page identifier of the first data page. In other words, the first data page and the second data page are different versions of the same data page. The first log records the modification of the first data page, so the first log can be used to recover the data of the second data page. The specific method for obtaining the first log is as follows: First, obtain at least one log from the recovery starting point in the log file, and the page identifier of the modified data page recorded in the log is the same as the page identifier of the second data page. Then, detect the log sequence number of the log in the log file and the log sequence number of the second data page. When the log sequence numbers of all logs in the log file are greater than or equal to the log sequence number of the second data page, it is determined that the second data page does not need to be recovered; when the first log exists in the log file, and the log sequence number of the second data page is less than the log sequence number of the first log, then enter step S3.

[0103] Furthermore, a method for generating a first log is described below. Specifically, in response to a modification of a first data page (or in response to a modification of the first data page), a version of the first data page is recorded, for example, an LSN of the first data page is recorded. Then, in response to the modification of the first data page, a first log is generated, the first log includes the first identifier, and the first identifier indicates the version of the first data page.

[0104] S2. Determine a version of a second data page, where the second data page is a data page of data to be recovered, a page identifier of the second data page is the same as a page identifier of the first data page, and a version of the second data page is smaller than a version of the first log.

[0105] In step S2, the version of the second data page is obtained. In a possible implementation, the version of the second data page is indicated by the log sequence number of the second data page. Specifically, the log sequence number of the second data page is first obtained; then the version of the second data page is determined by obtaining the log sequence number of the second data page. In step S1, in the process of obtaining the first log, the version of the second data page and the version of the first log are detected. Only when the version of the first log is greater than the version of the second data page, and the version indicated by the first identifier is the same as the version of the second data page, can the first log be used to perform data recovery on the second data page. Therefore, the version of the second data page needs to satisfy that the version of the second data page is less than the version of the first log.

[0106] It should be noted that there is no restriction on the execution order of step S1 and step S2. Step S1 may be executed first and then step S2; or step S2 may be executed first and then step S1.

[0107] S3. Detect whether the version indicated by the first identifier is the same as the version of the second data page.

[0108] In step S3, the version indicated by the first identifier is determined according to the first identifier, and after obtaining the version of the second data page, it is detected whether the version indicated by the first identifier is the same as the version of the second data page. If they are the same, proceed to step S4; if not, proceed to step S5.

[0109] S4. If the version indicated by the first identifier is the same as the version of the second data page, use the first log to perform data recovery on the second data page.

[0110] In step S4, since the page identifier of the first data page is the same as the page identifier of the second data page, the first data page and the second data page can be considered to be different versions of the same data page. And the version of the second data page is smaller than the version of the first log, which means that the modification of the first data page recorded in the first log is later than the modification of the second data page in terms of timing. If the version indicated by the first identifier is the same as the version of the second data page, the first data page can be considered to be the second data page, and the first log that records the modification of the first data page can be used to recover data of the second data page.

[0111] The specific data recovery method is as follows: redo the second data page using the first log to obtain a third data page. Write the third data page to the disk to replace the second data page stored in the disk. The version of the third data page is different from the version of the second data page.

[0112] For example, a possible application scenario is as follows: Fig. 9 As shown, Fig. 9 This is a schematic diagram of an application scenario in an embodiment of the present application. The page identifier of the second data page is: 0001, and the version of the second data page is: LSN#1. The page identifier of the first data page is: 0001, and the version of the first data page is: LSN#1. In response to modification operation 1, a modified first data page is generated, and the version of the modified first data page is: LSN#2. In response to modification operation 1, a first log is generated, and the first log records the modification operation on the first data page, and the version of the first log is: LSN#2. The first log also records the version indicated by the first identifier (i.e., the version of the first data page): LSN#1.

[0113] like Fig. 9As shown, in the data recovery process, first find the first log (the page identifier of the first data page is 0001) with the same page identifier according to the page identifier (0001) of the second data page. Check whether the version of the second data page is smaller than the version of the first log. Since LSN#1 (the version of the second data page) is smaller than LSN#2 (the version of the first log), the version of the second data page is smaller than the version of the first log. Further check whether the version indicated by the first identifier recorded in the first log is the same as the version of the second data page. Since LSN#1 (the version of the second data page) is the same as LSN#1 (the version indicated by the first identifier), it is considered that the first data page and the second data page are the same data page and have the same version. The first log can be used to perform data recovery on the first data page to generate a third data page. The third data page is the same as the modified first data page.

[0114] Optionally, after the third data page is generated, it is possible to continue to check whether there are other logs in the log file after the recovery starting point. If there are, the method shown in step S1 to step S5 is used to check whether other logs can be used in the log file to perform data recovery processing on the third data page. If there are other logs that can perform data recovery processing on the third data page, the data recovery processing on the third data page is performed similarly to the method shown in step S1 to step S4; if there are no other logs, the third data page is written to the disk.

[0115] S5. If the version of the first data page is different from the version of the second data page, an error message is generated.

[0116] In step S5, if the version of the first data page is different from the version of the second data page, it is considered that the first log cannot be used to recover data on the second data page, and an error message is generated, indicating that data on the second data page cannot be recovered.

[0117] Optionally, the error message may further indicate the specific type of error so that the user can further troubleshoot and recover data. The error message may specifically indicate any one or more of the following information: a log loss occurs before the first log in the log file, or a parallel recovery logic error occurs, or a data page error occurs.

[0118] For example, a possible application scenario is as follows: Fig.10 As shown, Fig.10This is a schematic diagram of an application scenario in an embodiment of the present application. The page identifier of the second data page is: 0001, and the version of the second data page is: LSN#1. The page identifier of the first data page is: 0001, and the version of the first data page is: LSN#1. In response to modification operation 1, a modified first data page is generated, and the version of the modified first data page is: LSN#2. In response to modification operation 1, a first log is generated, and the first log records the modification operation on the first data page, and the version of the first log is: LSN#2. The first log also records the version of the first data page: LSN#1.

[0119] like Fig.10 As shown, during the data recovery process, first find the first log (the page identifier of the first data page is 0001) with the same page identifier according to the page identifier (0001) of the second data page. Check whether the version of the second data page is smaller than the version of the first log. Since LSN#3 (the version of the second data page) is smaller than LSN#2 (the version of the first log), the version of the second data page is smaller than the version of the first log. Further check whether the version indicated by the first identifier recorded in the first log (the version of the first data page) is the same as the version of the second data page. Since LSN#3 (the version of the second data page) is different from LSN#1 (the version indicated by the first identifier), log loss may occur before the first log, and the first log cannot be used to recover data for the first data page.

[0120] The embodiment of the present application proposes a data recovery method, after the first data page is modified, a first log is generated, and the modification to the first data page is recorded in the first log. The first log includes a first identifier, and the first identifier indicates the version of the first data page. When the second data page needs to recover data, the first log can be used to recover data on the second data page only when the version of the first log is greater than the version of the second data page and the version indicated by the first identifier is the same as the version of the second data page. Through the first identifier, the use of a log with an incorrect version to recover data on the second data page is avoided, and the accuracy of data recovery is guaranteed. It is easy to find and locate the location where the log error occurs, and prevent the database from being restored to an erroneous state.

[0121] In combination with the above embodiments, an application scenario proposed by the embodiments of the present application is introduced below. Fig.11 , Fig.11 A schematic diagram of an application scenario proposed in an embodiment of the present application. An application scenario proposed in an embodiment of the present application includes:

[0122] F1, fault recovery, start the database. Specifically, after a database fault occurs, the database is started and the database fault recovery phase is entered.

[0123] F2. Determine the recovery starting point for data recovery based on the checkpoint. Then, find the corresponding redo log in the log file based on the recovery starting point.

[0124] F3. Read the next redo log.

[0125] F4. Find the corresponding page ID from the redo log.

[0126] F5. Read the corresponding data page according to the page identifier.

[0127] F6. Check whether the LSN of the data page is less than the LSN of the redo log. If so, proceed to step F7; if so, proceed to step F3.

[0128] F7. Check whether the LSN of the data page is equal to the LSN of the first identification record in the redo log. The LSN of the first identification record in the redo log is the LSN of the data page corresponding to the redo log before the modification occurs. If they are equal, go to step F8; if they are not equal, go to step F9.

[0129] F8. Use the redo log to recover the data page. After step F8, proceed to step F10.

[0130] F9, restore failed. Optionally, after confirming the restore failure, an error message may be generated.

[0131] F10, check whether there are more redo logs. If yes, go to step F3; if no, go to step F11.

[0132] F11. Write the data pages obtained after data recovery in step F8 to the disk. Then confirm that the data recovery is successful and start the database.

[0133] exist Figures 1 to 11 On the basis of the corresponding embodiments, in order to better implement the above-mentioned solutions of the embodiments of the present application, related devices for implementing the above-mentioned solutions are also provided below.

[0134] See also Fig.12 , Fig.12 A schematic diagram of the structure of a server provided in an embodiment of the present application. The server is applied to a database. The server includes: a receiving module 1201, a processing module 1202 and a sending module 1203, wherein the receiving module 1201 and the sending module 1203 are used to transmit data, and the processing module 1202 is used to process the data. The server includes:

[0135] The processing module 1202 is used to generate the first log in response to modifying the first data page, the first log including a first identifier, the first identifier indicating a version of the first data page, and the first log recording the modification to the first data page; the processing module 1202 is also used to determine the version of the second data page in the event of a failure of the database system, the second data page being a data page of data to be recovered, the page identifier of the second data page being the same as the page identifier of the first data page; the processing module 1202 is also used to use the first log to perform data recovery on the second data page when the version of the first log is greater than the version of the second data page and the version indicated by the first identifier is the same as the version of the second data page.

[0136] In a possible implementation, the first identifier is a log sequence number of the first data page.

[0137] In a possible implementation, the first log is a redo log.

[0138] In a possible implementation, the processing module 1202 is further configured to generate an error message when the version indicated by the first identifier is different from the version of the second data page, wherein the error message indicates that data recovery of the second data page is impossible.

[0139] In one possible implementation,

[0140] The error message is also used to indicate any one or more of the following information:

[0141] A log loss occurs before the first log, a parallel recovery logic error occurs, or a data page error occurs.

[0142] In one possible implementation,

[0143] The processing module 1202 is further configured to perform redo processing on the second data page using the first log to obtain a third data page;

[0144] The processing module 1202 is further configured to write the third data page to the disk to replace the second data page in the disk.

[0145] In one possible implementation,

[0146] The receiving module 1201 is used to obtain at least one log from a recovery starting point in a log file, where the log file includes one or more logs;

[0147] The processing module 1202 is further used to detect the log sequence number of the log in the log file and the log sequence number of the second data page;

[0148] The processing module 1202 is further configured to determine that the second data page does not need to be restored when the log sequence numbers of all logs in the log file are greater than or equal to the log sequence number of the second data page;

[0149] The processing module 1202 is also used to detect whether the version of the first data page before modification is the same as the version of the second data page when the first log exists in the log file and the log sequence number of the second data page is smaller than the log sequence number of the first log.

[0150] In one possible implementation,

[0151] The processing module 1202 is further used to determine the recovery starting point in the log file according to a checkpoint checkpoint, wherein the checkpoint indicates the storage location of the last refreshed dirty page, and the dirty page is a data page that has been modified and not refreshed to the disk.

[0152] In one possible implementation,

[0153] The receiving module 1201 is further used to obtain the log sequence number of the second data page;

[0154] The processing module 1202 is further configured to determine a version of the second data page according to the log sequence number of the second data page.

[0155] The embodiment of the present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned server.

[0156] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program for performing signal processing. When the program is run on a computer, the computer executes the steps executed by the aforementioned execution device, or the computer executes the steps executed by the aforementioned server.

[0157] The execution device or terminal device provided in the embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit so that the chip in the execution device executes the data recovery method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0158] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0159] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0160] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0161] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0162] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

Claims

1. A data recovery method, characterized in that: The method is applied to a server running a database system, and the method comprises: In response to modifying the first data page, generating the first log, the first log including a first identifier, the first identifier indicating a version of the first data page, the first log recording the modification to the first data page; In case of a failure of the database system, determining a version of a second data page, where the second data page is a data page of data to be recovered, and a page identifier of the second data page is the same as a page identifier of the first data page; When the version of the first log is greater than the version of the second data page, and the version indicated by the first identifier is the same as the version of the second data page, the first log is used to perform data recovery on the second data page.

2. The method according to claim 1, characterized in that The first identifier is a log sequence number of the first data page.

3. The method according to claim 1 or 2, characterized in that: The first log is a redo log.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When the version indicated by the first identifier is different from the version of the second data page, an error message is generated, where the error message indicates that data recovery of the second data page is impossible.

5. The method according to claim 4, characterized in that The error message is also used to indicate any one or more of the following information: A log loss, a parallel recovery logic error, or a data page error occurs before the first log.

6. The method according to any one of claims 1 to 5, characterized in that Determining a version of the second data page includes: Obtaining the log sequence number of the second data page; A version of the second data page is determined according to the log sequence number of the second data page.

7. A server, characterized in that: The server is applied to a database system, and the server includes: a processing module, configured to generate the first log in response to modifying the first data page, wherein the first log includes a first identifier, the first identifier indicates a version of the first data page, and the first log records the modification to the first data page; The processing module is further used to determine the version of a second data page when the database system fails, the second data page being a data page of data to be recovered, and the page identifier of the second data page being the same as the page identifier of the first data page; The processing module is further configured to use the first log to perform data recovery on the second data page when the version of the first log is greater than the version of the second data page and the version indicated by the first identifier is the same as the version of the second data page.

8. The server according to claim 7, characterized in that: The first identifier is a log sequence number of the first data page.

9. The server according to claim 7 or 8, characterized in that: The first log is a redo log.

10. The server according to any one of claims 7 to 9, characterized in that: The processing module is further configured to generate an error message when the version indicated by the first identifier is different from the version of the second data page, wherein the error message indicates that data recovery of the second data page is impossible.

11. The server according to claim 10, characterized in that: The error message is also used to indicate any one or more of the following information: A log loss occurs before the first log, a parallel recovery logic error occurs, or a data page error occurs.

12. The server according to any one of claims 7 to 11, characterized in that: The receiving module is further used to obtain the log sequence number of the second data page; The processing module is further used to determine the version of the second data page according to the log sequence number of the second data page.

13. A server, characterized in that: The server comprises a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the server executes the method according to any one of claims 1 to 6.

14. A database system, characterized in that: include: The server as claimed in claim 13.

15. A computer storage medium, characterized in that The computer storage medium stores instructions, which, when executed by a computer, enable the computer to implement the method of any one of claims 1 to 6.

Images