Data recovery method and device, program product and storage medium

Abstract

The invention provides a data recovery method and device based on a user mode file system FUSE, a program product and a storage medium, and the method comprises the steps: responding to a received recovery request for recovering an original data table in an original database instance to a historical moment, recovering a temporary data table of the original data table at a historical moment through a temporary database instance corresponding to the original database instance, and creating an empty recovery data table corresponding to the temporary data table in the original database instance for the original database instance to access; loading the temporary data table to a local temporary storage space of the original database instance through a FUSE process, and processing an access request of the original database instance to the recovery data table based on data of the temporary data table in the loading process; and after loading is completed, replacing the recovery data table with the data table stored in the local temporary storage space.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of storage, and in particular, to a data recovery method based on a user-mode file system (FUSE), a device, a program product and a storage medium. BACKGROUND

[0002] A database can manage one or more data tables, and database table recovery refers to recovering part of the data tables managed by a database instance to a historical time point through technical means. In the related art, a temporary database instance corresponding to the original database instance can be recovered, the temporary database instance manages the recovered temporary data table, the temporary database instance is different from the original database instance, and the storage space of the temporary data table is remote storage for the original database instance. Then, the temporary data table needs to be physically copied to the storage device where the original database instance is located, and after all the data is copied, the original database instance needs to be restarted to access the recovered data table. Therefore, the existing data table recovery scheme needs to wait for a long time. SUMMARY

[0003] To overcome the problems in the related art, the present disclosure provides a data recovery method based on a user-mode file system (FUSE), a device, a program product and a storage medium.

[0004] According to a first aspect of an embodiment of the present specification, a data recovery method based on a user-mode file system (FUSE) is provided, and the method comprises:

[0005] In response to receiving a recovery request for recovering an original data table in an original database instance to a historical time point, a temporary data table of the original data table at the historical time point is recovered through a temporary database instance corresponding to the original database instance, and an empty recovery data table corresponding to the temporary data table is created in the original database instance for the original database instance to access;

[0006] Through a FUSE process, the temporary data table is loaded into a local temporary storage space of the original database instance, and during the loading process, an access request of the original database instance to the recovery data table is processed based on the data of the temporary data table;

[0007] After the loading is completed, the data table stored in the local temporary storage space is replaced with the recovery data table.

[0008] According to a second aspect of an embodiment of the present specification, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method embodiment of the first aspect when executing the computer program.

[0009] According to a third aspect of the embodiments of the present specification, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the steps of the method embodiment of the first aspect.

[0010] According to a fourth aspect of the embodiments of the present specification, a computer program product is provided, and the computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of the method embodiment of the first aspect.

[0011] The technical solutions provided by the embodiments of the present specification can include the following beneficial effects:

[0012] In the embodiments of the present specification, after receiving the recovery request, the data recovery scheme based on the user-mode file system FUSE can first use the database instance recovery technology to recover the temporary database instance corresponding to the original database instance, and the temporary data table of the original data table at the historical time is recovered in the temporary database instance. At the same time, an empty recovery data table is created in the original database instance. Since the recovery data table is empty, there is no real data replication, so that the data access can be quickly recovered in the original database instance. Through the FUSE process, the temporary data table can be loaded into the local temporary storage space of the original database instance using the idea of lazy loading. If the original database instance needs to access the recovery data table during the loading process, the access request can be processed based on the temporary data table managed by the temporary database instance. After the loading is completed, the temporary data table loaded in the local temporary storage space is used to replace the recovery data table, and the data recovery is completed. In addition, the traditional idea considers that using the FUSE process will cause the performance of the database to decrease, but the data recovery based on the FUSE process can quickly recover the data access of the original instance after the recovery request is initiated, and compared with this, the decrease of the performance of the database is acceptable.

[0013] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings, which are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present specification and, together with the specification, serve to explain the principles of the present disclosure.

[0015] Figure 1 is a flowchart of a data recovery method based on a user-mode file system FUSE according to an exemplary embodiment of the present specification.

[0016] Figure 2A is a schematic diagram of a local temporary storage according to an exemplary embodiment of the present specification.

[0017] Figure 2B is an application scenario diagram of a user-mode file system FUSE-based data recovery method according to an example embodiment of the present specification.

[0018] Figure 2C is an application scenario diagram of another user-mode file system FUSE-based data recovery method according to an example embodiment of the present specification.

[0019] Figure 3 is a hardware structure diagram of a computer device where a user-mode file system FUSE-based data recovery apparatus according to an example embodiment of the present specification is located. DETAILED DESCRIPTION

[0020] The example embodiments will be described in detail below with reference to the accompanying drawings. When the description below refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following example embodiments do not represent all implementations consistent with the present specification. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present specification as detailed in the appended claims.

[0021] The terminology used in the present specification is for the purpose of describing particular embodiments only and is not intended to limit the present specification. As used in the present specification and the appended claims, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order. These terms are used only to distinguish one piece of information from another. For example, without departing from the scope of the present specification, first information can also be referred to as second information, and similarly, second information can also be referred to as first information. Depending on the context, the word “if’ as used herein can be interpreted as meaning “when” or “in response to determining” or “in response to ascertaining”.

[0023] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0024] First, the concepts involved in the embodiments of the present specification are described.

[0025] 1. Database backup: refers to creating one or more copies of database data through technical means to enhance data security and availability.

[0026] 2. Database recovery: refers to creating a usable database instance from backup data for user use through technical means.

[0027] In a database, a database instance generally refers to a running instance after the database management system (DBMS, DataBase Management System) is started, which represents a database service process loaded and running in the memory of the DBMS. A database instance includes database data, transaction logs, caches, and related processes and threads for processing user requests and executing database operations.

[0028] 3. Database library table recovery: refers to restoring a fine-grained library or table on a database instance to a historical time point through technical means.

[0029] A database instance can manage one or more databases (DataBase), and each library can contain one or more data tables (Table). Library table recovery is not the recovery of the entire database instance, but a recovery operation on a library or table. Specific libraries or tables can be selectively restored as needed without restoring the entire database to a certain historical time point.

[0030] 4. Database data page (Page): refers to the fact that databases store / manage data tables using Pages. Each Page contains metadata and data content. Database reading and writing are performed in Page dimensions. Typically, a database Page is 16KB (Kilobyte) or 8KB in size.

[0031] For example, when a database needs to read or write data from a storage medium, it performs I / O (Input / Output) operations in Page units, loading the entire page into memory or writing it back to the storage medium. A data table can be divided into one or more Pages for management.

[0032] For example, in a relational database, table data is actually stored in pages (Page), and the data of each table is stored in different pages. The database system tracks which pages the records are stored in through indexing and other mechanisms to quickly access and manipulate data.

[0033] 5. Point-in-Time Recovery (PITR): refers to restoring the database to the state of any historical point in time through various technical means.

[0034] 6. Recovery Time Objective (RTO) refers to the time period after a disaster, from the moment a system outage causes service interruption to the moment the system recovers to support all departments and services resume. RTO is an indicator of the timeliness of service recovery and reflects the maximum recovery time an enterprise can tolerate for its IT systems. A smaller RTO value indicates a stronger disaster recovery system, but also requires a higher enterprise investment.

[0035] As an example, suppose a failure occurs at 1 PM and the RTO is set to 1 hour, then recovery needs to be completed before 2 PM.

[0036] 7. Recovery Point Objective (RPO) refers to the point in time at which the disaster recovery system recovers data after a disaster. RPO is a metric that reflects the amount of data loss and represents the maximum amount of data loss an enterprise can tolerate. A lower RPO value indicates less data loss and lower losses for the enterprise.

[0037] As an example, if the RPO is set to 5 hours, data backup must be performed every 5 hours.

[0038] 8. Filesystem in Userspace (FUSE), which can obtain all data and metadata read and write operations of this file system from user-mode programs.

[0039] Specifically, FUSE is a file system framework implemented in user space. With the support of the FUSE kernel module, users only need to implement specific file operations according to the interface provided by FUSE to implement a file system. Before the emergence of FUSE, the file system in Linux (an operating system) was completely implemented in kernel state. Writing a file system with a specific function was not very convenient, both in terms of code writing and debugging. Even if it was just to add a small function to an existing traditional file system, it might still require a lot of work because it was implemented in the kernel. After the emergence of the user-state file system FUSE, the workload can be greatly reduced, and debugging will also be very convenient. When writing a FUSE file system, you only need to load the FUSE kernel module in the kernel, and there is no need to recompile the kernel.

[0040] 9、Database data logical replication refers to reading out data of a database through an SQL (Structured Query Language) interface and copying the logical structured data to other databases. For example, based on the SQL interface, data such as tables, rows in the tables, or columns in the tables in a source database is read out, and then based on the SQL interface, the data is written to a target database.

[0041] 10、Database data physical replication refers to backing up physical files of a database through a backup tool and copying the files to other databases. This replication mode directly operates physical files at a bottom layer of a database, rather than processing data through an SQL interface.

[0042] In the field of database backup and recovery, recovery is always an important link. In terms of database recovery means, there are approximately several kinds:

[0043] The first kind is database instance-level recovery, which recovers data to a new instance. This recovery scheme starts a new instance corresponding to the original instance, and recovers all data tables managed by the original database instance in the new instance. The new instance and the original instance are two different instances.

[0044] The second kind is database database-table-level recovery, which recovers data tables to a new instance or to the original instance.

[0045] The third kind is database row-level recovery, which recovers row data in a table to a new table or to the original table. Customers will select different recovery schemes according to different scenarios.

[0046] In the field of data recovery, there is a special index RTO (Recovery Time Objective) for measuring recovery speed, which refers to the time from starting recovery to completing recovery. In many scenarios, users are very sensitive to RTO, such as online emergency disaster recovery, instant analysis of historical data, and the like. Improving recovery time can not only solve the pain points of these scenarios, but also greatly improve the user experience of backup and recovery in other recovery scenarios, such as cloning instances, database-table recovery, and the like.

[0047] In the existing recovery scheme, the first kind of full-instance recovery scheme, customers can select distributed cloud storage, rely on snapshots and lazy loading of the underlying cloud storage, and can be restored to a new instance in seconds.

[0048] The second kind of database-table-level recovery scheme can also use distributed storage to restore to a new instance in seconds.

[0049] The third row-level recovery solution relies on the page multi-version of the database itself or the longer retention time transaction log (Undo Log) to achieve row-level data second-level recovery.

[0050] However, the second library table-level recovery solution cannot achieve fast recovery for the requirement of restoring the data table to the original instance. In the existing solution for restoring the data table to the original instance, physical replication and logical replication are combined to copy the restored data back to the original instance, and the recovery speed is limited by the machine hardware level, deployment density, network bottleneck, instance performance and other factors of the original instance, and the copying speed is usually within 100 MB / S (Megabytes per Second). If the amount of data to be restored is large, the RTO may be in the order of hours, which is unacceptable to users.

[0051] The related technologies are similar to the three technologies of logical replication recovery, physical replication recovery, and volume data delayed loading second-level recovery.

[0052] 1. Logical replication recovery is to extract data from the database through the most common SQL and write the data into the target database in the form of SQL. The advantage of this solution is that it is most common and can adapt to multiple database versions. However, the disadvantage of this method is that the speed of restoring and copying data is slow. In some common cases, the speed of logical replication is usually tens of MB / s, and it cannot achieve second-level data recovery and copying.

[0053] 2. Physical replication is to extract the database page from the database file through the backup tool, and then import the database page into the database through data replication and some database data import mechanisms. The advantage of this method is that the data extraction and import speed can be above 100 MB / s. The disadvantage is that there are differences between some database versions, which cannot adapt to all versions, and it cannot achieve second-level data recovery and copying.

[0054] 3. Volume data delayed loading second-level recovery technology. This technology does not look at the database application layer of the database file. From the perspective of volume storage, the database file cannot be seen, and only the data under the file system can be seen. This technology can achieve second-level recovery of a new volume from a volume snapshot, but this recovery method is for full database / full data volume-level fast recovery, and cannot achieve data table-level fine-grained recovery.

[0055] In summary, the existing technologies cannot achieve fast recovery of the data table to the original database instance. Based on this, the embodiments of the present specification provide a data recovery method, as shown in Figure 1As shown, the flowchart illustrates a data recovery method based on a user-mode file system (FUSE) according to an example embodiment of the present specification, which can include the following steps:

[0056] In step 102, in response to receiving a recovery request for recovering a data table in an original database instance to a historical time, a temporary data table of the data table at the historical time is recovered through a temporary database instance corresponding to the original database instance, and an empty recovery data table corresponding to the temporary data table is created in the original database instance for access by the original database instance.

[0057] In step 104, the temporary data table is loaded into a local temporary storage space of the original database instance through a FUSE process, and during the loading process, the access request of the original database instance to the recovery data table is processed based on the data of the temporary data table.

[0058] In step 106, after the loading is completed, the data table stored in the local temporary storage space is replaced with the recovery data table.

[0059] As can be seen from the above embodiment, the data recovery scheme of the present embodiment can first recover a temporary database instance corresponding to an original database instance using a database instance recovery technique after receiving a recovery request, and a temporary data table of a data table at a historical time is recovered in the temporary database instance. At the same time, an empty recovery data table is created in the original database instance. Since the recovery data table is empty, there is no real data replication, so that the data access in the original database instance can be quickly recovered. The temporary data table is loaded into the local temporary storage space of the original database instance through a delayed loading method. If the original database instance needs to access the recovery data table during the loading process, the access request can be processed based on the temporary data table managed by the temporary database instance. After the loading is completed, the recovery data table is replaced with the temporary data table loaded in the local temporary storage space, and the data recovery is completed. In addition, the traditional idea considers that using a FUSE process will cause a decrease in database performance, so it is necessary to avoid using a FUSE process to respond to a database request to prevent the response speed from being affected. However, the present embodiment uses a FUSE process to implement data recovery, which can quickly recover data access by the original instance after the recovery request is initiated. Compared with this, the decrease in database performance is acceptable, and the use of a FUSE process also reduces the workload of kernel modification.

[0060] As an example, the data recovery method of the embodiment can be applied to any computer device, including but not limited to a single server, a server group composed of multiple servers, or a cloud composed of a large number of hosts or servers based on cloud computing, etc. For example, the data recovery method can run on the host (which can be a physical machine or a virtual machine) where the original database instance is located, and is used to recover the data table of the original database instance running on the host.

[0061] The embodiment scheme can be applied to the recovery scenario of a data table. For example, the original database instance of a user can be used to manage multiple data tables, some (one or more) of which need to be recovered. Taking the example of a user needing to recover data table A, in these scenarios, because the user's service needs to use data table A, the user's requirement is not to delete data table A and cause the service to be suspended, but to immediately recover a data table A' identical to data table A, so that the original database instance can use the recovered data table A', thereby ensuring the normal operation of the service. In this way, when the user needs it, the user can trigger a recovery request to recover data table A to a historical time.

[0062] As an example, an existing database instance recovery technology can be used to recover a temporary database instance corresponding to the original database instance (hereinafter referred to as the original instance for convenience). At this time, the temporary instance and the original instance are two different instances, and the local storage of the temporary instance stores the temporary data table of the recovered original data table at a historical time. Currently, the storage space storing the temporary data table is remote storage for the original instance.

[0063] The idea of the embodiment is not to affect the user's use of the original instance, and there is no need to wait for all the data of the temporary data table to be transmitted to the original instance. The original instance can start accessing the recovered data table, so that the original instance can quickly use the recovered data. To achieve this purpose, the embodiment creates an empty recovery data table in the original instance after the recovery request is initiated, and uses the recovery data table to accept access requests of the original instance. The processing of the access request is based on the data of the temporary data table.

[0064] As an example, the database instance divides the data table into at least one data page Page for management. An empty recovery data table is created in the original database instance to replace the original data table. The recovery data table can have the same number of Pages as the temporary data table, and each Page of the recovery data table can correspond to each Page of the temporary data table. For example, the temporary data table can be divided into M Pages, and the recovery data table can also be divided into M Pages. The M Pages of the temporary data table and the M Pages of the recovery data table have a corresponding relationship, but the M Pages of the recovery data table do not store data content.

[0065] There are various ways to create an empty recovery data table; for example, in a Linux operating system, the empty recovery data table can be created by a system call fallocate(). The fallocate() is a system call for manipulating preallocated space for a file. It can preallocate a certain size of space for a file without actually writing data content. Therefore, in this embodiment, the fallocate() can be called to make the file system allocate a disk space with the same size as the temporary data table, and also make the original instance create a recovery data table with the same size as the temporary data table, and the data pages of the recovery data table have metadata but no data content.

[0066] As an example, a local temporary storage space (Local Temp Store) can be created in the original instance, and optionally, the local temporary storage space can be a persistent storage space, and the temporary storage space can be created in a persistent storage device corresponding to the original instance, and the size of the temporary storage space needs to be greater than the size of the temporary data table. As an example, the local temporary storage space can be recycled after the data recovery is completed.

[0067] In step 104, as an example, various ways can be used to load the data of the temporary data table into the local temporary storage space. For example, the temporary data table can be divided into multiple data pages, and one or more data pages can be loaded each time, and some data pages of the temporary data table can be randomly loaded each time, or some data pages can be preferentially loaded in a set manner, and the present embodiment does not limit this. Before the loading is completed, the original instance can access the recovery data table, and the recovery data table is empty at this time, but when the present embodiment receives an access request, it can process it based on the data of the temporary data table.

[0068] In some examples, the database instance manages the division of the data table into at least one data page, and the data page can contain metadata;

[0069] The loading of the temporary data table into the local temporary storage space of the original database instance can include:

[0070] Obtaining at least one temporary data page of the temporary data table that is currently to be loaded;

[0071] Obtaining, by the original database instance, metadata of a recovery data page of the recovery data table corresponding to the temporary data page;

[0072] Updating the metadata of the temporary data page based on the metadata of the recovery data page, and loading the updated temporary data page into the local temporary storage space.

[0073] In this embodiment, considering that in an actual database system, the database manages data tables according to pages, one data table can correspond to multiple pages; each page stores meta information Meta and data content, wherein the meta information can include a table space ID, an index ID, a transaction ID, and the like, and also includes a checksum of the data content stored in the page. The meta information stored in each page corresponding to the empty recovery data table in the original instance is related to the current running state of the original instance. On the other hand, the temporary data table managed by the restored temporary instance also has its own meta information in the page, which is different from the meta information in the page of the empty data table of the original instance. Because the temporary data table is also backup data backed up from the original database, it belongs to a historical state and may have problems such as space ID index ID reuse, the transaction ID is a historical small transaction ID, or the backup data is from a standby library, and the like, which may cause the Meta of the data page in the temporary instance to be inconsistent with the Meta of the data page in the empty table of the original instance.

[0074] Based on this, in order to enable the loaded data page in the local temporary storage space to be used by the original instance, when loading the to-be-loaded data page of the temporary data table into the local temporary storage space, the Meta corresponding to the to-be-loaded data page in the recovery data table can be used to update the Meta of the to-be-loaded data page, so that the Meta contained in the updated to-be-loaded data page can be directly used by the original instance, so that the original instance can use the loaded data page without restarting, and can also directly use the loaded data page in the local temporary storage space to respond to an access request.

[0075] In actual application, after the original instance creates an empty recovery data table, the Meta of all data pages of the recovery data table can be obtained in advance, and each data page of the recovery data table is mapped one by one with a data page of the temporary data table, which can be used in the loading process to implement the updating operation of the Meta of the to-be-loaded data page.

[0076] As an example, the data page also contains data content, the meta information includes a checksum and at least one other information in addition to the checksum; and the updating of the meta information of the temporary data page based on the meta information of the recovery data page can include:

[0077] corresponding replacement of the at least one other information in the temporary data page with at least one other information in the recovery data page;

[0078] A checksum is calculated based on the replaced at least one other information and data content in the temporary data page, and the checksum in the temporary data page is replaced by the calculated checksum.

[0079] In the field of databases, the checksum is used to detect whether a data page is damaged or tampered during storage or transmission. The checksum can be calculated byte by byte or in data blocks of a certain size for all contents of the data page, and can be calculated using a cyclic redundancy check or checksum algorithm. For example, the algorithm traverses the contents of the data page, takes the value of each byte or data block as input, and then generates a checksum value of a fixed length according to the calculation rule of the algorithm. This checksum value is stored in the metadata of the data page, so as to verify the data integrity when reading or writing the data page subsequently.

[0080] As an example, the metadata of the recovered data table Page_1 includes information 1 to information N, and a checksum;

[0081] Currently, the metadata of the temporary data table Page_1' is to be updated, information 1 to information N of Page_1' can be replaced by information 1 to information N in Page_1; that is, information 1 of Page_1' is replaced by information 1 in Page_1, information 2 of Page_1' is replaced by information 2 in Page_1, and so on;

[0082] Then, the checksum is calculated using the replaced information 1 to information N of Page_1' and the data content;

[0083] Finally, the checksum in Page_1' is replaced by the calculated checksum, and the metadata update of Page_1' is completed.

[0084] As an example, the data processing of the temporary data table based on the access request of the original database instance to the recovered data table can include:

[0085] In response to receiving a read request of the original database instance to a first data page of the recovered data table, it is determined whether a temporary data page corresponding to the first data page has been loaded in the local temporary storage space, if yes, the read request is responded using the temporary data page loaded in the local temporary storage space, otherwise, the read request is responded based on a temporary data page corresponding to the first data page in the temporary data table managed by the temporary database instance;

[0086] In response to receiving a write request of the original database instance to a second data page of the recovered data table, the second data page is written into the local temporary storage space.

[0087] As an example, the access requests of the original instance can include two types of read requests and write requests. In actual applications, based on different use scenarios, a processing scheme for only one type of access request can be implemented as needed, i.e., only read or only write is supported; or a processing scheme for both types of access requests can be implemented.

[0088] ①For the read request of the original instance, one idea is that all read requests directly use the data of the remote temporary data table to respond to the read request. Since the data of the temporary data table is continuously loaded into the local temporary storage space in this embodiment, in order to improve the response speed of some read requests, if the data to be accessed by the read request has been loaded in the local temporary storage space, the loaded data in the local temporary storage space can be used to respond to the read request; if not, the remote temporary data table is used for response. Therefore, the response speed of the read request can be improved in this embodiment.

[0089] ②For the write request of the original instance, it can be directly written into the local temporary storage space.

[0090] In this embodiment, for the write request of the original instance, it can be directly written into the local temporary storage space. Among them, the write request here can be new write data on the existing data of the temporary data table, or write operation on an existing data page of the temporary data table. Both of the two cases can be directly written into the local temporary storage space.

[0091] For example, the temporary data table and the recovery data table are corresponding, and both are 100 data pages. The write request of the original instance to the recovery data table can be to increase a new data page, i.e., the recovery data table is increased to 101 data pages. For the new data page, it can be directly written into the local temporary storage space.

[0092] In other cases, the write request of the original instance to the recovery data table is a write operation on an existing data page of the temporary data table. Whether the data page has been loaded in the local temporary storage space or not, it can also be directly written into the local temporary storage space, because the temporary data table is old data, and the write operation is currently performed. Directly writing makes the local temporary storage space store the new write data of the original instance to the recovery data table.

[0093] Therefore, for the write request, the embodiment can ensure that the new write data of the original instance to the recovery data table is stored in the local temporary storage space, and in step 106, the written data can replace the recovery data table as part of the data table, so the embodiment also supports the write function of the original instance to the recovery data table.

[0094] AsFigure 2A Fig. 1 is a schematic diagram of a local temporary storage space according to an example embodiment of the present disclosure, showing a local temporary storage space according to an example embodiment of the present disclosure, from Figure 2A As can be seen from Fig. 1, the local temporary storage space can store data pages, which include some data pages loaded from the temporary data table, and also include some data pages newly written in the process of the original instance writing to the recovery data table; and when there is a read request, if the data page to be read has been loaded, the read request can be responded by using the data page already loaded in the local temporary storage space. Of course, in the process of loading, there are some data pages in the temporary data table that have not been loaded into the local temporary storage space, and in Figure 2A Fig. 1 uses a dashed line to represent these data pages in the temporary data table that have not been loaded, and if the original instance needs to read these data pages, the read request for these data pages that have not been loaded can be responded by the temporary instance.

[0095] In some examples, after the step of loading the updated temporary data page into the local temporary storage space, the method further comprises:

[0096] recording the loaded state of the updated temporary data page in preset loaded state data;

[0097] The determining whether the temporary data page corresponding to the first data page has been loaded in the local temporary storage space comprises:

[0098] querying, by using the loaded state data, whether the temporary data page corresponding to the first data page in the temporary data table has been loaded in the local temporary storage space.

[0099] In this embodiment, a loaded state data is also maintained, which is used to record the loaded state of each data page in the temporary data table in the local temporary storage space. As an example, the loaded state data can be stored in the memory to realize fast query when needed.

[0100] The data structure of the loading state data can be flexibly configured according to actual needs. As an example, the data structure of the loading state data can be a bitmap. The size of the bitmap can be determined according to the number of data pages divided by the temporary data table. For each page in the temporary data table, it can be one-to-one corresponding to each binary bit of the bitmap. "1" and "0" can be used to represent the loading state of whether the data page has been loaded. Based on this, it can be quickly queried whether each data page of the temporary data table has been loaded in the local temporary storage space, thereby making the processing of the read request more efficient. As an example, in actual application, before starting loading, a loading state data indicating that all data pages of the temporary data table are in an unloaded state can be created in the memory space. For example, the temporary data table has M pages, and a bitmap containing M bits, each bit being 0, is created.

[0101] As an example, the loading state data can be stored in the memory space of the original instance, so that the data can be read from the memory when the loading state data needs to be queried, thereby improving the query efficiency. In order to prevent the data loss of the memory, the loading state data can also be periodically written into the persistent storage space of the original instance as needed, for example, in the local temporary storage space mentioned above.

[0102] For the loading state data, in actual application, the update of the loading state data can be performed after the data page in the temporary data table is loaded into the local temporary storage space in some examples, or the write operation of the original instance on a certain data page in the temporary data table in some other examples. The role of the loading state data can be to determine whether the data page to be read has been stored in the local temporary storage space when the read request occurs in some embodiments, or to determine whether the data page has been stored in the local temporary storage space before the certain data page of the temporary data table is loaded into the local temporary storage space in some other embodiments. Because if the write operation of the original instance on a certain data page in the temporary data table has occurred, the write operation of the original instance can be used as a reference, and the loading of the data page in the temporary data table can be terminated. The above embodiments will be described in the following.

[0103] In some examples, after the step of writing the second data page into the local temporary storage space, the method can further include:

[0104] If the temporary data table has a temporary data page corresponding to the second data page, the loaded state of the temporary data page corresponding to the second data page is recorded in the loading state data.

[0105] In this embodiment, the second data page of the recovery data table to which the write operation is directed can correspond to an existing data page of the temporary data table. For example, there is Page_1' in the temporary data table, and the current original instance wants to perform a write operation on Page_1 of the recovery data table, and Page_1 corresponds to Page_1'. In the case where the loading state data is maintained in the foregoing embodiment, the loaded state of the data page Page_1' can also be recorded in the loading state data. Here, because Page_1' of the temporary data table can have been loaded in the local temporary storage space or can not have been loaded before the write of Page_1 by the original instance occurs, the loaded state of Page_1' can or can not have been recorded in the loading state data. In actual application, it is an optional solution to first query whether the loaded state of Page_1' is recorded in the loading state data and then write the loaded state of Page_1' to the loading state data without querying the loading state data. Therefore, this embodiment can enable the data page to which the write request is directed to be queried by the loading state data to have been loaded in the local temporary storage space when a read request occurs, without accessing the remote storage, so that the processing of the read request is more efficient.

[0106] In some other examples, in the case of a write operation on an existing data page of the temporary data table, because the temporary data table is historical data and the current write operation is the latest operation, the latest write operation needs to be used as a reference; and the current write operation can be earlier than the loading operation of the data page of the temporary data table, so that the loading operation of the data page of the temporary data table should not cover the write operation of the original instance. Based on this, the writing of the updated temporary data page into the local temporary storage space can include:

[0107] querying, from the loading state data, whether the updated temporary data page has been loaded in the local temporary storage space;

[0108] if not, writing the updated temporary data page into the local temporary storage space;

[0109] if yes, discarding the updated temporary data page.

[0110] As an example, at 12:00:00, a write operation on a data page Page_2 of the recovery data table occurs, and there is a data page Page_2' corresponding to the data page Page_2 in the temporary data table;

[0111] Page_2 can be written in the local temporary storage space; and Page_2' is updated to a loaded state in the loading state data;

[0112] At the time of 12:01:00, Page_2' is ready to be loaded into the local temporary storage space as a data page to be loaded; in this embodiment, the loading state data is queried first, and it can be found that Page_2' is in a loaded state, so the loading of Page_2' is terminated directly, thereby ensuring that the latest data written by the original instance is stored in the local temporary storage space, and ensuring the normal use of the original instance to the recovery data table.

[0113] Therefore, through the design of the loading state data, on the one hand, the efficiency of reading data can be improved, and on the other hand, the latest data written by the original instance can be ensured to be stored in the local temporary storage space, and the normal use of the original instance to the recovery data table can be ensured.

[0114] In some examples, after the step of writing the updated temporary data page into the local temporary storage space, the method further comprises:

[0115] Accessing the loading state data stored in the memory space of the original database instance, recording the loaded state of the updated temporary data page in the loading state data, and then writing the loading state data into the local temporary storage space.

[0116] The loading state data stored in the local temporary storage space is used to obtain the loading state of the temporary data page in the temporary data table by the FUSE process after the FUSE process is restarted.

[0117] As an example, the loading state data maintained in the memory can be written into the local temporary storage space according to a set period as needed, and can also be refreshed into the local temporary storage space after each write operation is completed, so as to prevent data loss and ensure data consistency. In this embodiment, if a loading operation of a data page of the temporary data table occurs, the loading state data is updated, and then the loading state data in the memory is written into the local temporary storage space in time. In this way, if the process of this embodiment is lost due to crash or the like, the loading state data in the memory is lost, but the loading state data is still stored in the local temporary storage space, so that the loss of the loading state data is avoided. Therefore, if the FUSE process is closed, the memory space belonging to the FUSE process is recycled by the operating system, and the loading state data stored in the memory space is lost. However, by using this embodiment, after the FUSE process is restarted, the loading state data stored in the local temporary storage space can be accessed, so that the FUSE process can obtain the loading state of the temporary data page in the temporary data table.

[0118] In some examples, the recording of the loaded state of the temporary data page corresponding to the second data page in the loading state data can include:

[0119] After accessing the loading state data stored in the memory space of the original database instance, recording the loaded state of the temporary data page corresponding to the second data page in the loading state data, and writing the loading state data into the local temporary storage space, a write success message corresponding to the write request is returned.

[0120] When a write operation of the original instance is found, in the present embodiment, a write success message is returned after the three operations of writing a data page into the local temporary storage space, updating the loading state data in the memory, and writing the loading state data in the memory into the local temporary storage space are completed. Such processing can also prevent the write operation of the original instance on the existing data page in the temporary data table from being lost.

[0121] In actual applications, the database instance can adopt a master-standby or master-multiple-standby mode to ensure high availability of the database service. In the case of a standby database instance, as an example, the original database instance can include a master database instance and at least one standby database instance; wherein the at least one standby database instance is preferentially started the FUSE process, then the standby database instance and the master database instance that start the FUSE process are executed master-standby switchover, and the standby database instance after switchover is started the FUSE process again.

[0122] Taking two original instances, i.e., instance A and instance B, as an example; the current user uses instance A as the master instance, an empty table can be created in the current master instance A, and the operation of instance A is synchronized to the current standby instance B. Then instance B is closed, the FUSE process is started, and finally instance B is restarted; after instance B is restarted, the temporary data table can be mounted to start data recovery. Since the current user uses the master instance, the user will not perceive the restart of instance B.

[0123] The current instance B has started the FUSE process and successfully mounted the temporary data table in the persistent storage space, i.e., instance B can use the data of the temporary data table; while the current master instance has not completed the operation; therefore, the master-standby switchover is performed in the present embodiment; therefore, instance B currently serves as the master instance, and instance A serves as the standby instance. The current standby instance A also performs the operation of instance B: after being closed, the FUSE process is started, so as to mount the temporary data table. Therefore, the above operation involves the restart of the instance, but will not affect the user experience of the original instance and will not affect the service of the original instance.

[0124] In a master-multiple standby scenario, one of the standby instances can be selected to start the FUSE process first. After the selected standby instance starts the FUSE process, the master-standby switchover is performed between the selected standby instance and the master instance. The standby instance after the switchover starts the FUSE process again. The other standby instances can perform the FUSE process starting operation.

[0125] As an example, the loading of the temporary data table into the local temporary storage space of the original database instance, and the processing of the access request of the original database instance to the recovery data table based on the data of the temporary data table during the loading process, can include:

[0126] A preset file system is run in the host where the original data instance is located. The preset file system is configured to execute the FUSE process, and the temporary data table is loaded into the local temporary storage space of the original database instance by the preset file system. During the loading process, the access request of the original database instance to the recovery data table is processed based on the data of the temporary data table.

[0127] The method further includes:

[0128] After the loading is completed, the preset file system is unloaded.

[0129] In this embodiment, since the recovery data table is empty in the case where the temporary data table is not loaded, the access request needs to be processed specially. In addition to the data recovery, the access of the original instance to the data table can be implemented according to the existing scheme. Therefore, in order to reduce resource occupation, a preset file system for processing the access request of the original instance to the recovery data table can be implemented in this embodiment. The preset file system is configured to execute the FUSE process. During the loading process, the access request can be processed by the preset file system. After the loading is completed, the preset file system can be unloaded, thereby reducing the resource occupation of the host where the original instance is located.

[0130] In practical applications, the database instance can adopt a master-standby or master-multiple standby mode to ensure high availability of the database service. In the case of a standby database instance, as an example, the original database instance can include a master database instance and at least one standby database instance. The original database instance includes a master database instance and at least one standby database instance. The running of the preset file system in the host where the original data instance is located includes:

[0131] A corresponding remote storage space is created for each instance included in the original database instance, and the temporary data table is copied in the remote storage space.

[0132] After restarting the current standby database instance, running a preset file system in the host where the current standby database instance is located, and mounting the temporary data table in one of the remote storage spaces to the preset file system;

[0133] Triggering a master-standby switchover of the current master database instance and the current standby database instance, restarting the standby database instance after the switchover, running a preset file system in the host where the standby database instance after the switchover is located, and mounting the temporary data table in another remote storage space to the preset file system.

[0134] As an example, after the temporary database instance is restored, the temporary database instance manages the restored temporary data table, and the temporary data table is stored in the persistent storage device corresponding to the temporary database instance. In this embodiment, the temporary database instance can be closed first, and then the data of the temporary data table stored in the persistent storage device is copied to other different persistent storage devices. The number of copies is consistent with the number of masters and standbys in the original database instance, for example, if there are n masters and standbys, then n copies are copied to n different persistent storage spaces (such as solid state disks or other storage devices). Because the original database instance is running, the temporary data table stored in the n different persistent storage spaces cannot be used by each original database instance at present, so some operations are needed to realize data mounting without affecting the user experience of using the original instance.

[0135] Taking one master and one standby, i.e., two original instances, instance A and instance B, as an example; the current instance used by the user is instance A as the master instance, an empty table can be created in the current master instance A first, and the operation of instance A is synchronized to the current standby instance B. Then, the current instance B is closed, a preset file system is started, and finally instance B is restarted; after instance B is restarted, the preset file system can mount the temporary data table in one of the persistent storage spaces. Because the current user uses the master instance, the user will not perceive the restart of instance B.

[0136] The current instance B has run the preset file system and successfully mounted the temporary data table in the persistent storage space, i.e., the preset file system of instance B can use the data of the temporary data table; while the current master instance has not completed the operation; therefore, the master-standby switchover is performed in this embodiment; therefore, instance B currently serves as the master instance, and instance A serves as the standby instance.

[0137] Therefore, instance A currently serving as the standby instance also performs the operation of instance B: after being closed, a preset file system is started and restarted, so as to mount the temporary data table in another persistent storage space. Therefore, the operation of the above-mentioned operation involves the restart of the instance, but will not affect the user experience of using the original instance, nor will it affect the service of the original instance.

[0138] For the case of multiple standby instances, one of which is the primary and the other is the standby, the processing mode of the primary and the standby mentioned in the above embodiment is adopted, and the other standby instances can also execute the processing flow of the above-mentioned standby instance, that is, closing, restarting, and running the FUSE process to mount the temporary data table of the corresponding persistent storage space. In this way, the preset file system is running in the primary instance and the standby instance, and the temporary data table is mounted. The preset file system in the primary instance can load the temporary data table while processing the access request to the recovery data table, and the preset file system in each standby instance can also load the temporary data table. In the primary-standby scenario, the operation of the primary instance on the recovery data table can also be synchronized to each standby instance.

[0139] As shown in Figure 2B , it is a data recovery scenario diagram according to an exemplary embodiment of the present specification. In the present embodiment, the original database instance (referred to as the original instance) is taken as an example of one primary and one standby, which is referred to as the primary instance and the standby instance hereinafter. The data recovery process can include:

[0140] Step 1: A temporary database instance (hereinafter referred to as a temporary instance) can be quickly created through a data recovery technology, and the temporary database instance manages the recovered temporary data table.

[0141] Step 2: After successful recovery, the temporary instance can be closed, and the temporary data table is prepared to be mounted to the original instance.

[0142] Step 3: First-stage instance recovery flow

[0143] 3.1: Start recovering the data of the original instance; first, the standby instance can be closed, and after mounting the temporary data table in the preset file system of the standby instance, the standby instance is started.

[0144] Optionally, the temporary data table is mounted, and the specific process can be that after the temporary instance is closed, the temporary data table is copied and stored in the preset storage space. In the present embodiment, the original instance has two primary and standby instances, so two copies can be made, that is, as shown in the figure, the temporary data table managed by the temporary instance is copied to two different hard disks, such as a solid state drive (SSD), and the number of copies of the temporary data table corresponds to the number of original instances. After copying, the storage space storing the temporary data table is used to mount to the original instance. For example, one of the temporary data tables is mounted to the current standby instance, as shown in Figure 2C , hard disk 2. The other temporary data table in hard disk 1 is mounted to the current primary instance in the subsequent steps.

[0145] 3.2: After the standby instance is recovered, the primary-standby switchover is triggered, which can avoid the user perceiving the instance restart.

[0146] Specifically, in the present embodiment, the current standby instance triggers the master-standby switchover after completing mounting, i.e., the master instance and the standby instance perform the master-standby switchover.

[0147] 3.3, the current standby instance is closed, and the temporary data table in the hard disk 1 is mounted in the preset file system of the current standby instance, and the current standby instance is started.

[0148] After the above steps are completed, the master instance and the standby instance have both completed mounting recovery, and the user can perceive that the recovery of the data table of the original instance has been completed, i.e., can start using the recovered data table. At this time, the recovered data table is empty, and the temporary data table has not actually been completely copied from the SSD to the local file system of the original instance. If the original instance needs to access the data in the recovered data table, the preset file system processing is used to process the access request.

[0149] The instance recovery process of the first stage of step 3, the processing time of the process is relatively fixed, because no real data copying occurs. The data recovery time perceived by the user is irrelevant to the data volume of the data table. Therefore, from the user experience, it can be perceived that the data table is quickly recovered.

[0150] Step 4, the instance recovery process of the second stage

[0151] 4.1, in the process of this stage, the master instance and the standby instance will both continue to load the temporary data table to the local temporary storage space of each. If the data volume of the temporary data table is relatively large, the waiting time of this stage may be relatively long.

[0152] When the data copying of the standby instance is completed, the standby instance can be closed (i.e., the process of the standby instance in the host is closed), the preset file system is unloaded (i.e., the preset file system is unloaded in the host), the data stored in the local temporary storage space of the preset file system is completely moved to the local file system of the original instance, i.e., all the pages stored in the local temporary storage space are moved to the directory where the recovered data table is located; finally, the standby instance is restarted, and the unmounting is completed.

[0153] 4.2, trigger the master-standby switchover, which can avoid the customer perceiving the restart of the master instance.

[0154] Through the master-standby switchover, the standby instance that completes data copying before the switchover will serve as the current master instance.

[0155] 4.3. Wait for the current standby instance to complete data replication. Similarly, the process first uses the preset file system to complete data replication. If the replication is complete, shut down the current standby instance, unmount the preset file system, move all data stored in the preset file system's local temporary storage space to the local file system of the original instance, and finally restart the current standby instance to complete the unmounting.

[0156] This shows that without the user's knowledge, the second phase of the instance recovery process can use the preset file system to copy data to the local temporary storage space, and then restore the original instance's existing method of accessing data through the local file system, and the entire recovery process is completed.

[0157] like Figure 2C As shown, this specification shows another data recovery scenario schematic diagram according to an exemplary embodiment. As an example, the preset file system of this embodiment can be implemented based on FUSE. FUSE (Filesystem in Userspace) is a mechanism that allows non-privileged users to implement a file system in user space. Traditionally, the implementation of a file system needs to run in kernel space, and FUSE allows developers to write file systems in user space without modifying the kernel code of the operating system. With FUSE, developers can use standard programming languages ​​to write file systems without having to have an in-depth understanding of kernel programming. FUSE provides a user space library and a set of interfaces that developers can use to implement custom file system logic. Once the file system is mounted, the file system program in user space can handle file system operations, such as reading, writing, creating or deleting files.

[0158] In the related art, a FUSE user mode file system can be used to proxy the read and write I / O and some other I / O operations of the upper application system, so that the upper application system is not aware, but using the FUSE user mode file system will cause a performance degradation of the database system, which is about 10%-20%. In the data table recovery scenario of the embodiment, the data of some library tables is problematic, and if the FUSE is used, the read performance of these library tables will be reduced, but this cost is acceptable in the scenario where the user urgently needs to recover data. Of course, in actual applications, other ways to implement the embodiment are also optional, such as adding the processing mode of the embodiment to the file system function of the operating system, etc. For example, the step of "loading the temporary data table into the local temporary storage space of the original database instance, and processing the access request of the original database instance to the recovery data table based on the data of the temporary data table during the loading process" can not be implemented by the FUSE process, but can be improved by the file system function of the operating system, which can be implemented by the file system of the operating system, and the embodiment does not limit this.

[0159] In Figure 2C In the left side, the original instance (which can be a primary instance or a standby instance) in the foregoing embodiment is shown, Figure 2C In order to facilitate the example, only one is shown as an example in the original instance in which an empty recovery data table is created.

[0160] Through the preset file system of the embodiment, the database file managed by the temporary instance can be mounted to the original instance in seconds. The function of the preset file system is to provide the left original instance with the function of accessing the data in the data table of the right temporary instance, and the data in the data table is loaded into the local temporary storage space (local temp store) of the original instance slowly through delayed loading, without the user's awareness.

[0161] As an example, in a data recovery scenario, specifically, the user can initiate a recovery request to recover the original data table managed by the original instance to a historical time point during the use of the original instance.

[0162] The existing database recovery technology can be used to create a temporary database corresponding to the original database, and the temporary database manages a temporary data table, which is a data table of the original data table at a historical time point.

[0163] An empty new data table is created in the original instance, which is used to replace the original data table.

[0164] A local temporary storage space is created in the original instance, which is used to load the data of the temporary data table.

[0165] In order to make the user feel that the data has been quickly recovered, the embodiment creates an empty recovery data table in the original instance, so that the original instance can access the recovery data table, and the preset file system needs to respond to the access request of the recovery data table by using the data of the temporary instance. Therefore, how the preset file system responds to the access request is a problem to be faced.

[0166] Generally, a database manages a data table according to a page. One data table can correspond to multiple pages. Each page stores meta information and data content. The meta information can include a table space ID, an index ID, a transaction ID, and the like, and a checksum of the data content stored in the page. The meta information stored in each page corresponding to the empty table in the original instance is related to the current running state of the original instance. On the other hand, since the temporary data table is recovered in the temporary instance, the page also has its own meta information, which is different from the meta information in the page of the empty data table in the original instance. Because the temporary data table is also backup data from the original database, it belongs to a historical state, and there can be problems such as reuse of a space ID, an index ID, or the transaction ID is a historical small transaction ID, or the backup data is from a standby database, which causes the data meta in the temporary database to be different from the meta of the empty table in the original database. In this way, directly mounting the file, the original instance can not recognize the temporary data table in the temporary instance because the meta information cannot be matched.

[0167] Therefore, in the embodiment, the idea to solve the problem is that, although the meta information of the page in the original instance and the meta information of the page in the temporary instance cannot be matched, the data content stored in the page is the same. Therefore, in the embodiment, an empty table is created for the original instance to access. In the processing flow of the read I / O of the empty table in the original instance, the meta information of the data page read by the read I / O is converted into the meta of the original instance, and then the checksum is recalculated. The I / O reads the page data and returns the converted data. In this way, after the data conversion, the original instance can normally read the data content in the page according to the meta in the page.

[0168] See Figure 2C As an example, the preset file system can include the following functional modules:

[0169] 1. Data page determination module (Page align)

[0170] The data operation of the original instance to the data table is all based on the Page of the database as the operation object, for example, the data is converted according to the Page, such as judging the position of the data is in the local temporary directory or in the remote temporary database.

[0171] The access request of the original instance received in the preset file system is sent by the local file system of the original instance to the preset file system after receiving the access request sent by the original instance. The division granularity of the data by the local file system is usually different from the division granularity of the data by the database, for example, the division granularity of the data by the local file system is usually 4KB, and the division granularity of the data by the database is usually 16KB, and the like.

[0172] Taking the database system MySQL (My Structured Query Language, a name of a database system) as an example, the data page size of the storage engine of MySQL is set to 16KB, and the division granularity of the local file system is 4KB. Since it is necessary to determine which data page of the data table each access request is directed to, the access request can be converted based on the granularity of MySQL, assuming that the offset of the access request delivered by the local file system is 5KB, and the length is 9KB, the request can be converted to offset 0KB and length 16KB, so that the specific data page to which the access request is directed can be determined.

[0173] In actual application, the sizes of the data pages of different database systems can be different, therefore, in order to support different types of database systems, the preset file system of the embodiment can pre-configure different conversion rules of the access request according to the size of the data page of different database systems. For example, the conversion rules of other types of database systems such as MongoDB and Postgres can also be implemented in the module, and in actual application, other types of database systems can also be implemented, which are not limited in the embodiment.

[0174] 2、Data location judgment module (DataBase Page location)

[0175] After passing through the Page align module, all the access requests are converted into requests aligned with the database Page, and in the embodiment, the module judges the position of the Page to be accessed by the access request.

[0176] As the receiving, the data page of the temporary data table is needed to be delayed loaded, for example, the temporary data table has M pages, the identifier of the M pages can be set, for example, 1 to M. The page accessed by the access request can be the data page corresponding to the temporary data table, or can not be (the case of not being is that the access request accesses the page newly added by the original instance to the recovery data table), therefore, it is needed to judge whether the page accessed by the access request belongs to the M pages needed to be delayed loaded.

[0177] Because the temporary data table already recovered in the remote temporary instance will not be newly added data. Various ways can be used to judge whether the page of the access request belongs to the page needed to be delayed loaded. Because the number of data pages of the temporary data table is fixed, it can be determined from the access address carried by the write request whether the data page to be accessed is within the data page range of the temporary data table. For example, the access request can be determined to carry an offset and a length, and the temporary data table also has a size length, and whether the data page of the access request is within the length range of the temporary data table can be calculated based on this.

[0178] For the write request, if yes, the current written page and the asynchronously delayed loaded page need to be mutually exclusive to avoid that the delayed loaded page covers the newly written page. If not, it is indicated that the current writing is the newly added page, so mutual exclusion of the current writing and the delayed loading is not needed because the current written page is not in the remote temporary data table.

[0179] Meanwhile, the embodiment also maintains the loading state data Bitmap in the memory, the Bitmap records the loading state of each page of the temporary data table, and the loading state represents whether each page is in the remote temporary data table or in the local temporary storage space. The Bitmap in the memory will also periodically persist data to the local temporary storage space, and if the process of the preset file system hangs, the Bitmap can be found in the local temporary storage space after the process is restarted, so that the loading state of the page can be found after the process of the preset file system hangs and is restarted. For the page newly added by the original instance to the recovery data table, the loading state does not need to be recorded in the loading state data.

[0180] For read request, if the Page to be read belongs to the Page of delayed loading, the location of the Page can be determined according to the Bitmap in the memory, if the Page is in the local temporary storage space, the data in the local temporary storage space is read. If the Page is not loaded, it is in the remote temporary data table, the read request is directly forwarded to the remote request data. If the Page does not belong to the Page of delayed loading, it is the data page newly written before, the data in the local temporary storage space can be directly read, so that the processing efficiency of read request is improved.

[0181] 3. The meta information conversion module of the data page;

[0182] As an example, the module can load the Meta of the empty table of the original instance and the Meta of the remote temporary data table; the meta information of the Page read from the remote temporary data table can be converted by using the module, no matter the Page is loaded from the remote temporary data table to the local temporary storage space or the Page is read from the temporary data table based on the read request.

[0183] As an example, the module can support one or more different types of database systems; different types of database systems can have different designs for the fields in the meta information of the Page, and different meta information conversion modes can be implemented in advance according to the database systems to be supported. The specific meta information conversion mode can refer to the description of the foregoing embodiments. As an example, the module can be designed as a plug-in architecture, and the corresponding meta information conversion mode can be selected according to the type of the database instance in the actual application scenario. As an example, the conversion process can be that, the fields in the Meta of the Page of the temporary data table, except the checksum, are replaced by the corresponding fields in the Meta of the Page of the empty table; after the replacement, the checksum is calculated and updated according to the data content of the Page of the temporary data table and the replaced fields.

[0184] 4. The lazy load module lazyload;

[0185] The lazy load module of the embodiment is used to load the data page of the temporary data table to the local temporary storage space of the original instance, and it can be seen that the lazy load of the embodiment is the loading of the data table, and thus belongs to the file level lazy load.

[0186] As an example, the temporary data table that needs to be delayed loaded can be cut into N segments, and the specific cutting manner can be flexibly configured according to actual needs. Each time, a segment of data cut is loaded, and the size of each segment of the N segments can be the same or different, and the embodiment does not limit this. For example, each segment can be the same size as the data page, and the temporary data table has several data pages, and is cut into several segments. A larger cutting granularity can also be set, for example, the size of each segment cut is a multiple of the data page, so that each time loading is loading one or more data pages, and of course, the specific multiple is not limited by the embodiment. As an example, a loading task (such as a thread or a process, etc.) can be implemented, and the loading task is used to load one or more data pages each time, and after completing the loading of these data pages this time, the loading task obtains one or more data pages from the temporary data table again to load again.

[0187] In actual application, when loading, one or more data pages need to be loaded to a certain location range of the local temporary storage space, and the original instance can also have access request for the data page, so in order to maintain the consistency and integrity of the data, a lock mechanism can be used to achieve it. For example, when storing one or more data pages to be loaded in the local temporary storage space, the storage locations of these data pages need to be locked; or when the original instance accesses some data pages stored in the local temporary storage space, the storage locations of the accessed data pages also need to be locked. Therefore, the number of segments cut can also be considered in combination with the lock mechanism. If the granularity of the cut is relatively fine, more locks need to be maintained, and if the cutting granularity is relatively large, when concurrent access occurs, it can cause the response speed of the access request of the original instance to be relatively slow.

[0188] In the delayed loading process, the Bitmap described above can also be combined for design. If a data page to be loaded is recorded in the Bitmap as an already recorded state, it means that the write operation of the original instance to the data page has occurred, so the loading can be omitted, and the newly written data can be directly discarded to avoid the old data from being overwritten. If the Bitmap indicates an unloaded state, the data page to be loaded can be loaded into the local file temporary storage space of the original instance through the meta information conversion module, and the data page with updated meta information is marked as loaded in the Bitmap.

[0189] As another example, some strategies can be used for delayed loading. For example, the N segments cut can be loaded in a specific order, and the specific order can be set according to needs, such as a random order or an order from the first segment to the last segment.

[0190] Or, some data pages that the user is likely to access can be predicted, and these pages are loaded preferentially; the prediction method can be various, and the pages operated on the original data table before the user initiates the recovery request can be obtained as the preferentially loaded pages. Or, according to the historical use records of the original data table, the high-frequency used pages can be counted, and the like.

[0191] Or, the indexes created by the temporary instance for the temporary data table can also be loaded preferentially; as an example, taking the Innodb storage engine of MySQL as an example, the database instance adopts a B+ tree to implement the index structure of the data table when managing the data table, and the index is also stored in the form of an index file on the disk. In Innodb, the data table itself is an index structure organized in a B+ tree, and the leaf node data field of the tree saves complete data records. The key of this index is the primary key of the data table, so the InnoDB table data file itself is the primary index, and the leaf node of the B+ tree contains complete data records. When some data records of the data table are searched, the root node of the tree is searched down to the leaf node, therefore, the use probability of the non-leaf nodes in the B+ tree is high, and in the embodiment, the B+ tree corresponding to the temporary data table can also be obtained, and the non-leaf nodes from the root node to the leaf node in the B+ tree are loaded preferentially, that is, the primary key index is loaded to the non-primary key index. Therefore, the use probability of the index node is high, and the non-leaf nodes of the B+ tree in the local temporary storage space can be loaded preferentially, so that when the read request of the original instance occurs, the non-leaf nodes of the B+ tree loaded in the local temporary storage can be used and loaded, the Page required to be read in the read request is queried more quickly, and the read request is responded more efficiently.

[0192] 5. The local temporary storage space local temp store;

[0193] In the embodiment, the local temporary storage space can store two parts of data: Bitmap and the delayed loaded Page. The data position judgment module can periodically flush the Bitmap in the memory to the storage space. The data newly written by the original instance and the data page loaded by the delayed loading module can also be stored here. With the passage of time, the Pages stored in the local temp store for delayed loading will be more and more, and when all the delayed loading is completed, the Pages stored here can be restored to replace the data table by the process mentioned in the foregoing embodiment.

[0194] As an example, whether the data pages of the temporary data table are completed to be loaded can be determined by a Bitmap, in which all bits are 1, indicating that all data pages of the temporary data table have been completed to be loaded. After the completion of the loading, the data pages stored in the local temporary storage space can be used to replace the recovery data table. It can be understood that in actual application, when the loading is completed, all the Pages stored in the local temporary storage space correspond to all the Pages of the temporary data table, if there is no write operation of the original instance to the recovery data table. If the original instance has write operations to the recovery data table, and these write operations are only for the existing data table of the recovery data table, all the Pages stored in the local temporary storage space also correspond to all the Pages of the temporary data table, but the data content stored in the Pages may change. If the original instance has write operations to the recovery data table, but these write operations include new Pages added to the recovery data table, all the Pages stored in the local temporary storage space, in addition to those corresponding to the Pages of the temporary data table, also include the new Pages, and the new Pages and the Pages corresponding to the temporary data table jointly constitute a new data table and are used to replace the recovery data table.

[0195] The preset file system and the database of the embodiment can realize data non-loss and guarantee data consistency through cooperation. Because the original instance continuously has service write, the write data is also written into the local temp store, and it is necessary to ensure that the newly written data of the database cannot be lost due to the hanging of the preset file system process.

[0196] The database itself has a WAL mechanism; in a database system, the Write-Ahead Logging (WAL) mechanism is a common persistence technology used to ensure the transaction persistence and recovery ability of the database. The WAL mechanism guarantees the atomicity, consistency and durability of transactions by writing logs into a log file before writing data. Specifically, when the database has new write operations, the WAL mechanism processes them according to the following steps:

[0197] Write WAL logs: before executing transaction write operations, the database will first record these operations in the form of logs into the WAL log file. These logs contain the specific operations of the transaction, such as update, insert or delete operations.

[0198] Update the dirty page list: at the same time, the modified data pages are also marked as "dirty pages", indicating that the contents of these data pages have been modified but have not been persisted to the disk.

[0199] Continuous dirty page flushing logic: The database will continuously write the data in the dirty page back to the corresponding data file according to a certain strategy. This process is called "flushing logic", which ensures the persistence and consistency of data.

[0200] In addition, the database system also performs a checkpoint operation periodically, which writes the data in the memory to the disk and advances the recovery point of the database to reduce the amount of logs that need to be redone during database recovery. When performing a checkpoint, the database ensures that all previous data has been written to the local disk to ensure data integrity and consistency.

[0201] Therefore, when the original instance has new writes, the WAL mechanism will continuously flush to the disk according to the transaction. At the same time, the newly written Page will be added to the dirty page list, and the database will continuously write the Page to the disk according to the dirty page flushing logic. The preset file system can also receive the Page write caused by the dirty page flushing of the original instance. Page write is first saved in system cache, and in this embodiment, data can still be kept in system cache without being flushed to local temp store every time. Because the database will periodically do checkpoint, the database needs to ensure that the data in the dirty page is written back to the local disk when advancing the checkpoint. Therefore, the preset file system of this embodiment can receive these write operations, and the preset file system will write the Page to the local temp store according to the write operation at this time; after the write is successful, the local Bitmap is also written to the local temp store. After this part is written successfully, it returns a write success, and the checkpoint of the database is advanced. Therefore, the way of writing Page first and then writing Bitmap can ensure the consistency of Bitmap and data. The file system restarts at any time, and the newly written data of the database is not lost.

[0202] Corresponding to the foregoing embodiments of the data recovery method, the present specification also provides embodiments of a data recovery device and a computer device to which the data recovery device is applied.

[0203] The embodiment of the data recovery device of the present specification can be applied to a computer device, such as a server or a terminal device. The device embodiment can be realized by software, or realized by hardware or a combination of software and hardware. Taking software realization as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory to the memory for running by the processor where it is located. From the hardware level, as shown in Figure 3 Figure 3 ​In addition to the processor 310, the memory 330, the network interface 320, and the non-volatile memory 340 shown, the computer device in which the data recovery apparatus in the embodiment is located can also include other hardware according to the actual functions of the computer device, which will not be described again.

[0204] As an example, the data recovery apparatus based on the user-mode file system FUSE according to an exemplary embodiment illustrated in the specification can include:

[0205] The recovery module is configured to: in response to receiving a recovery request for recovering a data table in an original database instance to a historical time, recover a temporary data table of the data table at the historical time through a temporary database instance corresponding to the original database instance, and create an empty recovery data table corresponding to the temporary data table in the original database instance for the original database instance to access;

[0206] The processing module is configured to: load the temporary data table into a local temporary storage space of the original database instance through the FUSE process, and process an access request of the original database instance to the recovery data table based on data of the temporary data table during the loading process.

[0207] The replacement module is configured to: replace the data table stored in the local temporary storage space with the recovery data table after the loading is completed.

[0208] The implementation process of the functions and roles of each module in the above data recovery apparatus is specifically described in the implementation process of the corresponding steps in the above data recovery method, which will not be described again.

[0209] Correspondingly, the embodiments of the specification also provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps of the foregoing data recovery method embodiments.

[0210] Correspondingly, the embodiments of the specification also provide a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the data recovery method embodiments when executing the program.

[0211] Correspondingly, the embodiments of the specification also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the data recovery method embodiments.

[0212] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can be seen from the part of the method embodiment. The device embodiment described above is only illustrative, wherein the modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e., can be located in one place or distributed on multiple network modules. Part or all of the modules can be selected to achieve the purpose of the scheme of the present specification according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0213] The above embodiments can be applied to one or more computer devices, which are devices capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0214] The computer device can be any electronic product capable of human-computer interaction with the user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive Internet Protocol Television (IPTV), a smart wearable device, etc.

[0215] The computer device can also include a network device and / or a user device. The network device includes but is not limited to a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

[0216] The network in which the computer device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0217] The above-described embodiments of the application have several aspects, no single one of which is solely responsible for the application's desirable attributes. Without limiting the scope of this application, several non-limiting examples of aspects of the application are described in the following paragraphs. In some cases, actions or steps can be performed in an order different than presented in the examples, and still achieve the desired outcome. Also, descriptions of processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0218] The division of steps in the above methods is only for the sake of description, and in actual implementation, one step can be combined or some steps can be split into multiple steps, as long as the same logical relationship is included, which is within the protection scope of the patent; adding irrelevant modifications or introducing irrelevant designs in the algorithm or process, but not changing the core design of the algorithm and process, are within the protection scope of the application.

[0219] While this specification contains many specifics, these should not be construed as limitations on the scope of any invention or of what can be claimed, but rather as descriptors of features that can be combined to form specific embodiments of the invention. Some features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented or provided in the context of multiple embodiments separately or in any suitable sub-combination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a sub-combination or variation of a sub-combination.

[0220] described in connection with the described embodiments or examples. The illustrative examples set forth herein are by way of example and not limiting, as the scope of the application is defined by the claims. Moreover, none of the language in this specification should be considered as giving any clue, either directly or indirectly, as to description of the application. Accordingly, the application is not limited except as by the appended claims.

[0221] Those skilled in the art will readily recognize that other embodiments of the present specification can be used without departing from the scope of the application. The specification is intended to cover any adaptations or variations of the specific embodiments discussed in this specification including such further modifications as can occur to those skilled in the art in implementation of the application. The specification is intended to cover any and all adaptations or variations of preferred embodiments. The specification covers the goods and assets of the preferred embodiments including any patents, patent applications, publications, processes, trade secrets, copyrights, licenses, and the like.

[0222] It is to be understood that the present description is not limited to the precise construction herein described and as shown in the attached drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope thereof. The scope of the description is indicated only by the appended claims.

[0223] The above description is merely the preferred embodiments of the present description and is not intended to limit the present description. It is understood by those skilled in the art that any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present description shall be included in the scope of the present description.

Claims

1. A method for data recovery based on a user-mode file system (FUSE), the method comprising: in response to receiving a recovery request for recovering a data table in an original database instance to a historical time, recovering a temporary data table of the data table at the historical time through a temporary database instance corresponding to the original database instance, and creating an empty recovery data table corresponding to the temporary data table in the original database instance for access by the original database instance; loading the temporary data table into a local temporary storage space of the original database instance through a FUSE process, and during the loading, processing an access request of the original database instance to the recovery data table based on data of the temporary data table; after the loading is completed, replacing the data table stored in the local temporary storage space with the recovery data table. 2.The method of claim 1, wherein a database instance manages a data table in at least one data page, and the data page contains metadata. The loading of the temporary data table into the local temporary storage space of the original database instance comprises: obtaining at least one temporary data page of the temporary data table that is currently to be loaded; obtaining metadata of a recovery data page of the recovery data table corresponding to the temporary data page through the original database instance; updating the metadata of the temporary data page based on the metadata of the recovery data page, and loading the updated temporary data page into the local temporary storage space. 3.The method of claim 2, wherein the data page further contains data content, and the metadata comprises a checksum and at least one other information in addition to the checksum. The updating of the metadata of the temporary data page based on the metadata of the recovery data page comprises: correspondingly replacing the at least one other information in the temporary data page with the at least one other information in the recovery data page; and calculating a checksum according to the at least one other information and the data content in the temporary data page after the replacement, and replacing the checksum in the temporary data page with the calculated checksum. 4.The method of claim 2, wherein the processing of the access request of the original database instance to the recovery data table based on the data of the temporary data table comprises: in response to receiving a read request of a first data page of the recovery data table from the original database instance, determining whether a temporary data page corresponding to the first data page has been loaded in the local temporary storage space, if yes, responding to the read request with the temporary data page that has been loaded in the local temporary storage space, and if not, responding to the read request based on a temporary data page corresponding to the first data page in the temporary data table managed by the temporary database instance; and in response to receiving a write request of a second data page of the recovery data table from the original database instance, writing the second data page into the local temporary storage space. 5.The method of claim 4, after the loading of the updated temporary data page into the local temporary storage space, the method further comprises: record the loaded state of the updated temporary data page in the preset loaded state data; the determining whether the temporary data page corresponding to the first data page has been loaded in the local temporary storage space comprises: querying, by using the loaded state data, whether the temporary data page corresponding to the first data page in the temporary data table has been loaded in the local temporary storage space.

6. The method of claim 5, after the step of writing the second data page into the local temporary storage space, the method further comprises: if the temporary data table has the temporary data page corresponding to the second data page, recording the loaded state of the temporary data page corresponding to the second data page in the loaded state data.

7. The method of claim 6, the recording the loaded state of the temporary data page corresponding to the second data page in the loaded state data comprises: accessing the loaded state data stored in the memory space of the original database instance, after recording the loaded state of the temporary data page corresponding to the second data page in the loaded state data, writing the loaded state data into the local temporary storage space, and returning a write success message corresponding to the write request.

8. The method of claim 6, the writing the updated temporary data page into the local temporary storage space comprises: querying, from the loaded state data, whether the updated temporary data page has been loaded in the local temporary storage space; if not, writing the updated temporary data page into the local temporary storage space; if yes, discarding the updated temporary data page.

9. The method of claim 8, after the step of writing the updated temporary data page into the local temporary storage space, the method further comprises: accessing the loaded state data stored in the memory space of the original database instance, after recording the loaded state of the updated temporary data page in the loaded state data, writing the loaded state data into the local temporary storage space. The loaded state data stored in the local temporary storage space is used to obtain the loaded state of the temporary data page in the temporary data table by the FUSE process after the FUSE process is restarted.

10. The method of claim 1, the primary database instance comprising a primary database instance and at least one standby database instance, wherein, At least one of the standby database instances is preferentially started to start the FUSE process, then the standby database instance and the master database instance that start the FUSE process are executed to perform master-slave switchover, and the standby database instance after switchover is started to start the FUSE process.

11. The method of claim 1, the loading, by the FUSE process, the temporary data table into the local temporary storage space of the original database instance, and processing, during the loading process, the access request of the original database instance to the recovery data table based on the data of the temporary data table, comprises: running a preset file system in a host where the original data instance is located, the preset file system being configured to execute a FUSE process, and loading, by the preset file system, the temporary data table into a local temporary storage space of the original database instance, and processing, based on data of the temporary data table, an access request of the original database instance to the recovery data table during the loading process; The method further comprises: after the loading is completed, uninstalling the preset file system.

12. The method of claim 11, wherein the original database instance comprises a primary database instance and at least one standby database instance; and wherein the running a preset file system in a host where the original data instance is located comprises: creating a corresponding remote storage space for each of the instances included in the original database instance, and copying the temporary data table in the remote storage space; after restarting the current standby database instance, running a preset file system in a host where the current standby database instance is located, and mounting the temporary data table in one of the remote storage spaces to the preset file system; triggering a switchover of a primary database instance and a standby database instance, restarting a standby database instance after the switchover, running a preset file system in a host where the restarted standby database instance is located, and mounting the temporary data table in another of the remote storage spaces to the preset file system.

13. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor, when executing the computer program, implements the steps of the method of any one of claims 1 to 12.

14. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method of any one of claims 1 to 12.

15. A computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the method of any one of claims 1 to 12.

Images