Data recovery

Abstract

Provided are a data recovery method, a device, a program product, and a storage medium. The method comprises: in response to receiving a recovery request for recovering a temporary data table at a historical moment from an original data table in an original database instance, recovering the temporary data table of the original data table at the historical moment by means of a temporary database instance corresponding to the original database instance, and creating, in the original database instance, an empty recovered data table corresponding to the temporary data table for access by the original database instance; loading the temporary data table into a local temporary storage space of the original database instance by means of a FUSE process, and in the loading process, on the basis of data of the temporary data table, processing an access request of the original database instance for the recovered data table; and once the loading has been finished, replacing the data table stored in the local temporary storage space with the recovered data table.

Description

[0001] Data recovery technology

[0002]

[0001] This disclosure relates to the field of storage technology, and more particularly to data recovery. Background Art

[0003]

[0002] A database can manage one or more data tables. Database table recovery refers to restoring a portion of the data tables managed by a database instance to a historical point in time using technical means. In related technologies, a temporary database instance corresponding to the original database instance can be restored. This temporary database instance manages the restored temporary data tables. The temporary database instance is a different instance from the original database instance, and the storage space of the temporary data tables is remote storage for the original database instance. Then, the temporary data tables need to be physically copied to the storage device where the original database instance resides. After all the data is copied, the original database instance needs to be restarted to access the restored data tables. Therefore, existing data table recovery solutions require a long waiting time. Summary of the Invention

[0004]

[0003] This disclosure provides a data recovery method, device, program product and storage medium based on the user-mode file system FUSE.

[0005]

[0004] According to a first aspect of the embodiments of this specification, a data recovery method based on the user-space file system FUSE is provided. The method includes: in response to receiving a recovery request to restore an original data table in an original database instance to a historical time, restoring a temporary data table of the original data table at the historical time through a temporary database instance corresponding to the original database instance, and creating an empty recovery data table corresponding to the temporary data table in the original database instance for access by the original database instance; loading the temporary data table into the local temporary storage space of the original database instance through the FUSE process, and during the loading process, processing the access request of the original database instance to the recovery data table based on the data of the temporary data table; and after loading is completed, replacing the recovery data table with a data table stored in the local temporary storage space.

[0006]

[0005] According to a second aspect of the embodiments of this specification, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method embodiments described in the first aspect above.

[0007]

[0006] According to a third aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the method embodiments described in the first aspect above.

[0008]

[0007] According to a fourth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method embodiments described in the first aspect above.

[0009]

[0008] The technical solutions provided by the embodiments of this specification may include the following beneficial effects: In the embodiments of this specification, the data recovery scheme based on the user-space file system FUSE of this embodiment, after receiving a recovery request, can first use database instance recovery technology to recover the temporary database instance corresponding to the original database instance. The temporary database instance recovers the temporary data table of the original data table at a historical time. At the same time, an empty recovery data table is created in the original database instance. Since the recovery data table is empty and there is no copy of real data, data access can be quickly restored in the original database instance. Through the FUSE process, the temporary data table can be loaded into the local temporary storage space of the original database instance using the idea of ​​lazy loading. If the original database instance needs to access the recovery data table during the loading process, the access request can be handled based on the temporary data table managed by the temporary database instance. After loading is completed, the recovery data table is replaced by the temporary data table loaded in the local temporary storage space to complete the data recovery. Furthermore, traditional thinking holds that using the FUSE process will lead to a decrease in database performance. However, this embodiment uses the FUSE process to implement data recovery, which enables the original instance to quickly restore data access after a recovery request is initiated. Compared to this, the impact of the decrease in database performance is acceptable.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this disclosure. Brief Description of the Drawings

[0010]

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this specification and, together with the specification, serve to explain the principles of this disclosure.

[0011]

[0011] FIG1 is a flowchart illustrating a data recovery method based on the user-mode file system FUSE according to an exemplary embodiment of this specification.

[0012]

[0012] FIG2A is a schematic diagram illustrating a local temporary storage according to an exemplary embodiment of this specification.

[0013]

[0013] Figure 2B is an application scenario diagram of a data recovery method based on the user-mode file system FUSE, as illustrated in this specification according to an exemplary embodiment.

[0014]

[0014] Figure 2C is an application scenario diagram of another data recovery method based on the user-mode file system FUSE, as illustrated in this specification according to an exemplary embodiment.

[0015]

[0015] Figure 3 is a hardware structure diagram of a computer device containing a data recovery device based on a user-mode file system FUSE, according to an exemplary embodiment of this specification. Detailed Description

[0016]

[0016] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0017]

[0017] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0018]

[0018] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when..." or "in response to determination".

[0019]

[0019] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.

[0020]

[0020] First, the concepts involved in the embodiments of this specification will be explained.

[0021]

[0021] 1. Database backup: refers to creating one or more copies of the database data through technical means to enhance the security and availability of the data.

[0022]

[0022] 2. Database recovery: refers to creating a usable database instance from backup data for users to use through technical means.

[0023]

[0023] In a database, a database instance typically refers to a running instance of a Database Management System (DBMS) after it starts up. It represents a database service process that the DBMS loads and runs in memory. A database instance includes the database data, transaction logs, cache, and related processes and threads used to handle user requests, perform database operations, etc.

[0024]

[0024] 3. Database table recovery: refers to restoring the database instance to a historical point in time by means of technical means.

[0025]

[0025] A database instance can manage one or more databases, and each database can contain one or more tables. Database and table recovery is not the recovery of the entire database instance, but a recovery operation performed on a database or table. Specific databases or tables can be selectively recovered as needed, without having to restore the entire database to a certain historical point in time.

[0026]

[0026] 4. A database page refers to a database that uses pages to manage and store data tables. Each page contains metadata and data content. Database read and write operations are performed according to the page dimension. Typically, the size of a database page is 16KB (kilobyte) or 8KB, etc.

[0027]

[0027] For example, when a database needs to read or write data from a storage medium, it performs I / O (Input / Output) operations on a page-by-page basis, loading the entire page into memory or writing it back to the storage medium. A data table can be divided into one or more pages for management.

[0028]

[0028] Taking a relational database as an example, the data in a table is actually stored in pages, and the data of each table is distributed across different pages. The database system uses mechanisms such as indexes to track which pages the data is stored on, so as to quickly access and manipulate the data.

[0029]

[0029] 5. Point-in-Time Recovery (PITR): refers to restoring the database to any point in history through various technical means.

[0030]

[0030] 6. Recovery Time Objective (RTO) refers to the time period from the moment the system crashes and services are interrupted after a disaster until the system recovers to the point where it can support the operation of various departments and services are restored. RTO is an indicator reflecting the timeliness of service recovery and embodies the longest recovery time that an enterprise can tolerate for its IT system. The smaller the RTO value, the stronger the recovery capability of the disaster recovery system, but the higher the enterprise investment.

[0031]

[0031] As an example, suppose a failure occurs at 1 p.m. and the RTO is set to 1 hour, then recovery needs to be completed before 2 p.m.

[0032]

[0032] 7. Recovery Point Object (RPO) refers to the point in time corresponding to the data recovered by the disaster recovery system after a disaster. RPO is an indicator reflecting the amount of data loss and embodies the maximum amount of data loss that an enterprise can tolerate. The smaller the RPO value, the less data loss the enterprise suffers, and the smaller the enterprise's losses.

[0033]

[0033] As an example, assuming the RPO is set to 5 hours, data backup must be performed every 5 hours.

[0034]

[0034] 8> Filesystem in Userspace (FUSE) can obtain all data of this file system from user-space programs, and perform read and write operations on metadata.

[0035]

[0035] Specifically, FUSE is a file system framework implemented in user space. With the support of the FUSE kernel module, users only need to implement specific file operations according to the interfaces provided by FUSE to create a file system. Before FUSE, file systems in Linux (an operating system) were entirely implemented in kernel space. Writing a file system with specific functions was inconvenient, both in terms of code writing and debugging. Even adding a small function to an existing traditional file system could still require a significant amount of work because it was implemented in the kernel. With the advent of the user-space file system FUSE, the workload can be greatly reduced, and debugging is much more convenient. When writing a FUSE file system, only the kernel needs to load the FUSE kernel module; there is no need to recompile the kernel.

[0036]

[0036] 9. Logical data replication in databases refers to reading data from a database through an SQL (Structured Query Language) interface and copying the logically structured data to another database. For example, based on the SQL interface, data such as tables, rows, or columns in the source database are read and then written to the target database based on the SQL interface.

[0037]

[0037] 10. Physical database data replication refers to backing up the physical files of the database using backup tools and then copying the files to other databases. This replication method directly manipulates the underlying physical files of the database, rather than processing data through an SQL interface.

[0038]

[0038] In the field of database backup and recovery, recovery has always been an important step. Database recovery methods can be roughly divided into several categories.

[0039]

[0039] The first method is database instance-level recovery, which restores the data to a new instance. This recovery scheme starts a new instance corresponding to the original instance, and restores all the data tables managed by the original database instance in the new instance. The new instance and the original instance are two different instances.

[0040]

[0040] The second type is database table-level recovery, which restores the data table to a new instance or to the original instance.

[0041]

[0041] The third type is database row-level recovery, which restores the row data in the table to a new table or the original table. Customers will choose different recovery solutions depending on different scenarios.

[0042]

[0042] In the field of data recovery, there is a specific metric for measuring recovery speed: RTO (Recovery Time Objective), which refers to the time from the start of recovery to its completion. In many scenarios, users are very sensitive to RTO, such as online emergency disaster recovery and real-time analysis of historical data. Moreover, improving recovery time can not only solve the pain points in these scenarios, but also greatly improve the user's backup and recovery experience in other recovery scenarios, such as cloning instances and restoring database tables.

[0043]

[0043] In the existing recovery solutions, the first full instance recovery solution allows customers to choose distributed cloud storage, which relies on the snapshots and lazy loading of the underlying cloud storage to restore to a new instance in seconds.

[0044]

[0044] The second type of database table-level recovery scheme can also use distributed storage to restore to a new instance, and can also restore to a new instance in seconds.

[0045]

[0045] The third row-level recovery scheme relies on the database's own multi-version or longer retention time transaction log (Undo Log), which can also achieve row-level data recovery in seconds.

[0046]

[0046] However, the second type of database table-level recovery solution cannot achieve rapid recovery for the requirement of restoring data tables to the original instance. Existing solutions for restoring data tables to the original instance use a combination of physical and logical replication to copy the recovered data back to the original instance. The recovery speed is limited by factors such as the hardware level of the original instance, deployment density, network bottlenecks, and instance performance, and the replication speed is often within hundreds of MB / s (megabytes per second). If the amount of data to be restored is large, the RTO may be on the order of hours, which is unacceptable to users.

[0047]

[0047] Among the related technologies, the three similar technologies are logical copy recovery, physical copy recovery, and volume data delayed loading second-level recovery.

[0048]

[0048] K logical replication recovery extracts data from the database using the most common SQL and then writes the data back to the target database using SQL. The advantage of this approach is that it is the most universal and can be adapted to multiple database versions. However, the disadvantage of this approach is that the speed of data recovery is relatively slow. In some common cases, the speed of logical replication is usually tens of MB / s, which cannot achieve data recovery replication in seconds.

[0049]

[0049] 2> Physical replication involves using backup tools to extract the database Page from the database file, and then importing the database Page into the database through data replication and some database import mechanisms. The advantage of this method is that the data extraction and import speeds can reach hundreds of MB / s or more. The disadvantage is that there are differences between some database versions of Page, making it unsuitable for all versions, and it cannot achieve data recovery replication completed in seconds.

[0050] 3. Volume data lazy loading and second-level recovery technology: This technology does not look at the database files at the database application level. From the perspective of volume storage, the database files are no longer visible. What you see is the volume data under the file system. This technology can restore a new volume from a volume snapshot in seconds. However, this recovery method is for fast recovery at the volume level that can achieve full database / full data, but it cannot achieve fine-grained recovery of data tables.

[0050]

[0051] In summary, existing technologies cannot quickly restore a data table to its original database instance. Therefore, this specification provides a data recovery method, as shown in Figure 1, which is a flowchart illustrating a data recovery method based on the user-space file system FUSE according to an exemplary embodiment. The method may include the following steps.

[0051]

[0052] In step 102, in response to receiving a recovery request to restore the original data table in the original database instance to a historical time, a temporary data table of the original data table at the historical time is restored through a temporary database instance corresponding to the original database instance, and an empty recovery data table corresponding to the temporary data table is created in the original database instance for access by the original database instance.

[0052]

[0053] In step 104, the temporary data table is loaded into the local temporary storage space of the original database instance through the Fuse process, and during the loading process, the access request of the original database instance to the recovery data table is processed based on the data in the temporary data table.

[0053]

[0054] In step 106, after loading is complete, the data table stored in the local temporary storage space replaces the recovery data table.

[0054]

[0055] As can be seen from the above embodiments, the data recovery scheme of this embodiment, upon receiving a recovery request, can first employ database instance recovery technology to recover a temporary database instance corresponding to the original database instance. This temporary database instance recovers the temporary data table of the original data table at a historical moment. Simultaneously, an empty recovery data table is created in the original database instance. Since the recovery data table is empty and there is no copy of the actual data, data access can be quickly restored within the original database instance. The temporary data table is then loaded into the local temporary storage space of the original database instance using lazy loading. If the original database instance needs to access the recovery data table during the loading process, the access request can be handled based on the temporary data table managed by the temporary database instance. After loading is complete, the recovery data table is replaced using the loaded temporary data table in the local temporary storage space, completing the data recovery. Furthermore, traditional thinking suggests that using the FUSE process will degrade database performance; therefore, efforts are made to avoid using the FUSE process to respond to database requests, preventing any impact on response speed. However, this embodiment uses the FUSE process to implement data recovery, which allows the original instance to quickly restore data access after the recovery request is initiated. Compared to this, the impact of the database performance degradation is acceptable. Using the FUSE process also reduces the workload of modifying the kernel.

[0055]

[0056] As an example, the data recovery method of this embodiment can be applied to any computer device, including but not limited to a single server, a server group consisting of multiple servers, or a cloud based on cloud computing consisting of a large number of hosts or servers. For example, this data recovery method can run on the host (which can be a physical machine or a virtual machine) where the original database instance resides, and is used to recover the data tables of the original database instance running on that host.

[0056]

[0057] This embodiment can be applied to data table recovery scenarios. For example, a user's original database instance may manage multiple data tables, some of which (one or more) may need to be recovered. Taking the recovery of data table A as an example, in these scenarios, since the user's service requires data table A, the user does not want data table A to be deleted, causing service interruption. Instead, they want to immediately recover a data table A identical to data table A, so that the original database instance can use the recovered data table A, thereby ensuring the normal operation of the service. Thus, when needed, the user can trigger a recovery request to restore data table A to a historical point in time.

[0057]

[0058] As an example, existing database instance recovery techniques can be used to recover a temporary database instance (hereinafter referred to as the temporary instance) corresponding to the original database instance (hereinafter referred to as the original instance for convenience). At this time, the temporary instance and the original instance are two different instances. The local storage of the temporary instance stores the temporary data table of the recovered original data table at a historical moment. Currently, the storage space storing the temporary data table is remote storage for the original instance.

[0059] The idea behind this embodiment is to ensure that the user's use of the original instance is not affected. Without waiting for all the data from the temporary data table to be transferred to the original instance, the original instance can start accessing the recovery data table, allowing the original instance to quickly use the recovered data. To achieve this, this embodiment creates an empty recovery data table in the original instance after the recovery request is initiated. This recovery data table is used to handle the access request from the original instance, and the processing of the access request is based on the data in the temporary data table.

[0058]

[0060] As an example, a database instance manages a data table by dividing it into at least one data page. An empty recovery table is created in the original database instance to replace the original data table. This recovery table can have the same number of pages as the temporary data table, and the pages in the recovery table can correspond one-to-one with the pages in the temporary data table. For example, the temporary data table can be divided into M pages, and the recovery table can also be divided into M pages, with a correspondence between the M pages in the temporary data table and the M pages in the recovery data table. However, the M pages in the recovery data table do not store any data content.

[0059]

[0061] There are several ways to create an empty recovery data table. For example, in the Linux operating system, an empty recovery data table can be created using the system call `fallocate()`. `allocate()` is a system call used to manipulate the pre-allocated space of a file. It can pre-allocate a certain amount of space for a file without actually writing any data. Therefore, in this embodiment, `fallocate()` can be called to make the file system allocate disk space of the same size as the temporary data table, thus creating a recovery data table of the same size as the temporary data table in the original instance, with data pages containing metadata but no data content.

[0060]

[0062] As an example, a local temporary storage space can be created within the original instance. Optionally, this local temporary storage space can be persistent storage, created within the persistent storage device corresponding to the original instance. The size of this temporary storage space must be larger than the size of the temporary data table. As an example, this local temporary storage space can be reclaimed after data recovery is complete.

[0061]

[0063] In step 104, as an example, various methods can be used to load data from the temporary data table into the local temporary storage space. For example, the temporary data table can be divided into multiple data pages, and one or more data pages can be loaded at a time. Some data pages from the temporary data table can be loaded randomly each time, or certain data pages can be loaded preferentially using a set method. This embodiment does not limit this approach. Before loading is complete, the original instance can access the recovery data table, which is currently empty. However, when this embodiment receives an access request, it can process it based on the data in the temporary data table.

[0062]

[0064] In some examples, a database instance manages a data table by dividing it into at least one data page, and the data page may contain metadata. Loading the temporary data table into the local temporary storage space of the original database instance may include: obtaining at least one temporary data page of the temporary data table currently to be loaded; obtaining the metadata of the recovery data page corresponding to the temporary data page from the recovery data table through the original database instance; updating the metadata of the temporary data page based on the metadata of the recovery data page; and loading the updated temporary data page into the local temporary storage space.

[0063]

[0065] In this embodiment, considering that in a real database system, the database manages data tables according to Pages, one data table can correspond to multiple pages; each page stores metadata and data content. The metadata may include the table's space ID, index ID, transaction ID, etc., as well as the checksum and verification of the data content stored in that Page. The metadata stored in the Pages corresponding to the empty restored data tables in the original instance is related to the current running state of the original instance. On the other hand, the temporary data tables managed by the restored temporary instance also have their own set of metadata in their Pages. This metadata differs from the metadata in the Pages of the empty data tables in the original instance. This is because, although the temporary data tables are also backup data backed up from the original database, they belong to a historical state and may have issues such as reused space IDs and index IDs, transaction IDs being historically smaller transaction IDs, or backup data originating from a standby database. This can lead to discrepancies between the metadata of the data pages in the temporary instance and the metadata of the data pages of the empty tables in the original instance.

[0064]

[0066] Therefore, in order to enable the data pages loaded in the local temporary storage space to be used by the original instance, the embodiments of this specification, when loading the data pages to be loaded from the temporary data table to the local temporary storage space, can use the Meta corresponding to the data pages to be loaded in the recovery data table to update the Meta of the data pages to be loaded. This allows the Meta contained in the updated data pages to be loaded to be directly used by the original instance, so that the original instance can use the loaded data pages without restarting, and can also directly use the data pages loaded in the local temporary storage space to respond to access requests.

[0065]

[0067] In practical applications, after creating an empty recovery data table in the original instance, the Meta of all data pages of the recovery data table can be obtained in advance, and a mapping relationship can be established between each data page of the recovery data table and the data pages of the temporary data table. This mapping can then be used during the loading process to update the Meta of the data pages to be loaded.

[0066]

[0068] As an example, the data page also includes data content, and the metadata includes a checksum, as well as at least one other piece of information besides the checksum; updating the metadata of the temporary data page based on the metadata of the restored data page may include: replacing the at least one other piece of information in the temporary data page with the at least one other piece of information in the restored data page; calculating the checksum based on the replaced at least one other piece of information and the data content in the temporary data page, and replacing the checksum in the temporary data page with the calculated checksum.

[0067]

[0069] In the database field, checksums are used to detect whether data pages have been corrupted or tampered with during storage or transmission. Checksums can be calculated byte-by-byte or in blocks of a certain size across all content of the data page, using methods such as cyclic redundancy check (CRC) or checksum algorithms. For example, an algorithm might iterate through the data page, taking the value of each byte or data block as input, and then generate a fixed-length checksum according to its calculation rules. This checksum is stored in the data page's metadata for use in subsequent reads or writes to verify data integrity.

[0068]

[0070] As an example, the metadata of Page_l of the recovered data table, including information 1 to information N and a checksum, is obtained in advance. Now, the metadata of Page_r of the temporary data table needs to be updated. Information 1 to information N of Page_f can be replaced with information 1 to information N of Page_l; that is, information 1 of Page_l is replaced with information 1 of Page_l, information 2 of Page_r is replaced with information 2 of Page_1, and so on. Next, the checksum is calculated using the replaced information 1 to information N of Page_l and the data content. Finally, Page_l... 5 Replace the checksum in the table with the calculated checksum, and complete Page 1. 1 Meta information updates.

[0069]

[0071] As an example, the data processing based on the temporary data table, which involves the original database instance accessing the restored data table, may include: in response to receiving a read request from the original database instance for a first data page of the restored data table, determining whether a temporary data page corresponding to the first data page has been loaded in the local temporary storage space; if so, responding to the read request using the temporary data page already loaded in the local temporary storage space; otherwise, responding to the read request based on the temporary data page corresponding to the first data page in the temporary data table managed by the temporary database instance; and in response to receiving a write request from the original database instance for a second data page of the restored data table, writing the second data page to the local temporary storage space.

[0070]

[0072] As an example, the access requests of the original instance can include both read and write requests. In actual applications, based on different usage scenarios, a processing scheme for only one type of access request can be implemented as needed, i.e., only read or only write is supported; or a processing scheme that supports both types of access requests can be implemented.

[0071]

[0073] ① For read requests from the original instance, one approach is to directly utilize the data from the remote temporary data table to respond to all read requests. However, since this embodiment continuously loads data from the temporary data table into the local temporary storage space, to improve the response speed of some read requests, if the data requested by the read request is already loaded into the local temporary storage space, the already loaded data in the local temporary storage space can be used to respond to the read request; otherwise, the remote temporary data table is used to respond. Therefore, this embodiment can improve the response speed of read requests.

[0072]

[0074] ② For write requests to the original instance, the data can be written directly to the local temporary storage space.

[0075] In this embodiment, write requests for the original instance can be directly written to the local temporary storage space. These write requests may involve adding new data to existing data in the temporary data table, or performing a write operation on an existing data page of the temporary data table; both scenarios allow for direct writing to the local temporary storage space.

[0073]

[0076] For example, the temporary data table and the recovery data table are corresponding, and it is assumed that they both have 100 data pages. The original instance's write request to the recovery data table may add a new data page, that is, the recovery data table will increase to 101 data pages. This newly added data page can be directly written to the local temporary storage space.

[0074]

[0077] In other cases, the original instance's write request to the recovery table is a write operation on an existing data page of the temporary table. The local temporary storage space may or may not have loaded the data page. Regardless of whether it is loaded or not, it can be directly written to the local temporary storage space because the temporary table contains old data, and a write operation has occurred. Direct writing will make the local temporary storage space store the new data written by the original instance to the recovery table.

[0075]

[0078] Therefore, this embodiment ensures that the original instance's newly written data to the recovery data table is stored in the local temporary storage space for write requests. In step 106, this written data, as part of the data table, can replace the recovery data table. Therefore, this embodiment also supports the original instance's write function to the recovery data table.

[0076]

[0079] Figure 2A is a schematic diagram of a local temporary storage space according to an exemplary embodiment of this specification. As shown in Figure 2A, the local temporary storage space can store data pages. These data pages include both data pages loaded from the temporary data table and data pages newly written during the original instance's write operations on the restored data table. When a read request is received, if the data page to be read has already been loaded, the local temporary storage space can be used to respond to the read request. Of course, during the loading process, some data pages from the temporary data table are not yet loaded into the local temporary storage space. In Figure 2A, these unloaded data pages in the temporary data table are represented by dashed lines. If the original instance needs to read these data pages, the temporary instance can respond to the read request for these unloaded data pages.

[0077]

[0080] In some examples, after the step of loading the updated temporary data page into the local temporary storage space, the method further includes: recording the loaded status of the updated temporary data page in preset loading status data; determining whether the temporary data page corresponding to the first data page has been loaded in the local temporary storage space includes: using the loading status data to query whether the temporary data page corresponding to the first data page in the temporary data table has been loaded into the local temporary storage space.

[0078]

[0081] This embodiment also maintains a loading status data set, which records whether each data page in the temporary data table has been loaded into the local temporary storage space. As an example, this loading status data can be stored in memory for fast querying when needed.

[0079]

[0082] The data structure for loading status data can be flexibly configured according to actual needs. As an example, the data structure for loading status data can be a bitmap, and the size of the bitmap can be determined based on the number of pages in the temporary data table. For each page in the temporary data table, it can be mapped one-to-one with each binary bit of the bitmap, using "1" and "0" to represent whether the data page has been loaded. Based on this, it is possible to quickly query whether each data page of the temporary data table has been loaded in the local temporary storage space, thus making the processing of read requests more efficient. As an example, in practical applications, before loading begins, loading status data can be created in memory to represent that all data pages of the temporary data table are in an unloaded state; for example, if the temporary data table has M pages, a bitmap containing M bits, each bit being 0, can be created.

[0080]

[0083] As an example, load state data can be stored in the original instance's memory space, so that the data can be read from memory when needed, thereby improving query efficiency. To prevent data loss in memory, load state data can also be periodically written to the original instance's persistent storage space, such as the local temporary storage space mentioned above, as needed.

[0081]

[0084] Regarding load status data, in practical applications, updating the load status data can take several forms. In some examples, it might involve updating a data page from a temporary data table after it has been loaded into the local temporary storage space. In other examples, it might involve the original instance performing a write operation on a data page from the temporary data table. The role of the load status data, in some embodiments, is to determine whether the data page to be read is already stored in the local temporary storage space when a read request occurs. In other embodiments, it can be used to determine whether the data page is already stored in the local temporary storage space before loading it. This is because if a write operation has already occurred on a data page from the original instance, the original instance's write operation can be used as the reference, and loading that data page from the temporary data table can be terminated. The following will describe these embodiments.

[0082]

[0085] In some examples, after the step of writing the second data page to the local temporary storage space, the method may further include: if the temporary data table has a temporary data page corresponding to the second data page, recording the loaded status of the temporary data page corresponding to the second data page in the loading status data.

[0083]

[0086] In this embodiment, the second data page of the recovery data table targeted by the write operation may correspond to an existing data page of the temporary data table. For example, the temporary data table contains Page_r, and the current original instance wants to perform a write operation on Page_l of the recovery data table. Page_l corresponds to Page_l'. In the case of maintaining load status data as described in the previous embodiment, the loaded status of the data page Page_r can also be recorded in the load status data. Here, because before the original instance writes to Page_l, Page_r of the temporary data table may have already been loaded in the local temporary storage space, or it may not have been checked. Therefore, the loaded status of the data page Page_r may or may not have been recorded in the load status data. In practical applications, first querying the load status data to see if the loaded status of Page_r is recorded, and then writing it if it is not recorded, is an optional solution; or directly recording the loaded status of Page_l' to the load status data without querying the load status data is also an option. Therefore, this embodiment enables the data page targeted by the write request to be queried by the loading status data when a read request occurs, so that the data page has been loaded into the local temporary storage space without accessing the remote storage, thereby making the processing of read requests more efficient.

[0084]

[0087] In other examples, when performing a write operation on an existing data page of a temporary data table, since the temporary data table contains historical data and the current write operation is the latest one, the latest write operation must prevail. However, the current write operation may precede the loading operation of the temporary data table's data page; therefore, the loading operation of the temporary data table's data page should not overwrite the original instance's write operation. Based on this, writing the updated temporary data page to the local temporary storage space can include: querying the loading status data to see if the updated temporary data page has already been loaded into the local temporary storage space; if not, writing the updated temporary data page to the local temporary storage space; if yes, discarding the updated temporary data page.

[0085]

[0088] As an example, at 12:00:00, a write operation occurred to data page Page_2 in the recovery data table, and the temporary data table contains the corresponding data page Page_2. , Page_2 can be written to the local temporary storage space; and Page_2 can be updated in the loading status data to be in a loaded state; at 12:01:00, Page_2, as a data page to be loaded, is ready to be loaded into the local temporary storage space; in this embodiment, by first querying the loading status data, it can be found that Page_2 is in a loaded state, so the loading of Page_2 is directly terminated, ensuring that the local temporary storage space stores the latest data written by the original instance, and ensuring the normal use of the recovery data table by the original instance.

[0086]

[0089] Therefore, by designing the loading status data, we can improve the efficiency of reading data on the one hand, and at the same time ensure that the local temporary storage space stores the latest data written by the original instance, thus ensuring the normal use of the recovery data table by the original instance.

[0087]

[0090] In some examples, after the step of writing the updated temporary data page to the local temporary storage space, the method further includes: accessing loading status data stored in the memory space of the original database instance, recording the loaded status of the updated temporary data page in the loading status data, and then writing the loading status data to the local temporary storage space.

[0088]

[0091] The loading status data stored in the local temporary storage space is used by the FUSE process to obtain the loading status of the temporary data pages in the temporary data table after the FUSE process restarts.

[0092] As an example, the load state data maintained in memory can be flushed to the local temporary storage space at set intervals as needed, and can also be flushed to the local temporary storage space after each write operation to prevent data loss and ensure data consistency. In this embodiment, if a data page loading operation of the temporary data table occurs, the load state data will also be updated, and then the load state data in memory will be written to the local temporary storage space in a timely manner. Thus, if the process in this embodiment loses the load state data in memory due to a crash or other reasons, the load state data will not be lost because it is still stored in the local temporary storage space. Therefore, if the FUSE process is closed, the memory space belonging to the FUSE process will be reclaimed by the operating system, and the load state data stored in the memory space will be lost. However, using this embodiment, after the FUSE process restarts, it can access the load state data stored in the local temporary storage space, allowing the FUSE process to obtain the load state of the temporary data pages in the temporary data table.

[0089]

[0093] In some examples, recording the loaded status of the temporary data page corresponding to the second data page in the loaded status data may include: accessing the loaded status data stored in the memory space of the original database instance, recording the loaded status of the temporary data page corresponding to the second data page in the loaded status data, writing the loaded status data to the local temporary storage space, and returning a write success message corresponding to the write request.

[0090]

[0094] When a write operation from the original instance is detected, in this embodiment, a write success message is only returned after three operations are completed: writing the data page to the local temporary storage space, updating the load status data in memory, and writing the load status data in memory to the local temporary storage space. This process also prevents the loss of data already written to the temporary data table by the original instance.

[0091]

[0095] In practical applications, database instances can be configured as one primary and one standby, or one primary and multiple standby instances, to ensure high availability of the database service. As an example, in the case of standby database instances, the primary database instance may include a primary database instance and at least one standby database instance. Specifically, at least one standby database instance is given priority in starting the FUSE process. Then, the standby database instance that started the FUSE process and the primary database instance undergo a primary-standby switchover. The standby database instance after the switchover is then restarted with the FUSE process.

[0092]

[0096] Taking a primary and a backup instance (Instance A and Instance B) as an example; the user is currently using Instance A as the primary instance. First, an empty table can be created in the primary instance A, and operations on Instance A will be synchronously backed up to the backup instance B. Next, Instance B is shut down, the FUSE process is started, and finally, Instance B is restarted. After Instance B restarts, a temporary data table can be mounted to begin data recovery. Because the user is currently using the primary instance, the user will not be aware of the restart of Instance B.

[0093]

[0097] Instance B has already started the FUSE process and successfully mounted the temporary data table in the persistent storage space, meaning instance B can now use the data in the temporary data table. However, the primary instance has not yet completed this operation. Therefore, this embodiment will perform a primary-standby switchover. Thus, instance B is currently the primary instance, and instance A is the standby instance. Instance A, currently the standby instance, also performs the same operations as instance B: after shutdown, it starts the FUSE process, thereby mounting the temporary data table. Therefore, the above operations involve restarting the instance, but this will not affect the user experience of the original instance, nor will it affect the service of the original instance.

[0094]

[0098] The same principle applies to a primary-multiple-standby scenario. One standby instance can be selected to start the FUSE process first. After the selected standby instance starts the FUSE process, it performs a primary-standby switchover with the primary instance. The standby instance after the switchover then starts the FUSE process. The other standby instances simply need to perform the FUSE process startup operation.

[0095]

[0099] As an example, loading the temporary data table into the local temporary storage space of the original database instance, and processing the original database instance's access request to the recovery data table based on the data in the temporary data table during the loading process, may include: running a preset file system on the host where the original data instance resides, the preset file system being configured to execute a FUSE process, and the preset file system loading the temporary data table into the local temporary storage space of the original database instance; and processing the original database instance's access request to the recovery data table based on the data in the temporary data table during the loading process; the method further includes: unloading the preset file system after loading is completed.

[0096]

[0100] In this embodiment, since the recovery data table is empty before the temporary data table is fully loaded, specific processing is required for access requests. Considering that access to the data table by the original instance can be implemented according to the existing scheme after the data recovery is complete, in order to reduce resource consumption, this embodiment can implement a preset file system for handling access requests from the original instance to the recovery data table. The preset file system is configured to execute the FUSE process. During the loading process, the preset file system can handle access requests. After loading is completed, the preset file system can be unloaded, thereby reducing the resource consumption on the host where the original instance is located.

[0097]

[0101] In practical applications, database instances can be configured as a primary-standby or primary-multiple-standby setup to ensure high availability of the database service. As an example, with standby database instances, the primary database instance may include a primary database instance and at least one standby database instance. The step of running a preset file system on the host where the primary database instance resides includes: creating a corresponding remote storage space for each instance included in the primary database instance, and copying the temporary data table from the remote storage space; restarting the current standby database instance, and after running the preset file system on the host where the current standby database instance resides, mounting the temporary data table from one of the remote storage spaces to the preset file system; triggering a primary-standby switchover between the current primary database instance and the current standby database instance, restarting the switched standby database instance, and after running the preset file system on the host where the switched standby database instance resides, mounting the temporary data table from another of the remote storage spaces to the preset file system.

[0098]

[0102] As an example, after restoring the temporary database instance, the temporary database instance manages the restored temporary data tables, which are stored in the persistent storage device corresponding to the temporary database instance. In this embodiment, the temporary database instance can be shut down first, and then the data of the temporary data tables stored in the persistent storage device can be copied to other different persistent storage devices. The number of copies is consistent with the number of primary and secondary databases in the original database instance. For example, if there are n primary and secondary databases, then n copies are made to n different persistent storage spaces (such as solid-state drives or other storage devices). Because the original database instance is running, the temporary data tables stored in these n different persistent storage spaces are not yet used by the original database instance. Therefore, some operations are needed to mount the data without affecting the user experience of the original instance.

[0099]

[0103] Taking a primary and a backup instance (Instance A and Instance B) as an example; the user is currently using Instance A as the primary instance. First, an empty table can be created in the primary instance A, and operations on Instance A will be synchronously backed up to the backup instance B. Next, Instance B is shut down, a default file system is started, and finally, Instance B is restarted. After Instance B restarts, the default file system can mount a temporary data table from one of the persistent storage spaces. Since the user is currently using the primary instance, the user will not be aware of the restart of Instance B.

[0100]

[0104] The current instance B has already started the default file system and successfully mounted the temporary data table in the persistent storage space, meaning that the default file system of instance B can use the data in the temporary data table; however, the current primary instance has not yet completed this operation; therefore, this embodiment will perform a primary-backup switch; thus, instance B is currently the primary instance, and instance A is the backup instance.

[0101]

[0105] Therefore, instance A, currently serving as the backup instance, also performs the same operations as instance B: after shutdown, it starts the preset file system and restarts, thereby mounting a temporary data table in another persistent storage space. Thus, while the above operations involve restarting the instance, they do not affect the user experience of the original instance, nor do they affect the original instance's services.

[0102]

[0106] In scenarios with multiple standby instances, one primary and one standby instance employs the primary-standby processing method described in the previous embodiments. The other standby instances follow the same standby instance processing flow, performing shutdown, restart, and running the FUSE process to mount the corresponding temporary data tables in the persistent storage space. Thus, both the primary and standby instances run a preset file system and mount temporary data tables. The preset file system in the primary instance can load temporary data tables while processing access requests to the recovery data table, and the preset file system in each standby instance can also load temporary data tables. In a primary-standby scenario, the primary instance's operations on the recovery data table can also be synchronously backed up to each standby instance.

[0107] Figure 2B is a schematic diagram of a data recovery scenario illustrated in this specification according to an exemplary embodiment. In this embodiment, the original database instance (hereinafter referred to as the original instance) is described using a primary and a backup example, and will be referred to as the primary instance and the backup instance below. The data recovery process may include the following steps.

[0103]

[0108] Step 1: A temporary database instance (hereinafter referred to as a temporary instance) can be quickly created using data recovery technology. The temporary database instance manages the recovered temporary data tables.

[0104]

[0109] Step 2: After a successful recovery, the temporary instance can be shut down, and preparations can be made to mount the temporary data table to the original instance.

[0105]

[0110] Step 3: The first phase of the instance recovery process.

[0106]

[0111] 3.1. Start restoring the data of the original instance; First, you can shut down the standby instance, mount a temporary data table in the standby instance's default file system, and then start the standby instance.

[0107]

[0112] Optionally, a temporary data table can be mounted. Specifically, after shutting down the temporary instance, the temporary data table can be copied and stored in a preset storage space. In this embodiment, taking an original instance with two backups as an example, two copies can be made. That is, as shown in the figure, the temporary data table managed by the temporary instance is copied to two different hard drives, such as solid-state drives.

[0108] (SolID State Drive, SSD, etc.), the number of copies of the temporary data table corresponds to the number of original instances. After copying, the storage space for storing the temporary data table is used to mount it to the original instance. For example, one of the temporary data tables is first mounted to the current standby instance, as shown on disk 2 in Figure 2C. The temporary data table in the other disk 1 is mounted to the current primary instance in a subsequent step.

[0109]

[0113] 3.2 After the standby instance has recovered, a primary / standby switchover is triggered, which can prevent users from noticing the instance restart.

[0110]

[0114] Specifically, in this embodiment, the primary and backup instances are switched over after the current backup instance is mounted.

[0111]

[0115] 3.3. Shut down the current standby instance, mount the temporary data table on disk 1 in the default file system of the current standby instance, and start the current standby instance.

[0112]

[0116] After completing the above steps, both the primary and standby instances have been mounted and restored. Users will then be aware that the data tables of the original instance have been restored and can begin using them. However, at this point, the restored data tables are empty; the temporary data tables have not yet been fully copied from the SSD to the original instance's local file system. If the original instance needs to access data in the restored data tables, the access request will be handled through the preset file system of this embodiment.

[0113]

[0117] Step 3, the first stage of the instance recovery process, has a relatively fixed processing time because no actual data copying occurs. The data recovery time perceived by the user is independent of the amount of data in the table. Therefore, from a user experience perspective, the data table appears to be recovered quickly.

[0114]

[0118] Step 4: The second phase of the instance recovery process.

[0115]

[0119] 4.1> During this phase of the process, both the primary and standby instances will continuously load temporary data tables into their respective local temporary storage spaces. If the amount of data in the temporary data tables is large, the waiting time in this phase may be relatively long.

[0116]

[0120] Once the data replication for the standby instance is complete, you can shut down the standby instance (i.e., the standby instance process on the host machine where the standby instance resides is terminated), unmount the preset file system (i.e., unmount the preset file system on the host machine where the standby instance resides), and move all the data stored in the local temporary storage space of the preset file system to the local file system of the original instance. In other words, move all the pages stored in the local temporary storage space to the directory where the recovery data table is located. Finally, restart the standby instance to complete the unmounting process.

[0117]

[0121] 4.2 Triggering a primary / standby switchover can prevent customers from noticing the primary instance restarting.

[0118]

[0122] Through primary-standby failover, the standby instance that has completed data replication before the failover will become the current primary instance.

[0119]

[0123] 4.3 Wait for the current standby instance to complete data replication; Similarly, the process first uses the preset file system to complete data replication. If replication is complete, shut down the current standby instance, unmount the preset file system, move all data stored in the local temporary storage space of the preset file system to the local file system of the original instance, and finally restart the current standby instance to complete the unmounting.

[0120]

[0124] Therefore, it can be seen that the second-stage instance recovery process can be completed without the user's awareness. After copying the data to the local temporary storage space using the preset file system, the original instance can restore the way it accesses data through the local file system, and the entire recovery process ends.

[0121]

[0125] Figure 2C illustrates another data recovery scenario according to an exemplary embodiment of this specification. As an example, the default file system in this embodiment can be implemented based on FUSE. FUSE (Filesystem in Userspace) is a mechanism that allows non-privileged users to implement a file system in user space. Traditionally, file system implementations need to run in kernel space, while FUSE allows developers to write file systems in user space without modifying the operating system's kernel code. With FUSE, developers can write file systems using standard programming languages ​​without needing in-depth knowledge of kernel programming. FUSE provides a user-space library and a set of interfaces that developers can use to implement custom file system logic. Once the file system is mounted, the user-space file system program can handle file system operations, such as reading, writing, creating, or deleting files.

[0122]

[0126] In related technologies, the FUSE user-space file system can be used to proxy the read / write I / O and some other I / O operations of the upper-layer application system. This allows the upper-layer application system to be unaware of the changes. However, using the FUSE user-space file system will result in a performance degradation of the database system, which is approximately 10%-20%. In the data table recovery scenario of this embodiment, some database tables have data problems. If FUSE is used, although it will reduce the read performance of these database tables, this cost is acceptable in scenarios where users urgently need to recover data. Of course, in practical applications, it is also possible to implement this embodiment in other ways. For example, the processing method of this embodiment can be added to the file system function of the operating system. For example, the aforementioned steps of "loading the temporary data table into the local temporary storage space of the original database instance, and processing the access request of the original database instance to the recovery data table based on the data of the temporary data table during the loading process" do not have to be implemented by the FUSE process. They can be implemented by improving the file system function of the operating system. This embodiment does not limit this.

[0123]

[0127] In Figure 2C, the left side shows the original instance (which can be the primary instance or the backup instance) in the aforementioned embodiment. For the sake of illustration, only one instance is shown in Figure 2C as an example. An empty recovery data table has been created in this original instance.

[0124]

[0128] Using the preset file system in this embodiment, the database files managed by the temporary instance can be mounted to the original instance within seconds. The function of the preset file system is to allow the original instance on the left to access data in the data tables of the temporary instance on the right, while the data in the data tables is loaded lazily, so that the temporary data tables managed by the temporary instance on the right are gradually loaded into the local temporary storage space of the original instance without the user's awareness.

[0125] (local temp store).

[0126]

[0129] As an example, in a data recovery scenario, a user might initiate a recovery request to restore the original data tables managed by the original instance to a historical point in time while using the original instance.

[0127]

[0130] Existing database recovery techniques can be used to create a temporary database corresponding to the original database. The temporary database manages temporary data tables, which are the original data tables at historical points in time.

[0128]

[0131] Create a new empty data table in the original instance. This empty data table will replace the original data table.

[0129]

[0132] Create a local temporary storage space in the original instance to load data from the temporary data table.

[0130]

[0133] To ensure users perceive rapid data recovery, this embodiment creates an empty recovery data table in the original instance. Therefore, the original instance may access the recovery data table, and the default file system needs to utilize data from the temporary instance to respond to access requests. Thus, how the default file system responds to these access requests is a key challenge.

[0131]

[0134] Typically, databases manage data tables using pages, with one table corresponding to multiple pages. Each page stores metadata and data content. Meta information may include the table's space ID, index ID, transaction ID, etc., as well as the checksum of the data content stored in that page. The metadata stored in the pages corresponding to empty tables in the original instance is related to the current running state of the original instance. On the other hand, since the temporary data table is restored from a temporary instance, its page also has its own set of metadata, which differs from the metadata in the pages of the empty data table in the original instance. This is because, although the temporary data table is also backup data from the original database, it belongs to a historical state and may have issues such as reused space IDs and index IDs, transaction IDs from smaller historical transactions, or backup data originating from a standby database. This results in the metadata in the temporary database not matching the metadata of the empty table in the original database. Therefore, if the file is directly mounted, the original instance may not be able to recognize the temporary data table in the temporary instance due to the mismatch in metadata.

[0132]

[0135] Therefore, the solution to this problem in this embodiment is that although the Meta information of the Page in the original instance and the Meta information of the Page in the temporary instance cannot match, the data content stored in the Page is the same. Therefore, in this embodiment, an empty table is created for the original instance to access. In the original instance's read I / O process for the empty table, the Meta information of the Page data read by the read I / O is converted to the Meta information of the original instance, and finally the checksum is recalculated. The I / O reads the Page data and returns the converted data. After the data conversion, the original instance can read the data content in the Page normally according to the Meta in the Page.

[0133]

[0136] Please see Figure 2C. As an example, the default file system can contain the following functional modules.

[0134] 1. Data Page Determination Module (Page align)

[0135]

[0137] The original instance performed data operations on the data table based on the database Page. For example, it performed data meta transformations based on the Page, and determined whether the data was located in the local temporary directory or in a remote temporary database.

[0136]

[0138] The access requests from the original instance received by the default file system are actually sent from the original instance's local file system to the default file system. The granularity of data partitioning in the local file system typically differs from that in the database; for example, the local file system typically partitions data at a 4KB granularity, while the database typically partitions data at a 16KB granularity, and so on.

[0137]

[0139] Taking MySQL (My Structured Query Language) as an example, the data page size of MySQL's storage engine is set to 16KB, while the local file system's partition granularity is 4KB. Since it's necessary to determine which data page of the table each access request targets, the access request can be transformed based on MySQL's granularity. Assuming the access request from the local file system has an offset of 5KB and a length of 9KB, it can be transformed into a request with an offset of 0KB and a length of 16KB, thus determining the specific data page targeted by the access request.

[0138]

[0140] In practical applications, different database systems may have different settings for data page size. Therefore, to support different types of database systems, the preset file system in this embodiment can pre-configure different access request conversion rules based on the data page size settings of different database systems. For example, this module can also implement conversion rules for other types of database systems such as MongoDB and Postgres. In practical applications, other types of database systems can also be used, and this embodiment does not limit this.

[0139] 2. Data location determination module (DataBase Page location)

[0140]

[0141] After the Page align module is used, all access requests are converted into requests that are aligned with the database Page. In this embodiment, the module will determine the position of the Page that the access request wants to access.

[0141]

[0142] As a processing mechanism, data pages in a temporary data table need to be lazily loaded. For example, if a temporary data table has M pages, these M pages can be identified, such as 1 to M. The page that the access request wants to access may or may not be the data page corresponding to the temporary data table (if it's not, then the access request accesses a page added to the restored data table in the original instance). Therefore, it is necessary to determine whether the page that the access request wants to access belongs to one of the M pages that need to be lazily loaded.

[0142]

[0143] Because the temporary data table, already restored in the remote temporary instance, will not have any new data added, several methods can be used to determine whether the requested page belongs to the page that requires lazy loading. Since the number of data pages in the temporary data table is fixed, the access address carried in the write request can be used to determine whether the requested data page is within the range of the temporary data table's data pages. For example, the access request can be used to determine if it carries an offset and a length, and since the temporary data table also has a length, this can be used to calculate whether the requested data page is within the length range of the temporary data table.

[0143]

[0144] For write requests, if yes, it's necessary to mutually exclude the currently written Page and the asynchronously lazy-loaded Page to prevent the lazy-loaded Page from overwriting the newly written Page. If no, it means the currently written Page is a newly added Page, so mutual exclusion between the current write and lazy loading is not necessary, because the currently written Page is not in the remote temporary data table.

[0144]

[0145] In addition, this embodiment also maintains a loading status data bitmap in memory. The bitmap records the loading status of each page in the temporary data table. The loading status indicates whether each page is in a remote temporary data table or in the local temporary storage space. The bitmap in memory is also periodically persisted to the local temporary storage space. If the process of the preset file system crashes, the bitmap can be found in the local temporary storage space after restarting, preventing the page loading status from being lost after the preset file system process crashes and restarts. For pages newly added to the recovery data table in the original instance, there is no need to record the loading status in the loading status data.

[0145]

[0146] For read requests, if the page to be read is a lazy-loaded page, its location can be determined based on the bitmap in memory. If it's in local temporary storage, the data is read from that local temporary storage. If the page isn't loaded, it means it's in a remote temporary data table, and the read request is forwarded directly to request data from the remote table. If the page isn't a lazy-loaded page, it means the data to be read is a newly written page, and it can be read directly from local temporary storage. Therefore, this method improves the processing efficiency of read requests.

[0146] 3. Data page metadata conversion module

[0147]

[0147] As an example, this module can load the Meta of the empty table of the original instance and the Meta of the remote temporary data table; the Page read from the remote temporary data table, whether the Page to be loaded is loaded from the remote temporary data table into the local temporary storage space, or the Page is read from the temporary data table based on a read request, can use this module to convert the metadata of the Page read from the temporary data table.

[0148]

[0148] As an example, this module can support one or more different types of database systems; different types of database systems may have different designs for the fields of metadata in Page, and different metadata conversion methods can be implemented in advance according to the database systems to be supported. Specific metadata conversion methods can be referred to the description of the foregoing embodiments. As an example, this module can be designed as a plug-in architecture, and the corresponding metadata conversion method can be selected according to the type of database instance in the actual application scenario. As an example, the conversion process can be to replace all fields in the Meta of the temporary data table's Page except for the checksum with the corresponding fields in the Meta of the empty table's Page; after replacement, the checksum is calculated and updated based on the data content of the temporary data table's Page and the replaced fields.

[0149] 4. Lazyload module

[0150]

[0149] The lazy loading module in this embodiment is used to load the data pages of the temporary data table into the local temporary storage space of the original instance. It can be seen that the lazy loading in this embodiment is the loading of the data table, so it belongs to the file-level lazy loading.

[0151]

[0150] As an example, a temporary data table that requires delayed loading can be divided into N segments. The specific segmentation method can be flexibly configured according to actual needs. Each time a segment of data is loaded, the size of each of the N segments can be the same or different. This embodiment does not limit this. For example, each segment can be the same size as a data page, so the temporary data table is divided into as many segments as it has data pages. A larger segmentation granularity can also be set. For example, the size of each segment can be a multiple of the data page, so that each loading loads one or more data pages. Of course, this embodiment does not limit the specific multiple. As an example, a loading task (e.g., thread or process) can be implemented. The loading task is used to load one or more data pages each time. After completing the loading of these data pages, the loading task retrieves one or more data pages from the temporary data table and loads them again.

[0152]

[0151] In practical applications, during loading, one or more data pages need to be loaded into a certain range of locations in the local temporary storage space, and the original instance may also have access requests for that data page. Therefore, in order to maintain data consistency and integrity, a locking mechanism can be used. For example, when storing one or more data pages to be loaded in the local temporary storage space, these storage locations need to be locked; or, when the original instance accesses some data pages stored in the local temporary storage space, the storage location of the accessed data page also needs to be locked. Therefore, the number of segments can also be considered in conjunction with the locking mechanism. If the switching is fine-grained, more locks need to be maintained; if the segmentation granularity is large, the response speed of the original instance's access request may be slow when concurrent access occurs.

[0153]

[0152] During the lazy loading process, the aforementioned Bitmap can also be used in conjunction with the design. If a data page to be loaded is recorded as "loaded" in the Bitmap, it indicates that the original instance has already written to the data page. Therefore, it does not need to be loaded and can be discarded directly to avoid old data overwriting newly written data. If the Bitmap indicates an unloaded state, the data page to be loaded can be loaded into the original instance's local file temporary storage space through the aforementioned metadata conversion module, while marking the loaded state of this Page in the Bitmap.

[0154]

[0153] As another example, lazy loading can employ several strategies. For instance, the N segments can be loaded sequentially in a specific order, which can be set as needed, such as a random order or an order from the first segment to the last segment.

[0155]

[0154] Alternatively, some data pages that the user may access can be predicted, and these pages can be loaded first. There are various prediction methods. For example, the pages that the user operated on in the original data table before initiating the recovery request can be obtained as the pages to be loaded first. Alternatively, the pages that are frequently used can be identified based on the historical usage records of the original data table, etc.

[0156]

[0155] Alternatively, the indexes created by the temporary instance for the temporary data table can be loaded first. As an example, taking the InnoDB storage engine of MySQL, when the database instance manages the data table, it uses a B+ tree to implement the index structure of the data table, and the index is also stored on the disk in the form of an index file. Furthermore, in InnoDB, the data table itself is an index structure organized by a B+ tree, and the leaf node data field of this tree stores the complete data record. The key of this index is the primary key of the data table, so the InnoDB table data file itself is the primary index, and the leaf nodes in the B+ tree contain the complete data record. When searching for certain data records in the data table, the search will start from the root node of the tree and continue down to the leaf node. Therefore, the non-leaf nodes in the B+ tree have a high probability of being used. In this embodiment, the B+ tree corresponding to the temporary data table can also be obtained, and the non-leaf nodes from the root node to the leaf node in the B+ tree can be loaded first, that is, loaded sequentially from the primary key index to the non-primary key index. Therefore, the index node is used very frequently and can be loaded into the local temporary storage space first. This allows the non-leaf nodes of the B+ tree loaded in the local temporary storage to be used when a read request for the original instance occurs, so as to query the Page that needs to be read in the read request more quickly and thus respond to the read request more efficiently.

[0157] 5. Local temporary storage space

[0158]

[0156] In this embodiment, the local temporary storage space can store two parts of data: Bitmap and deferred-loaded Page. The data location determination module can periodically flush the Bitmap in memory to this storage space. Newly written data from the original instance and data pages loaded by the deferred-load module can also be stored here. As time progresses, the number of deferred-loaded Pages stored in the local temp store will increase. When all deferred loading is completed, the instance can be restarted using the process mentioned in the aforementioned embodiment, and all Pages stored here can be used to restore the data table.

[0159]

[0157] As an example, whether the data pages of the temporary data table have been loaded can be determined by a Bitmap. If all bits in the Bitmap are 1, it means that the data pages of the temporary data table have been loaded. After loading is complete, the data pages stored in the local temporary storage space can be used to replace the recovery data table. It can be understood that in practical applications, after loading is complete, all the pages stored in the local temporary storage space correspond to all the pages of the temporary data table if the original instance has not performed any write operations on the recovery data table. If the original instance has performed any write operations on the recovery data table, and these write operations are only for the existing data tables of the recovery data table, then all the pages stored in the local temporary storage space also correspond to all the pages of the temporary data table, only the data content stored in the pages may change. If the original instance has write operations on the recovery table, but these write operations include adding new pages to the recovery table, then all pages stored in the local temporary storage space, in addition to those corresponding to the temporary table, also include the newly added pages. These newly added pages and the pages corresponding to the temporary table together constitute a new table and are used to replace the recovery table.

[0160]

[0158] In this embodiment, the preset file system and database can work together to ensure that data is not lost and that data consistency is guaranteed. Because the original instance will continue to have service writes, and this written data will also be written to the local temp store, it is necessary to ensure that the newly written data in the database is not lost due to the crash of the preset file system process.

[0161]

[0159] The database itself has a WAL mechanism; in database systems, the Write-Ahead Logging (WAL) mechanism is a common persistence technique used to ensure the durability and recovery capabilities of database transactions. The WAL mechanism guarantees the atomicity, consistency, and durability of transactions by writing data to a log file before writing data. Specifically, when there is a new write operation in the database, the WAL mechanism will process it according to the following steps.

[0162]

[0160] Writing to the WAL log: Before performing transaction write operations, the database first records these operations in the WAL log file in the form of a log. These logs contain the specific operations of the transaction, such as update, insert, or delete operations.

[0163]

[0161] Update the dirty page list: At the same time, the modified data pages will also be marked as "dirty pages", indicating that the content of these data pages has been modified but has not yet been persisted to disk.

[0164]

[0162] Continuous Dirty Page Flush Logic: The database continuously writes data from dirty pages back to the corresponding data files according to a certain strategy. This process is called "dirty page flushing logic," which ensures data durability and consistency.

[0165]

[0163] In addition, the database system will periodically perform checkpoint operations, which write the data in memory to the disk, advancing the database recovery point to reduce the amount of logs that need to be redone during database recovery. When performing a checkpoint, the database will ensure that all previous data has been written to the local disk to ensure data integrity and consistency.

[0166]

[0164] Therefore, when the original instance has new writes, the WAL mechanism will be used to continuously flush the data to the disk according to the transaction. At the same time, the newly written Page will be added to the dirty page list, and the database will continue to write the Page to the disk according to the dirty page flushing logic. The default file system can also receive the Page writes brought by the dirty page flushing of the original instance. The Page writes are first stored in the system cache. In this embodiment, the data is not flushed to the local temp store for every write. The data can still be kept in the system cache. Because the database will perform checkpoints periodically, when the database advances the checkpoint, it needs to ensure that the data in the dirty pages is written back to the local disk. Therefore, the default file system in this embodiment can receive these write operations. At this time, the default file system will write the Page to the local temp store according to the write operations. After the write is successful, the local Bitmap will also be written to the local temp store. Only after all these parts are successfully written will the write success be returned, and the database checkpoint will be advanced. Therefore, the method of writing the Page first and then writing the Bitmap can ensure that the Bitmap and the data are consistent. The file system will also ensure that the newly written data in the database is not lost even if it restarts at any time.

[0167]

[0165] Corresponding to the embodiments of the aforementioned data recovery method, this specification also provides embodiments of a data recovery device and the computer equipment on which it is applied.

[0168]

[0166] The embodiments of the data recovery device in this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by its processor reading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, as shown in Figure 3, which is a hardware structure diagram of the computer device where the data recovery device of this specification is located, in addition to the processor 310, memory 330, network interface 320, and non-volatile memory 340 shown in Figure 3, the computer device where the data recovery device in the embodiment is located may also include other hardware depending on the actual function of the computer device, which will not be elaborated further.

[0169]

[0167] As an example, this specification illustrates a data recovery apparatus based on a user-mode file system FUSE according to an exemplary embodiment, which may include: a recovery module, configured to: in response to receiving a recovery request to restore an original data table in the original database instance to a historical time, recover a temporary data table of the original data table at the historical time through a temporary database instance corresponding to the original database instance, and create an empty recovery data table corresponding to the temporary data table in the original database instance for access by the original database instance; a processing module, configured to: load the temporary data table into the local temporary storage space of the original database instance through the FUSE process, and during the loading process, process the access request of the original database instance to the recovery data table based on the data of the temporary data table; and a replacement module, configured to: replace the recovery data table with the data table stored in the local temporary storage space after loading is completed.

[0170]

[0168] The implementation process of the functions and roles of each module in the above data recovery device is detailed in the implementation process of the corresponding steps in the above data recovery method, and will not be repeated here.

[0171]

[0169] Accordingly, this specification also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the aforementioned data recovery method embodiments.

[0172]

[0170] Accordingly, this specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the data recovery method embodiment.

[0173]

[0171] Accordingly, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the data recovery method embodiments.

[0174]

[0172] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0175]

[0173] The above embodiments can be applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0176]

[0174] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0177]

[0175] The computer equipment may further include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0178]

[0176] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0179]

[0177] Specific embodiments of this specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0180]

[0178] The step division of various methods in the h-plane is only for clear description. In implementation, they can be merged into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are all within the protection scope of this application.

[0181]

[0179] Although this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. On the other hand, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0182]

[0180] Wherein, the description of “specific example” or “some examples” means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0183]

[0181] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0184]

[0182] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

[0185]

[0183] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

Claims 1. A data recovery method based on the user-space file system FUSE, the method comprising: In response to receiving a recovery request to restore the original data table in the original database instance to a historical time, the temporary data table of the original data table at the historical time is restored through a temporary database instance corresponding to the original database instance, and an empty recovery data table corresponding to the temporary data table is created in the original database instance for the original database instance to access. The FUSE process loads the temporary data table into the local temporary storage space of the original database instance. During the loading process, the original database instance's access requests to the recovery data table are processed based on the data in the temporary data table. After loading is complete, the data table stored in the local temporary storage space replaces the recovery data table.

2. The method according to claim 1, wherein the database instance manages the data table by dividing it into at least one data page, and the data page contains metadata; the step of loading the temporary data table into the local temporary storage space of the original database instance includes: Obtain at least one temporary data page of the temporary data table that is currently to be loaded; The metadata of the recovery data page corresponding to the temporary data page is obtained from the original database instance; The metadata of the temporary data page is updated based on the metadata of the recovered data page, and the updated temporary data page is loaded into the local temporary storage space.

3. The method according to claim 2, wherein the data page further comprises data content; the metadata includes a checksum, and at least one other piece of information besides the checksum; the step of updating the metadata of the temporary data page based on the metadata of the recovered data page includes: Replace the at least one other piece of information in the temporary data page with at least one other piece of information in the recovery data page; Calculate the checksum based on at least one other piece of information and data content after replacement in the temporary data page, and replace the checksum in the temporary data page with the calculated checksum O.

4. The method according to claim 2, wherein the step of processing the access request of the original database instance to the recovery data table based on the data of the temporary data table includes: In response to receiving a read request from the original database instance for the first data page of the recovered data table, determine whether a temporary data page corresponding to the first data page has been loaded in the local temporary storage space. If so, respond to the read request using the temporary data page already loaded in the local temporary storage space; otherwise, respond to the read request based on the temporary data page corresponding to the first data page in the temporary data table managed by the temporary database instance. In response to receiving a write request from the original database instance for the second data page of the recovered data table, write the second data page to the local temporary storage space.

5. The method according to claim 4, further comprising, after the step of loading the updated temporary data page into the local temporary storage space: The loaded status of the updated temporary data page is recorded in the preset loading status data; Determining whether the temporary data page corresponding to the first data page has been loaded in the local temporary storage space includes: using the loading status data to query whether the temporary data page corresponding to the first data page in the temporary data table has been loaded in the local temporary storage space.

6. The method according to claim 5, further comprising, after the step of writing the second data page to the local temporary storage space: If the temporary data table contains a temporary data page corresponding to the second data page, the loaded status of the temporary data page corresponding to the second data page is recorded in the loading status data.

7. The method according to claim 6, wherein recording the loaded state of the temporary data page corresponding to the second data page in the loading state data includes: Access the loading status data stored in the memory space of the original database instance, record the loaded status of the temporary data page corresponding to the second data page in the loading status data, write the loading status data to the local temporary storage space, and return the write success message corresponding to the write request.

8. The method according to claim 6, wherein writing the updated temporary data page to the local temporary storage space comprises: Query the loading status data to see if the updated temporary data page has been loaded into the local temporary storage space; If not, write the updated temporary data page to the local temporary storage space; if yes, discard the updated temporary data page.

9. The method according to claim 8, after the step of writing the updated temporary data page to the local temporary storage space, the method further includes: Access the loading status data stored in the memory space of the original database instance, record the loaded status of the updated temporary data page in the loading status data, and then write the loading status data to the local temporary storage space. The loading status data stored in the local temporary storage space is used by the FUSE process to obtain the loading status of the temporary data pages in the temporary data table after the FUSE process restarts.

10. The method according to claim 1, wherein the original database instance includes a primary database instance and at least one backup database instance, wherein, At least one of the backup database instances is given priority to start the FUSE process. Then, the backup database instance that started the FUSE process and the primary database instance are switched over. The backup database instance after the switchover is then started by the FUSE process.

11. The method according to claim 1, wherein loading the temporary data table into the local temporary storage space of the original database instance via the FUSE process, and processing the access request of the original database instance to the recovery data table based on the data of the temporary data table during the loading process, includes: The method further includes running a preset file system on the host where the original data instance resides. The preset file system is configured to execute the FUSE process and load the temporary data table into the local temporary storage space of the original database instance. During the loading process, the method also includes processing the access requests of the original database instance to the recovery data table based on the data in the temporary data table. After the loading is completed, the preset file system is unloaded.

12. The method according to claim 11, wherein the original database instance includes a primary database instance and at least one backup database instance; the step of running a preset file system on the host where the original database instance resides includes: Create a corresponding remote storage space for each instance included in the original database instance, and copy the temporary data table out of the remote storage space; After restarting the current standby database instance, and running a preset file system on the host where the current standby database instance is located, a temporary data table in one of the remote storage spaces is mounted to the preset file system; a master-slave switch is triggered between the current master database instance and the current standby database instance, and the switched standby database instance is restarted. After running a preset file system on the host where the switched standby database instance is located, a temporary data table in another of the remote storage spaces is mounted to the preset file system.

13. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

14. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method of any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any one of claims 1 to 12.

Images