Data recovery method and device, equipment, medium and program product

Abstract

The invention discloses a data recovery method, device and equipment, a medium and a program product, and relates to the technical field of computers. The data recovery method comprises the following steps: analyzing a pre-write log WAL file in a database to obtain a log frame in the WAL file; the log frame comprises modification information, a page number, transaction metadata and page mapping information for a database page; according to the transaction metadata in the log frame, performing boundary identification on the modification information of the database page to obtain at least one item of transaction information; and according to the transaction information, the page number, the page mapping information and a data structure of the database, performing data recovery in the database to obtain a recovered database. According to the technical scheme provided by the embodiment of the invention, the problem of data loss easily occurring during data recovery can be solved by utilizing the characteristic that the WAL file is independently stored and is not easy to lose.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular, to a data recovery method, device, equipment, medium and program product. BACKGROUND

[0002] Data recovery of a database refers to repairing lost or damaged electronic data in the database through technical means, aiming to restore the database to a usable and consistent state.

[0003] Data recovery is required in many scenarios, such as recovering deleted evidence information in the process of judicial evidence collection, recovering application data of mobile terminal users, recovering lost data in network interruption in industrial equipment data collection, and recovering business data when the business data is maliciously deleted or tampered with.

[0004] A common data recovery method in the prior art is to backup data at regular intervals and use the backup data for data recovery. This method may cause data loss between the backup time point and the fault point, and the circular log of some databases will be overwritten by new data, so that the history operation cannot be traced back. SUMMARY

[0005] The present application provides a data recovery method, device, equipment, medium and program product to solve the problem of data loss that may occur when data recovery is performed by a regular backup method.

[0006] According to an aspect of the present application, a data recovery method is provided, comprising:

[0007] Parsing a write-ahead log (WAL) file in a database to obtain a log frame in the WAL file; the log frame includes modification information for a database page, a page number, transaction metadata, and page mapping information;

[0008] According to the transaction metadata in the log frame, boundary identification is performed on the modification information of the database page to obtain at least one item of transaction information;

[0009] According to the transaction information, the page number, the page mapping information, and the data structure of the database, data recovery is performed in the database to obtain a recovered database.

[0010] According to another aspect of the present application, a data recovery device is provided, comprising:

[0011] A WAL file parsing module is configured to parse a write-ahead log (WAL) file in a database to obtain a log frame in the WAL file; the log frame includes modification information for a database page, a page number, transaction metadata, and page mapping information;

[0012] A transaction identification module is configured to identify the boundary of the modification information of the database page according to the transaction metadata in the log frame, and obtain at least one item of transaction information.

[0013] A data recovery module is configured to recover the data in the database according to the transaction information, the page number, the page mapping information and the data structure of the database, and obtain the recovered database.

[0014] According to another aspect of the present application, an electronic device is provided, which comprises:

[0015] at least one processor; and

[0016] a memory connected to the at least one processor in communication; wherein

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the data recovery method according to any one of the embodiments of the present application.

[0018] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the data recovery method according to any one of the embodiments of the present application when executed by the processor.

[0019] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for implementing the data recovery method according to any one of the embodiments of the present application when executed by a processor.

[0020] The technical solution of the embodiments of the present application parses the pre-write-ahead log (WAL) file in the database to obtain the log frame in the WAL file, the log frame comprising the modification information of the database page, the page number, the transaction metadata and the page mapping information, and then identifies the boundary of the modification information of the database page according to the transaction metadata in the log frame to obtain at least one item of transaction information, and recovers the data in the database according to the transaction information, the page number, the page mapping information and the data structure of the database to obtain the recovered database. The WAL mechanism is introduced due to the requirements of transaction atomicity and durability, supports the historical operation retention function and is independently stored in the WAL file and is not easy to lose, thereby solving the problem of data loss that is prone to occur when data recovery is performed by the timing backup method.

[0021] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to make the technical solution in the embodiments of the present application clearer, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0023] Figure 1 is a flow chart of a data recovery method according to the first embodiment of the present application;

[0024] Figure 2 is a flow chart of a data recovery method according to the second embodiment of the present application;

[0025] Figure 3 is a structural schematic diagram of a data recovery device according to the third embodiment of the present application;

[0026] Figure 4 is a structural schematic diagram of an electronic device implementing the data recovery method according to the embodiments of the present application. DETAILED DESCRIPTION

[0027] In order to make the technical solution in the embodiments of the present application clearer, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0028] It should be noted that the terms "first", "second", and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to include all the steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0029] Embodiment One

[0030] Figure 1A flowchart of a data recovery method is provided for Embodiment One of the present application. The present embodiment can be applied to the case of data recovery based on a data WAL file. The method can be executed by a data recovery device, which can be implemented in the form of hardware and / or software, and can be configured in various general-purpose computing devices. As shown in FIG. 1, the method comprises the following steps. Figure 1

[0031] In S110, the pre-write log (WAL) file in the database is parsed to obtain the log frame in the WAL file. The log frame includes modification information for the database page, page number, transaction metadata, and page mapping information.

[0032] The pre-write log (WAL) of the database is a technology used to improve data reliability and integrity in a database system. In the WAL mechanism, all modification operations, such as INSERT, UPDATE, and DELETE, are first recorded in the log file before being actually written to the data file of the database. The sequential execution of writing the log first and then writing the data can restore the database to a consistent state through replaying the operations in the log file even when the database system fails, such as system crash or power failure.

[0033] The WAL file includes a file header (Header) and a log frame (Frame). The file header includes WAL version number, page size, and file state check character, etc. The log frame includes modification information for the database page, page number, transaction metadata, and page mapping information.

[0034] In the present embodiment, to perform data recovery, the metadata contained in the file header of the WAL file is first extracted, including the WAL version number, page size, and file state check character. Then the log frame is sequentially read from the WAL file, including the modification information for the database page, page number, transaction metadata, and page mapping information. Subsequently, the operations in the WAL file can be replayed based on the modification information for the database page, page number, transaction metadata, and page mapping information to perform data recovery.

[0035] In addition, before parsing and extracting the log frame, the file header of the WAL file can be parsed to obtain the file state check character contained therein. Then, it is determined whether the current WAL file is a valid file and whether the file is complete according to the file state check character. Only after the file state check, the log frame in the WAL file is extracted to ensure the reliability of data recovery.

[0036] In S120, the modification information of the database page is boundary-identified according to the transaction metadata in the log frame to obtain at least one item of transaction information.​

[0037] The transaction metadata includes mark information of the transaction involved in the log frame, such as a data commit mark and a data rollback mark.

[0038] In the embodiment of the present application, the modification information of the database page is identified according to the transaction metadata in the log frame, and at least one item of transaction information is obtained. Specifically, the start and end points of the transaction are identified according to the transaction metadata in the log frame. For example, the data commit mark is found in the log frame, which indicates the successful commit of the transaction (for example, the final COMMIT record of SQLite), and the position of the data commit mark is determined as the end point of the current transaction and the start point of the next transaction. The uncommitted transaction of the Rollback Segment needs to be discarded to avoid data inconsistency. The transaction information includes a group of database operations, such as addition, deletion, modification and query.

[0039] Optionally, before the modification information of the database page is identified according to the transaction metadata in the log frame, the following steps are further included:

[0040] The integrity check character is extracted in the log frame;

[0041] The validity of each log frame is checked according to the integrity check character.

[0042] In the optional embodiment, the specific steps performed before the modification information of the database page is identified according to the transaction metadata in the log frame are provided: the log frame further includes a check character, such as an integrity check character (checksum). The WAL file may be damaged due to storage damage or write interruption, resulting in data errors. Therefore, before the transaction boundary is identified, the validity of each log frame is first checked based on the integrity check character. In the case where the log frame is valid, the subsequent transaction boundary identification operation is continued to ensure the integrity of the log frame.

[0043] S130, according to the transaction information, the page number, the page mapping information and the data structure of the database, the data recovery in the database is performed to obtain the recovered database.

[0044] The page mapping information (Page Mapping) is used to establish the mapping of the page number and the physical storage location, and defines the target physical page (table index column or data page) corresponding to the page number in the WAL.

[0045] In the embodiment of the present application, after the WAL file is parsed, the transaction information, the page number and the page mapping information can be synchronized to the main database file. Subsequently, according to the time point defined by the user, the page number and the page mapping information are extracted from the main database file, and the target physical page associated with each log frame is determined based on the extracted information. Finally, based on the transaction information and the data structure (Schema) of the database, the target physical page in the database is reconstructed to generate the recovered database.

[0046] Optionally, after the data recovery in the database is performed based on the transaction information, the page number, the page mapping information and the data structure of the database, the recovered database is obtained, and the method further comprises:

[0047] analyzing the recovered database based on the data acquisition requirement of the user;

[0048] performing front-end display on the analysis result based on the data acquisition requirement and writing the analysis result into a result database.

[0049] In the optional embodiment, after the data recovery in the database is performed based on the transaction information, the page number, the page mapping information and the data structure of the database, the recovered database is obtained, and the specific steps are as follows: first, the recovered database is analyzed based on the data acquisition requirement of the user. For example, the user needs to acquire all the data associated with a target field in the database, and the data associated with the target field is queried in the recovered database. Then, the analysis result based on the data acquisition requirement is displayed on the front end and written into a result database, so that the user can intuitively acquire the required data.

[0050] In addition, the recovered data can also be displayed through a user interface, so that the user can view the list of recovered data, time, content and other related information.

[0051] The technical solution of the embodiment of the present application parses the pre-write log (WAL) file in the database to obtain the log frame in the WAL file, the log frame includes the modification information of the database page, the page number, the transaction metadata and the page mapping information, and then the boundary of the modification information of the database page is identified based on the transaction metadata in the log frame to obtain at least one item of transaction information. The data recovery in the database is performed based on the transaction information, the page number, the page mapping information and the data structure of the database to obtain the recovered database. The WAL mechanism is introduced due to the requirements of transaction atomicity and durability, supports the historical operation retention function and the WAL file is independently stored and is not easy to lose, thereby solving the problem of data loss that easily occurs when the data recovery is performed through the timing backup mode.

[0052] Embodiment two

[0053] Figure 2A flowchart of a data recovery method provided for the second embodiment of the present application is shown in FIG. 2B. The second embodiment is based on the first embodiment and further refines the steps of parsing the WAL file in the database to obtain the log frame in the WAL file, and the steps of performing data recovery in the database according to the transaction information, the page number, the page mapping information, and the data structure of the database to obtain the recovered database. As shown in FIG. 2B, the method comprises the following steps. Figure 2

[0054] In S210, the file header of the WAL file in the database is parsed to obtain a file status check character, and the WAL file is checked according to the file status check character.

[0055] The file status check character is used to check the validity of the WAL file. For example, the file status check character includes a file validity check character and a file integrity check character.

[0056] In the embodiment of the present application, the file header of the WAL file in the database can be parsed first to obtain the file status check character in the file header. For example, the characters at the offset of 0-3 in the file header are Magic Number, which is used to verify whether the WAL file is valid, i.e., whether it is a legal WAL file, and is referred to as a file validity check character hereinafter. The characters at the offset of 4-7 are version number, the characters at the offset of 8-11 are page size (e.g., usually 4096 bytes), and the characters at the offset of 12-15 are checksum, which is used to verify the file integrity and is referred to as a file integrity check character hereinafter.

[0057] Further, the WAL file is checked according to the file status check character. Specifically, the file validity is verified according to the file validity check character, and the file integrity is verified according to the file integrity check character. By parsing the file header of the WAL file, the validity and integrity of the WAL file can be quickly verified to ensure the accurate execution of data recovery.

[0058] Optionally, the file header of the WAL file in the database is parsed to obtain a file status check character, and the WAL file is checked according to the file status check character. The method comprises the following steps.

[0059] According to the pre-set association between the offset and the field information, the characters at the set offset in the file header of the WAL file are read to obtain the file validity check character and the file integrity check character;

[0060] The validity of the WAL file is checked according to the validity check character, and the integrity of the WAL file is checked according to the file integrity check character.

[0061] ​In a case where the WAL file passes the validity and integrity check, it is determined that the result of the WAL file state check is a valid state.

[0062] In the optional embodiment, the file header of the WAL file in the database is parsed to obtain the file state check character, and the specific steps of performing state check on the WAL file according to the file state check character are as follows: first, according to the pre-set association between the offset and the field information, the characters at the set offset in the file header of the WAL file are read to obtain the file validity check character and the file integrity check character. For example, the characters at the offset of 0-3 in the file header are the file validity check character, and the characters at the offset of 12-15 are the checksum, which is hereinafter referred to as the file integrity check character.

[0063] Then, the validity of the WAL file is checked according to the validity check character, and the integrity of the WAL file is checked according to the file integrity check character. In a case where the WAL file passes the validity and integrity check, it is determined that the result of the WAL file state check is a valid state. By first parsing the file header to verify the validity and integrity of the file, if the WAL file is found to be illegal or incomplete, timely processing measures can be taken to avoid resource loss caused by subsequent data recovery errors.

[0064] Optionally, if the database is enabled with encryption or WAL compression, the file decryption or file decompression needs to be performed before the WAL file is parsed.

[0065] S220, in a case where the result of the state check is a valid state, sequentially reading the log frames in the WAL file.

[0066] The log frames include the modification information of the database page, the page number, the transaction metadata and the page mapping information.

[0067] In the embodiment of the application, in a case where the result of the state check is a valid state, the log frames associated with a specific time period are sequentially read in the WAL file according to a user-defined time point.

[0068] S230, performing boundary identification on the modification information of the database page according to the transaction metadata in the log frame to obtain at least one transaction information.

[0069] S240, synchronizing the transaction information, the page number and the page mapping information to the main database file.

[0070] In the embodiment of the application, the database will periodically clean up the WAL file, synchronize the transaction information, the page number and the page mapping information to the main database file through Checkpoint, and the unsynchronized part can be manually merged.

[0071] In addition, after synchronizing to the master database file, the file header validity of the master database file can be verified to ensure the accuracy of data recovery, and the association completeness of indexes and data contained therein can also be verified.

[0072] S250, determining the target physical page associated with each log frame based on the page number and page mapping information in the master database file.

[0073] In the embodiments of the present application, since the WAL file records page-level changes, it is necessary to determine the target physical page associated with each log frame based on the page number and page mapping information of the time point involved log frame in the master database according to the user-defined data recovery time point.

[0074] S260, reconstructing the target physical page in the database based on the transaction information and the data structure of the database to obtain the recovered database.

[0075] In the embodiments of the present application, the target physical page in the database is reconstructed based on the transaction information and the data structure (Schema) of the database to generate the recovered database. Specifically, the committed transactions are re-executed, and the uncommitted transactions are rolled back.

[0076] Optionally, reconstructing the target physical page in the database based on the transaction information and the data structure of the database to obtain the recovered database includes:

[0077] Based on the transaction information, determining the committed transactions and the uncommitted transactions in the transaction;

[0078] Based on the transaction information and the data structure of the database, updating the target physical page in the database by re-executing the committed transactions and rolling back the uncommitted transactions to obtain the recovered database.

[0079] In the optional embodiments, specific steps are provided for reconstructing the target physical page in the database based on the transaction information and the data structure of the database to obtain the recovered database: first, based on the transaction information, determining the committed transactions and the uncommitted transactions in the transaction, and then based on the transaction information and the data structure of the database, updating the target physical page in the database by re-executing the committed transactions and rolling back the uncommitted transactions to obtain the recovered database, which can accurately perform data recovery, avoid data loss, and write dirty data, and is used for various data loss scenarios such as accidental deletion, system crash, and ransomware attack.

[0080] The technical scheme of the embodiment of the present application analyzes the pre-write-ahead log (WAL) file in the database to obtain a log frame in the WAL file, the log frame including modification information for a database page, a page number, transaction metadata, and page mapping information, and then performs boundary identification on the modification information of the database page according to the transaction metadata in the log frame to obtain at least one item of transaction information, synchronizes the transaction information, the page number, and the page mapping information to a main database file, determines a target physical page associated with each log frame based on the page number and the page mapping information in the main database file, and reconstructs the target physical page in the database based on the transaction information and a data structure of the database to obtain a restored database. The WAL mechanism is introduced due to the requirements of transaction atomicity and persistence, supports a historical operation retention function, and is independently stored in the WAL file and is not easy to lose, thereby solving the problem of data loss that easily occurs when data recovery is performed through a timing backup mode, and accurate data recovery can be performed, data loss is avoided, and dirty data writing is avoided.

[0081] Embodiment three

[0082] Figure 3 A structural schematic diagram of a data recovery device provided by the third embodiment of the present application is shown in FIG. 3. As shown in the figure, the device includes: Figure 3

[0083] The WAL file analysis module 310 is configured to analyze the pre-write-ahead log (WAL) file in the database to obtain a log frame in the WAL file, the log frame including modification information for a database page, a page number, transaction metadata, and page mapping information.

[0084] The transaction identification module 320 is configured to perform boundary identification on the modification information of the database page according to the transaction metadata in the log frame to obtain at least one item of transaction information.

[0085] The data recovery module 330 is configured to perform data recovery in the database according to the transaction information, the page number, the page mapping information, and a data structure of the database to obtain a restored database.

[0086] The technical scheme of the embodiment of the present application analyzes the pre-write-ahead log (WAL) file in the database to obtain a log frame in the WAL file, the log frame including modification information for a database page, a page number, transaction metadata, and page mapping information, and then performs boundary identification on the modification information of the database page according to the transaction metadata in the log frame to obtain at least one item of transaction information, synchronizes the transaction information, the page number, and the page mapping information to a main database file, determines a target physical page associated with each log frame based on the page number and the page mapping information in the main database file, and reconstructs the target physical page in the database based on the transaction information and a data structure of the database to obtain a restored database. The WAL mechanism is introduced due to the requirements of transaction atomicity and persistence, supports a historical operation retention function, and is independently stored in the WAL file and is not easy to lose, thereby solving the problem of data loss that easily occurs when data recovery is performed through a timing backup mode, and accurate data recovery can be performed, data loss is avoided, and dirty data writing is avoided.​

[0087] Optionally, the WAL file parsing module 310 comprises:

[0088] a state checking unit, configured to parse a file header of a WAL file in a database to obtain a file state checking character, and perform state checking on the WAL file according to the file state checking character;

[0089] a log frame reading unit, configured to sequentially read log frames in the WAL file in a case where a result of the state checking is a valid state.

[0090] Optionally, the data recovery module 330 comprises:

[0091] a main file synchronizing unit, configured to synchronize the transaction information, the page number and the page mapping information to a main database file;

[0092] a target physical page determining unit, configured to determine a target physical page associated with each log frame based on the page number and the page mapping information in the main database file;

[0093] a data recovery unit, configured to reconstruct the target physical page in the database based on the transaction information and a data structure of the database to obtain a recovered database.

[0094] Optionally, the data recovery unit is specifically configured to:

[0095] determine committed transactions and uncommitted transactions in the transactions based on the transaction information;

[0096] update the target physical page in the database by re-executing the committed transactions and rolling back the uncommitted transactions based on the transaction information and the data structure of the database to obtain the recovered database.

[0097] Optionally, the data recovery apparatus further comprises:

[0098] a database parsing module, configured to parse the recovered database based on a data acquisition requirement of a user after data recovery in the database is performed based on the transaction information, the page number, the page mapping information and the data structure of the database to obtain the recovered database;

[0099] a parsing result display module, configured to display a parsing result based on the data acquisition requirement in a front end and write the parsing result into a result database.

[0100] Optionally, the data recovery apparatus further comprises:

[0101] The check character advance module is configured to extract integrity check characters from the log frame before boundary identification of modification information of the database page according to transaction metadata in the log frame.

[0102] The log frame check module is configured to check validity of each log frame according to the integrity check characters.

[0103] Optionally, the state check unit is specifically configured to:

[0104] According to the pre-set offset and the association relationship of the field information, the file validity check characters and the file integrity check characters are obtained by reading the characters at the set offset in the file header of the WAL file.

[0105] According to the validity check characters, the WAL file is subjected to validity check, and according to the file integrity check characters, the WAL file is subjected to integrity check.

[0106] In the case that the WAL file passes the validity and integrity check, it is determined that the result of the WAL file state check is valid.

[0107] The data recovery device provided by the embodiment of the present application can execute the data recovery method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0108] In the technical solution of the present application, the collected information is information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the related data comply with the relevant laws, regulations and standards of the relevant countries and regions, necessary security measures are taken, public order and good customs are not violated, and corresponding operation entrances are provided for the user to select authorization or refusal.

[0109] Embodiment four

[0110] According to the embodiments of the present application, the present application further provides an electronic device, a readable storage medium and a computer program product.

[0111] Figure 4A structural diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, appliances, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0112] As shown, Figure 4 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., communicatively connected to the at least one processor 11, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0113] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0114] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the data recovery method.

[0115] In some embodiments, the data recovery method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the data recovery method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the data recovery method by way of other means, e.g., with the aid of firmware.

[0116] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0117] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or application.

[0118] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0120] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data application server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0121] A computing system can include a client and an applicator. The client and the applicator are generally remote from each other and typically interact through a communication network. The relationship of client and applicator arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The applicator can be a cloud applicator, also known as a cloud computing applicator or cloud host, which is a host product in the cloud computing application system to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS application.

[0122] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0123] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A data recovery method, characterized by, The method comprises the following steps: parsing a pre-write-ahead log (WAL) file in a database to obtain a log frame in the WAL file; the log frame comprises modification information of a database page, a page number, transaction metadata, and page mapping information; boundary identification is performed on the modification information of the database page according to the transaction metadata in the log frame, and at least one item of transaction information is obtained; data recovery in the database is performed according to the transaction information, the page number, the page mapping information, and a data structure of the database, and a recovered database is obtained.

2. The method of claim 1, wherein, The method comprises the following steps: parsing a file header of a WAL file in a database to obtain a file state check character, and performing state checking on the WAL file according to the file state check character; if the result of the state checking is a valid state, sequentially reading log frames in the WAL file.

3. The method of claim 1, wherein, The method comprises the following steps: synchronizing the transaction information, the page number, and the page mapping information to a main database file; determining a target physical page associated with each log frame based on the page number and the page mapping information in the main database file; reconstructing the target physical page in the database based on the transaction information and the data structure of the database, and obtaining a recovered database.

4. The method of claim 3, wherein, The method comprises the following steps: determining committed transactions and uncommitted transactions in a transaction based on the transaction information; updating the target physical page in the database by re-executing the committed transactions and rolling back the uncommitted transactions based on the transaction information and the data structure of the database, and obtaining a recovered database.

5. The method of claim 1, wherein, After the data recovery in the database is performed according to the transaction information, the page number, the page mapping information, and the data structure of the database, and a recovered database is obtained, the method further comprises the following steps: analyzing the recovered database based on a data acquisition requirement of a user; performing front-end display on an analysis result based on the data acquisition requirement, and writing the analysis result into a result database.

6. The method of claim 1, wherein, Before the boundary identification is performed on the modification information of the database page according to the transaction metadata in the log frame, the method further comprises the following steps: extracting an integrity check character in the log frame; checking the validity of each log frame according to the integrity check character.

7. The method of claim 2, wherein, The method comprises the following steps: according to a pre-set association between an offset and field information, reading characters at a set offset in the file header of the WAL file to obtain a file validity check character and a file integrity check character; performing validity checking on the WAL file according to the validity check character, and performing integrity checking on the WAL file according to the file integrity check character; In a case where the WAL file passes the validity and integrity check, it is determined that the result of the WAL file status check is a valid state.

8. A data recovery apparatus, characterized by comprising: Comprise: A WAL file parsing module is configured to parse a pre-write-ahead log (WAL) file in a database to obtain log frames in the WAL file. The log frames include modification information, a page number, transaction metadata, and page mapping information for a database page. A transaction identification module is configured to identify boundaries of the modification information of the database page according to the transaction metadata in the log frames to obtain at least one item of transaction information. A data recovery module is configured to perform data recovery in the database according to the transaction information, the page number, the page mapping information, and a data structure of the database to obtain a recovered database.

9. An electronic device, comprising: The electronic device comprises: At least one processor; and A memory connected in communication with the at least one processor; wherein The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the data recovery method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the processor to implement the data recovery method of any one of claims 1-7 when executed.

11. A computer program product, characterised in that, The computer program product comprises a computer program that, when executed by the processor, implements the data recovery method according to any one of claims 1-7.

Images