Data recovery method for damaged ibd file of MySQL8.0

Abstract

The invention discloses a data recovery method for a damaged ibd file of MySQL8.0. The data recovery method is characterized by comprising the following steps of S100, obtaining the ibd file, scanning the ibd file, and obtaining an SDI page and a DATA page according to data structure characteristics of pages; s200, serialized dictionary information is extracted from the SDI page; s300, obtaining a table structure in the serialized dictionary information; and S400, analyzing the data in the DATA page by adopting a table structure, recovering the table record and obtaining the data of the ibd file. According to the method disclosed by the invention, the data of the damaged ibd file of MySQL8.0 can be efficiently recovered without additional hardware and software and additional human input.

Description

Technical Field

[0001] The invention belongs to the field of data recovery and data forensics, and relates to a data recovery method for a damaged ibd file of MySQL 8.0. Background Art

[0002] As one of the most popular relational databases, MySQL has the advantages of small size, high speed, low cost, and open source code. It is selected by many companies as the preferred database, especially in Web applications. MySQL supports multiple storage engines, the most popular of which is InnoDB. The main data storage file of this engine is ibd file.

[0003] However, during the use of MySQL, ibd files are easily lost and damaged due to abnormal power outages, virus invasions, human errors, etc.

[0004] In the prior art, the traditional data recovery technology is used, but the damaged ibd file cannot be recovered or the recovery effect is poor. Therefore, a data recovery method for recovering damaged ibd files can fill this deficiency and has high practical value. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a data recovery method for a damaged MySQL8.0 ibd file: extracting SDI pages and DATA pages from the damaged ibd file, extracting serialized dictionary information from the damaged SDI page, and extracting the table structure from it; parsing data in the damaged DATA page and recovering table records. The method includes the following steps:

[0006] S100: Obtaining an ibd file and scanning the ibd file, obtaining an SDI page and a DATA page according to the data structure characteristics of the page, including the following steps:

[0007] S101: Retrieve the infimum pseudo record identifier in the ibd file;

[0008] S102: Determine whether the infimum pseudo record identifier is retrieved, if yes, execute step S103, otherwise, execute step S200;

[0009] S103: Record the starting address of the infinite pseudo record marker, take the starting address of the infinite pseudo record marker as the first address, shift backward by 0x0D bytes and obtain the content of 0x08 bytes in succession;

[0010] S104: Determine whether the content of the acquired 0x08 byte is a supremum pseudo record identifier, if yes, execute step S105, otherwise, execute step S101;

[0011] S105: Taking the starting address of the hypodetermined pseudo record marker as the first address, offset forward by 0x4B bytes and obtain the content of 0x02 bytes in succession;

[0012] S106: Determine whether the content of the acquired 0x02 byte is an SDI page identifier, if yes, execute step S108, otherwise, execute step S107;

[0013] S107: Determine whether the content of the acquired 0x02 byte is a DATA page identifier, if yes, execute step S109, otherwise, execute step S101;

[0014] S108: Mark the current page as an SDI page, and execute step S110;

[0015] S109: Mark the current page as a DATA page and execute step S110;

[0016] S110: Take the starting address of the hypofinite pseudo record marker as the first address, shift forward 0x73 bytes and obtain the content of 0x4000 bytes as the SDI page data / DATA page data, and execute step S101;

[0017] S200: extracting serialized dictionary information in the SDI page;

[0018] S300: Obtaining the table structure in the serialized dictionary information;

[0019] S400: Parse the data in the DATA page using the table structure, restore the table record and obtain the data of the ibd file.

[0020] Preferably, step S200 includes the following steps:

[0021] S201: Taking the start address of the SDI page as the first address, offset 0x61 bytes backward and obtain the content of 0x02 bytes in a big-endian format as the first value;

[0022] S202: Determine whether the sum of 0x63+first value+0x21 is less than 0x4000. If yes, execute step S203; otherwise, execute step S300.

[0023] S203: Taking the start address of the SDI page as the first address, offset backward by 0x63+first value+0x21 bytes and record the current addressing address as the first address;

[0024] S204: taking the first address as the first address, reading the content of 0x02 bytes continuously, and determining whether the content of the read 0x02 bytes is 0x789C, if yes, executing step S205, otherwise, executing step S300;

[0025] S205: Taking the first address as the first address, shifting forward 0x02 bytes and obtaining the content of 0x02 bytes in a big-endian format as the second value;

[0026] S206: Determine whether the sum of 0x63+first value+0x21+second value is less than 0x4000. If yes, execute step S207; otherwise, execute step S300.

[0027] S207: Taking the first address as the first address, continuously read the byte content whose byte length is the second value and decompress it, and use the decompressed data as the serialized dictionary information.

[0028] Preferably, step S300 includes the following steps:

[0029] S301: Retrieving a data dictionary object node in the serialized dictionary information according to the structural characteristics of the JSON format;

[0030] S302: According to the structural characteristics of the JSON format, the option node is searched in the data dictionary object node, the value of the option node is obtained and saved as a table attribute;

[0031] S303: According to the structural characteristics of the JSON format, a row format node is retrieved in the data dictionary object node, the value of the row format node is obtained and saved as the row format of the table;

[0032] 304. According to the structural characteristics of the JSON format, retrieve the column node in the data dictionary object node, obtain the value of the column node and save it as the field definition of the table.

[0033] Preferably, step S400 includes the following steps:

[0034] S401: Taking the starting address of the DATA page as the first address, offset 0x63 bytes backward and record the current addressing address as the second address;

[0035] S402: Taking the second address as the first address, offset forward 0x02 bytes and obtain the content of 0x02 bytes in a big-endian format as the third value;

[0036] S403: taking the second address as the first address, shifting backward by a byte length equal to a third value and recording the current addressing address as the second address;

[0037] S404: Using the second address as the first address, offset forward by 0x02 bytes and obtain the content of 0x02 bytes in a big-endian format as the third value;

[0038] S405: taking the second address as the first address, reading the content of 0x08 bytes continuously, and determining whether the content of the read 0x08 bytes is 0x73757072656D756D, if yes, executing step S406, otherwise, executing step S410;

[0039] S406: Using the start address of the DATA page as the first address, offset 0x2C bytes backward and obtain the content of 0x02 bytes in a big-endian format as the third value;

[0040] S407: Determine whether the current third value is equal to 0x0000, if yes, end the process, otherwise, execute step S408;

[0041] S408: Taking the start address of the DATA page as the first address, offset backward by the byte length of the current third value and record the current addressing address as the second address;

[0042] S409: Taking the second address as the first address, offset forward 0x02 bytes and obtain the content of 0x02 bytes in a big-endian format as the third value;

[0043] S410: According to the row format and field definition in the table structure, the data is parsed backwards with the current second address as the first address, and the parsing result is saved as a table record;

[0044] S411: Determine whether the current third value is equal to 0x0000. If so, end the process; otherwise, execute step S403.

[0045] Preferably, in step S101, a KMP algorithm is used to retrieve the infimum pseudo record identifier.

[0046] Preferably, the infimum pseudo record identifier is 0x696E66696D756D00, the supremum pseudo record identifier is 0x73757072656D756D, the SDI page identifier is 0x45BD, and the DATA page identifier is 0x45BF.

[0047] Preferably, the Inflate algorithm is used for decompression.

[0048] The beneficial effect of the present invention is that the data of the damaged MySQL8.0 ibd file can be efficiently restored without additional hardware and software and without additional manpower investment. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of the present invention;

[0050] Figure 2 A specific flow chart of obtaining SDI page and DATA page in an embodiment of the present invention;

[0051] Figure 3 A specific flow chart of extracting serialized dictionary information from an SDI page in an embodiment of the present invention;

[0052] Figure 4 A specific flow chart of obtaining the table structure in the serialized dictionary information in an embodiment of the present invention;

[0053] Figure 5 This is a specific flow chart of restoring table records and obtaining data of ibd files in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The terms used in this application are as follows:

[0055] InnoDB: A storage engine for MySQL database

[0056] ibd file: file where the InnoDB engine stores data

[0057] Page: The basic unit for storing data in an ibd file

[0058] SDI page: a page that stores serialized dictionary information

[0059] DATA page: a page that stores table records

[0060] Table structure: basic properties and field definitions of the table

[0061] Table record: User data stored in a certain format in the table

[0062] Inflate algorithm: decompression algorithm of Deflate compression algorithm

[0063] Big-endian format: a storage format in which high-byte data is stored at low addresses

[0064] KMP algorithm: an efficient character matching retrieval algorithm

[0065] JSON format: a lightweight data exchange format.

[0066] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0067] Figure 1 1 is a flow chart showing the present invention. Figure 1 As shown, the method of the present invention comprises the following steps:

[0068] S100: Obtain an ibd file and scan the ibd file, and obtain an SDI page and a DATA page according to the data structure characteristics of the page.

[0069] Figure 2FIG. 2 shows a specific flow chart of obtaining SDI page and DATA page in an embodiment of the present invention. Figure 2 As shown, step S100 includes the following steps:

[0070] S101: Using the KMP algorithm, the infimum pseudo record identifier is retrieved from the binary data of the ibd file. In this embodiment, the infimum pseudo record identifier is 0x696E66696D756D00;

[0071] S102: Determine whether the infimum pseudo record identifier is retrieved, if yes, execute step S103, otherwise, execute step S200;

[0072] S103: Record the starting address of the infinite pseudo record marker, take the starting address of the infinite pseudo record marker as the first address, shift backward by 0x0D bytes and obtain the content of 0x08 bytes in succession;

[0073] S104: Determine whether the content of the acquired 0x08 byte is a supremum pseudo record identifier, if yes, execute step S105, otherwise, execute step S101;

[0074] In this embodiment, the supremum pseudo record identifier is 0x73757072656D756D.

[0075] S105: Taking the starting address of the hypodetermined pseudo record marker as the first address, offset forward by 0x4B bytes and obtain the content of 0x02 bytes in succession;

[0076] S106: Determine whether the content of the acquired 0x02 byte is an SDI page identifier, if yes, execute step S108, otherwise, execute step S107;

[0077] In this embodiment, the SDI page identifier is 0x45BD.

[0078] S107: Determine whether the content of the acquired 0x02 byte is a DATA page identifier, if yes, execute step S109, otherwise, execute step S101;

[0079] In this embodiment, the DATA page identifier is 0x45BF.

[0080] S108: Mark the current page as an SDI page, and execute step S110;

[0081] S109: Mark the current page as a DATA page and execute step S110;

[0082] S110: Take the starting address of the hypofinite pseudo record marker as the first address, shift forward 0x73 bytes and obtain the content of 0x4000 bytes as the SDI page data / DATA page data, and execute step S101;

[0083] S200: Extract serialized dictionary information in the SDI page.

[0084] Figure 3 FIG. 4 shows a specific flow chart of extracting serialized dictionary information from an SDI page in an embodiment of the present invention. Figure 3 As shown, step S200 includes the following steps:

[0085] S201: Taking the start address of the SDI page as the first address, offset 0x61 bytes backward and obtain the content of 0x02 bytes in a big-endian format as the first value;

[0086] S202: Determine whether the sum of 0x63+first value+0x21 is less than 0x4000. If yes, execute step S203; otherwise, execute step S300.

[0087] S203: Taking the start address of the SDI page as the first address, offset backward by 0x63+first value+0x21 bytes and record the current addressing address as the first address;

[0088] S204: taking the first address as the first address, reading the content of 0x02 bytes continuously, and determining whether the content of the read 0x02 bytes is 0x789C, if yes, executing step S205, otherwise, executing step S300;

[0089] S205: Taking the first address as the first address, shifting forward 0x02 bytes and obtaining the content of 0x02 bytes in a big-endian format as the second value;

[0090] S206: Determine whether the sum of 0x63+first value+0x21+second value is less than 0x4000. If yes, execute step S207; otherwise, execute step S300.

[0091] S207: Taking the first address as the first address, continuously read the byte content whose byte length is the second value and decompress it using the Inflate algorithm, and use the decompressed data as the serialized dictionary information.

[0092] S300: Obtaining the table structure in the serialized dictionary information.

[0093] Figure 4 FIG. 2 shows a specific flow chart of obtaining the table structure in the serialized dictionary information in an embodiment of the present invention. Figure 4 As shown, step S300 includes the following steps:

[0094] S301: According to the structural characteristics of the JSON format, a dd_object node, that is, a data dictionary object node, is retrieved in the serialized dictionary information;

[0095] S302: According to the structural characteristics of the JSON format, the options node is retrieved in the dd_object node, that is, the option node, and the value of the options node is obtained and saved as a table attribute;

[0096] S303: According to the structural characteristics of the JSON format, a row_format node, that is, a row format node, is retrieved in the dd_object node, and the value of the row_format node is obtained and saved as the row format of the table;

[0097] 304. According to the structural characteristics of the JSON format, retrieve the columns node, that is, the column node, in the dd_object node, obtain the value of the columns node and save it as the field definition of the table.

[0098] S400: Parse the data in the DATA page using the table structure, restore the table record and obtain the data of the ibd file.

[0099] Figure 5 The specific flow chart of restoring table records and obtaining data of ibd files in an embodiment of the present invention is shown. Figure 5 As shown, step S400 includes the following steps:

[0100] S401: Taking the starting address of the DATA page as the first address, offset 0x63 bytes backward and record the current addressing address as the second address;

[0101] S402: Taking the second address as the first address, offset forward 0x02 bytes and obtain the content of 0x02 bytes in a big-endian format as the third value;

[0102] S403: taking the second address as the first address, shifting backward by a byte length equal to a third value and recording the current addressing address as the second address;

[0103] S404: Using the second address as the first address, offset forward by 0x02 bytes and obtain the content of 0x02 bytes in a big-endian format as the third value;

[0104] S405: taking the second address as the first address, reading the content of 0x08 bytes continuously, and determining whether the content of the read 0x08 bytes is 0x73757072656D756D, if yes, executing step S406, otherwise, executing step S410;

[0105] S406: Using the start address of the DATA page as the first address, offset 0x2C bytes backward and obtain the content of 0x02 bytes in a big-endian format as the third value;

[0106] S407: Determine whether the current third value is equal to 0x0000, if yes, end the process, otherwise, execute step S408;

[0107] S408: Taking the start address of the DATA page as the first address, offset backward by the byte length of the current third value and record the current addressing address as the second address;

[0108] S409: Taking the second address as the first address, shifting forward 0x02 bytes and obtaining the content of 0x02 bytes in a big-endian format as the third value;

[0109] S410: According to the row format and field definition in the table structure, the data is parsed backwards with the current second address as the first address, and the parsing result is saved as a table record;

[0110] S411: Determine whether the current third value is equal to 0x0000. If so, end the process; otherwise, execute step S403.

[0111] The method provided by the present invention solves the technical problem that there is no data recovery method for damaged MySQL 8.0 ibd files in the prior art.

[0112] It should be understood that the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A data recovery method for a damaged MySQL 8.0 ibd file, characterized in that The following steps are involved: S100: Obtaining an ibd file and scanning the ibd file, obtaining an SDI page and a DATA page according to the data structure characteristics of the page, including the following steps: S101: Retrieve the infimum pseudo record identifier in the ibd file; S102: Determine whether the infimum pseudo record identifier is retrieved, if yes, execute step S103, otherwise, execute step S200; S103: Record the starting address of the infinite pseudo record marker, take the starting address of the infinite pseudo record marker as the first address, shift backward by 0x0D bytes and obtain the content of 0x08 bytes in succession; S104: Determine whether the content of the acquired 0x08 byte is a supremum pseudo record identifier, if yes, execute step S105, otherwise, execute step S101; S105: Taking the starting address of the hypodetermined pseudo record marker as the first address, offset forward by 0x4B bytes and obtain the content of 0x02 bytes in succession; S106: Determine whether the content of the acquired 0x02 byte is an SDI page identifier, if yes, execute step S108, otherwise, execute step S107; S107: Determine whether the content of the acquired 0x02 byte is a DATA page identifier, if yes, execute step S109, otherwise, execute step S101; S108: Mark the current page as an SDI page, and execute step S110; S109: Mark the current page as a DATA page and execute step S110; S110: Take the starting address of the hypofinite pseudo record marker as the first address, shift forward 0x73 bytes and obtain the content of 0x4000 bytes as the SDI page data / DATA page data, and execute step S101; S200: extracting serialized dictionary information in the SDI page; S300: Obtaining the table structure in the serialized dictionary information; S400: Parse the data in the DATA page using the table structure, restore the table record and obtain the data of the ibd file.

2. A data recovery method for a damaged MySQL 8.0 ibd file according to claim 1, characterized in that: Step S200 includes the following steps: S201: Taking the start address of the SDI page as the first address, offset 0x61 bytes backward and obtain the content of 0x02 bytes in a big-endian format as the first value; S202: Determine whether the sum of 0x63+first value+0x21 is less than 0x4000. If yes, execute step S203; otherwise, execute step S300. S203: Taking the start address of the SDI page as the first address, offset backward by 0x63+first value+0x21 bytes and record the current addressing address as the first address; S204: taking the first address as the first address, reading the content of 0x02 bytes continuously, and determining whether the content of the read 0x02 bytes is 0x789C, if yes, executing step S205, otherwise, executing step S300; S205: Taking the first address as the first address, shifting forward 0x02 bytes and obtaining the content of 0x02 bytes in a big-endian format as the second value; S206: Determine whether the sum of 0x63+first value+0x21+second value is less than 0x4000. If yes, execute step S207; otherwise, execute step S300. S207: Taking the first address as the first address, continuously read the byte content whose byte length is the second value and decompress it, and use the decompressed data as the serialized dictionary information.

3. A data recovery method for a damaged MySQL 8.0 ibd file according to claim 1, characterized in that: Step S300 includes the following steps: S301: Retrieving a data dictionary object node in the serialized dictionary information according to the structural characteristics of the JSON format; S302: According to the structural characteristics of the JSON format, the option node is searched in the data dictionary object node, the value of the option node is obtained and saved as a table attribute; S303: According to the structural characteristics of the JSON format, a row format node is retrieved in the data dictionary object node, the value of the row format node is obtained and saved as the row format of the table; S304. According to the structural characteristics of the JSON format, retrieve the column node in the data dictionary object node, obtain the value of the column node and save it as the field definition of the table.

4. A data recovery method for a damaged MySQL 8.0 ibd file according to claim 1, characterized in that: Step S400 includes the following steps: S401: Taking the starting address of the DATA page as the first address, offset 0x63 bytes backward and record the current addressing address as the second address; S402: Taking the second address as the first address, offset forward 0x02 bytes and obtain the content of 0x02 bytes in a big-endian format as the third value; S403: taking the second address as the first address, shifting backward by a byte length equal to a third value and recording the current addressing address as the second address; S404: Using the second address as the first address, offset forward by 0x02 bytes and obtain the content of 0x02 bytes in a big-endian format as the third value; S405: taking the second address as the first address, reading the content of 0x08 bytes continuously, and determining whether the content of the read 0x08 bytes is 0x73757072656D756D, if yes, executing step S406, otherwise, executing step S410; S406: Using the start address of the DATA page as the first address, offset 0x2C bytes backward and obtain the content of 0x02 bytes in a big-endian format as the third value; S407: Determine whether the current third value is equal to 0x0000, if yes, end the process, otherwise, execute step S408; S408: Taking the start address of the DATA page as the first address, offset backward by the byte length of the current third value and record the current addressing address as the second address; S409: Taking the second address as the first address, offset forward 0x02 bytes and obtain the content of 0x02 bytes in a big-endian format as the third value; S410: According to the row format and field definition in the table structure, the data is parsed backwards with the current second address as the first address, and the parsing result is saved as a table record; S411: Determine whether the current third value is equal to 0x0000. If so, end the process; otherwise, execute step S403.

5. A data recovery method for a damaged MySQL 8.0 ibd file according to claim 1, characterized in that: In step S101, the KMP algorithm is used to retrieve the infimum pseudo record identifier.

6. A data recovery method for a damaged MySQL 8.0 ibd file according to claim 1, characterized in that: The infimum pseudo record identifier is 0x696E66696D756D00, the supremum pseudo record identifier is 0x73757072656D756D, the SDI page identifier is 0x45BD, and the DATA page identifier is 0x45BF.

7. A data recovery method for a damaged MySQL 8.0 ibd file according to claim 2, characterized in that: Use the Inflate algorithm to decompress.

Images