File metadata recovery methods, devices, electronic equipment, and storage media

Abstract

This invention provides a file metadata recovery method, apparatus, electronic device, and storage medium, relating to the field of distributed storage technology. First, the metadata object to be recovered is obtained, and the inode information of the target directory to which the metadata object belongs is determined. By applying the mapping relationship between the directory inode information and the inode information of files within the directory, the inode information of the target file in the target directory can be quickly determined. By improving the naming rules of each data object in the data pool and combining the inode information of the target file and the inode information of the target directory, the data object corresponding to the target directory can be quickly found in the data pool. This search process does not require traversing the entire data pool, greatly improving search efficiency and shortening the search time, thereby improving recovery efficiency, shortening recovery time, and making the file metadata recovery process imperceptible to the user, thus improving the user experience.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, and in particular to a method, apparatus, electronic device, and storage medium for recovering file metadata. Background Technology

[0002] With the explosive growth of data storage, traditional storage architectures have gradually become unable to meet the needs of modern applications in terms of data scale, performance, reliability, and scalability, leading to the emergence of distributed storage. Distributed file storage is a commonly used solution, which uses multiple storage devices to form a cluster and provides file storage services based on network protocols such as Server Message Block (SMB) and Network File System (NFS).

[0003] Currently, distributed file storage employs a metadata and data separation storage approach. File data is stored in a data pool, while file / directory metadata is stored in a metadata pool. The storage unit in the pool is an object, and each file in the data pool has 1 to n data objects storing its data, depending on its size. The metadata for all entries under a normal directory is recorded on the same metadata object in the metadata pool. The metadata for each entry is managed by the distributed file system's Metadata Server (MDS) cluster. Metadata is not directly written to the metadata object; instead, it is first written to a log object, updated in the MDS cache, and then written to the metadata object.

[0004] A typical file write operation can be summarized as follows: retrieve file metadata; write file data; update file metadata. For example... Figure 1 As shown, the clients in the distributed file system include device A and device B, each containing a kernel driver module. The object-based storage device (OSD) in the distributed file system includes a data pool and a metadata pool. When clients such as device A and device B perform file write operations, they write the file data to the data pool, store the file metadata through the MDS cache, and write the file metadata to the metadata pool.

[0005] In real-world production environments, data objects may be successfully written, but in situations such as device power failure, MDS anomalies, or log object corruption, metadata objects may become corrupted or lost. For corrupted or lost metadata objects, their directories may appear empty, but the file data that was actually written to the data object still exists in the data pool. Therefore, the conventional approach to recovering file metadata from metadata objects is to locate the corrupted or lost file metadata data object in the data pool, regenerate the file metadata, and then populate it back into the metadata object. However, this requires a global scan of all data objects in the data pool, which is not only time-consuming and inefficient, but also significantly impacts front-end business traffic and degrades user experience, especially in production environments where there is often substantial front-end business traffic. Summary of the Invention

[0006] This invention provides a method, apparatus, electronic device, and storage medium for recovering file metadata, in order to overcome the deficiencies existing in related technologies.

[0007] This invention provides a method for recovering file metadata, comprising:

[0008] Obtain the metadata object to be recovered, and determine the inode information of the target directory to which the metadata object to be recovered belongs;

[0009] Based on the index node information of the target directory, the mapping relationship between the directory index node information and the index node information of the files in the directory is applied to determine the index node information of the target files in the target directory;

[0010] Based on the inode information of the target file and the inode information of the target directory, find the data object corresponding to the target directory in the data pool;

[0011] Based on the metadata of the data object corresponding to the target directory, the file metadata is reconstructed, and the file metadata is written into the metadata object to be recovered;

[0012] Each data object in the data pool is named based on the corresponding file inode information and the file's parent directory inode information.

[0013] According to a file metadata recovery method provided by the present invention, the step of reconstructing the file metadata based on the metadata of the data object corresponding to the target directory includes:

[0014] Scan the data objects corresponding to the target directory, and add the metadata of the last data object of each file in the target directory to the extended attribute of the first data object of each file;

[0015] Read the extended attributes of the first data object of each file, and reconstruct the file metadata of each file based on the reading results.

[0016] According to a file metadata recovery method provided by the present invention, the mapping relationship is determined based on the following steps:

[0017] The mapping relationship is obtained by using the inode information of any non-soft link file with a non-zero reference count created in each directory as the value and the inode information of each directory as the key, stored in the form of key-value pairs.

[0018] According to a file metadata recovery method provided by the present invention, the step of searching for the data object corresponding to the target directory in the data pool based on the inode information of the target file and the inode information of the target directory includes:

[0019] Based on the inode information of the target file and the inode information of the target directory, the name of the reference data object is determined;

[0020] Based on the reference data object name, search for the specified data object name containing the index node information of the target directory in the RocksDB corresponding to the data pool, and use the data object corresponding to the specified data object name as the data object corresponding to the target directory;

[0021] In RocksDB, data objects are stored using their names as keys and their storage locations within the data pool as values.

[0022] According to a file metadata recovery method provided by the present invention, the step of searching for a specified data object name containing the same inode information of the target directory in the RocksDB corresponding to the data pool based on the reference data object name includes:

[0023] Construct a RocksDB iterator based on the name of the reference data object;

[0024] Use the RocksDB iterator to find the specified data object name in the RocksDB.

[0025] According to a file metadata recovery method provided by the present invention, the step of obtaining the metadata object to be recovered includes:

[0026] In response to a user's request to view the target directory, determine whether there is any abnormal information in the metadata object of the target directory;

[0027] If the metadata object of the target directory contains the abnormal information, then the metadata object of the target directory is determined to be the metadata object to be recovered.

[0028] According to a file metadata recovery method provided by the present invention, determining the inode information of the target directory to which the metadata object to be recovered belongs includes:

[0029] Determine the parent directory metadata object of the target directory;

[0030] Based on the parent directory metadata object, the inode information of the target directory is determined.

[0031] The present invention also provides a file metadata recovery device, comprising:

[0032] The acquisition module is used to acquire the metadata object to be recovered and determine the inode information of the target directory to which the metadata object to be recovered belongs;

[0033] The relationship application module is used to determine the index node information of the target file in the target directory by applying the mapping relationship between the index node information of the directory and the index node information of the files in the directory based on the index node information of the target directory.

[0034] The object lookup module is used to look up the data object corresponding to the target directory in the data pool based on the inode information of the target file and the inode information of the target directory;

[0035] The metadata reconstruction module is used to reconstruct the file metadata based on the metadata of the data object corresponding to the target directory, and write the file metadata into the metadata object to be recovered;

[0036] Each data object in the data pool is named based on the corresponding file inode information and the file's parent directory inode information.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the file metadata recovery method as described above.

[0038] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the file metadata recovery method as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the file metadata recovery method as described above.

[0040] The file metadata recovery method, apparatus, electronic device, and storage medium provided by this invention first acquire the metadata object to be recovered and determine the inode information of the target directory to which the metadata object belongs. By applying the mapping relationship between the directory inode information and the inode information of the files in the directory, the inode information of the target file in the target directory can be quickly determined. By improving the naming rules of each data object in the data pool and combining the inode information of the target file and the inode information of the target directory, the data object corresponding to the target directory can be quickly found in the data pool. This search process does not require traversing the entire data pool, greatly improving search efficiency and shortening the search time. Finally, using the metadata of the data object corresponding to the target directory, the file metadata is reconstructed and written into the metadata object to be recovered. This achieves the effect of quickly recovering the file metadata in the metadata object to be recovered, improving recovery efficiency, shortening recovery time, and making the file metadata recovery process imperceptible to the user, thus improving the user experience. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the architecture of an existing distributed file system.

[0043] Figure 2 This is a diagram illustrating the situation where there are metadata objects in the existing metadata pool whose content is missing.

[0044] Figure 3 This is a diagram illustrating the relationship between the restored metadata objects in the existing metadata pool and the data objects in the data pool.

[0045] Figure 4 This is one of the flowcharts illustrating the file metadata recovery method provided by the present invention.

[0046] Figure 5 This is the second flowchart illustrating the file metadata recovery method provided by this invention.

[0047] Figure 6 This is a schematic diagram of the file metadata recovery device provided by the present invention.

[0048] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] In current distributed file system applications, power outages and other anomalies can lead to corrupted or lost metadata objects. Existing recovery tools can only fully traverse every data object in the data pool. Since the extended attributes of data objects store backtracking information about file paths, this backtracking information can be used to confirm whether a data object belongs to a corrupted metadata object. Then, the file metadata is reconstructed using the data object's information and populated back into the metadata object. After this recovery process is complete, files in the directory corresponding to the previously corrupted or lost metadata object can be accessed normally. The following describes the recovery process in the current solution.

[0051] like Figure 2 As shown, assume the metadata pool contains three metadata objects named 10000000000.00000000 / , 10000000001.00000000 / dir1, and 10000000002.00000000 / dir2. The content of metadata object 10000000000.00000000 includes the metadata of dir1. The content of metadata object 10000000001.00000000 / dir1 includes the metadata of dir2. The content of metadata object 10000000002.00000000 / dir2 is lost, meaning that metadata object 10000000002.00000000 / dir2 is corrupted. The corrupted metadata object 10000000002.00000000 / dir2 belongs to the directory / dir1 / dir2 / . While the directory / dir1 / dir2 is still visible, the files within it are not, rendering / dir1 / dir2 an empty directory. Therefore, it is necessary to reconstruct the file metadata under / dir1 / dir2 by scanning the data pool and populate it into a new metadata object 10000000002.00000000.

[0052] Since there are many data objects in the data pool, the scanning process will use multiple threads. First, the data objects in the data pool will be divided into n groups, and then each thread will scan only one group of data objects.

[0053] First, a global scan of the data pool is performed to collect the metadata of the data objects at the end of each file, such as object identifier, size, and modification time. The object identifier can be an object ID. This metadata is then recorded in the extended attribute (xattr) of the first data object in the file. For example, a file named `file` has three data objects with names: 1000000000a.00000000, 1000000000a.00000001, and 1000000000a.00000002. Here, 1000000000a is the inode number of the file `file`, and 00000000, 00000001, and 00000002 are the object identifiers of the data objects.

[0054] ① When data object 1000000000a.00000001 is encountered during scanning, the metadata such as the object identifier (scan_ceiling), size (scan_max_size), and modification time (scan_max_mtime) are directly recorded in the extended attribute of the first data object of the file corresponding to data object 1000000000a.00000001. That is, the xattr of data object 1000000000a.00000000 is updated to:

[0055] scan_ceiling: 00000001

[0056] scan_max_mtime: "2020-07-12 16:31",

[0057] scan_max_size: 4MB.

[0058] ② When the data object 1000000000a.00000000 is scanned, since the object identifier of the data object is 00000000, which is smaller than the object identifier of the data object scanned in ①, the xattr of the data object 1000000000a.00000000 will not be updated.

[0059] ③ When data object 1000000000a.00000002 is scanned, its object identifier is 00000002, which is larger than the object identifier of the data object scanned in ①. Therefore, the xattr of data object 1000000000a.00000000 needs to be updated to:

[0060] scan_ceiling: 00000002

[0061] scan_max_mtime: "2020-07-12 16:32",

[0062] scan_max_size: 2 MB.

[0063] After the scan is complete, the extended attributes of the first and last data objects of the file `file` are read. The file metadata is reconstructed using the read results and written back to the metadata object `10000000002.00000000`. Since the file `file` with inode number 1000000000a has 3 data objects, and the size of the last data object is 2MB, and based on the default setting that the size of a fully written data object is 4MB, the size of the file `file` can be calculated to be 4MB. 2 + 2MB = 10MB, last modified on 2020-07-12 16:32.

[0064] In addition, the filename and directory to which the file belongs can be read from the extended properties of the first data object of the file.

[0065] like Figure 3 As shown, after the above operations, if everything goes smoothly, the directory / dir1 / dir2 / will be restored to its previous state and will no longer be displayed as an empty directory.

[0066] However, existing solutions require a global scan of all data objects in the data pool, which not only results in time-consuming and inefficient file metadata recovery, but also severely impacts front-end business operations and degrades user experience due to the typically high traffic volume in production environments. Therefore, this invention provides a file metadata recovery method.

[0067] Figure 4 This invention provides a method for recovering file metadata in an embodiment of the present invention. Figure 4 As shown, the method includes:

[0068] S1, obtain the metadata object to be recovered, and determine the inode information of the target directory to which the metadata object to be recovered belongs;

[0069] S2, based on the index node information of the target directory, apply the mapping relationship between the directory index node information and the index node information of the files in the directory to determine the index node information of the target files in the target directory;

[0070] S3, based on the index node information of the target file and the index node information of the target directory, find the data object corresponding to the target directory in the data pool;

[0071] S4, based on the metadata of the data object corresponding to the target directory, reconstruct the file metadata, and write the file metadata into the metadata object to be recovered;

[0072] Each data object in the data pool is named based on the corresponding file inode information and the file's parent directory inode information.

[0073] Specifically, the file metadata recovery method provided in this embodiment of the invention is executed by a file metadata recovery device, which can be configured in a computer. The computer can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.

[0074] First, execute step S1 to obtain the metadata object to be recovered. The metadata object to be recovered refers to the metadata object that has been damaged or lost due to power failure, MDS abnormality or log object corruption, and whose metadata needs to be recovered.

[0075] The metadata in the metadata object to be recovered includes the metadata of all entries contained in the target directory to which the metadata to be recovered belongs. Here, entries can include directories and / or files; that is, the target directory can include only directories, only files, or both. Correspondingly, the metadata in the metadata object to be recovered can include only directory metadata, only file metadata, or both. In this embodiment of the invention, only the recovery and backfilling of file metadata in the metadata object to be recovered is involved; the recovery and backfilling of directory metadata in the metadata object to be recovered is not limited.

[0076] The metadata object to be recovered can be determined when a device loses power, MDS becomes abnormal, or log object is corrupted during the file or metadata writing process, or when the user needs to access the target directory; no specific limitation is made here.

[0077] Furthermore, the inode information of the target directory to which the metadata object to be recovered belongs can be determined. The inode information is the identification information of the inode, such as the inode number. Here, the inode of the target directory is the data structure that stores the metadata of the target directory.

[0078] Then, step S2 is executed. The mapping relationship between directory inode information and the inode information of files within the directory is applied, and combined with the inode information of the target directory, the inode information of the target file in the target directory is determined. Here, in the mapping relationship, the directory inode information can be the inode number of each directory, and the inode information of the file within the directory can be the inode number of any non-soft link file within the directory with a non-zero reference count. In other words, in the mapping relationship, the inode number of each directory corresponds to the inode number of a file within that directory.

[0079] The mapping between directory inode information and the inode information of files within the directory can be constructed during file writing and stored in a metadata object called the recover_table. The object structure of recover_table can be shown in Table 1.

[0080] Table 1 Object structure of recover_table

[0081]

[0082] In Table 1, the inode number of dir1 represents the inode number of directory dir1, and the inode number of file1 represents the inode number of file1 under directory dir1. There is a mapping relationship between the two. Similarly, the inode number of dir2 represents the inode number of directory dir2, and the inode number of file2 represents the inode number of file2 under directory dir2. There is a corresponding relationship between the two.

[0083] By finding the inode number of the target directory in Table 1, the corresponding file inode number is determined, that is, the file inode number that is in the same row as the inode number of the target directory. This file inode number is used as the index node information of the target file in the target directory.

[0084] It is understandable that, since the mapping relationship is between a directory and a file under it, if the inode information of the target file is found, it means that the target directory contains at least the target file, and may also contain other files or directories.

[0085] Specifically, if no file inode information corresponding to the inode information of the target directory is found in the mapping relationship, it can be assumed that there is no file in the target directory, i.e., the target file does not exist. In this case, the process can be terminated directly without further recovery work.

[0086] Then, step S3 is executed, which uses the inode information of the target file and the inode information of the target directory to find the data object corresponding to the target directory in the data pool.

[0087] In this embodiment of the invention, each data object in the data pool is named based on the corresponding file index node information and the file parent directory index node information.

[0088] The naming format for each data object in the data pool can be represented as follows:

[0089] <File parent directory inode information>.<File inode information>.<Data object object identifier>;

[0090] For example, it could be <directory inode number>.<file inode number>.<object id>.

[0091] If a file named file2 exists under the path / dir1 / dir2, and assuming that the inode number of dir2 is 1000000000a and the inode number of file2 is 1000000000b, then the name of the data object of file2 in the data pool is:

[0092] 1000000000a.1000000000b.00000000.

[0093] If the size of file2 exceeds 4MB, then increment the data object id and continue generating new data objects. The name of the new data object is:

[0094] 1000000000a.1000000000b.00000001.

[0095] Similarly, for each new data object written after 4MB of content, the name only needs to include the data object id.

[0096] According to the naming rules of each data object in the data pool, the name of each data object contains the inode information of the file's parent directory and the file's inode information. Therefore, by combining the inode information of the target file and the inode information of the target directory, the data object corresponding to the target directory can be found in the data pool. That is, all data objects in the data pool whose names contain the inode information of the target directory. Here, the data objects corresponding to the target directory are all data objects used to store files under the target directory.

[0097] Finally, step S4 is executed to reconstruct the file metadata using the metadata of the data objects corresponding to the target directory. Here, the metadata of the data objects can include object identifiers, dimensions, and modification times, etc. The metadata of the first data object of a file also includes the filename and its directory. Based on this, the file metadata of each file in the target directory can be reconstructed using the metadata of the data objects corresponding to the target directory.

[0098] By writing the file metadata of each file into the metadata object to be recovered, the file metadata in the metadata object to be recovered can be recovered, resulting in a recovered metadata object. Through this recovered metadata object, the files it contains can be seen in the target directory.

[0099] The file metadata recovery method provided in this embodiment of the invention first obtains the metadata object to be recovered and determines the inode information of the target directory to which the metadata object belongs. By applying the mapping relationship between the directory inode information and the inode information of the files in the directory, the inode information of the target file in the target directory can be quickly determined. By improving the naming rules of each data object in the data pool and combining the inode information of the target file and the inode information of the target directory, the data object corresponding to the target directory can be quickly found in the data pool. This search process does not require traversing the entire data pool, which greatly improves search efficiency and shortens the search time. Finally, using the metadata of the data object corresponding to the target directory, the file metadata is reconstructed and written into the metadata object to be recovered. This achieves the effect of quickly recovering the file metadata in the metadata object to be recovered, improving recovery efficiency, shortening recovery time, and making the file metadata recovery process imperceptible to the user, thus improving the user experience.

[0100] Based on the above embodiments, reconstructing the file metadata based on the metadata of the data object corresponding to the target directory includes:

[0101] Scan the data objects corresponding to the target directory, and add the metadata of the last data object of each file in the target directory to the extended attribute of the first data object of each file;

[0102] Read the extended attributes of the first data object of each file, and reconstruct the file metadata of each file based on the reading results.

[0103] Specifically, in the process of reconstructing file metadata, you can first scan each data object corresponding to the target directory, and add the metadata of the last data object of each file in the target directory to the extended attributes of the first data object of each file.

[0104] For the current data object being scanned, if the object identifier of the current data object is greater than the object identifier of the previously scanned data object, then the metadata of the current data object is used to overwrite the metadata of the data object in the extended attributes of the first data object in the file to which the current data object belongs. If the metadata is less than or equal to the current data object, then no action is taken.

[0105] In particular, if the current data object is the first data object scanned, the data objects scanned previously do not exist, and the metadata of the current data object can be directly written into the extended attributes of the first data object.

[0106] Therefore, after scanning each data object corresponding to the target directory, the metadata of the last data object of each file is stored in the extended attribute of the first data object of each file, that is, the data object with the largest object identifier.

[0107] Next, the extended attributes of the first data object of each file are read. Each file's first data object's extended attributes contain information such as the filename, its directory, and metadata from the last data object. Using the information in the extended attributes, the file metadata of each file can be reconstructed. For example, the filename can be directly determined from the extended attributes; the size of the file metadata can be calculated based on the size of the last data object and the default size (4MB) of the previous data objects; and the modification time of the file metadata can be the modification time of the last data object, etc.

[0108] In this embodiment of the invention, by adding the metadata of the last data object of each file in the target directory to the extended attribute of the first data object of each file, the metadata of the data objects can be summarized, providing a data foundation for the subsequent reconstruction of the file metadata of each file. Moreover, the extended attribute of the first data object ultimately contains the metadata of the last data object, and it is not necessary to write the metadata of all data objects of each file into the extended attribute of the first data object, which can reduce the amount of data written and improve the reconstruction efficiency.

[0109] Based on the above embodiments, the mapping relationship is determined according to the following steps:

[0110] The mapping relationship is obtained by using the inode information of any non-soft link file with a non-zero reference count created in each directory as the value and the inode information of each directory as the key, stored in the form of key-value pairs.

[0111] Specifically, the mapping relationship provided in this embodiment of the invention can be that the directory index node information and the index node information of the files under the directory are stored in the form of key-value pairs, for example, in a metadata object named recovery_table. In this mapping relationship, the directory index node information serves as the key, and the index node information of the files under the directory serves as the value.

[0112] In the mapping relationship, files in the directory can be any non-symbolic link created within the directory with a non-zero reference count. symbolic links in the directory should be skipped because they don't have corresponding data objects in the data pool, so building a mapping relationship for them is meaningless. If a file has a zero reference count and is actually deleted, the inode information of any other file in the directory needs to be found to update the mapping relationship. If there are no other files in the directory, then simply deleting the mapping relationship for that file is sufficient.

[0113] Preferably, when the first non-symbolic link file is created in the directory, the mapping between the file's inode information and the directory's inode information can be written into a metadata object named `recover_table`. This way, subsequent newly written files do not need to have their mappings built, avoiding unnecessary delays.

[0114] In this embodiment of the invention, a mapping relationship is constructed by selecting any non-soft link type file in the directory with a non-zero reference count. This can reduce the workload of constructing the mapping relationship and improve its quality.

[0115] Based on the above embodiments, the step of finding the data object corresponding to the target directory in the data pool based on the inode information of the target file and the inode information of the target directory includes:

[0116] Based on the inode information of the target file and the inode information of the target directory, the name of the reference data object is determined;

[0117] Based on the reference data object name, search for the specified data object name containing the index node information of the target directory in the RocksDB corresponding to the data pool, and use the data object corresponding to the specified data object name as the data object corresponding to the target directory;

[0118] In RocksDB, data objects are stored using their names as keys and their storage locations within the data pool as values.

[0119] Specifically, in the process of finding the data object corresponding to the target directory in the data pool, the inode information of the target file and the inode information of the target directory can be used to determine the name of the reference data object. Here, since the name of the data object needs to contain the object identifier of the data object, and each file must have a first-order data object, and the object identifier of the first-order data object uses the same representation method, for example, it can be represented as 00000000.

[0120] Therefore, the inode information of the target file and the inode information of the target directory can be combined and then concatenated with the object identifier of the first and second data objects to obtain the name of the reference data object.

[0121] Subsequently, using the reference data object name, the specified data object name containing the inode information of the target directory can be found in the RocksDB corresponding to the data pool. RocksDB is a high-performance, persistent embedded key-value store based on an LSM-Tree, used to store the names of each data object in the data pool and their storage location within the data pool as key-value pairs. In RocksDB, the data object name is the key, and the storage location of the data object in the data pool is the value.

[0122] One of RocksDB's storage characteristics is that the keys of key-value pairs are ordered; that is, key-value pairs are stored in order according to their keys. Therefore, given the name of a reference data object, the starting search position in RocksDB can be quickly located. Furthermore, based on the ordered nature of the keys, the specified data object name in RocksDB, which contains the inode information of the target file from the reference data object name, can be quickly found.

[0123] Ultimately, the data object corresponding to the specified data object name can be used as the data object corresponding to the target directory, that is, the data object of each file in the target directory.

[0124] In this embodiment of the invention, by modifying the name structure of data objects and combining it with the storage characteristics of RocksDB, the effect of accelerating the search for data objects corresponding to the target directory can be achieved, thereby shortening the recovery time when the metadata objects to be recovered are damaged.

[0125] Based on the above embodiments, the step of searching for a specified data object name in the RocksDB corresponding to the data pool that contains the same inode information as the target directory, based on the reference data object name, includes:

[0126] Construct a RocksDB iterator based on the name of the reference data object;

[0127] Use the RocksDB iterator to find the specified data object name in the RocksDB.

[0128] Specifically, in this embodiment of the invention, a RocksDB iterator can be constructed using a reference data object name. That is, the parameters of the RocksDB iterator can be configured using the reference data object name so that the RocksDB iterator can locate the reference data object name.

[0129] Subsequently, using the RocksDB iterator, a search for a specified data object name can be performed within a small range before and after the reference data object name in RocksDB. This is because keys in RocksDB are stored sequentially. Therefore, after locating the reference data object name, the RocksDB iterator can continuously expand the search range before and after the reference data object name until the search ends when no data object containing the inode information of the target directory is found. The data object names that contain the inode information of the target directory are then used as the specified data object name. The smaller range can be the area bounded by data object names that do not contain the inode information of the target directory before or after the reference data object name.

[0130] In this embodiment of the invention, the RocksDB iterator can be used to perform bidirectional lookups before and after the reference data object name, thereby improving lookup efficiency and improving the recovery efficiency of file metadata, greatly increasing the availability of the recovery solution in the production environment.

[0131] Based on the above embodiments, obtaining the metadata object to be recovered includes:

[0132] In response to a user's request to view the target directory, determine whether there is any abnormal information in the metadata object of the target directory;

[0133] If the metadata object of the target directory contains the abnormal information, then the metadata object of the target directory is determined to be the metadata object to be recovered.

[0134] Specifically, in this embodiment of the invention, the metadata object to be restored can be determined when the user needs to access the target directory. When the user needs to access the target directory, the user can send a request to view the target directory by clicking the icon of the target directory.

[0135] At this point, the executing entity can respond to the user's request to view the target directory and determine whether there is any abnormal information in the metadata object of the target directory. This abnormal information can be feedback information automatically generated by the system when the device loses power, the MDS is abnormal, or the log object is corrupted during the file or metadata writing process, which is used to indicate that the metadata object is corrupted or lost.

[0136] If the metadata object of the target directory contains abnormal information, the metadata object of the target directory is determined to be a metadata object to be recovered, and steps S2-S4 are executed to recover the metadata object to be recovered, so that the files in the target directory can be displayed normally on the display interface after the user clicks the icon of the target directory, avoiding the situation where the target directory is an empty directory on the display interface.

[0137] In this embodiment of the invention, by responding to the user's request to view the target directory, it is determined whether there is abnormal information in the metadata object of the target directory, and then the metadata object to be recovered is identified. This can avoid the situation of inaccurate recovery, large recovery workload, and redundant work caused by immediately recovering when the metadata object to be recovered is damaged or lost.

[0138] Based on the above embodiments, determining the inode information of the target directory to which the metadata object to be recovered belongs includes:

[0139] Determine the parent directory metadata object of the target directory;

[0140] Based on the parent directory metadata object, the inode information of the target directory is determined.

[0141] Specifically, when determining the inode information of the target directory to which the metadata object to be recovered belongs, the parent directory metadata object of the target directory can be determined first. This parent directory metadata object refers to the metadata object of the parent directory of the target directory. For example, the target directory is / dir1 / dir2, and the parent directory of the target directory is / dir1.

[0142] Since the metadata object of the target directory's parent directory stores the inode information of the target directory, the inode information of the target directory can be determined by reading the contents of the metadata object of the target directory's parent directory.

[0143] In this embodiment of the invention, the index node information of the target directory is determined by the metadata object of the parent directory of the target directory, which can make full use of the known data and reduce the difficulty of obtaining the index node information of the target directory.

[0144] like Figure 5 The diagram illustrates the specific implementation steps of a file metadata recovery method provided in this embodiment of the invention, including:

[0145] 1) Determine that the metadata object of the directory / dir1 / dir2 is corrupted, that is, the metadata object of the directory / dir1 / dir2 is the metadata object to be recovered, and the directory / dir1 / dir2 is the target directory;

[0146] 2) The inode number of the directory / dir1 / dir2 can be obtained through the metadata object of the parent directory / dir1 of the target directory, i.e., the parent directory metadata object.

[0147] 3) Search for the inode number of the target file in the target directory in the metadata object named recover_table using the inode number of the directory / dir1 / dir2. If no inode number is found, it means that the directory / dir1 / dir2 is an empty directory and does not need to be recovered. Otherwise, determine the name of the reference data object based on the inode number of the directory / dir1 / dir2 and the inode number of the target file.

[0148] 4) Create a RocksDB iterator by referencing the data object name. Apply the RocksDB iterator to search for the specified data object name in RocksDB that contains inode information for the target directory. When the search key contains inode information for the target directory, expand the search scope and continue searching. When the search key does not contain inode information for the target directory, the search ends, indicating that the data object names of all files under the directories / dir1 / dir2 have been obtained.

[0149] 5) Organize the metadata of the data objects at the end of each file and write it into the extended attributes of the data objects at the beginning of each file. Reconstruct the file metadata based on the contents of the extended attributes of the data objects at the beginning of each file, and write the file metadata into the metadata objects of the directories / dir1 / dir2. That is, the recovery of the metadata objects to be recovered is completed.

[0150] In this embodiment of the invention, by reasonably modifying the naming format of data objects in the data pool and utilizing the characteristic of the underlying RocksDB to store key-value pairs in an ordered manner, it is only necessary to search a small range of data objects to complete the recovery of file metadata in lost or damaged metadata objects. It can quickly search for the data objects needed to recover the metadata objects to be recovered, achieving the effect of short recovery time and reduced impact on normal business, which can greatly increase the availability of the recovery solution in the production environment.

[0151] like Figure 6 As shown, based on the above embodiments, this embodiment of the invention provides a file metadata recovery device, including:

[0152] The acquisition module 61 is used to acquire the metadata object to be recovered and determine the inode information of the target directory to which the metadata object to be recovered belongs;

[0153] The relationship application module 62 is used to determine the index node information of the target file in the target directory by applying the mapping relationship between the index node information of the directory and the index node information of the files in the directory based on the index node information of the target directory.

[0154] The object lookup module 63 is used to look up the data object corresponding to the target directory in the data pool based on the index node information of the target file and the index node information of the target directory.

[0155] Metadata reconstruction module 64 is used to reconstruct the file metadata based on the metadata of the data object corresponding to the target directory, and write the file metadata into the metadata object to be recovered;

[0156] Each data object in the data pool is named based on the corresponding file inode information and the file's parent directory inode information.

[0157] Based on the above embodiments, the file metadata recovery device provided in this embodiment of the invention, wherein the metadata reconstruction module is specifically used for:

[0158] Scan the data objects corresponding to the target directory, and add the metadata of the last data object of each file in the target directory to the extended attribute of the first data object of each file;

[0159] Read the extended attributes of the first data object of each file, and reconstruct the file metadata of each file based on the reading results.

[0160] Based on the above embodiments, the file metadata recovery device provided in this embodiment of the invention further includes a relationship determination module, used for:

[0161] The mapping relationship is obtained by using the inode information of any non-soft link file with a non-zero reference count created in each directory as the value and the inode information of each directory as the key, stored in the form of key-value pairs.

[0162] Based on the above embodiments, the file metadata recovery device provided in this embodiment of the invention, wherein the object search module is specifically used for:

[0163] Based on the inode information of the target file and the inode information of the target directory, the name of the reference data object is determined;

[0164] Based on the reference data object name, search for the specified data object name containing the index node information of the target directory in the RocksDB corresponding to the data pool, and use the data object corresponding to the specified data object name as the data object corresponding to the target directory;

[0165] In RocksDB, data objects are stored using their names as keys and their storage locations within the data pool as values.

[0166] Based on the above embodiments, the file metadata recovery device provided in this embodiment of the invention, wherein the object search module is further specifically used for:

[0167] Construct a RocksDB iterator based on the name of the reference data object;

[0168] Use the RocksDB iterator to find the specified data object name in the RocksDB.

[0169] Based on the above embodiments, the file metadata recovery device provided in this embodiment of the invention, wherein the acquisition module is specifically used for:

[0170] In response to a user's request to view the target directory, determine whether there is any abnormal information in the metadata object of the target directory;

[0171] If the metadata object of the target directory contains the abnormal information, then the metadata object of the target directory is determined to be the metadata object to be recovered.

[0172] Based on the above embodiments, the file metadata recovery device provided in this embodiment of the invention, wherein the acquisition module is further specifically used for:

[0173] Determine the parent directory metadata object of the target directory;

[0174] Based on the parent directory metadata object, the inode information of the target directory is determined.

[0175] Specifically, the functions of each module in the file metadata recovery device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0176] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the file metadata recovery methods provided in the above embodiments.

[0177] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the file metadata recovery method provided in the above embodiments.

[0179] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the file metadata recovery method provided in the above embodiments. This computer-readable storage medium can be either a non-transitory computer-readable storage medium or a transient computer-readable storage medium, and is not specifically limited herein.

[0180] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0181] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recovering file metadata, characterized in that, include: Obtain the metadata object to be recovered, and determine the inode information of the target directory to which the metadata object to be recovered belongs; Based on the index node information of the target directory, the mapping relationship between the directory index node information and the index node information of the files in the directory is applied to determine the index node information of the target files in the target directory; Based on the inode information of the target file and the inode information of the target directory, find the data object corresponding to the target directory in the data pool; Based on the metadata of the data object corresponding to the target directory, the file metadata is reconstructed, and the file metadata is written into the metadata object to be recovered; Each data object in the data pool is named based on the corresponding file inode information and the file's parent directory inode information; The mapping relationship is determined based on the following steps: The mapping relationship is obtained by using the inode information of any non-soft link file with a non-zero reference count created in each directory as the value and the inode information of each directory as the key, stored in the form of key-value pairs.

2. The file metadata recovery method according to claim 1, characterized in that, The process of reconstructing the file metadata based on the metadata of the data object corresponding to the target directory includes: Scan the data objects corresponding to the target directory, and add the metadata of the last data object of each file in the target directory to the extended attribute of the first data object of each file; Read the extended attributes of the first data object of each file, and reconstruct the file metadata of each file based on the reading results.

3. The file metadata recovery method according to claim 1, characterized in that, The step of finding the data object corresponding to the target directory in the data pool based on the inode information of the target file and the inode information of the target directory includes: Based on the inode information of the target file and the inode information of the target directory, the name of the reference data object is determined; Based on the reference data object name, search for the specified data object name containing the index node information of the target directory in the RocksDB corresponding to the data pool, and use the data object corresponding to the specified data object name as the data object corresponding to the target directory; In RocksDB, data objects are stored using their names as keys and their storage locations within the data pool as values.

4. The file metadata recovery method according to claim 3, characterized in that, The step of searching for a specified data object name in the RocksDB corresponding to the data pool that contains the same inode information as the target directory, based on the reference data object name, includes: Construct a RocksDB iterator based on the name of the reference data object; Use the RocksDB iterator to find the specified data object name in the RocksDB.

5. The file metadata recovery method according to any one of claims 1-4, characterized in that, The process of obtaining the metadata object to be recovered includes: In response to a user's request to view the target directory, determine whether there is any abnormal information in the metadata object of the target directory; If the metadata object of the target directory contains the abnormal information, then the metadata object of the target directory is determined to be the metadata object to be recovered.

6. The file metadata recovery method according to any one of claims 1-4, characterized in that, The process of determining the inode information of the target directory to which the metadata object to be recovered belongs includes: Determine the parent directory metadata object of the target directory; Based on the parent directory metadata object, the inode information of the target directory is determined.

7. A file metadata recovery device, characterized in that, include: The acquisition module is used to acquire the metadata object to be recovered and determine the inode information of the target directory to which the metadata object to be recovered belongs; The relationship application module is used to determine the index node information of the target file in the target directory by applying the mapping relationship between the index node information of the directory and the index node information of the files in the directory based on the index node information of the target directory. The object lookup module is used to look up the data object corresponding to the target directory in the data pool based on the inode information of the target file and the inode information of the target directory; The metadata reconstruction module is used to reconstruct the file metadata based on the metadata of the data object corresponding to the target directory, and write the file metadata into the metadata object to be recovered; Each data object in the data pool is named based on the corresponding file inode information and the file's parent directory inode information; The mapping relationship is determined based on the following steps: The mapping relationship is obtained by using the inode information of any non-soft link file with a non-zero reference count created in each directory as the value and the inode information of each directory as the key, stored in the form of key-value pairs.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the file metadata recovery method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the file metadata recovery method as described in any one of claims 1-6.

Images