Method, device, computing device and storage medium for obtaining index node number

Abstract

The present application provides a method, apparatus, computing device and storage medium for obtaining an index node number, and the technical field to which it belongs is the field of storage technology. The method includes: when parsing the path information to be parsed, for the target path component in the path information, the identifier of the metadata file corresponding to the target path component can be obtained, and the first index block corresponding to the identifier of the metadata file is determined in the index block loaded into the memory, the first index block stores the correspondence between the identifier of the metadata file and the index node number, and the index node number corresponding to the identifier of the metadata file is determined based on the correspondence, and the data block corresponding to the first index block stores the data of the metadata file. By adopting the present application, the delay in parsing path information can be reduced.

Description

[0001] This application claims priority to Chinese patent application No. 201910695741.9 filed on July 30, 2019, entitled “Method, computer device and system for accelerating file system path resolution”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of storage technology, and in particular to a method, apparatus, computing device and storage medium for obtaining an index node number. Background Art

[0003] Using a key-value storage system to store file system metadata is one of the effective solutions to resolve the file system metadata performance bottleneck. The key-value storage system uses a "key" and "value" (Key Value, KV) approach to manage the data stored in it. This data management approach is very suitable for small data such as file system metadata. When the file system metadata is stored in a key-value storage system, the file name or other logical identifier of the file is generally used as the key, and the file system index node (inode) of the file is used as the value.

[0004] Before the file system performs various operations, it first needs to obtain the metadata of the target file. The file system parses the incoming path information (path resolution) and obtains the metadata of the target file through the directory entry (dentry, that is, a pair consisting of the file name and the inode number corresponding to the file name). Although existing file systems generally use a directory entry cache (dentry cache, dcache) to accelerate the performance of file system metadata, the space of the file system directory entry cache is limited, and the directory entry is a necessary parameter for many operations in the file system. Therefore, the directory entry cache cannot meet all the file system's search requests for directory entries. When the file system does not find the required directory entry in the directory entry cache, the file system needs to access the storage system by parsing the path information to obtain the required directory entry and file metadata.

[0005] Most key-value storage systems are based on the Log-Structured Merge tree (LSM tree). Key-value storage systems can be divided into memory structures and disk structures. The memory structure includes memory tables (memtable), immutable memory tables (immutable), sorted string tables (SST), and sorted string table information caches (SST Info Cache). The disk structure includes SST. SST consists of two parts: SST data blocks and SST index blocks. SST data blocks store "keys" and "values" (i.e., key-value pairs). SST index blocks index key-value pairs in SST data blocks to quickly find the addresses of key-value pairs in SST data blocks.

[0006] In the related art, when the host parses any path component in the path information to be parsed, the host can obtain the data of the metadata file corresponding to the previous path component of the path component by accessing the hard disk, and find the index node number of the metadata file corresponding to the path component from the data. Then, based on the index node number of the metadata file corresponding to the path component, the host obtains the index node block of the metadata file corresponding to the path component, and based on the index node block, obtains the data of the metadata file corresponding to the path component.

[0007] Thus, when resolving path information, the related technical solution requires access to the hard disk to obtain the index node number of the metadata file corresponding to the path component. Since accessing the hard disk takes a long time, the delay in resolving the path information is relatively high. Summary of the invention

[0008] In order to reduce the overall delay in parsing path information, embodiments of the present application provide a method, apparatus, computing device, and storage medium for obtaining an index node number.

[0009] In a first aspect, a method for obtaining an index node number is provided, the method comprising: obtaining an identifier of a metadata file corresponding to a target path component, wherein the target path component is any path component of multiple path components in the path information to be parsed, determining a first index block corresponding to the identifier of the metadata file in an index block loaded into a memory, wherein the first index block stores a correspondence between the identifier of the metadata file and the index node number, and the memory is a memory of a device executing the method. According to the correspondence, determining the index node number corresponding to the identifier of the metadata file, wherein a data block corresponding to the first index block stores an index node block of the metadata file.

[0010] The path information to be parsed includes multiple path components.

[0011] The scheme shown in the present application is that when parsing any path component of the path information to be parsed (which can be called the target path component), the host can obtain the identifier of the metadata file corresponding to the target path component (the identifier can be the file name of the metadata file of the target path component). The host can then determine the first index block corresponding to the identifier of the metadata file in the index block loaded into the memory. The host then uses the identifier of the metadata file corresponding to the target path component in the first index block to search for the index node number corresponding to the identifier of the metadata file. In this way, since the host can obtain the index node number of the metadata file corresponding to the target path component in the host's memory without accessing the hard disk to obtain it, the delay in parsing the path information can be reduced.

[0012] In one possible implementation, in the index block loaded into the memory, the index block corresponding to the data block where the first index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the first index node block is the index node block of the metadata file corresponding to the previous path component, and the previous path component is the path component arranged before the target path component in the path information.

[0013] In the scheme shown in the present application, the data volume of the metadata file of the file corresponding to the path information to be parsed is relatively small, and the index node block of the metadata file corresponding to the path component under the same path information is more likely to be stored in the data block under one index block. In this way, the index node block of the metadata file corresponding to the target path component will also be stored in the data block under the index block, and then the index block stores the correspondence between the identifier of the metadata file corresponding to the target path component and the index node number. The host can determine the index block loaded into the host's memory as the first index block corresponding to the identifier of the metadata file, and then obtain the index node number of the metadata file corresponding to the target path component from the first index block. In this way, the host can obtain the index node number of the metadata file corresponding to the target path component without accessing the hard disk, which can reduce the delay in parsing the path information.

[0014] In one possible implementation, in the index block loaded into the memory, the index block corresponding to the data block where the second index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

[0015] In the scheme shown in the present application, the number of sub-files under the previous path component of the target path component is relatively small, so that the sub-files under the previous path component can be stored using only a data block under an index block. Then the index node block of the metadata file corresponding to the target path component is stored in the data block under the index block of the previous path component. Therefore, in the index block where the metadata file corresponding to the previous path component is located, the index node number corresponding to the identifier of the metadata file corresponding to the target path component is stored. In this way, the host can determine the index block as the first index block corresponding to the identifier of the metadata file, and then obtain the index node number of the metadata file corresponding to the target path component from the first index block. Therefore, the host does not need to access the hard disk to obtain the index node number of the metadata file corresponding to the target path component, which can reduce the delay in parsing the path information.

[0016] In one possible implementation, a first index block corresponding to an identifier of a metadata file is determined among other index blocks belonging to the index blocks loaded into the memory, wherein the other index blocks are index blocks other than the second index block among the index blocks loaded into the memory, the second index block is an index block corresponding to a data block where a second index node block is located, the second index node block is an index node block of the metadata file corresponding to a first path component, and the first path component is a path component preceding a target path component in the path information.

[0017] In the scheme shown in the present application, the number of sub-files under the previous path component of the target path component is relatively large, so that the sub-files under the previous path component are stored using data blocks under multiple index blocks. Then, the index block under the index block of the previous path component (the index block can be called the second index block) does not store the index node block of the metadata file corresponding to the target path component, so the index block where the metadata file corresponding to the previous path component is located does not store the index node number corresponding to the identifier of the metadata file corresponding to the target path component. The host can determine the first index block corresponding to the identifier of the metadata file corresponding to the target path component in other index blocks loaded into the memory (index blocks other than the second index block in the index blocks loaded into the memory). In this way, the host can also obtain the first index block corresponding to the identifier of the metadata file from the index block loaded into the memory, and then obtain the index node number of the metadata file corresponding to the target path component from the first index block. Therefore, the host does not need to access the hard disk to obtain the index node number of the metadata file corresponding to the target path component, which can reduce the delay in parsing the path information.

[0018] In one possible implementation, among other index blocks belonging to the index block loaded into the memory, each index block in the other index blocks is searched in turn to determine the first index block corresponding to the identifier of the metadata file, wherein the third index block in the other index blocks is searched before the fourth index block in the other index blocks, the third index block is the index block corresponding to the data block where the first metadata file is located, the fourth index block is the index block corresponding to the data block where the second metadata file is located, and the write time point of the first metadata file is earlier than the write time point of the second metadata file.

[0019] In the scheme shown in the present application, when the host determines the first index block corresponding to the identifier of the metadata file corresponding to the target path component among other index blocks loaded into the memory, the host first searches for the index block that was most recently written into the metadata file. Since the probability of newly stored data being searched is high, the host can determine the first index block corresponding to the identifier of the metadata file corresponding to the target path component by searching relatively fewer index blocks, and the time spent is relatively short.

[0020] In a possible implementation manner, the target path component is any path component in the path information except the root directory path component.

[0021] In a second aspect, a device for obtaining an index node number is provided, the device comprising:

[0022] An acquisition module, used to acquire an identifier of a metadata file corresponding to a target path component, wherein the target path component is any path component among multiple path components in the path information to be parsed;

[0023] Identify modules for:

[0024] Determine a first index block corresponding to the identifier of the metadata file in the index blocks loaded into the memory, wherein the first index block stores a correspondence between the identifier of the metadata file and the index node number, and the memory is the memory of the device executing the method;

[0025] According to the corresponding relationship, an index node number corresponding to the identifier of the metadata file is determined, wherein the data block corresponding to the first index block stores the index node block of the metadata file.

[0026] In a possible implementation manner, the determining module is used to:

[0027] In the index blocks loaded into the memory, the index block corresponding to the data block where the first index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the first index node block is the index node block of the metadata file corresponding to the previous path component, and the previous path component is the path component arranged before the target path component in the path information.

[0028] In a possible implementation manner, the determining module is used to:

[0029] In the index block loaded into the memory, the index block corresponding to the data block where the second index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

[0030] In a possible implementation manner, the determining module is used to:

[0031] Among other index blocks belonging to the index blocks loaded into the memory, determine the first index block corresponding to the identifier of the metadata file, wherein the other index blocks are index blocks other than the second index block in the index blocks loaded into the memory, the second index block is the index block corresponding to the data block where the second index node block is located, the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

[0032] In a possible implementation manner, the determining module is used to:

[0033] In other index blocks belonging to the index blocks loaded into the memory, each index block in the other index blocks is searched in turn to determine the first index block corresponding to the identifier of the metadata file, wherein the third index block in the other index blocks is searched before the fourth index block in the other index blocks, the third index block is the index block corresponding to the data block where the first metadata file is located, the fourth index block is the index block corresponding to the data block where the second metadata file is located, and the writing time point of the first metadata file is earlier than the writing time point of the second metadata file.

[0034] In a possible implementation manner, the target path component is any path component in the path information except the root directory path component.

[0035] In a third aspect, a device for obtaining an index node number is provided, the device comprising a processor and a memory, wherein the processor is used to execute computer instructions included in the memory to implement the method for obtaining an index node number described in the first aspect.

[0036] In a fourth aspect, a computing device for obtaining an index node number is provided, the computing device comprising a processor and a memory, wherein:

[0037] The memory stores computer instructions; the processor executes the computer instructions to implement the method for obtaining the index node number described in the first aspect above.

[0038] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions. When the computer instructions in the computer-readable storage medium are executed by a computing device, the computing device executes the method for obtaining the index node number described in the first aspect.

[0039] In a sixth aspect, a computer program product comprising instructions is provided, which, when executed on a computing device, enables the computing device to execute the method for obtaining the index node number described in the first aspect above, or enables the computing device to implement the functions of the apparatus of the second aspect above and its possible implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a system architecture diagram of an LSM tree provided by an exemplary embodiment of the present application;

[0041] Figure 2 is a schematic diagram of the structure of an SST provided by an exemplary embodiment of the present application;

[0042] Figure 3 It is a structural diagram of a system for metadata storage service provided by an exemplary embodiment of the present application;

[0043] Figure 4 is a schematic diagram of the structure of a host provided by an exemplary embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of metadata key-value storage provided by an exemplary embodiment of the present application;

[0045] Figure 6 It is a schematic diagram of the structure of an SST index block provided by an exemplary embodiment of the present application;

[0046] Figure 7 It is a schematic diagram of the structure of an SST index block provided by an exemplary embodiment of the present application;

[0047] Figure 8 is a flowchart of a method for obtaining an index node number provided by an exemplary embodiment of the present application;

[0048] Fig. 9 It is a structural diagram of an apparatus for obtaining an index node number provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0050] In order to facilitate the understanding of the embodiments of the present application, the concepts of the terms involved in the embodiments of the present application are first introduced below:

[0051] A file system is used to manage data storage on a storage device. The file system can be any file system, such as the 4th extended file system (Ext4), the Flashfriendly file system (F2FS), etc.

[0052] Parsing path information (also called path resolution) is the process of retrieving a file by retrieving the file's path information. Specifically, in the process of retrieving the file's path information, each path component in the file's path information is sequentially resolved to obtain the address information of the file to be retrieved, and the file is retrieved based on the address information.

[0053] LSM trees are widely used in key-value storage systems (key-value storage systems use key-value pairs to store data). Figure 1 As shown in the figure, the index tree structure of the LSM tree is a large and a small tree. The smaller tree is stored in the host's memory C0 (which can be called the memory cache area), and the larger tree is persisted to the hard disk (such as C1, C2, ..., C k ). The data write operation will first operate the host's memory. As the tree in the host's memory continues to grow, it will trigger a merge operation with the tree in the hard disk. There are many ways to implement LSM trees in databases (such as the key-value databases LevelDB and RocksDB based on LSM trees). LSM trees can be divided into memory structures and disk structures. The memory structure can include memory tables (memtable), non-writable memory tables (immutable) (memtable and immutable can be called memory cache areas), and SST information caches (SST InfoCache). The memory structure is C0 mentioned above, and each layer of the disk structure is equivalent to C1, C2, ..., C k .

[0054] SST is the way the key-value storage system organizes key-value pairs when persisting them to the hard disk, such as Figure 2As shown, the LSM tree includes multiple SSTs, each of which can be composed of two parts, one part is the SST index block, and the other part is the SST data block (there are one or more SST data blocks in an SST). Data is stored in the SST data block in the form of "key" and "value", and the "key" and "value" of the SST data block are indexed in the SST index block to quickly locate the location information of the key-value pair in the SST data block. Therefore, when generating a new SST, the key-value storage system first determines the SST data block, and then generates an SST index block based on the SST data block, and records the location information of the key-value pair in the SST data block in the generated SST index block. For a certain SST, when the host accesses the SST, the index block of the SST will be loaded into the host's memory, specifically loaded into the SST information cache.

[0055] The index node block of the metadata file is stored in the SST data block and stores the information of the metadata file of the file. For example, the metadata file may include the creator and permissions of the file, the creation date of the file, the size of the file, and the data identifier of the file. The index node block of the metadata file is used to find the data body of the metadata file.

[0056] The inode number of the metadata file is used to index the inode block of the metadata file corresponding to the path component. For a path component, regardless of whether the information in the inode block of the metadata file corresponding to the path component is updated, the inode number used to index the inode block of the metadata file corresponding to the path component will not change.

[0057] Currently, in the file system, the host converts the metadata of the file system to be stored using the key-value storage system. Taking the key-value storage system as LevelDB as an example, the specific processing is as follows: when the metadata file of the file system is stored using the key-value storage system, the host needs to convert the metadata file of the file system into the form of key-value pairs. The data of each metadata file needs two key-value pairs to be stored. One key-value pair stores the index node block of the metadata file. The key of the key-value pair is the identifier of the index node block of the metadata file (which can be referred to as the index node number later), and the value corresponding to the key is the index node block; the other key-value pair stores the data body of the metadata file. The key of the key-value pair is the data identifier of the metadata file, and the value corresponding to the key is the data of the metadata file.

[0058] For example, the path information of the metadata file of file 1.txt is / home / foo / 1.txt. After the metadata file is stored as a key value, the structure diagram is as follows Figure 3As shown, the leftmost " / " is the root directory, "home" is the subdirectory of " / ", and "foo" is the subdirectory of "home". " / ", "home" and "foo" are directories, and their corresponding metadata files can be represented by two key-value pairs. One key-value pair stores the index node block of the metadata file, and the other key-value pair stores the data body of the metadata file (e.g. for "home", one key-value pair stores the index node block of the metadata file corresponding to home, and the other key-value pair stores the data body of the metadata file corresponding to home). Figure 3 In the example, "etc" indicates another subdirectory of the root directory " / ", and "bar" indicates another subdirectory of "home". It should be noted that for a common file 1.txt, since the data of 1.txt can be stored in any way, there may be only one key-value pair for 1.txt, that is, only the key-value pair of the inode number and the inode block.

[0059] After the host has keyed the metadata file, the host can store the key-value pair of the metadata file in the data block. After the host stores the key-value pair in the data block, it creates an index item in the index block corresponding to the data block (the index item key includes the index item key and the index item value), and one index item corresponds to one key-value pair. Specifically, a key-value pair of the index node number of the metadata file and the index node block of the metadata file is stored in a certain data block, and the index item of the key-value pair in the index block corresponding to the data block is: the index item of the index node number of the metadata file and the location information of the data block where the index node block of the metadata file is located (the index item key is the index node number of the metadata file, and the index item value is the location information of the data block where the index node block of the metadata file is located). A key-value pair of the data identifier of the metadata file and the data of the metadata file is stored in a certain data block, and the index item of the key-value pair in the index block corresponding to the data block is: the index item of the data identifier of the metadata file and the location information of the data of the metadata file (the index item key is the data identifier of the metadata file, and the index item value is the location information of the data of the metadata file).

[0060] It should be noted here that, because the file system uses " / " to split the path information under the file system, the path information is split into multiple path components, and each path component actually corresponds to a file (the file can be a directory file or an ordinary file). If the path component corresponds to a directory file, then the directory file is the metadata file corresponding to the path component. If the path component corresponds to an ordinary file, then the index node block of the ordinary file is the metadata file corresponding to the path component, and the data of the ordinary file is not the metadata file corresponding to the path component. For example, the path information is / home / foo / 1.txt, the leftmost " / ", "home", "foo" and "1.txt" can all be called path components, the leftmost " / ", "home", "foo" correspond to directory files, and the file corresponding to "1.txt" is an ordinary file.

[0061] In this way, since the index block stores the index item of the index node number of the metadata file and the location information of the data block where the index node block of the metadata file is located, when parsing a certain path component later, the host can use the index node number of the metadata file corresponding to the path component to find the location information of the data block where the index node block is located, thereby obtaining the data block where the index node block is located, and then obtaining the index node from the data block. Then, the data identifier of the metadata file corresponding to the path component can be obtained from the index node block. Moreover, since the index block stores the index item of the data identifier of the metadata file corresponding to the path component and the location information of the data of the metadata file, the host can use the data identifier of the metadata file corresponding to the path component to find the corresponding location information, and use the location information to find the data of the metadata file corresponding to the path component.

[0062] It should be noted here that, for the convenience of subsequent description, the "index node number of the metadata file corresponding to the path component" is referred to as the "index node number of the path component"; the "index node block of the metadata file corresponding to the path component" is referred to as the "index node block of the path component"; the "data identifier of the metadata file corresponding to the path component" is referred to as the "data identifier of the path component"; and the "data of the metadata file corresponding to the path component" is referred to as the "data of the path component".

[0063] When a host obtains data of a certain file, it first obtains the index node block of the file. The specific processing steps in the related art are as follows (taking the use of path information / home / foo / 1.txt to find the data of file 1.txt as an example (the root directory / , home, foo and 1.txt can be called path components, and the metadata information of the root directory / is generally known and cached in the content of the host)):

[0064] Step 1, when the host starts, the metadata file corresponding to the root directory " / " has been stored in the host's memory. The host uses the identifier of the path component home to find the index node number of the path component home in the metadata file corresponding to the root directory " / ".

[0065] Step 2: The host uses the inode number of the path component home to search for the inode block corresponding to the inode number in the memory cache (memtable, immutable) of the key-value storage system. If the inode block corresponding to the inode number does not exist in the memory cache, the host searches for the inode block in layers C1 to C k In the index item of the SST index block of the layer, the index node number of the path component home is used to find the location information of the SST data block where the index node block of the path component home is located. The host uses this location information to determine the SST data block in the hard disk, and then the host uses this index node number to read the index node block of the path component home in the SST data block. It should be noted here that the SST index block of the SST will be cached in the host's memory only when a certain SST is read.

[0066] Step 3: The host reads the data identifier of the path component home from the index node block of the path component home.

[0067] Step 4: The host uses the data identifier to search for the location information corresponding to the data identifier of the path component home in the index item of the path component data identifier and the location information of the path component data in the SST index block. The host uses the location information to read the data of the path component home from the hard disk.

[0068] Step 5: The host finds the index node number of the path component foo in the data of the path component home.

[0069] Step 6: The host uses the inode number of the path component foo to search for the inode block corresponding to the inode number in the memory cache (memtable, immutable) of the key-value storage system. If the inode block corresponding to the inode number does not exist in the memory cache, the host searches for the inode block in layers C1 to C k In the index item of the SST index block of the layer, the index node number of the path component foo is used to find the location information of the SST data block where the index node block of the path component foo is located. The host uses this location information to determine the SST data block in the hard disk, and then the host uses this index node number to read the index node block of the path component foo in the SST data block.

[0070] Step 7: The host reads the data identifier of the path component foo from the index node block of the path component foo.

[0071] Step 8: The host uses the data identifier to search for the location information corresponding to the data identifier of the path component foo in the index item of the path component data identifier and the location information of the path component data in the SST index block. The host uses the location information to read the data of the path component foo from the hard disk.

[0072] Step 9, the host searches the data of path component foo for the inode number of path component 1.txt.

[0073] Step 10: The host uses the inode number of path component 1.txt to search for the inode block corresponding to the inode number in the memory cache (memtable, immutable) of the key-value storage system. If the inode block corresponding to the inode number does not exist in the memory cache, the host searches for the inode block in layers C1 to C2 in turn. k In the index item of the SST index block of the layer, the index node number of the path component 1.txt is used to find the location information of the SST data block where the index node block of the path component 1.txt is located. The host uses the location information to determine the SST data block in the hard disk, and then the host uses the index node number to read the index node block of the path component 1.txt in the SST data block.

[0074] The host can then execute steps 11 and 12 to read the data in 1.txt. The specific processing is as follows:

[0075] Step 11, read the address of the 1.txt data block in the index node block of 1.txt.

[0076] Step 12: The host uses the address to read the data in 1.txt.

[0077] From the above description, it can be seen that the above steps 2, 4, 6, 8 and 10 all need to access the hard disk. In this way, when the path components of the parsed path information are relatively large, the number of hard disk accesses is relatively large, which makes the overall delay of parsing the path information relatively high, so it is necessary to provide a method for reducing the delay of parsing the path information.

[0078] It should also be noted that the above-mentioned memory cache area is set in the memory of the host.

[0079] Before introducing the method for obtaining the index node number provided in the embodiment of the present application, the application scenarios and systems to which the embodiment of the present application is applicable are first introduced.

[0080] In the embodiment of the present application, a key-value storage system is used to provide a storage service for storing and managing metadata files of a file system. The system architecture used is as follows: Figure 4As shown, a wrapping layer of the file system interface is set between the Portable Operating System Interface of UNIX (POSIX) and the key-value storage system (the wrapping layer can be understood as a piece of code, and the key-value interface is converted in the wrapping layer, which is actually implemented by the host running this piece of code). When the application calls the storage service of the metadata file of the file system, what is seen and used is POSIX, and what the key-value storage system exposes to the wrapping layer of the file system interface is the key-value interface. After being wrapped by the wrapping layer of the file system interface, the key-value interface of the key-value storage system is converted to POSIX used by the file system, thereby providing the storage service of the metadata file of the file system to the application using POSIX. The key-value pairs organized by the key-value storage system are stored in the storage device (such as a hard disk, etc.), and the data in the storage device is operated accordingly according to the call of the key-value interface (for example, when the key-value storage system is LevelDB, the key-value pairs can be organized in batches into the form of SST and stored in the storage device, and the data in the SST in the storage device is operated accordingly according to the call of the key-value interface). The wrapping layer of the file system interface completes the conversion between the key-value interface and the file system POSIX. Under the above system architecture, the application is unaware that the storage service that provides metadata files for it is actually a key-value storage system.

[0081] It should be noted that when the above metadata files are stored in the key-value storage system, they can be stored based on the file system or other methods, which is not limited in the embodiments of this application. The above storage device can be a storage device in a local storage service or a storage device in a cloud storage service, such as a storage device in an elastic block storage service (ElasticBlock Store, EBS), a storage device in an object storage system S3, etc.

[0082] It should also be noted that the above-mentioned key-value storage system is any storage system, such as LevelDB, TiKV, etc. In the subsequent process, the key-value storage system LevelDB is used as an example for explanation.

[0083] It should also be noted that the standard interface of POSIX can be open(A), where A is the path information of the file to be opened. The open() interface means opening a file, and the specific target file to be opened is specified by A. The key-value interface is an interface of the key-value storage system, such as GET(key), etc., which means getting the value (Value) corresponding to the key (Key). The conversion of the package layer is: taking the path information / home / foo / 1.txt as an example, the POSIX standard interface submits the open(" / home / foo / 1.txt") operation, which is converted into multiple ordered key-value operations through the package layer: GET( / ), GET(home), GET(foo), GET(1.txt). This is only an example and is not a limitation of the embodiments of the present application.

[0084] The method for obtaining the index node number can be executed by a device for obtaining the index node number. The device for obtaining the index node number can be a hardware device, such as a host, or a software device (such as a set of software programs running on the hardware device).

[0085] When the method for obtaining the inode number is executed by the host, Figure 5 A possible architecture diagram of the host of the present application is provided as an example. The host may include a processor 501, a memory 502, a communication interface 503 and a bus 504. In the host, the number of processors 501 may be one or more. Figure 5 Only one of the processors 501 is shown. Optionally, the processor 501 may be a central processing unit (CPU). If the host has multiple processors 501, the types of the multiple processors 501 may be different or the same. Optionally, the multiple processors of the host may also be integrated into a multi-core processor.

[0086] The memory 502 stores computer instructions and data. The memory 502 may store computer instructions and data required to implement the method for obtaining the index node number provided in the present application. For example, the memory 502 stores instructions for implementing the steps of obtaining the module execution in the method for obtaining the index node number provided in the present application. For another example, the memory 502 stores instructions for determining the steps of the module execution in the method for obtaining the index node number provided in the present application, etc. The memory 502 may be any one or any combination of the following storage media: non-volatile memory (such as read-only memory (ROM), solid state disk (SSD), hard disk (HDD), optical disk, etc.), volatile memory.

[0087] The communication interface 503 may be any one or any combination of the following devices: a network interface (such as an Ethernet interface), a wireless network card, or other devices with a network access function.

[0088] The communication interface 503 is used for the host to perform data communication with other hosts or terminals.

[0089] The bus 504 can connect the processor 501 with the memory 502 and the communication interface 503. Thus, through the bus 504, the processor 501 can access the memory 502, and can also use the communication interface 503 to exchange data with other hosts or terminals.

[0090] In the present application, the host executes the computer instructions in the memory 502, and uses the host to implement the method for obtaining the index node number provided in the present application. For example, the host executes the steps performed by the determination module in the above method for obtaining the index node number.

[0091] The method for obtaining the index node number provided in an embodiment of the present application will be described below. The scheme will be described below by taking the subject that executes the method for obtaining the index node number as a host as an example.

[0092] Before obtaining the inode number, the host converts the file system metadata file to be stored using the key-value storage system.

[0093] In this embodiment, the process of the host writing the metadata file in the key-value storage system is exactly the same as the process described above, except that when establishing the index item, if the data block stores the key-value pair of the index node number of the metadata file and the index node block of the metadata file, then on the basis of the index item of the index node number of the metadata file and the location information of the data block where the index node block of the metadata file is located, the index block corresponding to the data block is added with: the identifier of the metadata file (also known as the file name of the metadata file) and the index item of the index node number of the metadata file (the index item key is the identifier of the metadata file, and the index item value is the index node number of the metadata file), that is, the correspondence between the identifier of the metadata file and the index node number of the metadata file is added.

[0094] Specifically, for the metadata file corresponding to a certain path component, when establishing the identifier of the metadata file corresponding to the path component (which can be called the identifier of the path component) and the index item of the index node number of the metadata file corresponding to the path component, the host can obtain the index node number of the metadata file corresponding to the path component from the data of the previous path component of the path component (the host can obtain it when the data of the previous path component is written into the key-value storage system). In this way, when parsing a certain path component, the host can directly find the index node number of the metadata file corresponding to the path component in the index block without having to obtain it from the data of the previous path component adjacent to the path component. In addition, there is no need to access the hard disk to obtain the data of the previous path component, so the number of hard disk accesses can be reduced. How to search specifically will be described later. Figure 8 The process shown is described in detail.

[0095] Specifically, in the embodiment of the present application, in the key-value storage system LevelDB, the metadata file is stored in the SST data block of the SST using a key-value pair. Taking the path information of the metadata file as / home / foo / 1.txt as an example, first establish a key-value pair of the metadata file in the SST data block of the SST, which can be specifically: the key is the index node number of the path component home, and the value is the index node block of the path component home; the key is the index node number of the path component foo, and the value is the index node block of the path component foo; the key is the index node number of the path component 1.txt, and the value is the index node block of the path component 1.txt. The SST data block also stores a key-value pair in which the key is the data identifier of the path component home and the value is the data of the path component home. The path component foo is similar to the path component home, and will not be repeated here. For the path component 1.txt, since the file 1.txt is an ordinary file, the data of the file 1.txt is not metadata, and thus the data of the file 1.txt may not be stored in the manner of a metadata file.

[0096] Correspondingly, such as Figure 6As shown, the index items of each key-value pair in the SST data block are recorded in the SST index block, and one key-value pair corresponds to one index item in the SST data block. When the key-value pair in the SST data block is the index node number of the path component home and the index node block of the path component home, in the SST index block corresponding to the SST data block, the index items include: the index item key is the identifier of the path component home, and the index item value corresponding to the index item key is the index node number of the path component; the index item key is the index node number of the path component home, and the index item value corresponding to the index item key is the location information of the SST data block where the index node block of the path component home is located. When the key-value pair in the SST data block is the data identifier of the path component home and the data of the path component home, in the SST index block corresponding to the SST data block, the index items include: the index item key is the data identifier of the path component home, and the index item value corresponding to the index item key is the location information of the data of the path component home. Path component foo and path component 1.txt are similar to path component home and will not be repeated here. In this way, the host can find the index node number of the path component from the SST index block corresponding to the SST data block where the index node block of the path component is located, without obtaining it from the data of the previous path component of the path component. Therefore, there is no need to obtain the data of the previous path component, and then there is no need to access the hard disk, which can reduce the number of hard disk access times.

[0097] In addition, when the index item key is an identifier of a path component, the index item value may also include access permission information of the metadata file, etc.

[0098] It should be noted here that for any metadata file, since the key-value storage system first stores the key-value pairs corresponding to the metadata file in the data block and then creates the index information of these key-value pairs, the host can determine the index node number of any metadata file and the data block where the index node block is located. For example, the index node number of the path component foo is stored in the data of the metadata file corresponding to the path component home, so the index node number of the metadata file corresponding to the path component foo can be obtained.

[0099] It should also be noted here that, assuming the path information is / home / foo / 1.txt, the path component preceding the path component foo is the path component home, and the path component preceding the path component 1.txt is the path component foo.

[0100] The above method of storing the index information of the key-value pairs of the data blocks in the index block is only one feasible method, and other methods are also possible. Figure 7As shown, taking the path information / home / foo / 1.txt as an example, the three index items in the SST index block described above are organized in a correspondence relationship between an index item key and an index item value.

[0101] The following combination Figure 8 The parsing process of path information is described below:

[0102] Step 801: The host obtains an identifier of a metadata file corresponding to a target path component.

[0103] In this embodiment, an application wants to read a file from a host (the file may be referred to as a file to be obtained later), and the application transmits the path information of the file to the key-value storage system of the host. The path information of the file to be obtained includes path information that meets the requirements of the corresponding POSIX interface. When the key-value storage system of the host parses any path component in the path information (which may be referred to as a target path component), it can obtain the identifier of the metadata file corresponding to the target path component, which may be the file name of the metadata file corresponding to the target path component, or it may be a logical identifier.

[0104] For example, the application submits the open(" / home / foo / 1.txt") operation to the package layer on the host through POSIX. The package layer on the host converts the open(" / home / foo / 1.txt") operation into key-value operations GET( / ), GET(home), GET(foo), and GET(1.txt) that can be recognized by the key-value storage system. In this way, the host's key-value storage system can receive the GET( / ), GET(home), GET(foo), and GET(1.txt) operations.

[0105] It should be noted that the above application can be an application installed on the host or not. Before the file system operates on a file, it must first obtain the metadata file of the file to be obtained. In this application, the metadata file of the file system is stored using a key-value storage system, so the application will transmit the path information of the file to the key-value storage system of the host.

[0106] It should also be noted that, since there is a wrapper layer between POSIX and the key-value storage system to convert operations, the application will not perceive that the storage service that provides the metadata of the file system is actually a key-value storage system. The path component arranged at the front in the path information here is the root directory. When the host starts, the metadata file of the root directory will be cached in the host's memory, so the host does not obtain the inode block of the root directory based on the inode number, so the target path component is any path component in the path information except the root directory path component.

[0107] Step 802: The host determines a first index block corresponding to the identifier of the metadata file among the index blocks loaded into the memory.

[0108] In this embodiment, the host can use the identifier of the metadata file corresponding to the target path component in the index block loaded into the host's memory to determine the first index block corresponding to the identifier of the metadata file. As described above, the first index block stores the correspondence between the identifier of the metadata file and the index node number, and the data block corresponding to the first index block stores the index node block of the metadata file.

[0109] Step 803: The host determines the index node number corresponding to the identifier of the metadata file according to the corresponding relationship.

[0110] In this embodiment, the host can find the index node number corresponding to the identifier of the metadata file corresponding to the target path component in the correspondence between the identifier of the metadata file and the index node number, that is, obtain the index node number of the metadata file corresponding to the target path component.

[0111] In this way, since the host can obtain the index node number of the metadata file corresponding to the target path component in the host's memory without accessing the hard disk, the delay of parsing the path information can be reduced.

[0112] In a possible implementation of the above step 802, the host determines, in the index block loaded into the memory, the index block corresponding to the data block where the first index node block is located, as the first index block corresponding to the identifier of the metadata file corresponding to the target path component, the first index node block is the index node block of the metadata file corresponding to the previous path component, and the previous path component is the path component arranged before the target path component in the path information.

[0113] In this embodiment, for the target path component, the host first uses the identifier of the metadata file corresponding to the target path component to search the memory cache area of ​​the key-value storage system for the index node number of the metadata file corresponding to the target path component. If the index node number of the metadata file corresponding to the target path component exists in the memory cache area, the index node number is returned to POSIX; if the index node number of the metadata file corresponding to the target path component does not exist in the memory cache area, the host determines the path component before the target path component in the path information. The host then determines the data block where the first index node block of the path component before the target path component is located, and determines the index block corresponding to the data block. The host determines the index block as the first index block corresponding to the identifier of the metadata file corresponding to the target path component.

[0114] In this way, the data volume of the metadata file of the file corresponding to the path information to be parsed is relatively small, and the index node block of the metadata file corresponding to the path component under the same path information is more likely to be stored in the data block under one index block. In this way, the index node block of the metadata file corresponding to the target path component will also be stored in the data block under the index block, and then the index block stores the correspondence between the identifier of the metadata file corresponding to the target path component and the index node number. The host can determine the index block loaded into the host's memory as the first index block corresponding to the identifier of the metadata file, and then the host can obtain the index node number of the metadata file corresponding to the target path component from the first index block. In this way, the host can obtain the index node number of the metadata file corresponding to the target path component without accessing the hard disk, which can reduce the delay in parsing the path information.

[0115] In a possible implementation of the above step 802, the host determines, in the index block loaded into the memory, the index block corresponding to the data block where the second index node block is located, as the first index block corresponding to the identifier of the metadata file, wherein the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

[0116] In this embodiment, when the index node number of the metadata file corresponding to the target path component does not exist in the host's memory cache area, the host can determine the previous path component of the target path component in the path information. The host uses the identifier of the metadata file corresponding to the previous path component to determine the index block where the second index node block of the metadata file corresponding to the previous path component is located. This index block can be referred to as the second index block later. The host loads the second index block into the memory and determines it as the first index block corresponding to the identifier of the metadata file corresponding to the target path component. For example, the host uses the identifier of the metadata file corresponding to the previous path component to determine the SST index block of the previous path component, and in the SST index block, uses the identifier of the metadata file corresponding to the target path component to find the index node number of the metadata file corresponding to the target path component.

[0117] The reason why the second index block is determined as the first index block corresponding to the identifier of the metadata file corresponding to the target path component in the above process is:

[0118] The number of sub-files under the previous path component of the target path component is relatively small, so that the sub-files under the previous path component can be stored using only a data block under one index block. Then the index node block of the metadata file corresponding to the target path component is stored in the data block under the index block of the previous path component. Therefore, in the second index block where the metadata file corresponding to the previous path component is located, the index node number corresponding to the identifier of the metadata file corresponding to the target path component is stored. In this way, the host can determine the second index block as the first index block corresponding to the identifier of the metadata file, and then obtain the index node number of the metadata file corresponding to the target path component from the first index block. Therefore, the host does not need to access the hard disk to obtain the index node number of the metadata file corresponding to the target path component, which can reduce the delay in parsing the path information.

[0119] For some other situations, the index node number of the metadata file corresponding to the target path component is not stored in the second index block, so the host cannot find the index node number of the metadata file corresponding to the target path component in the second index block. Among them, the above-mentioned other situations include but are not limited to the following situations: the number of sub-files under the previous path component of the target path component is relatively large, so that the storage of the sub-files of the previous path component requires the use of data blocks under multiple index blocks to be completely stored. Thus, in this case, the index node block of the metadata file corresponding to the target path component and the index node block of the previous path component do not belong to the data block under one index block, so the index node number of the metadata file corresponding to the target path component is not stored in the second index block. In this case, the search can be performed in the following manner:

[0120] When the host fails to find the index node number of the metadata file corresponding to the target path component in the second index block, it can determine other index blocks other than the second index block in the index blocks loaded into the memory (i.e., in the key-value storage system, other index blocks other than the second index block in the index blocks loaded into the memory). In the memory cache area of ​​the key-value storage system and the other index blocks, the host uses the identifier of the metadata file corresponding to the target path component to find out whether the index node number corresponding to the identifier exists in the memory cache area. If the index node number corresponding to the identifier exists, the index node number of the metadata file corresponding to the target path component is obtained and returned to POSIX. If the index node number corresponding to the identifier does not exist, the first index block corresponding to the identifier of the metadata file corresponding to the target path component is determined in the other index blocks loaded into the host's memory. In this way, the host can also obtain the first index block corresponding to the identifier of the metadata file from the index block loaded into the memory, and then obtain the index node number of the metadata file corresponding to the target path component from the first index block. Therefore, the host can obtain the index node number of the metadata file corresponding to the target path component without accessing the hard disk, which can reduce the delay in parsing the path information.

[0121] It should be noted here that, corresponding to some other situations mentioned above, using the relevant technology, the data of the previous path component of the target path component may also be stored in the data block of other index blocks. At this time, the host needs to use the data identifier of the previous path component to search for the location information of the data of the previous path component in the memory cache area and other index blocks in turn. After the host finds the location information of the data of the previous path component, it is also necessary to access the hard disk to obtain the data of the metadata file corresponding to the previous path component, and read the index node number of the metadata file corresponding to the target path component from the data. In the present application, the host can search for the index node number of the metadata file corresponding to the target path component in the memory cache area or other index blocks loaded into the memory, without having to access the hard disk to obtain the data of the metadata file corresponding to the previous path component, which can reduce the number of hard disk access times.

[0122] It should also be noted here that when a metadata file corresponding to a path component is stored in the memory cache area, the key-value pair of the index node number and the index node block of the metadata file corresponding to the path component is stored, as well as the key-value pair of the identifier and the index node number of the metadata file corresponding to the path component is stored.

[0123] In addition, if the inode number of the metadata file corresponding to the target path component does not exist in the memory cache area and in any index block of other index blocks, it means that the inode block of the metadata file corresponding to the target path component does not exist. The host can return to POSIX the indication information that the inode block of the metadata file corresponding to the target path component is missing.

[0124] In a possible implementation, the host searches for each index block in turn in the other index blocks loaded into the memory to determine the first index block corresponding to the identifier of the metadata file. The specific process is:

[0125] The host can determine the writing order of the metadata files corresponding to other index blocks in the data blocks written into the key-value storage system. According to the writing time order of the metadata files, the location with the earliest writing time of the metadata files is searched first. Specifically, the memory cache area is searched first (described in the previous text, and will not be repeated here), and then other index blocks are searched. When the host searches for other index blocks, the host loads other index blocks into the host's memory in sequence according to the writing time order of the metadata files. For each index block loaded into the host's memory, the identifier of the metadata file corresponding to the target path component is used to determine whether it is the first index block corresponding to the identifier. If it is the first index block corresponding to the identifier, the host executes step 803. If it is not the first index block corresponding to the identifier, the host continues to load the next index block into the host's memory according to the writing time order of the metadata files until all other index blocks are found, or until the first index block corresponding to the identifier of the metadata file corresponding to the target path component is found. For example, the third index block among other index blocks is searched before the fourth index block among other index blocks. The third index block is the index block corresponding to the data block where the first metadata file is located, and the fourth index block is the index block corresponding to the data block where the second metadata file is located. The writing time point of the first metadata file is earlier than the writing time point of the second metadata file, that is, the first duration corresponding to the first metadata file is less than the second duration corresponding to the second metadata file. The first duration is the duration from the start writing time point of the first metadata file to the current time point, and the second duration is the duration from the start writing time point of the second metadata file to the current time point.

[0126] In this way, when the host determines the first index block corresponding to the identifier of the metadata file corresponding to the target path component among other index blocks loaded into the memory, it first searches for the index block that was most recently written into the metadata file. Since the probability of newly stored data being searched is high, the host can determine the first index block corresponding to the identifier of the metadata file corresponding to the target path component by searching relatively fewer index blocks, and it takes less time.

[0127] For example, in a key-value storage system based on an LSM tree, the memory cache area is C0 (including memtable and immutable), and the latest written data is written to C0. With subsequent write operations, the data will be gradually moved from C0 to C1 until C k The process of searching from new data to old data is to search from C0 to C k , it should be noted here that memtable is searched first, and then immutable is searched.

[0128] In a possible implementation, when the host obtains the index node block of the metadata file corresponding to the previous path component of the target path component, the host can obtain the data volume of the metadata file corresponding to the previous path component in the index node block. The host determines whether the data volume is greater than a preset value. If the data volume is less than or equal to the preset value, the host can use the index node number of the target path component when parsing the index node number of the target path component. Figure 8 If the amount of data is greater than the preset value, the host can use the inode number of the target path component when parsing it. Figure 8 The process shown can also obtain the data of the metadata file corresponding to the previous path component based on the index node block of the metadata file corresponding to the previous path component. The host obtains the index node number of the metadata file corresponding to the target path component from the data.

[0129] In one possible implementation, after the host finds the index node number of the metadata file corresponding to the target path component, the host uses the index node number of the metadata file corresponding to the target path component as the index item key, searches for the index item value corresponding to the index item key in the key-value storage system, and obtains the index node block of the metadata file corresponding to the target path component based on the index item value.

[0130] For example, when the key-value storage system is LevelDB, assuming the path information is / home / foo / 1.txt and the target path component is foo, the host uses the inode number of the metadata file corresponding to foo as the index item key, and sequentially in C0 (memtable, immutable), C1 layer SST, C1 layer SST, and finally C kIn the SST of the C1 layer, search for the index item value corresponding to the index node number of the metadata file corresponding to the path component foo. If the index item value corresponding to the index node number of the metadata file corresponding to foo is found in a certain layer, there is no need to continue searching backwards. Specifically, when searching for the value corresponding to the index node number in the memtable and immutable, since the memtable and immutable are stored in the form of key-value pairs, the index item value corresponding to the index node number of the metadata file corresponding to the path component foo in the memtable and immutable is the index node block corresponding to the index node number of the metadata file corresponding to the path component foo. In the SST of the C1 layer, the SST of the C2 layer, and up to the C k When searching for the index item value corresponding to the index node number of the metadata file corresponding to the path component foo in the SST of any layer in the layer, the host first finds the position information corresponding to the index node number of the metadata file corresponding to the path component foo in the SST index block of the SST. The position information is the index item value corresponding to the index node number of the metadata file corresponding to the path component foo. The host uses the position information to determine the SST data block where the index node block of the metadata file corresponding to the path component foo is located. The host then uses the index node number of the metadata file corresponding to the path component foo to determine the content corresponding to the index node number of the metadata file corresponding to the path component foo in the SST data block. The content is the index node block of the metadata file corresponding to the path component foo.

[0131] It should be noted here that, for a certain path component, although the SST index block of the SST to which the index node block of the path component belongs records the index item of the index node number of the metadata file corresponding to the path component and the location information of the SST data block to which the index node block of the metadata file corresponding to the path component belongs, the data block where the index node block is located will not be found in the SST to which the SST index block belongs using the location information. The reason is that the index node block in the current SST may not be the latest index node block, which will make the acquired index node block incorrect.

[0132] from Figure 8 As can be seen from the process shown above, Figure 8 In the process, the target path component in the path information is used as an example to illustrate the process of obtaining the index node number. For the path components other than the root directory in the path information to be parsed, the host can use Figure 8 The process of obtaining the index node number is not limited in this embodiment of the present application.

[0133] In addition, since the POSIX specification is met in the embodiment of the present application, when parsing the path information to be parsed, each path component is parsed in sequence according to the arrangement order of each path component of the path information, and the index node number of the metadata file corresponding to each path component is obtained. When the index node number of the metadata file corresponding to the last path component of the path information is parsed, the index node block of the metadata file corresponding to the last path component is obtained through the index node number of the metadata file corresponding to the last path component, and then the file corresponding to the path information is obtained.

[0134] In this embodiment, the host can first parse the path component in the front row according to the arrangement order of each path component in the path information, and then parse the second path component in the arrangement to obtain the index node number and index node block of the metadata file corresponding to the second path component. The host then parses the third path component in the arrangement to obtain the index node number and index node block of the metadata file corresponding to the third path component, until all path components in the path information are parsed, and the index node number and index node block of the last path component of the path information are obtained. The host obtains the address information of the file corresponding to the path information in the index node block of the last path component. The host then uses the address information to determine the location where the file corresponding to the path information is stored, and obtains the file corresponding to the path information from the location.

[0135] It should be noted that, during the above path information parsing process, although the host obtains the index node block of the metadata file corresponding to each path component, the host only uses the index node block of the metadata file corresponding to the last path component to search for the data of the file to be obtained. The reasons why the host obtains the index node blocks of the metadata files corresponding to the other path components except the last path component in the path information are: 1) The solution of the embodiment of the present application complies with the POSIX specification. Since the operation submitted by the application to the key-value storage system through POSIX is an operation for obtaining the index node blocks of the metadata files corresponding to each path component, the key-value storage system of the host must return the index node blocks to POSIX to meet the requirements of POSIX; 2) Taking the path information / home / foo / 1.txt as an example, the key-value storage system of the host returns the index node blocks of the metadata files corresponding to the path component home to POSIX. POSIX confirms that the upper directory of the path component foo (i.e., the path component home) is correct before continuing to parse the path component foo; 3) The path components in the path information may be updated (such as the path information is / home / foo / 1.txt, which becomes / home / aa / bb / foo / 1.txt after the update). Only by parsing in sequence can the correct path components be parsed, ensuring that the index node blocks of the metadata files corresponding to the correct path components are returned to POSIX. Therefore, in order to ensure that the parsing process of the path information proceeds normally, the key-value storage system of the host must return the index node blocks of the metadata files corresponding to each path component to POSIX.

[0136] In order to better understand the embodiment of the present application, the embodiment of the present application also provides a process for parsing the path information of / home / foo / 1.txt when the key-value storage system is LevelDB, assuming that the data of the SST index block and the root directory / of the file system have been cached in the host's memory. The following steps need to be completed to obtain the index node block of file 1.txt (in this process, the "index node number of the metadata file corresponding to the path component" is referred to as the "index node number of the path component"; the "index node block of the metadata file corresponding to the path component" is referred to as the "index node block of the path component"; the "data identifier of the metadata file corresponding to the path component" is referred to as the "data identifier of the path component"; the "data of the metadata file corresponding to the path component" is referred to as the "data of the path component"):

[0137] Step S1, the host uses the identifier of the path component home in the data of the root directory " / " to obtain the index node number of the path component home.

[0138] Step S2: The host uses the inode number of the path component home to find the path in memtable, immutable, SST of C1 layer, SST of C2 layer, and finally SST of C k In the SST of the C1 layer, the index node number of the path component home is used as the index item key to find the index item value corresponding to the index node number of the path component home. If the index item value corresponding to the index node number of the path component home is found in the memtable or immutable, the host can determine the index item value as the index node block of the path component home. If the host does not find the index item value corresponding to the index node number of the path component home in the memtable or immutable, the host can search the SST of the C1 layer, the SST of the C2 layer, and the C k In the SST of the layer, the index node number of the path component home is used as the index item key to search for the index item value corresponding to the index node number of the path component home. If the host finds the location information corresponding to the index node number of the path component home in the SST index block of the SST of a certain layer (the location information is the index item value corresponding to the index node number of the path component home), the host ends the search, and the SST index block is the index node block of the path component home. The host uses the location information to determine the SST data block where the index node block of the path component home is located in the hard disk. The host then uses the index node number of the path component home to determine the content corresponding to the index node number of the path component home in the SST data block, and the content is the index node block corresponding to the path component home. The host then returns the index node block of the path component home to POSIX.

[0139] Step S3, if the host fails to find the index node number of the path component foo in the memtable and immutable of the key-value storage system, the host searches for the index node number of the path component foo in the SST index block of the SST data block where the metadata file corresponding to the path component home is located. Specifically, the host can use the identifier of the path component foo as the index item key to search for the corresponding index item value, which is the index node number of the path component foo.

[0140] Step S4, if the host finds the index node number of the path component foo in the SST index block of the SST data block where the metadata file corresponding to the path component home is located, then the SST index block is determined to be the index block corresponding to the identifier of the path component foo, and the hard disk is accessed using the index node number to obtain the index node block of the path component foo (see the process of obtaining the index node block of the path component home for the processing process), and returned to POSIX. If the host does not find the index node number of the path component foo in the SST index block of the SST data block where the metadata file corresponding to the path component home is located, then the index node number of the path component foo is searched in memtable, immutable, and other SST index blocks except the SST index block of the SST data block where the metadata file corresponding to the path component home is located (see the description in step S1 for specific processing). If the host finds the index node number of the path component foo in any of the memtable, the immutable, or the other SST index blocks, then the processing of searching for the index node block corresponding to the index node number is executed (the specific processing has been described above and will not be repeated here), and the processing of step S5 is executed. If the host fails to find the index node number of the path component foo in the memtable, the immutable and the other SST index blocks, it returns an indication that the metadata file is missing and no longer executes step S5 and subsequent steps.

[0141] Step S5, if the host fails to find the index node number of path component 1.txt in the memtable and immutable of the key-value storage system, the host searches for the index node number of path component 1.txt in the SST index block of the SST data block where the metadata file corresponding to the path component foo is located. Specifically, the host can use the identifier of path component 1.txt as the index item key to search for the corresponding index item value, which is the index node number of path component 1.txt.

[0142] Step S6, if the host finds the index node number of path component 1.txt in the SST index block of the SST data block where the metadata file corresponding to path component foo is located, then the SST index block is determined to be the index block corresponding to the identifier of path component 1.txt, and the process of searching for the index node block of path component 1.txt is performed (the specific process has been described above and will not be repeated here), and the process of step S7 is performed. If the host does not find the index node number of path component 1.txt in the SST index block of the SST data block where the metadata file corresponding to path component foo is located, then the identifier of path component 1.txt is used to search for the index node number of path component 1.txt in memtable, immutable and other SST index blocks except the SST index block where path component foo is located. If the host finds the index node number of the path component 1.txt in any of the SST index blocks in the memtable, the immutable, or the other SST index blocks, it executes the process of searching for the index node block corresponding to the index node number (the specific process has been described above and will not be repeated here), and executes the process of step S7. If the host does not find the index node number of the path component 1.txt in the memtable, the immutable, and the other SST index blocks, it returns the indication information that the metadata file is missing, and does not execute step S7.

[0143] Step S7: the host searches the index node block of the path component 1.txt for the address of the data block of the path component 1.txt. Based on the address, the host determines the location of the file of the path component 1.txt and reads the file 1.txt from the location.

[0144] Thus, in the process of parsing the path information / home / foo / 1.txt, for the path component home, since the host does not need to obtain the data of the path component home based on the index node block of the path component home (for specific processing, see the processing of the above step 4), it is not necessary to access the hard disk to read the data of the path component home, thereby reducing one access to the hard disk. Similarly, for the path component foo, since the host does not need to obtain the data of the path component foo based on the index node block of the path component foo (for specific processing, see the processing of the above step 8), it is not necessary to access the hard disk to read the data of the path component foo, thereby reducing one access to the hard disk. It can be seen that for the parsing process of the path information / home / foo / 1.txt, the solution of the related art needs to access the hard disk five times (step 2, step 4, step 6, step 8 and step 10), while the solution of the present application reduces the process of accessing the hard disk to obtain the data of the path component home and the data of the path component foo, that is, it is not necessary to perform the processing of the above steps 4 and 8, so it only needs to access the hard disk three times to obtain the data of 1.txt, so the solution of the present application can reduce the number of hard disk accesses. The above description is based on the assumption that the path information includes four path components. When the path information includes more path components, the number of hard disk access times can also be reduced.

[0145] In an embodiment of the present application, the index block stores the index node number of the metadata file corresponding to the path component, and the index block is loaded into the host memory. In this way, when the host parses the path information, for a certain path component, since the host can obtain the index node number of the metadata file corresponding to the path component from the index block in the host's memory, without having to obtain it from the data of the metadata file corresponding to the previous path component of the path component, the number of hard disk access times can be reduced, thereby accelerating the parsing of the file system's path information.

[0146] It should be noted that the present application is aimed at the situation where the directory entry cache of the file system is missing. This is because if the directory entry cache of the file system exists, the inode number of the file to be obtained can be directly read using the directory entry of the file system, and the file to be obtained can be read based on the inode number (the process of obtaining the file to be obtained based on the inode number has been described above and will not be repeated here). Among them, the situation where the directory entry of the file system is missing may include insufficient storage space for the directory entry of the file system, the directory entry of the file system cannot be loaded, etc.

[0147] Fig. 91 is a structural diagram of a device for obtaining an index node number provided in an embodiment of the present application. The device can be implemented as part or all of the device through software, hardware, or a combination of both. The device provided in an embodiment of the present application can implement the embodiment of the present application. Figure 8 The process described above, the device includes: an acquisition module 910 and a determination module 920, wherein:

[0148] An acquisition module 910 is used to acquire an identifier of a metadata file corresponding to a target path component, wherein the target path component is any path component among multiple path components in the path information to be parsed, and can be specifically used to implement the acquisition function in step 801 and the implicit steps contained therein;

[0149] The determination module 920 is used to:

[0150] Determine a first index block corresponding to the identifier of the metadata file in the index blocks loaded into the memory, wherein the first index block stores a correspondence between the identifier of the metadata file and the index node number, and the memory is the memory of the device executing the method;

[0151] According to the corresponding relationship, the index node number corresponding to the identifier of the metadata file is determined, wherein the data block corresponding to the first index block stores the index node block of the metadata file, which can be specifically used to implement the determination function in step 802 and step 803 and the implicit steps contained therein.

[0152] In a possible implementation, the determining module 920 is configured to:

[0153] In the index blocks loaded into the memory, the index block corresponding to the data block where the first index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the first index node block is the index node block of the metadata file corresponding to the previous path component, and the previous path component is the path component arranged before the target path component in the path information.

[0154] In a possible implementation, the determining module 920 is configured to:

[0155] In the index block loaded into the memory, the index block corresponding to the data block where the second index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

[0156] In a possible implementation, the determining module 920 is configured to:

[0157] Among other index blocks belonging to the index blocks loaded into the memory, determine the first index block corresponding to the identifier of the metadata file, wherein the other index blocks are index blocks other than the second index block in the index blocks loaded into the memory, the second index block is the index block corresponding to the data block where the second index node block is located, the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

[0158] In a possible implementation, the determining module 920 is configured to:

[0159] In other index blocks belonging to the index blocks loaded into the memory, each index block in the other index blocks is searched in turn to determine the first index block corresponding to the identifier of the metadata file, wherein the third index block in the other index blocks is searched before the fourth index block in the other index blocks, the third index block is the index block corresponding to the data block where the first metadata file is located, the fourth index block is the index block corresponding to the data block where the second metadata file is located, and the writing time point of the first metadata file is earlier than the writing time point of the second metadata file.

[0160] In a possible implementation manner, the target path component is any path component in the path information except the root directory path component.

[0161] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0162] In an embodiment of the present application, the index node number of the path component is stored in the index block, and the index block is loaded into the host memory. In this way, when the host parses the target path component in the path information, since the host can determine the index node number of the metadata file corresponding to the target path component from the index block in the host's memory, without having to obtain it from the data of the metadata file corresponding to the previous path component of the target path component, the number of times the hard disk is accessed can be reduced, thereby reducing the delay in parsing the path information.

[0163] In an embodiment of the present application, a computer-readable storage medium is also provided, which stores computer instructions. When the computer instructions stored in the computer-readable storage medium are executed by a computing device, the computing device executes the method for obtaining the index node number provided above.

[0164] In an embodiment of the present application, a computer program product including computer instructions is also provided. When the computer program product is run on a computing device, the computing device executes the above-mentioned method for obtaining the index node number.

[0165] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a server or terminal, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a server or terminal or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (such as a floppy disk, a hard disk and a tape, etc.), or an optical medium (such as a digital video disk (Digital Video Disk, DVD), etc.), or a semiconductor medium (such as a solid-state hard disk, etc.).

Claims

1. A method for obtaining an index node number, characterized in that: The method comprises: Obtaining an identifier of a metadata file corresponding to a target path component, wherein the target path component is any path component among multiple path components in the path information to be parsed; In the index blocks loaded into the memory, the index block corresponding to the data block where the first index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the first index node block belongs to the index node block of the metadata file corresponding to the previous path component, the previous path component is the path component arranged before the target path component in the path information, the first index block stores the correspondence between the identifier of the metadata file and the index node number, and the memory is the memory of the device executing the method; According to the corresponding relationship, an index node number corresponding to the identifier of the metadata file is determined, wherein the data block corresponding to the first index block stores the index node block of the metadata file.

2. The method according to claim 1, characterized in that The step of determining, in the index blocks loaded into the memory, the index block corresponding to the data block where the first index node block is located as the first index block corresponding to the identifier of the metadata file comprises: In the index block loaded into the memory, the index block corresponding to the data block where the second index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

3. The method according to claim 1, characterized in that The step of determining, in the index blocks loaded into the memory, the index block corresponding to the data block where the first index node block is located as the first index block corresponding to the identifier of the metadata file comprises: If the index node number corresponding to the identifier of the metadata file is not found in the second index block, the first index block corresponding to the identifier of the metadata file is determined in other index blocks belonging to the index blocks loaded into the memory, wherein the other index blocks are index blocks other than the second index block in the index blocks loaded into the memory, the second index block is the index block corresponding to the data block where the second index node block is located, the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

4. The method according to claim 3, characterized in that The step of determining, among other index blocks belonging to the index blocks loaded into the memory, a first index block corresponding to the identifier of the metadata file comprises: In other index blocks belonging to the index blocks loaded into the memory, each index block in the other index blocks is searched in turn to determine the first index block corresponding to the identifier of the metadata file, wherein the third index block in the other index blocks is searched before the fourth index block in the other index blocks, the third index block is the index block corresponding to the data block where the first metadata file is located, the fourth index block is the index block corresponding to the data block where the second metadata file is located, and the writing time point of the first metadata file is earlier than the writing time point of the second metadata file.

5. The method according to any one of claims 1 to 4, characterized in that: The target path component is any path component in the path information except the root directory path component.

6. A device for obtaining an index node number, characterized in that: The device includes a memory and a processor, wherein the memory stores computer instructions, and the processor executes the computer instructions to implement the following steps: Obtaining an identifier of a metadata file corresponding to a target path component, wherein the target path component is any path component among multiple path components in the path information to be parsed; In the index blocks loaded into the memory, the index block corresponding to the data block where the first index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the first index node block belongs to the index node block of the metadata file corresponding to the previous path component, the previous path component is the path component arranged before the target path component in the path information, the first index block stores the correspondence between the identifier of the metadata file and the index node number, and the memory is the memory of the device used by the apparatus; According to the corresponding relationship, an index node number corresponding to the identifier of the metadata file is determined, wherein the data block corresponding to the first index block stores the index node block of the metadata file.

7. The device according to claim 6, characterized in that The processor is configured to: In the index block loaded into the memory, the index block corresponding to the data block where the second index node block is located is determined as the first index block corresponding to the identifier of the metadata file, wherein the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

8. The device according to claim 6, characterized in that The processor is configured to: If the index node number corresponding to the identifier of the metadata file is not found in the second index block, the first index block corresponding to the identifier of the metadata file is determined in other index blocks belonging to the index blocks loaded into the memory, wherein the other index blocks are index blocks other than the second index block in the index blocks loaded into the memory, the second index block is the index block corresponding to the data block where the second index node block is located, the second index node block is the index node block of the metadata file corresponding to the first path component, and the first path component is the previous path component of the target path component in the path information.

9. The device according to claim 8, characterized in that The processor is configured to: In other index blocks belonging to the index blocks loaded into the memory, each index block in the other index blocks is searched in turn to determine the first index block corresponding to the identifier of the metadata file, wherein the third index block in the other index blocks is searched before the fourth index block in the other index blocks, the third index block is the index block corresponding to the data block where the first metadata file is located, the fourth index block is the index block corresponding to the data block where the second metadata file is located, and the writing time point of the first metadata file is earlier than the writing time point of the second metadata file.

10. The device according to any one of claims 6 to 9, characterized in that The target path component is any path component in the path information except the root directory path component.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions. When the computer instructions in the computer-readable storage medium are executed by a computing device, the computing device is enabled to execute the method of any one of claims 1 to 5.

Images