File system management method and device, cluster, medium and product

Abstract

The invention relates to a file system management method and device, a cluster, a medium and a product. The file system management method comprises the steps that a super block is allocated to a first address field of a disk of a file system, and a super block area is used for recording system metadata of the file system; the method further includes allocating at least one subspace region in a second address field of the disk, the at least one subspace region including a hash index region for managing the file system in a hash structure and a data region, the hash index region including N buckets, each bucket being used for storing at least one key whose hash value matches a bucket index value, the at least one key is used for indicating the storage position of data of the file system in the data area, and N is a positive integer greater than 1. The method further comprises the step of distributing a super block backup in a third address field of the disk, wherein the super block backup is the backup of the system metadata of the file system recorded in the super block. Therefore, the disaster tolerance of the file system can be improved, and the number of lost files is reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of storage, and more specifically to a management method, device, cluster, product, and medium for a file system. Background Art

[0002] The file system is used to organize and manage files and directories. It provides a structured way to store and access files, allowing users and applications to easily manage and operate files. The file system abstracts physical storage devices (such as mechanical hard disks, solid-state drives, etc.) into a logical space and stores files by allocating and managing disk space. The file system usually uses a hierarchical directory structure so that files are organized in an orderly manner based on the nested relationship between directories and files. The file system also provides operations such as naming, deleting, copying, moving, and modifying files. In addition, the file system can track the attributes of files (for example, creation time, modification time, permissions, etc.), and provide security and access control functions to ensure that only authorized users can access and modify files.

[0003] Typically, a file system manages the directories and files of the file system in a tree structure, such as a B+ tree. When a directory or file needs to be found, the search target can be located based on the information in the tree structure. However, when a node in the tree structure on the disk is damaged, the file will be lost. If the problem node is a node close to the root node or the root node itself, a large number of or even all files will be lost. Summary of the invention

[0004] According to some embodiments of the present disclosure, a file system management solution is provided. Through the file system management solution according to the embodiments of the present disclosure, the disaster tolerance of the file system can be improved, so that when degradation occurs, the number of lost files is reduced and the damage radius is reduced.

[0005] In a first aspect of the present disclosure, a method for managing a file system is provided. The method includes allocating a super block in the first address segment of a disk of the file system, and the super block is used to record the system metadata of the file system. The method also includes allocating at least one subspace area in the second address segment of the disk, and the at least one subspace area includes a hash index area and a data area, and the hash index area is used to manage the file system with a hash structure, wherein the hash index area includes N buckets, each bucket is used to store at least one key whose hash value matches the bucket index value, and at least one key is used to indicate the storage location of the data of the file system in the data area, and N is a positive integer greater than 1. The method also includes allocating a super block backup in the third address segment of the disk, and the super block backup is a backup of the system metadata of the file system recorded in the super block. In this way, the disaster recovery capability of the file system can be improved while ensuring the consistency, integrity and reliability of the file system, so that even when degradation occurs, the degradation is limited to a smaller range, the number of lost files is reduced, and the damage radius is reduced.

[0006] In some embodiments, the first address segment, the second address segment and the third address segment form a continuous address segment on the disk, which helps to identify the disk structure corresponding to the file system and further facilitates the search and retrieval of files and directories.

[0007] In some embodiments, the method further includes: determining whether the file system meets the hash expansion condition according to the expansion threshold of the file system; if the hash expansion condition is met, expanding the hash index area according to the linear hash algorithm. In this way, the conflict problem caused by metadata distribution can be solved, and the utilization rate of the bucket will not be too low, and it will not cause too many elements to be rehashed.

[0008] In some embodiments, the method further includes allocating the expanded hash index area to one or more subspace areas of the at least one subspace area, thereby facilitating flexible allocation of the hash index area.

[0009] In some embodiments, the method further includes: receiving a request to expand at least one subspace area; if a first subspace area of ​​at least one subspace area is smaller than a preset space size, increasing the first subspace area to the preset space size; if each subspace area of ​​at least one subspace area is equal to the preset space size, adding a second subspace area. In this way, space expansion can be performed depending on usage requirements, and each subspace area after expansion has the preset space size, thereby facilitating addressing of files and directories.

[0010] In some embodiments, the method also includes: adding a fourth address segment and a fifth address segment after the third address segment of the disk, the third address segment, the fourth address segment and the fifth address segment constitute a continuous address segment; allocating a super block backup in the fifth address segment of the disk, and copying the backup of the system metadata of the super block backup of the third address segment to the fifth address segment; recycling the backup of the system metadata of the super block backup of the third address segment; allocating a second subspace area in the third address segment and the fourth address segment of the disk.

[0011] In some embodiments, at least one subspace area further includes a log area, and the log area expands the at least one subspace through a transaction model, and the transaction model is used to isolate the process of expanding the at least one subspace area. This helps to ensure the consistency and reliability of the expansion process and prevent data conflicts or unexpected errors during the expansion from causing adverse effects on the system.

[0012] According to a second aspect of the present disclosure, a management device for a file system is provided. The device includes: a super block allocation module, which is used to allocate a super block in the first address segment of the disk of the file system, and the super block area is used to record the system metadata of the file system. The device also includes a subspace area allocation module, which is used to allocate at least one subspace area in the second address segment of the disk, and at least one subspace area includes a hash index area and a data area. The hash index area is used to manage the file system with a hash structure, wherein the hash index area includes N buckets, each bucket is used to store at least one key whose hash value matches the bucket index value, and at least one key is used to indicate the storage location of the file system data in the data area, and N is a positive integer greater than 1. The device also includes a super block backup allocation module, which is used to allocate a super block backup in the third address segment of the disk, and the super block backup is a backup of the system metadata of the file system recorded in the super block.

[0013] According to a third aspect of the present disclosure, a computing device is provided, which includes a processor and a memory, wherein the memory stores instructions, and when the instructions are executed by the processor, the computing device executes the method according to the first aspect of the present disclosure.

[0014] According to a fourth aspect of the present disclosure, a computing device cluster is provided. The computing device cluster includes at least one computing device, each computing device including a processor and a memory. The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in the first aspect of the present disclosure. In some embodiments, the computing device cluster includes one computing device. In other embodiments, the computing device cluster includes multiple computing devices. In some embodiments, the computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop, a laptop, or a smart phone.

[0015] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method according to the first aspect of the present disclosure.

[0016] According to a sixth aspect of the present disclosure, a computer program product is provided. The computer program product includes instructions, and when these instructions are executed by a computing device cluster, the computing device cluster performs the method according to the first aspect of the present disclosure. In some embodiments, the program product may include one or more software installation packages, and when it is necessary to use the method provided by the aforementioned first aspect or its possible variations, the software installation packages can be downloaded or copied and executed on the computing device.

[0017] Please note that the invention summary is provided to introduce a series of concepts in a simplified form, which will be further described in the detailed description below. The invention summary is not intended to identify key features or essential features of the present disclosure, nor is it intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other purposes, features and advantages of the present disclosure will become more clearly understood by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings, in which:

[0019] Figure 1 is a diagram schematically illustrating an example environment in which methods and / or processes according to embodiments of the present disclosure may be implemented;

[0020] Figure 2 is a flowchart schematically illustrating a file system management method according to an embodiment of the present disclosure;

[0021] Figure 3 An exemplary diagram schematically illustrates a disk corresponding to a file system according to an embodiment of the present disclosure;

[0022] Figure 4 schematically illustrates an exemplary diagram of a subspace region according to an embodiment of the present disclosure;

[0023] Figure 5 Schematically illustrates an exemplary diagram of a hash index area according to an embodiment of the present disclosure;

[0024] Fig. 6A The example process of expanding the capacity of a hash index area according to an embodiment of the present disclosure is schematically illustrated;

[0025] Figure 6B Schematically illustrates an example process of linear hashing according to an embodiment of the present disclosure;

[0026] Figure 7 Schematically illustrates an example arrangement of an expanded hash index area according to an embodiment of the present disclosure;

[0027] Fig. 8A Schematically illustrates an example process for expanding the capacity of a subspace region according to an embodiment of the present disclosure;

[0028] Figure 8B Another example process for expanding the capacity of a subspace region according to an embodiment of the present disclosure is schematically illustrated;

[0029] Figure 8C Schematically illustrates another example process for expanding the capacity of a subspace region according to an embodiment of the present disclosure;

[0030] Fig. 9 An example of distributed storage deployment for a file system according to an embodiment of the present disclosure is schematically illustrated;

[0031] Fig.10 A schematic block diagram of an apparatus for capacity expansion according to some embodiments of the present disclosure is shown;

[0032] Fig.11 A schematic diagram of the structure of a computing device provided in an embodiment of the present application is shown;

[0033] Fig.12 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application is shown;

[0034] Fig.13 A schematic diagram of a structure in which a computing device cluster provided by an embodiment of the present application is connected via a network is shown. DETAILED DESCRIPTION

[0035] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0036] In the description of the embodiments of the present disclosure, the term "including" and its variations should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects, unless explicitly indicated to be different.

[0037] As mentioned above, the file system is a mechanism for operating systems or storage devices, etc., which is configured to organize and manage files and directories. The file system provides a structured way to store and access files, so that users and applications can easily manage and operate files. The file system abstracts physical storage devices (such as mechanical hard disks, solid-state drives, etc.) into a logical space, and stores files by allocating and managing disk space. By using a hierarchical directory structure, the file system organizes and manages files in an orderly manner based on the nested relationship between directories and files. The file system also provides operations such as naming, deleting, copying, moving, and modifying files. In addition, the file system can track the attributes of files (for example, creation time, modification time, permissions, etc.), and provide security and access control functions to ensure that only authorized users can access and modify files.

[0038] Typically, a file system manages directories and files of the file system in a tree structure, such as a B+ tree. Each node in the tree structure includes a key value and a pointer to the next node. When a directory or file needs to be found, the search target can be located based on the information in the tree structure. However, when a node in the tree structure on the disk is damaged or the hard disk metadata is inconsistent, files will be lost. If the problem node is a node close to the root node or the root node itself, a large number of or even all files will be lost. In addition, as the number of files increases, the file system needs to expand its components in order to adapt to the changing storage environment.

[0039] In order to solve at least some of the above problems and other potential problems, a management scheme for a file system according to an embodiment of the present disclosure is proposed. The scheme includes allocating a super block in the first address segment of the disk of the file system, and the super block area is used to record the system metadata of the file system. The scheme also includes allocating at least one subspace area in the second address segment of the disk, and the at least one subspace area includes a hash index area and a data area. The hash index area is used to manage the file system with a hash structure, wherein the hash index area includes N buckets, each bucket is used to store at least one key whose hash value matches the bucket index value, and at least one key is used to indicate the storage location of the data of the file system in the data area, and N is a positive integer greater than 1. The scheme also includes allocating a super block backup in the third address segment of the disk, and the super block backup is a backup of the system metadata of the file system recorded in the super block. In this way, the disaster tolerance capability of the file system can be improved while ensuring the consistency, integrity and reliability of the file system, so that even when degradation occurs, the degradation is limited to a smaller range, the number of lost files is reduced, and the damage radius is reduced.

[0040] Reference below Figures 1 to 13 It should be understood that these exemplary embodiments are provided only to enable those skilled in the art to better understand and implement the embodiments of the present disclosure, and are not intended to limit the scope of the present disclosure in any way.

[0041] Figure 1 1 is a schematic diagram illustrating an example environment 100 in which methods and / or processes according to embodiments of the present disclosure may be implemented. Figure 1 As exemplarily shown in FIG. 1 , the example environment 100 may include a file system 110 and storage devices 121, 122, 123, and 124. According to an embodiment of the present disclosure, the file system 110 is a plurality of storage devices (e.g., Figure 1 The storage devices 121-124 shown in the figure abstract a logical storage space, and store files by allocating and managing the disk space of these storage devices. Such abstraction enables the file system 110 to shield the details of the underlying physical devices and provide a unified interface for users and applications to access and manage files. The multiple storage devices managed by the file system 110 may include a mechanical hard disk HDD or a solid-state drive SSD, etc. It should be understood that the number and type of the above-mentioned multiple storage devices are only exemplarily shown for the convenience of illustration and description, and the multiple storage devices according to the embodiments of the present disclosure may include a greater or lesser number and other different types.

[0042] According to an embodiment of the present disclosure, the physical storage device corresponding to the file system 110 may be deployed locally, may be deployed in a distributed manner, or a portion may be deployed locally and the rest may be deployed in a distributed manner, etc. For example, Figure 1 As shown, storage devices 121 and 122 can be deployed locally, while storage devices 123 and 124 can be deployed in the cloud. Local deployment can reduce network latency and bandwidth limitations, providing lower access latency and higher throughput. At the same time, distributed deployment can disperse data across multiple storage nodes, thereby enhancing the scalability of the system. Through distributed deployment, it is possible to balance the load, improve the capacity and processing power of the system, and support more concurrent accesses. Deploying part of the physical storage device locally and part in a distributed environment can be flexibly configured according to actual needs. The appropriate deployment method can be selected based on factors such as data characteristics, access frequency, security requirements, etc., in order to provide optimal performance and resource utilization. It should be understood that Figure 1 The deployment shown is merely exemplary and non-limiting, and embodiments of the present disclosure may also include other different deployments.

[0043] According to an embodiment of the present disclosure, the file system 110 performs space management with a fixed block size, such as 4KB. Blocks of fixed size ensure space utilization and avoid a large amount of fragmented space waste. Blocks of fixed size also facilitate the rapid allocation and release of space, so that the file system 110 can allocate continuous blocks to store files as needed, and quickly release these blocks when files are deleted. In addition, blocks of fixed size also facilitate data reading and writing operations. The file system 110 can locate and access specific data blocks through the index of the block without scanning the entire storage space. This improves the efficiency and speed of data access. It should be understood that the block size described herein is exemplary and non-restrictive, and other different sizes can be set according to actual needs, such as 8KB, etc. In the following, the file system 110 according to an embodiment of the present disclosure will be described in further detail.

[0044] Figure 2 The flowchart schematically illustrates a method 200 for managing a file system 110 according to an embodiment of the present disclosure. In order to improve the disaster tolerance of the file system, so as to reduce the number of lost files and reduce the damage radius when degradation occurs, a method 200 for managing a file system 110 according to an embodiment of the present disclosure is proposed. Through the management method 200, it is possible to limit the undesired degradation to a small range while ensuring the consistency, integrity and reliability of the file system, thereby realizing a file system 110 with improved security.

[0045] Step 210, a super block is allocated in the first address segment of the disk of the file system 110, and the super block is used to record the system metadata of the file system 110. The super block on the first address segment is configured to record system-level metadata (i.e., system metadata). In an exemplary and non-restrictive manner, the system metadata recorded by the super block may include the type, version, size, or other parameters related to the file system 110 of the file system 110. The consistency and integrity of the file system 110 can be guaranteed by the system metadata recorded by the super block. In the following, the super block according to the embodiment of the present disclosure will be further described in detail.

[0046] Step 220, at least one subspace area is allocated in the second address segment of the disk, and the at least one subspace area includes a hash index area and a data area. The hash index area is used to manage the file system 110 with a hash structure, wherein the hash index area includes N buckets, each bucket is used to store at least one key whose hash value matches the bucket index value, and at least one key is used to indicate the storage location of the data of the file system in the data area, and N is a positive integer greater than 1. The hash-form index area on the second address segment includes multiple buckets, and the key stored in each bucket is used to identify the location of the user data in the data area. By using such a hash structure to organize the files and directories of the file system, even when a small amount of metadata is damaged, it will often only affect one or a small number of buckets and the conflict chain behind them, and will not affect the entire or most of the index area, thereby reducing the total number of files lost when the metadata is damaged. In this way, the disaster tolerance of the file system 110 is improved and a relatively short damage radius is obtained. In addition, the metadata information in the hash index area can quickly locate the location of the target data in the data area, thereby improving the read and write performance of the file system 110. Hereinafter, at least one subspace area and the hash index area and data area included therein according to an embodiment of the present disclosure will be described in further detail.

[0047] Step 230, a super block backup is allocated in the third address segment of the disk, and the super block backup is a backup of the system metadata of the file system 110 recorded in the super block. The super block backup on the third address segment is configured to back up and store the system-level metadata (i.e., system metadata) recorded in the file system 110 to ensure the integrity and reliability of the file system 110. In this way, when the file system 110 needs to be restored (such as when a problem or damage occurs), these backups in the super block backup can be used to restore the file system 110, such as its structure and data.

[0048] In some embodiments, the first address segment, the second address segment, and the third address segment may form a continuous address segment on the disk of the file system 110 to facilitate identification and search operations. In addition, the disk in the present disclosure is not limited to a magnetic disk, but may include not only a magnetic disk but also other types of disks such as a solid state disk. Figure 2 The management method 200 of the file system 110 according to the embodiment of the present disclosure is described below. Figure 3 An exemplary diagram of a storage structure (also referred to as an on-disk structure) corresponding to the file system 110 according to an embodiment of the present disclosure is described below.

[0049] Figure 3 The exemplary diagram of the disk 300 and the file system 110 according to an embodiment of the present disclosure is schematically illustrated. The disk 300 corresponds to a plurality of areas divided for the file system 110. The file system 110 may abstract (a plurality of) physical storage devices into Figure 3 The logical storage structure shown in the figure stores files by allocating and managing disk space. The physical storage device corresponding to the file system 110 can be deployed locally, can be deployed in a distributed manner, or a part is deployed locally and the remaining part is deployed in a distributed manner, and so on. Such flexible deployment enables that when part of the physical storage device fails, data can still be accessed and recovered through other storage nodes, thereby improving the reliability and fault tolerance of the system. By deploying the physical storage device in a distributed manner and applying appropriate data replication and backup strategies, the security of the data can be enhanced. Even if a certain storage node or local storage device fails or data is lost, data can still be recovered through other storage nodes or distributed backups. It should be understood that the description of the deployment of the physical storage device herein is only exemplary and non-restrictive, and the embodiments of the present disclosure may also include the deployment of other different physical storage devices.

[0050] like Figure 3 As shown, the disk 300 corresponding to the file system 110 may include a super block 310, at least one subspace area 320, and a super block backup 330. The super block 310, at least one subspace area 320, and the super block backup 330 may be continuous on the disk 300. It should be understood that Figure 3 The illustrated sizes of the super block 310, the at least one subspace region 320, and the super block backup 330 are only exemplary and non-restrictive, and appropriate sizes can be set for the super block 310, the at least one subspace region 320, and the super block backup 330 according to actual use needs. In addition, for the purpose of ease of understanding and easy illustration, Figure 3In the figure, one subspace region is used to indicate at least one subspace region 320, but the at least one subspace region 320 may also include a greater number of subspace regions, which will be further illustrated below.

[0051] According to an embodiment of the present disclosure, a super block 310 may be arranged before at least one subspace area 320 to record system metadata of the file system 110. The super block 310 may be located at the beginning of the disk 300 to facilitate identification of the disk 300 corresponding to the file system 110. The metadata recorded by the super block 310 may be metadata at the file system level, such as the type, version, size, or other parameters related to the file system 110, and the recorded metadata may promote consistency and integrity of the file system 110.

[0052] According to an embodiment of the present disclosure, a superblock backup 330 may be arranged after at least one subspace area 320, and is a backup of the system metadata of the file system 110 recorded in the superblock 310. The superblock 310 may be located at the end position (also referred to as the last position) of the disk 300, so as to facilitate identification of the disk 300 corresponding to the file system 110. In order to ensure the integrity and reliability of the file system 110, during maintenance or backup of the file system 110, a backup may be performed on the superblock 310, and the backed-up data may be saved in the superblock backup 330. In this way, when the file system 110 needs to be restored (such as when a problem or damage occurs), these backups in the superblock backup 330 may be used to restore the file system 110, such as its structure and data.

[0053] According to an embodiment of the present disclosure, at least one subspace area 320 may include one or more subspace areas, and at least one subspace area 320 may be arranged between the superblock 310 and the superblock backup 330. The superblock 310, at least one subspace area 320, and the superblock backup 330 may form a logically continuous storage space, even if the data included in the superblock 310, at least one subspace area 320, and the superblock backup 330 are not necessarily physically continuous on the underlying storage device. Since the file is not stored continuously on the physical device, even if a physical block fails or is damaged, it will only affect the data on the physical block without affecting the integrity of the entire file. This distributed storage method provides redundancy and fault tolerance mechanisms to protect the security of file data. By mapping logically continuous file data to different locations on the physical device, the file system 110 can dynamically adjust the size and position of the file without moving the entire file content. This can achieve more efficient file operations, reduce space fragmentation, and provide better storage management and maintenance capabilities. In the following, with the help of Figure 4 The at least one subspace region 320 is described in further detail.

[0054] Figure 4 FIG. 4 schematically illustrates an exemplary diagram of one of the subspace regions 400 included in the at least one subspace region 320 according to an embodiment of the present disclosure. Figure 4 As shown, the subspace area 400 may include a sub-superblock 410, a space management area 420, a log area 430, a hash index area 440, a data area 450, and a metadata backup 460. The sub-superblock 410, the space management area 420, the log area 430, the hash index area 440, the data area 450, and the metadata backup 460 may be logically continuous, even though the data included in the sub-superblock 410, the space management area 420, the log area 430, the hash index area 440, the data area 450, and the metadata backup 460 may not be physically continuous on the underlying storage device. It should be noted that in Figure 4 In the figure, at least one subspace region 320 includes one subspace region (i.e., subspace region 400) as an example for illustration and description, but this is only for the purpose of convenience of illustration and description. According to an embodiment of the present disclosure, at least one subspace region 320 may include a greater number of subspace regions, and these subspace regions may have the same or similar arrangement as that of subspace region 400.

[0055] According to an embodiment of the present disclosure, a sub-super block 410 may be arranged at the starting position of a subspace region 400 to record metadata of the subspace region 400. A metadata backup 460 may be arranged at the end position (also referred to as the last position) of the subspace region 400, which is a backup of metadata recorded in the subspace region 400. Arranging the sub-super block 410 at the beginning of the subspace region 400 and arranging the metadata backup 460 at the end of the subspace region 400 is conducive to identifying the subspace region to which the sub-super block 410 and the metadata backup 460 belong. In addition, the sub-super block 410 and the metadata backup 460 included in the subspace region 400 further improve the consistency, integrity, reliability, etc. of the file system 110.

[0056] According to an embodiment of the present disclosure, the subspace area 400 also includes a space management area 420. The space management area 420 may be arranged after the sub-super block, and may be configured to record the space usage of the subspace area 400. The space management area 420 may be configured to record, for example, the total space size, the used space size, the free space size, etc. of the subspace area 400, which information is helpful for understanding the subspace area 400. The subspace area 400 may also include a data area 450, and the data area 450 is configured to store user data. The file system 110 may allocate a data area 450 to each space in at least one subspace area 320 to meet the user's file storage needs.

[0057] According to an embodiment of the present disclosure, the subspace area 400 further includes a log area 430 and a hash index area 440. The log area 430 may be global to the file system 110, and may be arranged in the first subspace area among one or more subspace areas included in the at least one subspace area 320. In other words, in this example, it is assumed that the subspace area 400 is the first subspace area, and the subspace areas after the first subspace area (the second subspace area, the third subspace area, etc.) may not include the log area 430.

[0058] The log area 430 is configured to store information about the operation of the file system 110, and the recorded log may be a transaction log. In some embodiments, the log area 430 may expand the capacity of at least one subspace area 320 through a transaction model, wherein the transaction model is used to isolate the process of expanding the capacity of at least one subspace area 320. The transaction model helps to ensure the consistency and reliability of the expansion process, and it provides a protection mechanism to prevent data conflicts or unexpected errors during expansion from adversely affecting the system. By isolating the transaction model, it can be ensured that the expansion operation can proceed as expected and will not interfere with other tasks or subspaces. In this way, even in the event of a power outage, it can help to back up and restore the file system 110, thereby improving the consistency and integrity of the file system 110.

[0059] According to an embodiment of the present disclosure, the hash index area 440 may also be global to the file system 110. However, unlike the log area 430, the hash index area 440 may be arranged in the first subspace area of ​​one or more subspace areas in at least one subspace area 320, or may be arranged in a subspace area after the first subspace area (the second subspace area, the third subspace area, etc.). This depends on the size of the hash index area 440, the conflict rate on the hash index area 440, and the like. In other words, in this example, assuming that the subspace area 400 is the first subspace area, the hash index area 440 arranged in the first subspace area may be used to identify user data in the data area 450 in the first subspace area, and may also be used to identify user data in the data area 450 in the subspace area after the first subspace area (the second subspace area, the third subspace area, etc.). In addition, the hash index area 440 arranged in a subspace area other than the subspace area 400 may also be used to identify user data in the data area 450 in one or more other subspace areas included in the at least one subspace area 320. In the following, with the help of Figure 5 Hash index area 440 is described in further detail.

[0060] Figure 5 An exemplary diagram of a hash index area 440 according to an embodiment of the present disclosure is schematically illustrated. The hash index area 440 according to an embodiment of the present disclosure is in a hash form. In other words, the hash index area 440 manages and organizes directories and files of the file system 110 in a hash structure. Such a hash index area 440 may include a group of buckets, wherein the group of buckets may include at least one bucket, such as Figure 5 Bucket 1, bucket 2, ..., bucket N shown in, N is an integer greater than or equal to 1. By way of example, the size of each bucket can be 512 bytes. Each bucket included in the buckets in the hash index area 440 may include metadata associated with user data (distributed in the data area 450 in one or more subspace areas in the at least one subspace area 320) for identifying the user data. It should be understood that Figure 5 The number of buckets shown in is exemplarily shown for the purpose of convenience of illustration and ease of explanation, and the hash index area 440 according to an embodiment of the present disclosure may also include other different numbers of buckets.

[0061] Further, the bucket index value is an identifier for identifying and locating the bucket where the metadata is stored. The file data in the data area 450 can be stored in a key-value pair, and the key can be stored in a bucket in the hash index area 440 to index the file data in the data area 450. A hash value can be calculated for the key according to a hash algorithm, and if its hash value matches the bucket index value, it is stored in the bucket. If the hash values ​​calculated for multiple keys are the same, they are stored in one bucket, resulting in a hash conflict.

[0062] According to an embodiment of the present disclosure, each bucket included in the buckets in the hash index area 440 may include a chain consisting of a group of nodes. The node group of each bucket may include one or more nodes. For example, Figure 5 As shown, bucket 1 includes a chain consisting of nodes A, B, and C, bucket 2 includes a chain consisting of nodes A' and B', and so on. Each node in the chain includes metadata for identifying user data. For example, nodes B and C of bucket 1 include metadata for identifying data segments 530 and 540, respectively. In other words, a hash bucket can record a file. When a hash conflict occurs, a field can be used on the hash bucket to store the current conflicting file, thereby forming a hash conflict chain. It should be understood that in Figure 5 For the purpose of illustration and ease of explanation, only barrels 1 and Figure 2 The corresponding chains are shown, but buckets 3-N may also include their corresponding chains. Figure 5 The chain and node modes shown are exemplary and non-limiting, and embodiments of the present disclosure may also include Figure 5 Chain and node patterns other than the examples shown in , such as more or fewer nodes.

[0063] According to an embodiment of the present disclosure, one or more nodes in a set of nodes of each bucket include a subchain consisting of at least one child node. When the data segment to be searched or retrieved (hereinafter also referred to as the target data segment) is relatively large or has relatively many associated indexes, the node itself may include a portion of metadata, and other remaining metadata may be located on the child node corresponding to the node. Assume that the target data segment is the sum of data segment 510 and data segment 520. For example, Figure 5 As shown, the metadata included in node A of bucket 1 is used to identify data segment 510, and node A includes the address of the child node A corresponding thereto, wherein child node A includes additional metadata for identifying data segment 520. In this way, while or after determining data segment 510, the address of child node A can be used to locate child node A, and the additional metadata included in child node A can be used to identify data segment 520. In this way, the entire target data segment (i.e., data segment 510 plus data segment 520) can be effectively identified. It should be understood that in Figure 5For the purpose of facilitating illustration and explanation, only the child nodes of node A are shown and the number is 1, but other nodes may also include corresponding child nodes and the number of child nodes may also be different. In addition, the child nodes may also include further lower-level child nodes.

[0064] As described above, by using an index area in the form of a hash, the directories and files of the file system 110 can be organized and managed by buckets and chains. According to an embodiment of the present disclosure, such a hash index area 440 improves the disaster tolerance of the file system 110. Even when a node or even a bucket on the chain is damaged and causes the loss of files, the degradation can be limited to the bucket and its corresponding chain without affecting other buckets and corresponding chains. In other words, when a small amount of metadata is damaged, it often only affects one or several buckets and the conflict chains behind them, thereby reducing the total number of files lost when the metadata is damaged. In this way, the number of lost files is significantly reduced and the damage radius is reduced. In addition, through such a hash index area 440, the bucket or node where the directory and file are located can be quickly located, thereby improving the read and write performance of the file system 110.

[0065] According to an embodiment of the present disclosure, based on the expansion threshold in the hash structure, it can be determined whether the file system 110 meets the hash expansion condition. When the hash expansion condition is met, the hash index area 440 is expanded. The expansion threshold can be or based on the hash conflict rate. In some embodiments, reaching the expansion threshold can indicate that the number of files has reached a certain threshold, or can indicate that the length of the conflict chain has reached a certain threshold.

[0066] As the number of files increases and reaches the file number threshold, more and more metadata will fall on the same chain. Such uneven distribution will cause the conflict chain to reach the chain length threshold, or each chain is relatively long and causes the number of conflict chains to reach the chain number threshold, etc., which will lead to an increase in the hash conflict rate in the hash structure. The high hash conflict rate of the hash structure is disadvantageous to the file system 110, for example, it increases the time cost and computing cost of finding or retrieving the target file. In order to solve the high hash conflict rate in the hash structure, the hash index area 440 can be expanded. According to an embodiment of the present disclosure, when the hash expansion conditions for the hash index area 440 are met, the file system 110 can trigger the expansion of the hash index area 440 to solve the conflict problem caused by the metadata distribution.

[0067] By way of example, the hash structure includes N conflict chains, and the conflict rate of N / 2 of the conflict chains is greater than or equal to the conflict rate threshold. Assuming that the number threshold for the conflict chains is N / 2, the number of conflict chains with a conflict rate greater than or equal to the conflict rate threshold is equal to the number threshold for the conflict chains, which indicates that the hash conflict rate in the hash structure is relatively high at this time, and the hash expansion condition for the hash index area 440 is met. Fig. 6A The expansion of the hash index area 440 is described in further detail.

[0068] Fig. 6A The exemplary process 600A for expanding the capacity of the hash index area 440 according to an embodiment of the present disclosure is schematically illustrated. According to an embodiment of the present disclosure, another group of buckets (such as Fig. 6A Another group of buckets 610 is shown, which is also referred to as the second group of buckets hereinafter), and the number of the other group of buckets is equal to the number of the above group of buckets (hereinafter also referred to as the first group of buckets). In addition, part of the metadata for user data included in the first group of buckets can be migrated to the newly created second group of buckets, thereby reducing the hash conflict rate in the hash structure.

[0069] The number of the second group of buckets created by the linear hash algorithm may be a power of 2 times the number of the first group of buckets. In some embodiments, the hash index area 440 originally has N buckets, and N more buckets may be created each time the hash expansion is performed. Fig. 6A As shown, based on the original first group of buckets (bucket 1, bucket 2, ..., bucket N), a second group of buckets (bucket N+1, bucket N+2, ..., bucket 2N) can be created. It should be noted that during the expansion process, the hash value of each element needs to be recalculated (that is, the hash value of each key of the original N buckets needs to be recalculated) because the size of the new bucket array has changed. At the same time, in order to ensure the performance of the hash table, it is necessary to avoid rehashing of elements as much as possible during expansion, because rehashing will increase the time complexity. In some embodiments, the size of the new bucket array can be set to twice the original size, so that the utilization rate of the bucket is not too low, and it will not cause the rehashing of too many elements.

[0070] As described above, after creating a new set of buckets, part of the metadata for user data in the original set of buckets can be migrated to the new set of buckets, thereby reducing the hash collision rate in the hash structure. In some embodiments, linear hashing can be used to perform the migration process of metadata for user data from the original set of buckets to the new set of buckets. Figure 6B Let's further describe linear hashing.

[0071] According to the recalculated hash value and bucket index value, one or more keys (parts) in the original N buckets are migrated to the newly created N buckets. Figure 6B An example process 600B of linear hashing according to an embodiment of the present disclosure is schematically illustrated. According to an embodiment of the present disclosure, the hash index is first expanded exponentially, then the old index is traversed, and then the linear hash expansion algorithm is executed. As shown in 600B (1) of Figure 6, the first column, the first row and the second row highlighted indicate buckets, respectively, and the "0" and "1" therein indicate the hash value of the metadata for the user data included in the bucket. The "0", "10" and "110" after the "0" in the first column and the first row indicate the hash value of the metadata for the user data included in the chain node of the bucket, and the "1", "11" and "101" after the "1" in the second row of the first column indicate the hash value of the metadata for the user data included in the chain node of the bucket. According to an embodiment of the present disclosure, in order to redistribute the metadata, two new buckets can be created, as shown in 600B (2) and 600B (3) of Figure 6.

[0072] As shown in 600B (2) of Figure 6, part of the metadata for the user data can be migrated to the newly created first bucket (generally, the third bucket) and its on-chain node. For example, the bucket located in the third row of the first column may include a hash value "10", and using the modulo method, "10" and "110" can be migrated to the on-chain node of the bucket.

[0073] Then, as shown in 600B (3) of FIG. 6 , the metadata for the user data can be migrated to the newly created second bucket (generally, the fourth bucket) and its on-chain node. For example, the bucket located in the fourth row of the first column may include a hash value of "11", and by taking the remainder, "11" can be migrated to the on-chain node of the bucket. By using the linear hash expansion process, only individual file chains are locked at the same time, and the business input and output IO of the entire file system will not be interrupted.

[0074] Figure 7 An example arrangement of an expanded hash index area according to an embodiment of the present disclosure is schematically illustrated. Due to the relatively large size of the expanded hash index area, etc., the size originally allocated to the hash index area 440 may be insufficient. Therefore, after the expansion of the hash index area 440, the expanded hash index area may be distributed in multiple spaces.

[0075] Figure 7 Two spaces are shown as an example. Figure 7The upper part of the hash index area 440 is an example before the capacity is expanded. The first subspace area is composed of a sub-super block 410, a space management area 420, a log area 430, a hash index area 440, a data area 450 and a metadata backup 460, as shown in FIG. Figure 4 As shown, the first subspace region is followed by the second subspace region 710 .

[0076] Figure 7 The lower part indicates an example of hash index area 440 after expansion. A portion of the expanded hash index area may be arranged in the first subspace area, such as hash index areas 440 and 440'. In addition, a portion of the expanded index area may be arranged in the second subspace area 710, such as hash index area 440".

[0077] The following will introduce the expansion of the file system 110 (hereinafter also referred to as space expansion and capacity expansion). The space expansion can be triggered by the user, for example, the user will initiate an expansion request according to the usage requirements. According to an embodiment of the present disclosure, in response to receiving an expansion request for at least one subspace area 320 from the user, it can be determined whether the space expansion condition for the file system 110 is met based on the size of the at least one subspace area 320. If the space expansion condition for the file system 110 is met, the at least one subspace area 320 can be expanded. In some embodiments, the space expansion condition may include at least one of the following items: the data area of ​​the file system 110 cannot write more files, the remaining space of the file system 110 is less than or equal to the remaining space threshold, the number of files in the file system 110 is greater than or equal to the file number threshold, a new storage device is added, or the reserved space of the file system 110 is less than or equal to the reserved space threshold. It should be noted that the space expansion conditions described herein are only exemplary and non-restrictive.

[0078] Fig. 8A An example process 800A for expanding the capacity of at least one subspace area 320 according to an embodiment of the present disclosure is schematically illustrated. As described above, at least one subspace area 320 may include one or more subspace areas. According to an embodiment of the present disclosure, a preset space size for a subspace area in at least one subspace area 320 may be predefined, such as 8GB. Ideally, one or more subspace areas included in at least one subspace area 320 may all have a preset space size, so that the file system 110 can facilitate various operations and processes. For example, the fact that the subspace areas in at least one subspace area 320 all have a preset space size facilitates addressing of files. In addition, it is also beneficial to the recovery of the file system 110 even in the event of a power outage.

[0079] Fig. 8AThe upper part of the at least one subspace region 320 indicates an example before the expansion, where the at least one subspace region 320 includes one subspace region, namely, the first subspace region 810. Fig. 8A As shown in the upper part of , the disk 300 corresponding to the file system 110 may include a super block 310, a first subspace area 810, and a super block backup 330. Assume that the size of the first subspace area 810 in this example is smaller than the preset space size. For example, the current size of the first subspace area 810 is 4GB, which is smaller than 8GB. According to an embodiment of the present disclosure, if the size of the first subspace area 810 is smaller than the preset space size, the first subspace area 810 may be increased to the preset space size. Fig. 8A The lower part of the diagram indicates an example of at least one subspace region 320 being expanded. Fig. 8A As shown in the lower part of , the first subspace area 810 can be increased to a preset space size, that is, the enlarged first subspace area 810'. For example, it is increased from 4GB to 8GB. A new address segment will be allocated after the enlarged first subspace area 810', and a super block backup will be allocated on the address segment for copying the backup of the system metadata of the super block backup 330. Then, the backup of the system metadata on the address segment of the super block backup 330 will be recycled.

[0080] According to an embodiment of the present disclosure, when each subspace area in at least one subspace area 320 has a preset space size, a fourth address segment and a fifth address segment can be added after the third address segment of the disk (on which the superblock backup 330 is originally allocated), wherein the third address segment, the fourth address segment and the fifth address segment can form a continuous address segment. Next, a superblock backup can be allocated in the fifth address segment of the disk, and the backup of the system metadata of the superblock backup 330 of the third address segment can be copied to the fifth address segment. Then, the backup of the system metadata of the superblock backup 330 of the third address segment can be recovered. In the following, Figure 8B and Figure 8C The space expansion in this case is further described.

[0081] Figure 8B Another example process 800B for expanding at least one subspace region 320 according to an embodiment of the present disclosure is schematically illustrated. Figure 8BThe upper part indicates an example before the at least one subspace area 320 is expanded, in which the at least one subspace area 320 includes one subspace area, namely, the first subspace area 820. The disk 300 corresponding to the file system 110 may include a super block 310, a first subspace area 820, and a super block backup 330. Assume that the size of the first subspace area 820 in this example is a preset space size. For example, the current size of the at least one subspace area 320 is 8GB. According to an embodiment of the present disclosure, if the size of the at least one subspace area 320 is a preset space size, a second subspace area 830 of the preset space size may be added after the at least one subspace area 320. Figure 8B The lower part of the diagram indicates an example of at least one subspace region 320 being expanded. Figure 8B As shown in the lower part of the figure, a second subspace area 830 of a preset space size can be added after at least one subspace area 320. The size of the second subspace area 830 is, for example, 8GB. A new address segment will be allocated after the second subspace area 830, and a super block backup will be allocated on the address segment for migrating the backup of the system metadata of the super block backup 330. Then, the backup of the system metadata on the address segment of the super block backup 330 will be recycled.

[0082] Figure 8C Another example process 800C for expanding at least one subspace region 320 according to an embodiment of the present disclosure is schematically illustrated. Figure 8C The upper part indicates an example before the at least one subspace area 320 is expanded, in which the at least one subspace area 320 includes two subspace areas, namely, a first subspace area 840 and a first subspace area 850. The disk 300 corresponding to the file system 110 may include a super block 310, a first subspace area 840, a first subspace area 850, and a super block backup 330. Assume that the size of the first subspace area 820 in this example is an integer multiple of the preset space size. For example, the current size of the at least one subspace area 320 is 16GB. According to an embodiment of the present disclosure, if the size of the at least one subspace area 320 is an integer multiple of the preset space size, a second subspace area 860 of the preset space size may be added after the at least one subspace area 320. Figure 8C The lower part of the diagram indicates an example of at least one subspace region 320 being expanded. Figure 8CAs shown in the lower part of FIG. 1 , a second subspace area 860 of a preset space size can be added after at least one subspace area 320 (i.e., the first subspace area 840 and the first subspace area 850). The size of the second subspace area 860 is, for example, 8GB. A new address segment will be allocated after the second subspace area 860, and a super block backup will be allocated on the address segment for migrating the backup of the system metadata of the super block backup 330. Then, the backup of the system metadata on the address segment of the super block backup 330 will be recycled.

[0083] Fig. 9 Schematically illustrates an example of a file system distributed storage deployment 900 according to an embodiment of the present disclosure. Fig. 9 As shown, the distributed storage deployment 900 for the file system 110 may include a network attached storage (NAS) 910. The NAS 910 may be coupled to a metadata server 920 and a data server 930. The metadata server 920 may be configured to perform read and / or write operations associated with metadata of user data on files on the file system 110, and the data server 930 may be configured to perform read and / or write operations associated with user data on files on the file system 110.

[0084] Fig.10 FIG. 1 is a schematic block diagram of a file system management device 1100 according to some embodiments of the present disclosure. Fig.10 As shown, the management device 1000 includes a super block allocation module 1010 , a subspace area allocation module 1020 , a super block backup allocation module 1030 , a hash index area expansion module 1040 and a subspace area expansion module 1050 .

[0085] The super block allocation module 1010 is used to allocate a super block in the first address segment of the disk of the file system. The super block is used to record the system metadata of the file system.

[0086] The subspace area allocation module 1020 is used to allocate at least one subspace area in the second address segment of the disk, and the at least one subspace area includes a hash index area and a data area. The hash index area is used to manage the file system with a hash structure, wherein the hash index area includes N buckets, each bucket is used to store at least one key whose hash value matches the bucket index value, and at least one key is used to indicate the storage location of the file system data in the data area, and N is a positive integer greater than 1.

[0087] The super block backup allocation module 1030 is used to allocate a super block backup in the third address segment of the disk, where the super block backup is a backup of the system metadata of the file system recorded in the super block.

[0088] In some embodiments, the first address segment, the second address segment and the third address segment may form a continuous address segment on the disk, which can facilitate identification and search operations.

[0089] The hash index area expansion module 1040 is used to determine whether the file system meets the hash expansion condition according to the expansion threshold of the file system; if the hash expansion condition is met, the hash index area can be expanded according to the linear hash algorithm.

[0090] The subspace area expansion module 1050 is used to receive a request for expanding at least one subspace area; if the first subspace area of ​​at least one subspace area is smaller than the preset space size, the first subspace area can be increased to the preset space size; if each subspace area of ​​at least one subspace area is equal to the preset space size, a second subspace area can be added.

[0091] The subspace area expansion module 1050 is used to add a fourth address segment and a fifth address segment after the third address segment of the disk, the third address segment, the fourth address segment and the fifth address segment constitute a continuous address segment, a super block backup is allocated to the fifth address segment of the disk, the backup of the system metadata in the super block backup of the third address segment is copied to the fifth address segment, the backup of the system metadata in the super block backup of the third address segment is recovered, and a second subspace area is allocated in the third address segment and the fourth address segment of the disk.

[0092] Among them, the super block allocation module 1010, the subspace area allocation module 1020, the super block backup allocation module 1030, the hash index area expansion module 1040 and the subspace area expansion module 1050 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the super block allocation module 1010 is introduced below by taking the super block allocation module 1010 as an example. Similarly, the implementation of the subspace area allocation module 1020, the super block backup allocation module 1030, the hash index area expansion module 1040 and the subspace area expansion module 1050 can refer to the implementation of the super block allocation module 1010.

[0093] As an example of a software functional unit, the super block allocation module 1010 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the super block allocation module 1010 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0094] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0095] As an example of a hardware functional unit, the super block allocation module 1010 may include at least one computing device, such as a server, etc. Alternatively, the super block allocation module 1010 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0096] The multiple computing devices included in the super block allocation module 1010 can be distributed in the same region or in different regions. The multiple computing devices included in the super block allocation module 1010 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the super block allocation module 1010 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0097] It should be noted that, in other embodiments, the super block allocation module 1010 can be used to execute any step in the file system management method according to the embodiment of the present disclosure, the subspace area allocation module 1020 can be used to execute any step in the file system management method according to the embodiment of the present disclosure, the super block backup allocation module 1030 can be used to execute any step in the file system management method according to the embodiment of the present disclosure, the hash index area expansion module 1040 can be used to execute any step in the file system management method according to the embodiment of the present disclosure, and the subspace area expansion module 1050 can be used to execute the implementation of the present disclosure. For any step in the method for managing a file system of the example, the steps implemented by the super block allocation module 1010, the subspace area allocation module 1020, the super block backup allocation module 1030, the hash index area expansion module 1040 and the subspace area expansion module 1050 can be specified as needed, and all functions of the file system management device 1100 can be realized by respectively implementing different steps in the method for managing a file system according to an embodiment of the present disclosure through the super block allocation module 1010, the subspace area allocation module 1020, the super block backup allocation module 1030, the hash index area expansion module 1040 and the subspace area expansion module 1050.

[0098] The present disclosure also provides a computing device 1100 . Fig.11 1 shows a schematic block diagram of an example computing device 1100 that can be used to implement an example implementation of the present disclosure. Fig.11 As shown, the computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate with each other through the bus 1102. The computing device 1100 can be a server or a terminal device. It should be understood that the present disclosure does not limit the number of processors and memories in the computing device 1100.

[0099] The bus 1102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 The bus 1102 may include a path for transmitting information between various components of the computing device 1100 (eg, the memory 1106, the processor 1104, and the communication interface 1108).

[0100] The processor 1104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0101] The memory 1106 may include a volatile memory, such as a random access memory (RAM). The processor 1104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0102] The memory 1106 stores executable program codes, and the processor 1104 executes the executable program codes to respectively implement the functions of the aforementioned super block allocation module 1010, the subspace area allocation module 1020, the super block backup allocation module 1030, the hash index area expansion module 1040, and the subspace area expansion module 1050, thereby implementing the file system management method according to the embodiment of the present disclosure. That is, the memory 1106 stores instructions for executing the file system management method according to the embodiment of the present disclosure.

[0103] Alternatively, the memory 1106 stores executable codes, and the processor 1104 executes the executable codes to respectively implement the functions of the aforementioned file system management device 1000, the AA device, and the BB device, thereby implementing the file system management method according to the embodiment of the present disclosure. That is, the memory 1106 stores instructions for executing the file system management method according to the embodiment of the present disclosure.

[0104] The communication interface 1108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0105] The disclosed embodiment also provides a computing device cluster. Fig.12 A schematic block diagram of an example computing device cluster 1200 that can be used to implement an exemplary implementation of the present disclosure is shown. The computing device cluster 1200 includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0106] like Fig.12 As shown, the computing device cluster 1200 includes at least one computing device 1100. The memory 1106 in one or more computing devices 1100 in the computing device cluster 1200 may store the same instructions for executing the file system management method according to the embodiment of the present disclosure.

[0107] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster 1200 may also respectively store partial instructions for executing the file system management method according to the embodiment of the present disclosure. In other words, the combination of one or more computing devices 1100 may jointly execute the instructions for executing the file system management method according to the embodiment of the present disclosure.

[0108] It should be noted that the memory 1106 in different computing devices 1100 in the computing device cluster 1200 can store different instructions, which are respectively used to execute part of the functions of the file system management device 1000. That is, the instructions stored in the memory 1106 in different computing devices 1100 can implement the functions of one or more modules among the super block allocation module 1010, the subspace area allocation module 1020, the super block backup allocation module 1030, the hash index area expansion module 1040 and the subspace area expansion module 1050.

[0109] In some possible implementations, one or more computing devices in the computing device cluster 1200 may be connected via a network, which may be a wide area network or a local area network. Fig.13 A possible implementation is shown. Fig.13 As shown, two computing devices 1100A and 1100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1106 in the computing device 1100A stores instructions for executing the functions of the super block allocation module 1010, the subspace area allocation module 1020, and the super block backup allocation module 1030. At the same time, the memory 1106 in the computing device 1100B stores instructions for executing the functions of the hash index area expansion module 1040 and the subspace area expansion module 1050.

[0110] Fig.13 The connection method between the computing device cluster 1200 shown may be based on the consideration that the file system management method provided by the present disclosure may be targeted at distributed storage devices, and therefore the functions implemented by the hash index area expansion module 1040 and the subspace area expansion module 1050 may be handed over to the computing device 1100B for execution.

[0111] It should be understood that Fig.13 The functions of the computing device 1100A shown in FIG. 1100A may also be completed by multiple computing devices 1100. Similarly, the functions of the computing device 1100B may also be completed by multiple computing devices 1100.

[0112] The embodiment of the present disclosure also provides another computing device cluster 1200. The connection relationship between the computing devices in the computing device cluster 1200 can be similarly referred to as Fig.12 and Fig.13 The connection mode of the computing device cluster 1200. The difference is that the memory 1106 in one or more computing devices 1100 in the computing device cluster 1200 may store the same instructions for executing the file system management method according to the embodiment of the present disclosure.

[0113] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster 1200 may also respectively store partial instructions for executing the file system management method according to the embodiment of the present disclosure. In other words, the combination of one or more computing devices 1100 may jointly execute the instructions for executing the file system management method according to the embodiment of the present disclosure.

[0114] The embodiment of the present disclosure also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a file system management method according to an embodiment of the present disclosure.

[0115] The embodiment of the present disclosure also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the file system management method according to the embodiment of the present disclosure.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for managing a file system, characterized in that: The method comprises: Allocate a super block in the first address segment of the disk of the file system, wherein the super block is used to record the system metadata of the file system; Allocating at least one subspace area in the second address segment of the disk, the at least one subspace area including a hash index area and a data area, the hash index area being used to manage the file system in a hash structure, wherein the hash index area includes N buckets, each bucket being used to store at least one key whose hash value matches the bucket index value, the at least one key being used to indicate a storage location of data of the file system in the data area, and N is a positive integer greater than 1; A super block backup is allocated in the third address segment of the disk, and the super block backup is a backup of the system metadata of the file system recorded in the super block.

2. The method according to claim 1, characterized in that: The method comprises: The first address segment, the second address segment and the third address segment constitute continuous address segments on the disk.

3. The method according to claim 1 or 2, characterized in that: After allocating at least one subspace area in the second address segment of the disk, wherein the at least one subspace area includes a hash index area and a data area, the method includes: Determining whether the file system meets a hash expansion condition according to an expansion threshold of the file system; If the hash expansion condition is met, the hash index area is expanded according to a linear hash algorithm.

4. The method according to claim 3, characterized in that After the hash index area is expanded according to the linear hash algorithm, the method includes: The expanded hash index area is allocated to one or more subspace areas of the at least one subspace area.

5. The method according to claim 1 or 2, characterized in that: After allocating at least one subspace area to the second address segment of the disk, the method further includes: Receiving a request to expand the at least one subspace region; If a first subspace area of ​​the at least one subspace area is smaller than a preset space size, increasing the first subspace area to the preset space size; If each of the at least one subspace region is equal to the preset space size, a second subspace region is added.

6. The method according to claim 5, wherein if each subspace region of the at least one subspace region is equal to the preset space size, adding a second subspace region comprises: Adding a fourth address segment and a fifth address segment after the third address segment of the disk, wherein the third address segment, the fourth address segment and the fifth address segment constitute a continuous address segment; Allocate a super block backup in the fifth address segment of the disk, and copy the backup of the system metadata in the super block backup in the third address segment to the fifth address segment; Reclaiming the backup of the system metadata in the super block backup of the third address segment; The second subspace area is allocated in the third address segment and the fourth address segment of the disk.

7. The method according to claim 1, wherein the at least one subspace region further comprises a log area, the method comprising: The log area expands the at least one subspace through a transaction model, and the transaction model is used to isolate the process of expanding the at least one subspace area.

8. A file system expansion device, characterized in that: The device comprises: A super block allocation module, used to allocate a super block in the first address segment of the disk of the file system, wherein the super block is used to record the system metadata of the file system; a subspace area allocation module, configured to allocate at least one subspace area in the second address segment of the disk, wherein the at least one subspace area includes a hash index area and a data area, wherein the hash index area is used to manage the file system in a hash structure, wherein the hash index area includes N buckets, each bucket is used to store at least one key whose hash value matches the bucket index value, and the at least one key is used to indicate a storage location of data of the file system in the data area, and N is a positive integer greater than 1; The super block backup allocation module is used to allocate a super block backup in the third address segment of the disk, and the super block backup is a backup of the system metadata of the file system recorded in the super block.

9. The device according to claim 8, characterized in that: The first address segment, the second address segment and the third address segment constitute continuous address segments on the disk.

10. The device according to claim 8 or 9, characterized in that After allocating at least one subspace area in the second address segment of the disk, wherein the at least one subspace area includes a hash index area and a data area, the device includes: Hash index area expansion module, used for: Determining whether the file system meets a hash expansion condition according to an expansion threshold of the file system; If the hash expansion condition is met, the hash index area is expanded according to a linear hash algorithm.

11. The device according to claim 10, characterized in that After the hash index area is expanded according to the linear hash algorithm, the hash index area expansion module: The expanded hash index area is allocated to one or more subspace areas of the at least one subspace area.

12. The device according to claim 8 or 9, characterized in that After allocating at least one subspace area to the second address segment of the disk, the device further comprises: Subspace area expansion module, used for: Receiving a request to expand the at least one subspace region; If a first subspace area of ​​the at least one subspace area is smaller than a preset space size, increasing the first subspace area to the preset space size; If each of the at least one subspace region is equal to the preset space size, a second subspace region is added.

13. The device according to claim 12, wherein if each subspace region of the at least one subspace region is equal to the preset space size, a second subspace region is added, and the subspace region expansion module: Adding a fourth address segment and a fifth address segment after the third address segment of the disk, wherein the third address segment, the fourth address segment and the fifth address segment constitute a continuous address segment; Allocate a super block backup in the fifth address segment of the disk, and copy the backup of the system metadata in the super block backup in the third address segment to the fifth address segment; Reclaiming the backup of the system metadata in the super block backup of the third address segment; The second subspace area is allocated in the third address segment and the fourth address segment of the disk.

14. The apparatus according to claim 8, wherein the at least one subspace region further comprises a log area, The log area expands the at least one subspace through a transaction model, and the transaction model is used to isolate the process of expanding the at least one subspace area.

15. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method as claimed in any one of claims 1 to 7.

17. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to any one of claims 1 to 7.

Images