Distributed file system management system and file system extension method

Abstract

The invention discloses a distributed file system management system and a file system extension method. The distributed file system management system comprises a distributed file system management system and a file information acquisition system. The distributed file system management system comprises distributed management blocks and file system metadata blocks. The distributed management block comprises a file system index block (including file system metadata storage index addresses), a cross-file system index (pointing to a cross-file system management address) and a cross-file system management block (including cross-file system file metadata including file quantity, file name, distribution, distribution condition and the like). The file information acquisition system acquires file metadata (including attributes such as name, size and creation time). The system is used for managing a plurality of isomorphic or heterogeneous file systems in a unified manner, supports storage of a single file across multiple systems, adopts a distributed management block as a core, and can realize parallel storage and reading.

Description

Technical Field

[0001] The present invention belongs to the technical field of file systems, and in particular relates to a distributed file system management system and a file system expansion method. Background Art

[0002] In current computer technology, various operating systems offer a diverse range of file systems, including NTFS and extFat in Windows, jffs2 and yaffs2 in Linux, and tffs, commonly used in embedded devices. These file systems can meet basic storage requirements when operating independently, but existing management methods present significant drawbacks in large systems encompassing multiple file systems. First, heterogeneous file systems lack unified interfaces, resulting in a lack of unified management tools, a non-vertical management model, and inefficient resource scheduling. Second, file storage is limited to a single file system, making it impossible to distribute storage across systems, significantly limiting storage capacity expansion. Third, traditional expansion methods rely on hardware expansion (such as adding disk slots or upgrading interface speeds), which, due to physical constraints, makes it difficult to achieve both large capacity and high bandwidth. Fourth, existing interconnection tools (such as Samba) rely on network interfaces and are unable to adapt to file systems without network ports or using low-speed interfaces (such as I2C and SPI), resulting in limited compatibility. Fifth, the lack of cross-file system parallel read and write mechanisms creates bandwidth bottlenecks when storing and reading single files, making it impossible to meet the high-speed data processing requirements of large systems.

[0003] To address the above problems, the present invention proposes a distributed file system management system and a corresponding expansion method, which realizes unified control, flexible expansion and efficient collaboration of multiple file systems by reconstructing the management architecture and storage logic. Summary of the Invention

[0004] The purpose of the present invention is to provide a distributed file system management system and a file system expansion method to solve the problems in the prior art proposed in the background art, such as the difficulty in unified management of multiple file systems (including homogeneous and heterogeneous file systems), the lack of cross-system storage capabilities, the limited expansion and the bandwidth bottleneck.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: A distributed file system management system includes a distributed management block, multiple file system metadata blocks, and a file information collection system; the distributed management block is configured to implement index management of each file system and collaborative management and control across file systems, and the file information collection system is configured to collect file metadata from each file system and upload it to the file system metadata block.

[0006] According to the above technical solution, the distributed management block includes a file system index block, a cross-file system index and a cross-file system management block; the file system index block is used to record the starting address of each file system metadata block, and the cross-file system index is used to point to the storage address of the cross-file system management block.

[0007] According to the above technical solution, the file system index block adopts a pointing addressing method, and directly locates the corresponding file system metadata block through the starting address. The file system metadata block stores file attribute information, which includes file name, file size and creation time.

[0008] According to the above technical solution, the cross-file system index adopts a pointing addressing method, and the cross-file system management block is configured to manage the storage interval of a single file in multiple file systems, which includes the total number of cross-file system files and detailed information of each cross-file system file. The detailed information includes the file name, storage distribution and allocation details in each file system.

[0009] According to the above technical solution, the file information acquisition system and the distributed management block exchange data through a hardware interface, and the hardware interface includes one or more of a network interface, a PCIe interface, an I2C interface or an SPI interface.

[0010] Furthermore, the file information collection system is designed for each independent file system. It is a software system that can obtain the information of each file in each file system. The software system calls the underlying file interface to obtain the metadata information of each file, such as fstat in the Linux system, GetFileInformationByHandle in the Windows system, statfs in vxWorks, etc.

[0011] According to the above technical solution, the file metadata collected by the file information collection system is consistent with the storage structure of the file system metadata block, and both are encapsulated in a preset unified format that includes file identification, capacity parameters and timestamp fields.

[0012] According to the above technical solution, the cross-file system management block supports collaborative storage management of homogeneous and heterogeneous file systems. When a single file is distributed across multiple heterogeneous file systems, the cross-file system management block implements the association mapping of each file fragment through detailed information.

[0013] According to the above technical solution, the cross-file system management block is configured to support parallel storage and parallel reading operations, locates the storage location of each file segment by parsing detailed information, and realizes collaborative data reading and writing of multiple file systems.

[0014] A distributed file system expansion method, the method comprising the following steps: S1, collects file metadata from each file system through the file information collection system, encapsulates it in a unified format, and uploads it to the distributed file system management system; S2, the distributed file system management system stores the received file metadata in the corresponding file system metadata block and constructs a file system index block based on the starting address of each file system metadata block to achieve unified management and control of metadata; S3: When a single file needs to be stored across multiple file systems, the file data is fragmented and each fragment is stored in the target file system. At the same time, a cross-file system management block is created through the distributed management block, and its storage address is written into the cross-file system index. S4, recording the storage information of the corresponding file fragments in each file system into the cross-file system management block according to a preset format, where each fragment information corresponds to a cross-file system file record; S5, in the process of adding a new file system to expand storage capacity, the file fragment information in the new file system is added to the cross-file system management block according to a preset format to maintain the integrity of the cross-file system file record.

[0015] According to the above technical solution, in step S3, the sharding processing of the file data is independent of the file system type. The target file system includes a homogeneous file system and a heterogeneous file system. Each shard is stored as an independent file in the target file system.

[0016] Compared with the prior art, the present invention has the following beneficial effects: In the present invention, with the help of the standardized index and metadata unified structure of the distributed management block, vertical unified management of multiple homogeneous and heterogeneous file systems is achieved, solving the management and control problems caused by interface incompatibility; the cross-file system management block supports the distributed storage of a single file in any number of file systems, theoretically breaking through the physical limitations of storage capacity and achieving unlimited expansion. It is compatible with a variety of hardware interfaces such as network interfaces, PCIe, I2C, SPI, and adapts to multiple scenarios such as server-level and embedded devices. Through the parallel storage and reading mechanism of file sharding, the data transmission bandwidth is greatly improved, effectively solving the bandwidth bottleneck of traditional storage. The sharding processing is independent of the file system type, ensuring that both homogeneous and heterogeneous systems can work together, significantly improving the compatibility and flexibility of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a diagram showing the overall composition of the distributed file system management system of the present invention; Figure 2 This is a structural diagram of the distributed file system management system of the present invention; Figure 3 This is a diagram of the distributed management block structure of the present invention; Figure 4This is a diagram of the file system index block structure of the present invention; Figure 5 This is a cross-file system index structure diagram of the present invention; Figure 6 This is a diagram of the cross-file system management block structure of the present invention; Figure 7 This is a diagram of the cross-file system file information structure of the present invention; Figure 8 This is a diagram of the cross-file system file situation of the present invention; Figure 9 This is a structural diagram of the file information collection system of the present invention; Figure 10 This is a file information structure diagram of the file information collection system of the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] Example 1 like Figure 1 As shown, a distributed file system management system and a file system expansion method include a distributed file system management system and a file information collection system, one located at the server side and the other located at the file system side; like Figure 2 As shown, the distributed file system management system mainly includes a distributed management block and each file system metadata block. The metadata of each file system is the metadata information of each file system file reported by the file information collection system.

[0020] like Figure 3 As shown, the distributed management block includes a file system index block, a cross-file system index, and a cross-file system management block. It has two functions: one is the index block information of each file system, and the other is cross-file system management.

[0021] like Figure 4 As shown, the file system index block is used to index the starting address of each file system information, using a pointing addressing method.

[0022] like Figure 5 As shown, the cross-file system index is used to index the starting address of the cross-file system management block, using a pointing addressing method.

[0023] like Figure 6As shown, the cross-file system management block is mainly used to manage the storage intervals occupied by a single file in multiple file systems, including the number of cross-file system files, cross-file system file information, etc.

[0024] like Figure 7 As shown, cross-file system file information includes file name, file distribution, and specific allocation status in each file system.

[0025] like Figure 8 As shown, when storing files, the management information of each level is filled in step by step through the cross-file system management block, and then the data is stored in each file system according to the distribution of the source end; when reading files, the management information of each level is read step by step in reverse through the cross-file system management block, and finally the position of each file is located, and then the file is read through each file system.

[0026] like Figure 9 As shown, the file information collection system is mainly responsible for the metadata information of files in each file system. The file metadata information of the file system includes the attribute information of all files in the file system.

[0027] like Figure 10 As shown, file information includes file name, file capacity, file creation time and other information, which needs to be organized and stored according to a fixed structure. The structure is basically consistent with the file system metadata block information in the distributed file system management system.

[0028] Step S1: The file information collection system collects metadata information of each file in the file system and reports it to the distributed file system management system; Step S2: The distributed file system management system stores the file information in each file system in the file system metadata area and establishes a file system index block in the distributed management block. The file system metadata has a unified structure and is managed in a unified manner. Step S3: When a single file occupies multiple file systems (regardless of homogeneity or heterogeneity), the collected data is first divided and stored in each file system. Each file system is stored as a separate file. At the same time, the distributed management block creates a cross-file system management block and assigns the address to the cross-file system index. Step S4: The information corresponding to the cross-file system file in each file system is stored in the cross-file system management block in a unified format, and each file is stored as a piece of cross-file system file information.

[0029] Step S5: When the cross-file system file of the present invention is expanded, the information corresponding to the file in each file system is stored in the cross-file system management block in a unified format, and each file is stored as a cross-file system file information.

[0030] The present invention proposes a distributed file system management system and a set of file system expansion methods, which redesign the collaborative work between different file systems (homogeneous or heterogeneous), improve the file management and storage management capabilities of the large system for each small system and even at the module level, and at the same time, the large system can uniformly manage all subordinate file systems.

[0031] The system has richer hardware interfaces and is not limited to interface speeds. That is, it provides the ability for embedded file systems to access the system. This management method can be used for both server-level and board-level management.

[0032] Through cross-file system file management, a single file can exist in multiple file systems, enabling dynamic expansion of the file system and greatly improving storage capacity. In theory, there is no limit to the system's ability to expand the file system.

[0033] Through cross-file system file management, a single file can be stored in multiple file systems and exist as an independent file in each file system. The overall file management is achieved only through cross-file system file management. After diversion at the source, parallel storage can be achieved, which greatly improves the storage bandwidth, meets the high-speed bandwidth requirements of large systems, and solves the storage bottleneck.

[0034] Example 2 This embodiment provides a method for a dynamic consistency protocol.

[0035] To address the data consistency issue of shards across heterogeneous file systems, a hybrid consistency protocol module is added to the cross-file system management block of the distributed management block. By integrating Paxos and ZAB protocols through shard status marking and version vectors, the atomicity and consistency of shard updates are achieved, ensuring data integrity during collaborative storage across multiple systems.

[0036] To achieve data consistency management across heterogeneous file systems, the present invention adds three sets of status fields to the detailed information of the cross-file system management block, specifically including: LockStatus (read-write lock status), whose value is defined as 0 for unlocked, 1 for read locked, and 2 for write locked, which is used to control concurrent access rights to the shard; Checksum (checksum), which uses the SHA256 hash algorithm to calculate the shard data to verify the integrity of the data during transmission or storage; VersionVector uses the (system ID, version number) array format to record the latest version information of the corresponding fragments in each file system. For example, [(NTFS,3),(ext4,3),(jffs2,3)] means that the three fragments of the file are in the 3rd state in the NTFS, ext4, and jffs2 file systems.

[0037] During the data status synchronization process, when a file system initiates a shard update request, the cross-file system management block first sets the LockStatus of the shard to 2 (write lock) to block other concurrent modification operations; after the update operation is completed, the system automatically calculates a new Checksum value and increments the version number corresponding to the file system initiating the update in the VersionVector by 1; then, the cross-file system management block broadcasts the version update notification to all associated file systems storing the file shards until confirmation feedback is received from all associated file systems, and then resets the LockStatus to 0 (releases the lock) to complete a status synchronization.

[0038] Regarding the triggering mechanism of the hybrid consistency protocol (a fusion of Paxos and ZAB), the system automatically activates the hybrid protocol when the number of shards for a single file is greater than or equal to 2; when the number of shards is 1, the native consistency mechanism of the corresponding file system is directly used. The implementation process of this hybrid protocol is as follows: First, the proposal phase (based on the Paxos protocol) begins. The cross-file system management block acts as the proposer and sends an update proposal to all file systems storing the file shards (acting as acceptors). The proposal contains information such as the file ID, shard ID, new VersionVector, and data summary. After receiving the proposal, the acceptor verifies whether the VersionVector of the local shard is consistent with the old version carried in the proposal. If so, it returns a ready-to-accept response; otherwise, it rejects the proposal.

[0039] Next, the confirmation phase begins (based on the ZAB protocol). The proposer collects feedback from all recipients. When the number of recipients who agree to the proposal exceeds 1 / 2 of the total (i.e., k>n / 2, where n is the total number of file systems storing the shard), the atomic broadcast mechanism is triggered. The proposer sends a commit instruction to all recipients, and the recipients return a committed confirmation after performing the update operation. After receiving all committed confirmations, the cross-file system management block updates the global VersionVector to complete the consistency confirmation.

[0040] The handling mechanism for abnormal situations is as follows: If the number of acceptors who agree to the proposal does not reach the above threshold, the proposer will wait for 100ms and re-initiate the proposal, and the number of retries will be up to 3 times; if an acceptor does not respond after the timeout, the system will mark it as an "abnormal node" and temporarily exclude it from the consistency confirmation range. After the heartbeat test confirms that the node has returned to normal, its shard version information will be synchronized.

[0041] To quantify the threshold for consistency confirmation, let n be the total number of file systems storing a certain file shard, and the number of nodes k that need to agree to the proposal satisfy the formula: (For example, when n=3, k≥2; when n=4, k≥3).

[0042] When the version vectors V1=(s1,v1) and V2=(s2,v2) of two shards conflict (i.e., s1≠s2 and v1≠v2), the timestamp priority principle is used to resolve the conflict, that is, the version with the latest timestamp is retained first. Its mathematical expression is: , where T1 and T2 are the timestamps of the two shards respectively, T new Timestamp for the final preserved version.

[0043] Example 3 This embodiment provides a load sensing module to address load imbalance between low-speed interfaces (I2C / SPI) and high-speed interfaces (PCIe / network). This module is added to the file information collection system and, combined with metadata from the distributed management block, optimizes read and write traffic across the file system through LSTM load prediction, dynamic weight allocation, and elastic circuit breaking.

[0044] To achieve intelligent load balancing across file systems, this embodiment builds an integrated strategy that includes a load prediction model, a dynamic weight allocation mechanism, and an elastic circuit breaker mechanism. The specific implementation is as follows: The load prediction model uses an LSTM neural network architecture. Its input features are six-dimensional monitoring data from the past hour, sampled every five minutes. These data include: LoadCPU (CPU usage, in %), LoadMem (memory usage, in %), IOPS (number of I / O operations per second), Latency (average response delay, in milliseconds), InterfaceSpeed ​​(theoretical interface bandwidth, in Mbps, such as 10 Mbps for the SPI interface and 8000 Mbps for the PCIe interface), and ErrorRate (error rate, in ‰).

[0045] During model training and prediction, historical data is first preprocessed using a sliding window method with a window size of 12 (covering one hour of data). A normalization formula is used to map the data to the interval [0, 1] to eliminate dimensionality effects. An LSTM model is then trained, consisting of three hidden layers with 64, 32, and 16 nodes in each hidden layer, respectively. The model outputs predicted LoadCPU and Latency values ​​for the next two hours. To adapt to real-time load fluctuations, the model parameters are updated every 30 minutes.

[0046] Based on the above load prediction results, this embodiment designs a dynamic weight allocation mechanism, in which the read and write weight of each file system i is calculated by weighting the four dimensions, and the formula is:

[0047] Where, is the predicted CPU usage normalized to the [0,1] interval, To predict the response delay (in ms), Indicates health status (1 corresponds to normal, 0.5 corresponds to degraded, 0 corresponds to abnormal), is the interface priority (high-speed interfaces such as PCIe are set to 1, low-speed interfaces such as SPI are set to 0.5). When allocating traffic, read and write requests are prioritized according to the priorities of each file system. Proportional distribution, for example, when , 40% of the traffic will be allocated to system 1; for the low-speed interface file system, if its , the system will automatically trigger traffic transfer to compensate for the load, and the transfer ratio is calculated according to the formula as follows:

[0048] in, Indicates the traffic transfer ratio, which is used to compensate for the load of the file system on the slow interface.

[0049] The weight adjustment of high-speed interface system j after the transfer is as follows:

[0050] in, is the proportion of total flow diverted from the low-speed system; represents the sum of the initial weights of all low-speed systems; is the weight ratio of high-speed system j in all high-speed systems, which is used to distribute the total transferred traffic to each high-speed system j according to the load capacity ratio of the high-speed system.

[0051] To ensure system stability, this mechanism also includes an elastic circuit breaker policy, which triggers a circuit breaker when a file system i meets any of the following conditions five times in a row: (error rate is too high), (delay exceeded) or (Load too high).

[0052] After the fuse is blown, the file system Temporarily set it to 0 to stop receiving new requests, and at the same time start the shard migration process across the file system management block to migrate its stored shards to a file system with a higher weight; the system checks the recovery status of the file system every 1 minute. When the circuit breaker is broken, the weight is gradually restored according to the circuit breaker recovery curve, that is, the weight value of the circuit breaker at minute t (t=1,2,...) is , until the weights return to normal levels.

[0053] Example 4 This embodiment is a further refinement of the first embodiment. In this embodiment, a distributed file system management system is deployed on the server side, and file information collection systems are deployed on three file system sides: the NTFS file system for Windows, the ext4 file system for Linux, and the jffs2 file system for embedded devices. These three file systems constitute a heterogeneous file system cluster. The file information collection system is connected to the server side via hardware interfaces: the NTFS and ext4 file systems are connected via a gigabit network interface, and the jffs2 file system is connected via an SPI interface. Initially, each file information collection system initiates the metadata collection process: the NTFS file system's collection system collects internal files "doc1.txt" (2MB) and "img1.jpg" (5MB), the ext4 file system's collection system collects "data1.bin" (10MB), and the jffs2 file system's collection system collects "log1.txt" (1MB). The collected metadata is encapsulated in a unified format, including fields for "file name, size, creation time, and file system identifier." After encapsulation, each collection system uploads the metadata to the server-side distributed file system management system via the corresponding interface. After receiving the metadata, the server stores NTFS-related data in file system metadata block 1 (starting at address 0x00001000), ext4-related data in file system metadata block 2 (starting at address 0x00002000), and jffs2-related data in file system metadata block 3 (starting at address 0x00003000). It also constructs a file system index block, recording the starting addresses of the three metadata blocks and forming index entries: 0x00001000 (NTFS), 0x00002000 (ext4), and 0x00003000 (jffs2). When a 20MB file "bigfile.dat" needs to be stored across the three file systems, the system first fragments the file into three fragments in a 4:3:3 ratio, with sizes of 8MB, 6MB, and 6MB, respectively. The first fragment is then stored in the NTFS file system as "bigfile_part1.dat," the second fragment is stored in the ext4 file system as "bigfile_part2.dat," and the third fragment is stored in the jffs2 file system as "bigfile_part3.dat." Simultaneously, the distributed management block creates a cross-file system management block, assigns it storage address 0x00005000, and writes this address into the cross-file system index.Next, the system enters the storage information of each shard into the cross-file system management block according to a preset format, forming three records: the first record contains "bigfile.datpart1 NTFS storage path / usr / ntfs / starting address 0x00010000 size 8MB"; the second record contains "bigfile.datpart2 ext4 storage path / var / ext4 / starting address 0x00020000 size 6MB"; and the third record contains "bigfile.datpart3 jffs2 storage path / mnt / jffs2 / starting address 0x00030000 size 6MB." The total number of cross-file system files recorded in the cross-file system management block is 1. If storage expansion is subsequently required and a new Linux system yaffs2 file system is added, the file information collection system collects the system metadata and uploads it. The server then adds the address 0x00004000 (yaffs2) to the file system index block. If "bigfile.dat" needs to be expanded to store 4MB of data, the system creates a new shard, "bigfile_part4.dat," and stores it in the yaffs2 file system. Its information, "bigfile.datpart4yaffs2, storage path: / home / yaffs2 / , starting address: 0x00040000, size: 4MB," is added to the cross-file system management block in the same format. At this point, the total number of cross-file system files remains at 1, but the number of shard records is updated to 4, achieving storage expansion. When reading a file, the system uses the cross-file system index to locate the cross-file system management block (address: 0x00005000), parses it, and obtains the storage information for the four shards. The file system index block then locates the file system metadata block to which each shard belongs. Finally, the metadata block information accurately locates the physical location of each shard within the corresponding file system, enabling parallel reading and significantly improving read efficiency.

[0054] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0055] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A distributed file system management system, characterized by: It includes a distributed management block, multiple file system metadata blocks and a file information collection system; the distributed management block is configured to implement index management of each file system and collaborative control across file systems, and the file information collection system is configured to collect file metadata of each file system and upload it to the file system metadata block.

2. A distributed file system management system according to claim 1, characterized in that: The distributed management block includes a file system index block, a cross-file system index, and a cross-file system management block; the file system index block is used to record the starting address of each file system metadata block, and the cross-file system index is used to point to the storage address of the cross-file system management block.

3. A distributed file system management system according to claim 2, characterized in that: The file system index block uses a pointing addressing method to directly locate the corresponding file system metadata block through the starting address. The file system metadata block stores file attribute information, which includes file name, file size and creation time.

4. A distributed file system management system according to claim 3, characterized in that: The cross-file system index adopts a pointing addressing method. The cross-file system management block is configured to manage the storage interval of a single file in multiple file systems. It includes the total number of cross-file system files and detailed information of each cross-file system file. The detailed information includes the file name, storage distribution and allocation details in each file system.

5. A distributed file system management system according to claim 1, characterized in that: The file information acquisition system and the distributed management block exchange data through a hardware interface, which includes one or more of a network interface, a PCIe interface, an I2C interface, or an SPI interface.

6. A distributed file system management system according to claim 5, characterized in that: The file metadata collected by the file information collection system is consistent with the storage structure of the file system metadata block, and both are encapsulated in a preset unified format that includes file identification, capacity parameters and timestamp fields.

7. A distributed file system management system according to claim 1, characterized in that: The cross-file system management block supports collaborative storage management of homogeneous and heterogeneous file systems. When a single file is distributed across multiple heterogeneous file systems, the cross-file system management block implements the associated mapping of each file fragment through detailed information.

8. A distributed file system management system according to claim 7, characterized in that: The cross-file system management block is configured to support parallel storage and parallel read operations. It locates the storage location of each file segment by parsing detailed information, thereby achieving collaborative data reading and writing of multiple file systems.

9. A distributed file system expansion method, characterized by: The distributed file system management system according to any one of claims 1 to 8 comprises the following steps: S1, collects file metadata from each file system through the file information collection system, encapsulates it in a unified format, and uploads it to the distributed file system management system; S2, the distributed file system management system stores the received file metadata in the corresponding file system metadata block and constructs a file system index block based on the starting address of each file system metadata block to achieve unified management and control of metadata; S3: When a single file needs to be stored across multiple file systems, the file data is fragmented and each fragment is stored in the target file system. At the same time, a cross-file system management block is created through the distributed management block, and its storage address is written into the cross-file system index. S4, recording the storage information of the corresponding file fragments in each file system into the cross-file system management block according to a preset format, where each fragment information corresponds to a cross-file system file record; S5, in the process of adding a new file system to expand storage capacity, the file fragment information in the new file system is added to the cross-file system management block according to a preset format to maintain the integrity of the cross-file system file record.

10. A distributed file system expansion method according to claim 9, characterized in that: In S3, the sharding of file data is independent of the file system type. The target file system includes homogeneous file systems and heterogeneous file systems. Each shard is stored as an independent file in the target file system.

Images