Fast file expansion method for exFAT file system
By traversing the cluster linked list in the exFAT file system, allocating new clusters and linking to the tail of the cluster linked list, the problems of slow file expansion speed and low efficiency are solved, and efficient expansion and compatibility improvement of large files are achieved.
Patent Information
- Application Number
- CN202411442055.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-02-07
AI Technical Summary
The existing exFAT file system is slow when expanding files and requires writing a large amount of empty data, resulting in inefficiency and serious fragmentation.
By traversing the cluster linked list of files to be expanded, calculating the number of new clusters required, allocating new clusters, and linking them to the tail of the cluster linked list, forming a new complete cluster linked list, updating the file size and attributes, and achieving rapid file expansion.
This method realizes efficient expansion of large files without affecting the normal operation of the system, avoids the inefficiency problem of writing empty data, and improves the speed and compatibility of file expansion.
Smart Images
Figure CN119396773A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of rapid file expansion, and in particular to a rapid file expansion method for an exFAT file system. Background Art
[0002] exFAT (Extended File Allocation Table) is a file system developed by Microsoft, designed for portable devices and embedded systems, to address the limitations of the FAT32 file system in file size and partition size. As an alternative to FAT32, exFAT provides larger storage capacity and higher performance, especially for flash storage devices such as USB flash drives and SD cards.
[0003] There are two existing exFAT file extension methods:
[0004] Standard file system calls
[0005] Fallocate is a system call used to preallocate or extend the space of a file. Normally, fallocate can effectively preallocate disk space without writing data to the file content, but the exfat file system currently does not support this function. In actual tests, fallocate still expands the file size by appending 0 to the end of the file. This method of writing 0 may take more than ten minutes to create large files such as hundreds of GB files, which will greatly delay the system process. Ftruncate is another system call, which is mainly used to adjust the size of a file. If the new size is larger than the original file, the file will be extended at the end, and the newly added part will usually be filled with 0. Fseek is also a system call. It completes the extension by moving the file pointer to the instruction size and writing \0 to determine the file boundary. In actual tests, the expansion speed is the same as the expansion method of writing 0.
[0006] Command line method
[0007] truncate is a userspace command line tool used to resize files. The underlying execution is similar to the ftruncate system call, and in the exFAT file system, it still appends zeros to the end of the file.
[0008] Existing technical means can create files when the file system is mounted, but its underlying logic is to write empty data, which is inefficient and severely fragmented. There are also technologies that can change the size of files by directly rewriting file information in the file system when the file system is not mounted on the operating system, but this requires unmounting the file system in advance, which will affect other operations that access the hard disk at the same time. Summary of the invention
[0009] In view of the above problems, the present invention discloses a method for rapid file expansion for an exFAT file system, which relates to the technical field of rapid file expansion, effectively solves the problem of slow file expansion speed, and improves the compatibility and flexibility of the file expansion method.
[0010] The method comprises the following steps:
[0011] S1. Check whether the file system has sufficient storage space and mark the file system as dirty.
[0012] S2, traverse the cluster list of the file to be extended, and find the last cluster of the file to be extended;
[0013] S3, calculating the number of new clusters required for the file to be expanded and allocating new clusters;
[0014] S4, linking the tail cluster of the to-be-allocated file cluster linked list with the head cluster of the new cluster linked list to form a new complete cluster linked list;
[0015] S5, updating the size of the file to be expanded;
[0016] S6, set file attributes, update directory entries and inode;
[0017] S7. Align the expanded files.
[0018] Furthermore, the method for confirming whether the file system has sufficient storage space is to obtain information on disk space usage by querying the super block of the file system, and compare it with the target cluster number of the file to be expanded to confirm whether the file system has sufficient storage space.
[0019] Furthermore, the method for traversing the cluster linked list of the file to be extended is to query the starting cluster of the file to be extended through the information stored in the inode of the file to be extended, and then traverse the cluster linked list of the file to be extended by reading the FAT table of the file system to be extended.
[0020] Furthermore, the number of new clusters required for the file to be expanded is the target number of clusters of the file to be expanded minus the original number of clusters of the file to be expanded, the new clusters include continuous clusters and discrete clusters, and the allocation of new clusters includes searching for new clusters in the cluster bitmap, verifying the validity of the cluster, and allocating the cluster bitmap and the FAT table.
[0021] Furthermore, the linking method is to set the next member of the tail cluster of the to-be-allocated file cluster linked list as the head cluster of the new cluster linked list.
[0022] Furthermore, the method for updating the size of the file to be expanded is to write the new size of the file to be expanded into the file node through the size member in the inode.
[0023] Furthermore, the file attribute is set to a "modified" state, and the updating of the directory entry, i.e., the updating of the directory entry, cluster bitmap and FAT table is completed simultaneously in the same write-back operation, and the updating of the inode specifically updates the size occupied by the file on the physical storage medium and the number of blocks actually occupied by the file on the disk.
[0024] Furthermore, the method for aligning the extended file is to round up the extended file to the boundary of the file system cluster.
[0025] The beneficial effects of the present invention are:
[0026] This method is the same as the command line method. It does not require the file system to be formatted or unmounted in advance. It can be widely used to achieve efficient expansion of large files without significantly increasing system resource consumption. It is particularly suitable for large-capacity storage devices and embedded systems that need to frequently expand file sizes.
[0027] This method overcomes the technical bias of the existing technology that requires writing empty data when expanding files. Such operations are compatible with the system and can be performed synchronously when the system is operating normally. There is no need to perform special formatting operations on the storage device in advance, nor is there a need to suspend system operation, cancel mounting, or interrupt other normal operations of the file system when expanding files. The implementation of this method has no dependence on the existing file system architecture and can be seamlessly connected with the existing file management process, ensuring its wide application in various common usage scenarios, and effectively solving the problems of low efficiency and severe fragmentation in the file expansion process. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 This is a flow chart of the method described in an embodiment of the present invention; DETAILED DESCRIPTION
[0029] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0030] This embodiment provides a method for quickly expanding files in an exFAT file system. The flowchart of the method is as follows: Figure 1 , the method comprises the following steps:
[0031] S1. Confirm whether the file system has sufficient storage space and mark the file system as dirty.
[0032] The relevant operations of step S1 are introduced with a specific example:
[0033] Query the super block of the file system to obtain the information of the disk occupied space, and compare it with the target number of clusters of the file to be expanded. In this embodiment, the remaining disk space is 1TB, the size of the file to be expanded is 1B, and the target file size is 500GB. The cluster size (clu_size) of the exFAT file system can be set by itself, that is, the target number of clusters of the file to be expanded is 500GB / clu_size, the original number of clusters of the file to be expanded is 1B / clu_size (1 when the original number of clusters of the file to be expanded is less than 1), and the number of clusters of the remaining disk space is 1TB / clu_size. It can be obtained that the remaining disk storage space is sufficient to complete the comparison result of storing the 500GB file extension content, and the super block of the file system is set to dirty to prevent data inconsistency problems during file expansion.
[0034] S2. Traverse the cluster linked list of the file to be extended and find the last cluster of the file to be extended.
[0035] The relevant operations of step S2 are introduced with a specific example:
[0036] The starting cluster of the file to be extended is queried through the information stored in the inode of the file to be extended, and then the cluster linked list of the file to be extended is traversed by reading the FAT table of the file system to be extended, and the position of the end cluster (clast) used by the file to be extended is found.
[0037] S3. Calculate the number of new clusters required for the file to be expanded and allocate new clusters.
[0038] The relevant operations of step S3 are introduced with a specific example:
[0039] The number of new clusters required for the file to be extended is the number of target clusters of the file to be extended minus the number of original clusters of the file to be extended, that is, the number of new clusters (cnew) = 500GB / clu_size-1B / clu_size, and the number of cnew is 499.99GB / clu_size. The allocated new clusters are as continuous as possible to improve the reading and writing performance of the file. Allocating new clusters includes searching for new clusters in the cluster bitmap, verifying the validity of the clusters, and allocating the cluster bitmap and FAT table.
[0040] S4. Link the tail cluster of the to-be-allocated file cluster list with the head cluster of the new cluster list to form a new complete cluster list.
[0041] The relevant operations of step S4 are introduced with a specific example:
[0042] Set the next member of the to-be-allocated file cluster list clast as the first cluster of the cnew list to obtain a new complete cluster list.
[0043] S5. Update the size of the file to be expanded.
[0044] The relevant operations of step S5 are introduced with a specific example:
[0045] The new size of the file to be expanded, 500GB, is written into the file node through the size member in the inode.
[0046] S6. Set file attributes and update directory entries and inodes.
[0047] The relevant operations of step S6 are introduced with a specific example:
[0048] Set the file attributes to the "modified" state, set the archive bit in the file attributes to 1, and synchronously update the directory entry, cluster bitmap, and FAT table in the same write-back operation. Update the inode by updating the i_blocks and i_size_ondisk members in the inode in order to update the size of the file on the physical storage medium and the number of blocks actually occupied by the file on the disk.
[0049] S7. Align the expanded files.
[0050] The relevant operations of step S7 are introduced with a specific example:
[0051] The expanded file is rounded up to the cluster boundary, that is, a multiple of the block, so that it meets the boundary requirements of the file system block and ensures that the data can be written to the storage device in the best way.
Claims
1. A method for rapidly expanding files in an exFAT file system, characterized in that: The method comprises the following steps: S1. Check whether the file system has sufficient storage space and mark the file system as dirty. S2, traverse the cluster list of the file to be extended, and find the last cluster of the file to be extended; S3, calculating the number of new clusters required for the file to be expanded and allocating new clusters; S4, linking the tail cluster of the to-be-allocated file cluster linked list with the head cluster of the new cluster linked list to form a new complete cluster linked list; S5, updating the size of the file to be expanded; S6, set file attributes, update directory entries and inode; S7. Align the expanded files.
2. A method for rapid file expansion for exFAT file system according to claim 1, characterized in that: The method for confirming whether the file system has sufficient storage space is to obtain information on disk space usage by querying the super block of the file system, and compare it with the target cluster number of the file to be expanded, so as to confirm whether the file system has sufficient storage space.
3. The method for rapidly expanding files for the exFAT file system according to claim 1, characterized in that: The method for traversing the cluster linked list of the file to be extended is to query the starting cluster of the file to be extended through the information stored in the inode of the file to be extended, and then traverse the cluster linked list of the file to be extended by reading the FAT table of the file system to be extended.
4. The method for rapidly expanding files for the exFAT file system according to claim 2, characterized in that: The number of new clusters required for the file to be expanded is the target number of clusters of the file to be expanded minus the original number of clusters of the file to be expanded. The new clusters include continuous clusters and discrete clusters. The allocation of new clusters includes searching for new clusters in the cluster bitmap, verifying the validity of the clusters, and allocating the cluster bitmap and the FAT table.
5. The method for rapidly expanding files for exFAT file system according to claim 3, characterized in that: The linking method is to set the next member of the tail cluster of the to-be-allocated file cluster linked list as the head cluster of the new cluster linked list.
6. The method for rapidly expanding files for exFAT file system according to claim 3, characterized in that: The method for updating the size of the file to be expanded is to write the new size of the file to be expanded into the file node through the size member in the inode.
7. The method for rapidly expanding files for exFAT file system according to claim 1, characterized in that: The file attribute is set to the "modified" state, and the directory entry update, that is, the update of the directory entry, cluster bitmap and FAT table is completed simultaneously in the same write-back operation, and the inode update specifically updates the size occupied by the file on the physical storage medium and the number of blocks actually occupied by the file on the disk.
8. The method for rapidly expanding files for exFAT file system according to claim 7, characterized in that: The method for aligning the extended file is to round up the extended file to the boundary of the file system cluster.