A file management method, device, storage medium and terminal

Abstract

Embodiments of the present application disclose a file management method and device, a storage medium and a terminal. In the present application, an arbitration control module with a pre-compression function is added in the EROFS system image formatting process. The arbitration control module calculates the file compression ratio of the to-be-compressed file, compares the file compression ratio with the optimal file compression ratio, and selects a target file with a larger file compression ratio from the to-be-compressed file based on the comparison result. No compression processing is performed on the target file, which can improve the negative optimization problem of file compression in the EROFS system, thereby improving the sequential read-write performance of the EROFS.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a file management method, apparatus, storage medium, and terminal. Background Technology

[0002] Currently, extended read-only file systems (EROFS) are frequently used in terminals. Data in EROFS is stored in compressed form, which reduces the space occupied by the data and improves random read performance.

[0003] In related technologies, the default processing of EROFS formatting tools for image files is to use a "brute force" compression strategy that compresses all types of files in the image file. This compression strategy can improve the overall compression ratio of the image file. However, different files perform differently after compression. In EROFS, sequential read performance is strongly correlated with the quality of file compression. When the compression effect of some files is poor, the compression strategy of compressing all files will affect the sequential read and write performance of EROFS. Summary of the Invention

[0004] This application provides a file management method, apparatus, storage medium, and terminal that can improve the sequential read performance of a file system.

[0005] This application provides a file management method, including:

[0006] Obtain at least one file to be compressed;

[0007] Based on the file index information of each file to be compressed, predict the target cache size to be occupied by each file after compression processing by the compression algorithm of the specified file system;

[0008] The file compression ratio of each file to be compressed is determined based on the initial file size of each file to be compressed and the target cache size.

[0009] From the at least one file to be compressed, determine a target file whose file compression ratio is greater than the reference compression ratio of the specified file system;

[0010] The target file is removed from the at least one file to be compressed, so that the target file is not compressed using the specified compression algorithm.

[0011] Accordingly, embodiments of this application also provide a file management device, including:

[0012] The acquisition unit is used to acquire at least one file to be compressed.

[0013] The prediction unit is used to predict the target cache size occupied by each file to be compressed after being compressed by the compression algorithm of the specified file system, based on the file index information of each file to be compressed.

[0014] The first determining unit is used to determine the file compression ratio of each file to be compressed based on the initial file size of each file to be compressed and the target cache size;

[0015] The second determining unit is used to determine, from the at least one file to be compressed, a target file whose file compression ratio is greater than the reference compression ratio of the specified file system;

[0016] The removal unit is used to remove the target file from the at least one file to be compressed, so that the target file is not compressed by the specified compression algorithm.

[0017] In some embodiments, the prediction unit includes:

[0018] The first determining subunit is used to determine the file data of the file to be compressed based on the file index information;

[0019] The first acquisition subunit is used to determine the buffer in the specified file system that is used for file compression processing;

[0020] A processing subunit is used to pass the file data into the buffer for compression processing and determine the number of buffer blocks occupied by the compression processing in the buffer.

[0021] The second determining subunit is used to determine the target cache size based on the cache size of the buffer block and the number of blocks.

[0022] In some embodiments, the processing subunit is specifically used for:

[0023] Obtain the starting buffer block number of the buffer;

[0024] The specified compression algorithm is invoked to perform compression processing on the file data, and the target buffer block number corresponding to the compression processing is obtained;

[0025] The number of blocks is calculated based on the starting buffer block number and the target buffer block number.

[0026] In some embodiments, the first determining unit includes:

[0027] The calculation subunit is used to calculate the ratio of the target cache size to the initial file size to obtain the file compression ratio.

[0028] In some embodiments, the second determining unit includes:

[0029] The third determining subunit is used to determine the reference compression ratio of the specified file system;

[0030] The fourth determining subunit is used to compare the file compression ratio of each file to be compressed with the reference compression ratio, determine the file to be compressed whose file compression ratio is greater than the reference compression ratio, and obtain the target file.

[0031] In some embodiments, the third determining subunit is specifically used for:

[0032] Get the actual file compression ratio of each file in the target file partition of the specified file system;

[0033] The actual file compression ratios of each file are sorted according to preset rules to obtain the sorted actual file compression ratios;

[0034] Obtain the first read / write performance of each file in the target partition on the specified file system, and the second read / write performance on other file systems;

[0035] The first read / write performance is compared with the second read / write performance to obtain the comparison result;

[0036] The reference compression ratio is determined from the actual file compression ratios after sorting based on the comparison results.

[0037] In some embodiments, the device further includes:

[0038] The processing unit is configured to compress files other than the target file in the at least one file to be compressed using the specified compression algorithm.

[0039] Accordingly, embodiments of this application also provide a storage medium storing a plurality of instructions adapted for loading by a processor to execute the file management method described above.

[0040] Accordingly, this application also provides a terminal, including a processor and a memory, wherein the memory stores multiple instructions, and the processor loads the instructions to execute the file management method described above.

[0041] This application embodiment adds an arbitration control module with a "pre-compression" function during the EROFS system image formatting process. The arbitration control module calculates the file compression ratio of the file to be compressed and compares the file compression ratio with the optimal file compression ratio. Based on the comparison result, the target file with the larger file compression ratio is selected from the file to be compressed. The target file is not compressed. This can improve the negative optimization problem of file compression in the EROFS system, thereby improving the sequential read and write performance of EROFS. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a file management method provided in an embodiment of this application.

[0044] Figure 2 This is a flowchart illustrating another file management method provided in an embodiment of this application.

[0045] Figure 3 This is a structural block diagram of a file management device provided in an embodiment of this application.

[0046] Figure 4 This is a schematic diagram of the terminal structure provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] This application provides a file management method, apparatus, storage medium, and computer device. Specifically, the file management method of this application can be executed by a terminal. The terminal can be a smartphone, tablet computer, laptop computer, touch screen, personal computer (PC), personal digital assistant (PDA), or other terminal device.

[0049] For example, the terminal can acquire at least one file to be compressed; predict the target cache size to be occupied by each file after compression by a specified file system compression algorithm based on the file index information of each file to be compressed; determine the file compression ratio of each file to be compressed according to the initial file size and the target cache size; identify a target file from the at least one file to be compressed whose file compression ratio is greater than the reference compression ratio of the specified file system; and remove the target file from the at least one file to be compressed so that the target file is not compressed by the specified compression algorithm.

[0050] To address the aforementioned problems, this application provides a file management method, apparatus, storage medium, and terminal, which can improve the sequential read performance of the file system. These will be described in detail below. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0051] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a file management method provided in an embodiment of this application. Taking the application of this file management method to a terminal as an example, the specific process of the file management method can be as follows:

[0052] 101. Obtain at least one file to be compressed.

[0053] The file to be compressed refers to the file that needs to be compressed through the specified file system. The file to be compressed can be a file inside the specified file system.

[0054] 102. Based on the file index information of each file to be compressed, predict the target cache size to be occupied by each file after compression processing by the compression algorithm of the specified file system.

[0055] In this context, file index information refers to inodes, which are data structures, essentially data structures, containing important information related to each file in the file system. Inodes store file information for each file or directory, such as file creation time and file size. In a file system, a file or directory corresponds to a unique inode number on the disk layout, serving as its identifier.

[0056] The specified file system can be an EROFS (Extendable Read-Only File System) file system.

[0057] Specifically, by obtaining the file index information of the file to be compressed in the specified file, the file to be compressed can be pre-compressed.

[0058] In this embodiment of the application, pre-compression processing can represent a simulated compression process of the file to be compressed. Pre-compression processing is different from the actual compression process. After compressing the file, pre-compression processing does not write the compressed file data to the disk block, which can save disk space. The compression ratio of each file to be compressed can be calculated through pre-compression processing.

[0059] The file compression ratio refers to the ratio of the size of the compressed file to the size of the original file. For example, if a file with an initial size of 100m is compressed to a size of 90m, then the file compression ratio is 90 / 100*100%=90%. Generally, the lower the file compression ratio, the better.

[0060] In some embodiments, to improve the accuracy of calculating the file compression ratio of the files to be compressed, the step "predicting the target cache size occupied by each file after compression processing by the compression algorithm of the specified file system based on the file index information of each file to be compressed" may include the following operations:

[0061] Determine the file data of the file to be compressed based on the file index information;

[0062] Determines the buffer in the specified file system that is used for file compression processing;

[0063] The file data is passed into the buffer for compression, and the number of buffer blocks occupied by the compression process is determined.

[0064] The target cache size is determined based on the cache size and number of buffer blocks.

[0065] Specifically, to determine the file data of the file to be compressed based on the file index information, you can first obtain the file descriptor of the file to be compressed based on the file index information, and then read the file data of the file to be compressed through the file descriptor.

[0066] A file descriptor is an identifier returned by the `open` function (used to open and create files) for each file, used for file reading and writing. While it can be a non-negative integer, a file descriptor is actually an index value pointing to a table maintained by the kernel for each process, recording the files opened by that process. When a program opens an existing file or creates a new file, the kernel returns a file descriptor to the process.

[0067] Specifically, the file descriptor is obtained by opening the inode's src_path. Here, src_path refers to the actual path of the source directory to be compressed into the corresponding image file when the image file corresponding to the read-only partition of the specified file system is formatted.

[0068] Specifically, file data of the file to be compressed can be read through file descriptors. The length of the read data can be determined based on the total length of the file to be compressed. Since the buffer size for compression processing in a given file system is limited, when the file data to be compressed is too large, the file data can be divided into multiple data packets and then processed in multiple rounds of compression.

[0069] Furthermore, the buffer used for file compression processing in the specified file system is determined, and then the file data read through the file descriptor is passed into the buffer for compression processing to determine the number of buffer blocks occupied by the file to be compressed in the buffer after compression processing.

[0070] In some embodiments, the step "passing file data into a buffer for compression and determining the number of buffer blocks occupied by the compression process in the buffer" may include the following operations:

[0071] Get the starting buffer block number of the buffer;

[0072] The specified compression algorithm is invoked to compress the file data, and the corresponding target buffer block number after compression is obtained.

[0073] The number of blocks is calculated based on the starting buffer block number and the target buffer block number.

[0074] The starting buffer block number can be obtained from the starting address of the buffer. The buffer can be divided into multiple buffer blocks, and different buffer blocks can correspond to different block numbers.

[0075] Furthermore, the compression module corresponding to the specified compression algorithm in the specified file system is called to perform the compression process on the file data of the file to be compressed. After the compression process is completed, the updated buffer block number, which is also the target buffer block number, is obtained.

[0076] In this embodiment, the starting buffer block number of the requested buffer is retained before the file to be compressed is compressed. The buffer block number is updated during each round of compression until the last round of compression is completed. The number of blocks calculated based on the starting buffer block number and the target buffer block number can be obtained by subtracting the previously retained starting buffer block number from the target buffer block number, thereby obtaining the number of buffer blocks occupied by the entire file to be compressed after compression.

[0077] Specifically, the target cache size can be determined by multiplying the number of buffer blocks by the number of blocks occupied after file compression, which is the size of the entire file to be compressed.

[0078] For example, the number of buffer blocks occupied can be 4, and the size of the buffer block can be 4 * 1024 bytes, which is 4KB. Then the target cache size can be calculated as 4 * 4KB = 16KB.

[0079] 103. Determine the file compression ratio of each file to be compressed based on the initial file size and the target cache size.

[0080] In some embodiments, the step "determine the file compression ratio of each file to be compressed based on the initial file size of each file to be compressed and the target cache size" may include the following steps:

[0081] The file compression ratio is obtained by calculating the ratio of the target cache size to the initial file size.

[0082] The target cache size refers to the size of the compressed file after pre-compression of the file to be processed. The file compression ratio of the file to be compressed can be calculated as: compressed file size / original file size.

[0083] The file size before compression is the initial file size of the file to be compressed, which can be obtained through the file index information of the file to be compressed.

[0084] For example, the compressed file size after pre-compression of the file to be processed can be 16KB, and the file size before compression can be 20KB. Then the file compression ratio of the file to be compressed can be calculated as: 16KB / 20KB*100%=80%.

[0085] 104. Determine the target file from at least one file to be compressed, whose compression ratio is greater than the reference compression ratio of the specified file system.

[0086] In this embodiment of the application, during the formatting process of a specified file system image, a judgment logic is designed to determine whether the file to be compressed needs to be actually compressed. This can improve the "negative optimization" problem that is common in some scenarios of sequential read performance of the specified file system.

[0087] Specifically, after calculating the file compression ratio of each file to be compressed, it can be determined whether the file to be compressed needs to be actually compressed based on the file compression ratio.

[0088] In some embodiments, to improve the accuracy of the judgment, the step "determine the target file whose file compression ratio is greater than the reference compression ratio of the specified file system from the file to be compressed" may include the following operations:

[0089] Determine the reference compression ratio for the specified file system;

[0090] The compression ratio of each file to be compressed is compared with the reference compression ratio. The file to be compressed with the compression ratio greater than the reference compression ratio is determined, and the target file is obtained.

[0091] First, it's necessary to obtain the reference compression ratio for the specified file system, which is also the optimal file compression ratio. This reference compression ratio is then used to determine whether the file to be compressed actually needs to be compressed.

[0092] Specifically, the calculated compression ratio of each file to be compressed is compared with the reference compression ratio. The main goal of this scheme is to filter out files with poor compression performance. Files with a compression ratio greater than the reference compression ratio can be selected to obtain target files that do not require actual compression processing.

[0093] In some embodiments, to further improve the accuracy of the determination, the step "determine the reference compression ratio of the specified file system" may include the following steps:

[0094] Get the actual file compression ratio of each file in the target file partition of the specified file system;

[0095] The actual file compression ratios of each file are sorted according to preset rules to obtain the sorted actual file compression ratios;

[0096] Get the first read / write performance of each file in the target partition on the specified file system, and the second read / write performance on other file systems;

[0097] The first read / write performance is compared with the second read / write performance, and the comparison results are obtained.

[0098] A reference compression ratio is determined from the actual file compression ratios after sorting, based on the comparison results.

[0099] In this application embodiment, determining the optimal file compression ratio for each file is crucial. The optimal file compression ratio serves as a standard for measuring whether to actually compress the file. This optimal file compression ratio can be determined using rigorous big data statistical methods to obtain this standard value.

[0100] The specific method for determining the optimal file compression ratio is as follows: In a system environment with a specified file system where all files in the image file have been compressed, use a dedicated script tool to sort and count all files of various types under the / system partition and / vendor partition according to their actual file compression ratio. Then, use a file system-specific IO performance testing tool (fio) to test the sequential read performance of these files. Compare this performance with other file systems to find the specific turning point, that is, the point where the sequential read performance of the specified file system first shows "negative optimization" compared to other file systems. Finally, obtain the actual file compression ratio corresponding to the point of "negative optimization" as the optimal file compression ratio.

[0101] The specific implementation steps are as follows: With the specified file system in default full compression mode, use appropriate scripts to calculate the file compression ratio of all .apk and .so files under the read-only partitions / system and / vendor. Then, arrange the file compression ratios in descending order, and divide the files into compression ratio ranges of 90%–100%, 80%–90%, 70%–80%, etc. Next, use the fio performance testing tool to compare the sequential read performance of files within each specific range with other file systems before importing the specified file system. The general principle behind the improved read performance of the specified file system is that the smaller the file compression ratio (i.e., the better the file compression effect), the more significant the improvement in IO performance compared to other file systems. Therefore, there is a "turning point," where when reading files with compression ratios lower than this turning point, IO performance is improved compared to other file systems. However, when reading files with a compression ratio higher than the compression ratio at this "turning point" (i.e., the files themselves have poor compression), there will be a "negative optimization" situation where the IO performance is not as good as other file systems. Based on the file compression ratio corresponding to the "turning point" when the "negative optimization" situation occurs, the optimal file compression ratio is obtained.

[0102] In this embodiment, the principle for determining the reference compression ratio is as follows: the key improvement of the specified file system's performance compared to other file systems can be summarized as follows: compression reduces the actual I / O volume during file reading, thereby reducing I / O time. However, this also introduces a decompression process, which incurs additional time. The final reduction in I / O time, achieved by balancing these two factors, is the main reason for the performance improvement. Therefore, the better the file compression, the more significant this improvement will be, because compared to other uncompressed file systems, the actual I / O volume during file reading will be significantly reduced, while the additional I / O overhead from the decompression process is minimal. Conversely, for files with poor compression, the specified file system's performance not only fails to improve compared to other native file systems but may even decrease to some extent. Similarly, because the file compression is poor, meaning the compressed file size is not significantly smaller than the uncompressed file size, the reduction in the actual I / O volume during file reading is also very limited. However, the added decompression process introduces additional I / O time, and the final performance, achieved by balancing these two factors, will be slightly worse than that of other file systems.

[0103] 105. Remove the target file from at least one file to be compressed so that the target file is not compressed using the specified compression algorithm.

[0104] Specifically, by filtering out target files from all files to be compressed, compression of target files will not be performed during the subsequent actual compression process. This avoids the "negative optimization" problem of target file compression and thus improves the sequential read performance of the specified file system.

[0105] In some embodiments, the method may further include the following steps:

[0106] Compress at least one file in the file to be compressed, excluding the target file, by specifying a compression algorithm;

[0107] Specifically, the remaining files to be compressed after filtering out the target files are actually compressed to achieve file compression processing for the specified file system.

[0108] This application discloses a file management method, which includes: acquiring at least one file to be compressed; predicting the target cache size to be occupied by each file after compression using a specified file system compression algorithm based on the file index information of each file to be compressed; determining the file compression ratio of each file to be compressed based on the initial file size and the target cache size; identifying a target file from the at least one file to be compressed whose file compression ratio is greater than the reference compression ratio of the specified file system; and removing the target file from the at least one file to be compressed so that it is not compressed using the specified compression algorithm. By adding an arbitration control module with a "pre-compression" function during the specified file system image formatting process, the arbitration control module calculates the file compression ratio of the files to be compressed and compares the file compression ratio with the optimal file compression ratio. Based on the comparison result, the target file with the larger file compression ratio is selected from the files to be compressed. The target file is not compressed, which can improve the negative optimization problem of file compression in the specified file system, thereby improving the sequential read / write performance of the specified file system.

[0109] refer to Figure 2 , Figure 2 This is a flowchart illustrating another file management method provided in an embodiment of this application. Taking the application of this file management method to a terminal as an example, the specific details of the file management method are as follows:

[0110] 201. The terminal receives a file compression command for the EROFS file system and obtains the file to be compressed according to the file compression command.

[0111] The file compression command instructs the EROFS file system to compress files within the image during the image file formatting process. The files to be compressed refer to the files within the image.

[0112] Upon receiving a file compression command, the compression initialization is first performed, including: parsing the passed options through the default configuration items of the EROFS file system initialization tool, loading the hints related to the compression process (hints refer to a configuration file in the EROFS file system formatting tool that is related to the compression strategy of each file. The relevant content in the file needs to be read and loaded before compression. It is not enabled by default), initializing the compression module and the main compression algorithm, and completing various configurations related to the compression process.

[0113] Furthermore, relevant information about the file to be compressed is obtained, including: constructing the tree used for formatting via src_path (referring to the "tree mapping" relationship of each subdirectory and specific file under the root directory obtained through the passed-in root directory path, similar to hierarchical directories and hierarchical files), obtaining the inode of the file to be compressed via src_path, and performing specific tree construction work based on the passed-in inode.

[0114] Then, based on the inode of the file to be compressed, it is determined whether the file needs to be compressed. This includes: for the inode corresponding to the file to be compressed, determining whether a specific compression algorithm (LZ4, LZ4HC, LZMA, etc.) was specified when the formatting tool was called, and whether the file to be compressed belongs to the category of compressible files (the compression strategy of each file is controlled by a dedicated configuration file to determine whether each file belongs to the category of compressible files), as well as the compression processing control strategy designed in this scheme.

[0115] 202. The terminal performs pre-compression processing on the file to be compressed to obtain the actual file compression ratio of the file to be compressed.

[0116] Specifically, the compression processing control strategy is implemented as follows: Open the src_path of the file to be compressed to obtain the file descriptor, obtain the starting address (starting block number) of the buffer used for compression processing in the file system, then obtain the initial file size of the file to be compressed, that is, the file size before compression, through the inode, and read the file data of the file to be compressed of a specified length through the file descriptor and put it into the buffer used for pre-compression processing.

[0117] The pre-compression process can be as follows: First, perform the first round of compression, obtain the length of the file data to be compressed in the first round of compression, obtain the number of buffer blocks to be compressed each time when calling the specific compression algorithm, call the compression module corresponding to the specific compression algorithm to execute the compression process, after completing the first round of compression, update the block number of the buffer, and perform post-processing after the first round of compression (including the start and end pointers and data length).

[0118] Then, it checks if the data length is 0. If the data length is 0, it checks if the entire file to be compressed has completed pre-compression. If the entire file to be compressed has completed pre-compression, it performs the final round of compression. The number of blocks occupied by the entire file after pre-compression is calculated using the block number of the compressed buffer. (Before the file compression begins, the starting block address (number) of the allocated data buffer is retained. During each round of compression, the block address (number) of the data buffer is updated. After the final round of compression, the current data buffer is used...) The number of blocks occupied by the entire file after compression is obtained by subtracting the previously reserved starting block address from the block address of the buffer. The size of the compressed file is calculated by multiplying the number of blocks occupied by the compressed file by the size of each block (a fixed value of 4 * 1024 bytes, or 4KB). Finally, the file compression ratio of the file to be compressed is calculated based on the file size before compression and the file size after pre-compression, so as to obtain the actual file compression ratio of the file to be compressed.

[0119] After the first round of compression, if the data length is not 0, or the entire file to be compressed is not fully compressed, the second round, the third round of compression, and so on, can continue until the entire file to be compressed is fully compressed, so as to obtain the accurate number of buffer blocks occupied by the compression process.

[0120] 203. The terminal compares the actual file compression ratio of the file to be compressed with the optimal file compression ratio of the EROFS file system to determine whether the file to be compressed needs to be actually compressed.

[0121] After calculating the actual file compression ratio of the file to be compressed, the actual compression ratio can be compared with the optimal file compression ratio. Based on the comparison result, it can be determined whether the file to be compressed needs to be actually compressed.

[0122] The calculation process for the optimal compression ratio can be found in the above embodiments, and will not be elaborated further here.

[0123] In this embodiment, the optimal file compression ratio parameter N (threshold) can be designed to be configurable and passed in when calling the formatting tool on different platforms. Specifically, the optimal file compression ratio parameter N (threshold) may differ for different platforms due to differences in underlying storage devices, etc. After obtaining the specific optimal file compression ratio parameter N (threshold) for each platform using big data statistical methods, it can be passed as a script option when calling the EROFS unified formatting script during the compilation process. This enables manual configuration and flexible control of the optimal file compression ratio parameter N (threshold), as well as the strong portability of EROFS across platforms.

[0124] Specifically, if the comparison result indicates that the actual file compression ratio is greater than the optimal file compression ratio, it can be determined that the file to be compressed needs to be actually compressed, and step 204 is further executed; if the comparison result indicates that the actual file compression ratio is less than or equal to the optimal file compression ratio, it can be determined that the file to be compressed does not need to be actually compressed, and step 205 is further executed.

[0125] 204. The terminal compresses the file to be compressed and writes the compressed file to the storage space.

[0126] Specifically, the compressed file is compressed to obtain a compressed file, and the file data of the compressed file is written to the underlying disk block.

[0127] 205. The terminal directly writes the uncompressed file to be compressed into the storage space.

[0128] Specifically, the compressed file is not compressed; instead, the file data is written directly to the underlying disk block.

[0129] This solution improves EROFS's performance in sequential reads by adding an arbitration control module with "pre-compression" functionality, which strictly controls whether each file is actually compressed during image formatting. This addresses the common "negative optimization" issue in EROFS's sequential read performance in some scenarios. On the other hand, it rigorously utilizes big data statistical methods to find the optimal file compression ratio parameter N (threshold) in the newly added judgment logic. This maximizes the system IO performance improvement brought by EROFS's greatest feature, "compression," while effectively avoiding the potential loss in system sequential read performance due to poor compression of some files.

[0130] The proposed solution incorporates arbitration logic to assess the compression ratio of each file after compression. Files with poor compression are left uncompressed, while only those with good compression are actually compressed. Statistical analysis shows that these poorly compressed files constitute a small proportion of all files on the system's read-only partition. Therefore, this approach has minimal impact on the overall compression ratio of EROFS for the image file. Furthermore, this method largely resolves the previously observed "negative optimization" of EROFS compared to the original EXT4 sequential read performance. Since EXT4 doesn't compress files, EROFS also avoids compressing poorly compressed files. Finally, when testing the overall sequential read performance of such files, the results obtained by both methods should be similar.

[0131] This application discloses a file management method. This method receives a file compression command for the EROFS file system via a terminal, obtains the file to be compressed according to the command, performs pre-compression on the file to be compressed to obtain the actual file compression ratio, compares the actual file compression ratio with the optimal file compression ratio of the EROFS file system, and determines whether actual compression is needed. If actual compression is needed, the file is compressed and written to storage space; otherwise, the uncompressed file is directly written to storage space. This improves the "negative optimization" problem commonly found in EROFS for sequential read performance in certain scenarios.

[0132] To facilitate better implementation of the file management method provided in this application, this application also provides an apparatus based on the above-described file management method. The meanings of the terms used are the same as in the file management method described above, and specific implementation details can be found in the descriptions within the method embodiments.

[0133] Please see Figure 3 , Figure 3 This application provides a structural block diagram of a file management device. This file management device can be applied to mobile terminals such as mobile phones, tablets, laptops, PDAs, and portable media players (PMPs), as well as fixed terminals such as desktop computers. The device may include:

[0134] Acquisition unit 301 is used to acquire at least one file to be compressed;

[0135] Prediction unit 302 is used to predict the target cache size occupied by each file to be compressed after being compressed by the compression algorithm of the specified file system, based on the file index information of each file to be compressed.

[0136] The first determining unit 303 is used to determine the file compression ratio of each file to be compressed based on the initial file size of each file to be compressed and the target cache size;

[0137] The second determining unit 304 is used to determine, from the at least one file to be compressed, a target file whose file compression ratio is greater than the reference compression ratio of the specified file system;

[0138] The removal unit 305 is used to remove the target file from the at least one file to be compressed, so that the target file is not compressed by the specified compression algorithm.

[0139] In some embodiments, the prediction unit 302 may include:

[0140] The first determining subunit is used to determine the file data of the file to be compressed based on the file index information;

[0141] The first acquisition subunit is used to determine the buffer in the specified file system that is used for file compression processing;

[0142] A processing subunit is used to pass the file data into the buffer for compression processing and determine the number of buffer blocks occupied by the compression processing in the buffer.

[0143] The second determining subunit is used to determine the target cache size based on the cache size of the buffer block and the number of blocks.

[0144] In some embodiments, the processing subunit may specifically be used for:

[0145] Obtain the starting buffer block number of the buffer;

[0146] The specified compression algorithm is invoked to perform compression processing on the file data, and the target buffer block number corresponding to the compression processing is obtained;

[0147] The number of blocks is calculated based on the starting buffer block number and the target buffer block number.

[0148] In some embodiments, the first determining unit 303 may include:

[0149] The calculation subunit is used to calculate the ratio of the target cache size to the initial file size to obtain the file compression ratio.

[0150] In some embodiments, the second determining unit includes:

[0151] The third determining subunit is used to determine the reference compression ratio of the specified file system;

[0152] The fourth determining subunit is used to compare the file compression ratio of each file to be compressed with the reference compression ratio, determine the file to be compressed whose file compression ratio is greater than the reference compression ratio, and obtain the target file.

[0153] In some embodiments, the third determining subunit may specifically be used for:

[0154] Get the actual file compression ratio of each file in the target file partition of the specified file system;

[0155] The actual file compression ratios of each file are sorted according to preset rules to obtain the sorted actual file compression ratios;

[0156] Obtain the first read / write performance of each file in the target partition on the specified file system, and the second read / write performance on other file systems;

[0157] The first read / write performance is compared with the second read / write performance to obtain the comparison result;

[0158] The reference compression ratio is determined from the actual file compression ratios after sorting based on the comparison results.

[0159] In some embodiments, the device may further include:

[0160] The processing unit is configured to compress files other than the target file in the at least one file to be compressed using the specified compression algorithm.

[0161] This application discloses a file management device. The device acquires at least one file to be compressed via an acquisition unit 301; a prediction unit 302 predicts the target cache size to be occupied by each file after compression using a specified file system's compression algorithm, based on the file index information of each file; a first determination unit 303 determines the file compression ratio of each file based on its initial file size and the target cache size; a second determination unit 304 determines target files from the at least one file to be compressed whose compression ratio is greater than a reference compression ratio of the specified file system; and a rejection unit 305 removes the target files from the at least one file to be compressed, so that the target files are not compressed using the specified compression algorithm. This improves the sequential read / write performance of the specified file system.

[0162] This application also provides a terminal. For example... Figure 4As shown, the terminal may include a radio frequency (RF) circuit 601, a memory 602 including one or more storage media, an input unit 603, a display unit 604, a sensor 605, an audio circuit 606, a wireless fidelity (WiFi) module 607, a processor 608 including one or more processing cores, and a power supply 609, among other components. Those skilled in the art will understand that... Figure 4 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0163] RF circuit 601 can be used for signal reception and transmission during information transmission and reception. Specifically, it receives downlink information from the base station and hands it over to one or more processors 608 for processing; additionally, it transmits uplink data to the base station. Typically, RF circuit 601 includes, but is not limited to, an antenna, at least one amplifier, a tuner, one or more oscillators, a Subscriber Identity Module (SIM) card, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 601 can also communicate wirelessly with networks and other devices.

[0164] The memory 602 can be used to store software programs and modules. The processor 608 executes various functional applications and file management by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.). In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide access to the memory 602 for the processor 608 and the input unit 603.

[0165] Input unit 603 can be used to receive input numerical or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, in one embodiment, input unit 603 may include a touch-sensitive surface and other input devices. A touch-sensitive surface, also known as a touch display or touchpad, can collect touch operations by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface), and drive corresponding connected devices according to a pre-set program. In addition to a touch-sensitive surface, input unit 603 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0166] Display unit 604 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the server. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 604 may include a display panel, optionally configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. Furthermore, a touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to processor 608 to determine the type of touch event. Subsequently, processor 608 provides corresponding visual output on the display panel according to the type of touch event. Although in Figure 4 In this context, the touch-sensitive surface and the display panel are two separate components for implementing input and output functions. However, in some embodiments, the touch-sensitive surface and the display panel can be integrated to achieve both input and output functions.

[0167] The terminal may also include at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel according to the ambient light level, and the proximity sensor can turn off the display panel and backlight when the server is moved close to the ear.

[0168] Audio circuitry 606, a speaker, and a microphone provide an audio interface between the user and the server. Audio circuitry 606 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 606, converted back into audio data, and processed by processor 608. The audio data is then transmitted via RF circuitry 601 to, for example, a terminal, or output to memory 602 for further processing. Audio circuitry 606 may also include an earphone jack to facilitate communication between peripheral headphones and the server.

[0169] WiFi is a short-range wireless transmission technology. A terminal using a WiFi module 607 can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 4 WiFi module 607 is shown, but it is understood that it is not a necessary component of the terminal and can be omitted as needed without changing the nature of the application.

[0170] The processor 608 is the control center of the terminal, connecting various parts of the mobile phone through various interfaces and lines. It performs overall monitoring of the mobile phone by running or executing software programs and modules stored in the memory 602, and by calling data stored in the memory 602, thus executing various server functions and processing data. Optionally, the processor 608 may include one or more processing cores; preferably, the processor 608 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may not be integrated into the processor 608.

[0171] The terminal also includes a power supply 609 (such as a battery) to power various components. Preferably, the power supply can be logically connected to the processor 608 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 609 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0172] Specifically, in this embodiment, the processor 608 in the terminal loads the executable files corresponding to the processes of one or more applications into the memory 602 according to the following instructions, and the processor 608 runs the applications stored in the memory 602 to achieve various functions:

[0173] Obtain at least one file to be compressed;

[0174] Based on the file index information of each file to be compressed, predict the target cache size to be occupied by each file after compression processing by the compression algorithm of the specified file system;

[0175] The file compression ratio of each file to be compressed is determined based on the initial file size and the target cache size.

[0176] Identify a target file from at least one file to be compressed, whose compression ratio is greater than a reference compression ratio for the specified file system.

[0177] Remove the target file from at least one file to be compressed so that the target file is not compressed using the specified compression algorithm.

[0178] This application discloses a file management method, apparatus, storage medium, and terminal. The file management method includes: acquiring at least one file to be compressed; predicting the target cache size to be occupied by each file after compression using a compression algorithm of a specified file system based on the file index information of each file to be compressed; determining the file compression ratio of each file to be compressed based on the initial file size and the target cache size; identifying a target file from the at least one file to be compressed whose file compression ratio is greater than a reference compression ratio of the specified file system; and removing the target file from the at least one file to be compressed so that the target file is not compressed using the specified compression algorithm.

[0179] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be accomplished by instructions, or by controlling related hardware with instructions. These instructions can be stored in a storage medium and loaded and executed by a processor.

[0180] Therefore, embodiments of this application provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the file management methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0181] Obtain at least one file to be compressed; based on the file index information of each file to be compressed, predict the target cache size to be occupied by each file after compression using the specified file system's compression algorithm; determine the file compression ratio of each file to be compressed based on the initial file size and the target cache size; identify a target file from the at least one file to be compressed whose file compression ratio is greater than the reference compression ratio of the specified file system; remove the target file from the at least one file to be compressed so that the target file is not compressed using the specified compression algorithm.

[0182] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0183] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0184] Since the instructions stored in the storage medium can execute the steps of any of the file management methods provided in the embodiments of this application, the beneficial effects that any of the file management methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0185] The file management method, apparatus, storage medium, and terminal provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A file management method, characterized in that, The method includes: Obtain at least one file to be compressed; Based on the file index information of each file to be compressed, predict the target cache size to be occupied by each file after compression processing by the compression algorithm of the specified file system; The file compression ratio of each file to be compressed is determined based on the initial file size of each file to be compressed and the target cache size. From the at least one file to be compressed, determine a target file whose file compression ratio is greater than the reference compression ratio of the specified file system; The target file is removed from the at least one file to be compressed, so that the target file is not compressed using the compression algorithm; The step of predicting the target cache size occupied by each file after compression using a specified file system compression algorithm, based on the file index information of each file to be compressed, includes: The file data of the file to be compressed is determined based on the file index information; Determine the buffer in the specified file system that is used for file compression processing; The file data is passed into the buffer for compression, and the number of buffer blocks occupied by the compression process in the buffer is determined. The target cache size is determined based on the cache size of the buffer block and the number of blocks.

2. The method according to claim 1, characterized in that, The step of passing the file data into the buffer for compression processing, and determining the number of buffer blocks occupied by the compression processing in the buffer, includes: Obtain the starting buffer block number of the buffer; The compression algorithm is invoked to compress the file data, and the target buffer block number corresponding to the compression is obtained; The number of blocks is calculated based on the starting buffer block number and the target buffer block number.

3. The method according to claim 1, characterized in that, The step of determining the file compression ratio of each file to be compressed based on the initial file size of each file to be compressed and the target cache size includes: The file compression ratio is obtained by calculating the ratio of the target cache size to the initial file size.

4. The method according to claim 1, characterized in that, The step of determining the target file from the file to be compressed whose file compression ratio is greater than the reference compression ratio of the specified file system includes: Determine the reference compression ratio for the specified file system; The compression ratio of each file to be compressed is compared with the reference compression ratio. The file to be compressed with a compression ratio greater than the reference compression ratio is determined, and the target file is obtained.

5. The method according to claim 4, characterized in that, Determining the reference compression ratio of the specified file system includes: Get the actual file compression ratio of each file in the target file partition of the specified file system; The actual file compression ratios of each file are sorted according to preset rules to obtain the sorted actual file compression ratios; Obtain the first read / write performance of each file in the target partition on the specified file system, and the second read / write performance on other file systems; The first read / write performance is compared with the second read / write performance to obtain the comparison result; The reference compression ratio is determined from the actual file compression ratios after sorting based on the comparison results.

6. The method according to claim 1, characterized in that, The method further includes: The compression algorithm is used to compress files other than the target file in the at least one file to be compressed.

7. A document management device, characterized in that, The device includes: The acquisition unit is used to acquire at least one file to be compressed. The prediction unit is used to predict the target cache size occupied by each file to be compressed after being compressed by the compression algorithm of the specified file system, based on the file index information of each file to be compressed. The first determining unit is used to determine the file compression ratio of each file to be compressed based on the initial file size of each file to be compressed and the target cache size; The second determining unit is used to determine, from the at least one file to be compressed, a target file whose file compression ratio is greater than the reference compression ratio of the specified file system; The rejection unit is used to reject the target file from the at least one file to be compressed, so that the target file is not compressed by the compression algorithm; The prediction unit includes: The first determining subunit is used to determine the file data of the file to be compressed based on the file index information; The first acquisition subunit is used to determine the buffer in the specified file system that is used for file compression processing; A processing subunit is used to pass the file data into the buffer for compression processing and determine the number of buffer blocks occupied by the compression processing in the buffer. The second determining subunit is used to determine the target cache size based on the cache size of the buffer block and the number of blocks.

8. A storage medium, characterized in that, The storage medium stores multiple instructions, which are adapted for loading by a processor to execute the file management method according to any one of claims 1 to 6.

9. A terminal, characterized in that, The system includes a processor and a memory, the memory storing multiple instructions, and the processor loading the instructions to execute the file management method according to any one of claims 1 to 6.

Images