A disk file compression method and terminal
By dividing the disk into sector groups and reading small files sequentially, the problem of low efficiency in compressing a large number of small files in existing technologies is solved, achieving a fast compression effect.
Patent Information
- Application Number
- CN202210832654.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-14
- Publication Date
- 2025-12-05
- Estimated Expiration
- 2042-07-14
AI Technical Summary
When compressing a large number of small files in Windows systems, existing technologies require repeatedly scanning the entire disk, resulting in low efficiency and slow compression speed.
By dividing the disk into a first sector group and a second sector group, all data in the first sector group is read sequentially, and when the storage limit is reached, files smaller than the threshold are read from the second sector group and compressed, thus avoiding reading each file one by one.
This reduces disk read time and compression time for each file, thereby improving the compression speed of a large number of small files.
Smart Images

Figure CN115407934B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of disk storage technology, in particular to a disk file compression method and terminal. BACKGROUND
[0002] At present, when compressing a large number of small files on Windows, each small file needs to be read, and since a file is stored in scattered sectors, multiple repeated scans of the entire disk are required for compression of each file, which is inefficient.
[0003] Therefore, the waiting time for compressing a large number of small files is long, and the compression speed is slow. SUMMARY
[0004] The present application solves the technical problem of providing a disk file compression method and terminal, which can reduce the compression time of each file and improve the compression speed of a large number of small files.
[0005] In order to solve the above technical problems, the technical scheme adopted by the present application is:
[0006] A disk file compression method, comprising the steps of:
[0007] obtaining a file compression request, reading all data in a first sector group of a disk in order, and compressing the read data, the file size of the files in the first sector group being less than a threshold value;
[0008] determining whether the data amount stored in the first sector group of the disk reaches the storage upper limit of the first sector group, if so, reading files with a file size less than the threshold value from a second sector group, and compressing the files;
[0009] storing all compressed files uniformly.
[0010] In order to solve the above technical problems, another technical scheme adopted by the present application is:
[0011] A disk file compression terminal, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
[0012] obtaining a file compression request, reading all data in a first sector group of a disk in order, and compressing the read data, the file size of the files in the first sector group being less than a threshold value;
[0013] determining whether the data amount stored in the first sector group of the disk reaches the storage upper limit of the first sector group, if so, reading files with a file size less than the threshold value from a second sector group, and compressing the files;
[0014] all the compressed files are stored uniformly.
[0015] The present application has the beneficial effect that after obtaining the file compression request, the file sizes of the files in the first sector group are all less than the threshold value, so all the data in the first sector group of the disk is read in sequence; and it is determined whether the amount of data stored in the first sector group reaches the upper limit, if yes, the files with file sizes less than the threshold value are read from the second sector group, so that the small files in the second sector group can also be read out and compressed. In this way, when a large number of small files need to be compressed, it is not necessary to read each file one by one, but the first sector group specially storing small files is divided, the first sector group is read in sequence, so that the small files are quickly read; therefore, repeated reading of the disk is not necessary, the time for reading data from the disk is greatly reduced, and the compression time of each file is reduced, so that the compression speed of a large number of small files is improved. BRIEF DESCRIPTION OF DRAWINGS
[0016] Figure 1 a flow chart of a disk file compression method according to an embodiment of the present application;
[0017] Figure 2 a schematic diagram of a disk file compression terminal according to an embodiment of the present application;
[0018] Figure 3 a schematic diagram of file storage of a disk in the prior art;
[0019] REFERENCE NUMERALS
[0020] 1. A disk file compression terminal; 2. a memory; 3. a processor. DETAILED DESCRIPTION
[0021] To make the technical content, the achieved purposes and effects of the present application clear, the following will be described in detail in combination with the embodiments and the accompanying drawings.
[0022] Please refer to Figure 1 The embodiment of the present application provides a disk file compression method, which comprises the following steps:
[0023] obtaining a file compression request, reading all the data in the first sector group of the disk in sequence, and compressing the read data, the file sizes of the files in the first sector group being all less than a threshold value;
[0024] determining whether the amount of data stored in the first sector group of the disk reaches the storage upper limit of the first sector group, if yes, reading the files with file sizes less than the threshold value from the second sector group, and compressing the files;
[0025] all the compressed files are stored uniformly.
[0026] From the above description, the beneficial effects of the present application are that after obtaining a file compression request, the file sizes of the files in the first sector group are all less than the threshold value, so all the data in the first sector group of the disk is read in sequence; and it is determined whether the amount of data stored in the first sector group reaches the upper limit, if so, files with a file size less than the threshold value are read from the second sector group, so that small files in the second sector group can also be read out and compressed. In this way, when a large number of small files need to be compressed, it is not necessary to read each file one by one, but by dividing the first sector group specially storing small files, the first sector group is read in sequence, so that the small files are quickly read; therefore, repeated reading of the disk is not needed, the time for reading data from the disk is greatly reduced, and the compression time of each file is reduced, thereby improving the compression speed of a large number of small files.
[0027] Further, the reading of the files with a file size less than the threshold value from the second sector group and the compression of the files include:
[0028] determining whether the files with a file size less than the threshold value in the second sector group reach half of the disk capacity, if so, after removing the files with a file size greater than or equal to the threshold value in the second sector group, the second sector group is read in sequence, and the read data is compressed;
[0029] otherwise, the files with a file size less than the threshold value in the second sector group are read one by one, and file compression is performed.
[0030] From the above description, if the small files stored in the second sector group reach half of the disk capacity, the large files in the second sector group are removed and then read in sequence, otherwise, the small files in the second sector group are read one by one. In this way, the reading method of the small files in the second sector group can be selected according to the file storage situation in the second sector group, and the flexibility and efficiency of data reading are improved.
[0031] Further, the obtaining of the file compression request comprises:
[0032] obtaining a file storage request, determining whether the first sector group reaches the storage upper limit, if so, storing the files in the file storage request to the second sector group;
[0033] otherwise, determining whether the file size in the file storage request is less than the threshold value, if so, storing the files in the file storage request to the first sector group, otherwise, storing the files in the file storage request to the second sector group.
[0034] From the above description, if the first sector group is full, the small files are stored in the second sector group, and flexible storage of files can be performed according to the actual storage situation.
[0035] Further, the obtaining of the file compression request comprises:
[0036] reading a master file table of the disk, loading path information and size information of all files in the first sector group.
[0037] As can be seen from the above description, reading the master file table of the disk can facilitate the statistics of the occupied size of the files according to the file path information and the size information in the master file table.
[0038] Further, the unified storage of the compressed files comprises:
[0039] unified compression of the compressed files to obtain a compressed file;
[0040] attaching the master file table of the disk to the compressed file.
[0041] As can be seen from the above description, attaching the master file table of the disk to the compressed file facilitates the decompression of the compressed file in combination with the master file table of the disk.
[0042] Please refer to Figure 2 Another embodiment of the present application provides a disk file compression terminal, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the computer program:
[0043] obtaining a file compression request, reading all data in a first sector group of a disk in sequence, and compressing the read data, wherein the file size of the files in the first sector group is less than a threshold value;
[0044] judging whether the data amount stored in the first sector group of the disk reaches the storage upper limit of the first sector group, if yes, reading files with a file size less than the threshold value from a second sector group, and compressing the files;
[0045] unified storage of the compressed files.
[0046] From the above description, the beneficial effects of the present application are that after obtaining the file compression request, the file sizes of the files in the first sector group are all less than the threshold value, so all the data in the first sector group of the disk is sequentially read; and it is judged whether the amount of data stored in the first sector group reaches the upper limit, if yes, the files with file sizes less than the threshold value are read from the second sector group, so that the small files in the second sector group can also be read out for compression. In this way, when a large number of small files need to be compressed, it is not necessary to read each file one by one, but by dividing the first sector group specially storing small files, the first sector group is sequentially read, so that the small files are quickly read; therefore, repeated reading of the disk is not necessary, the time for reading data from the disk is greatly reduced, and the compression time of each file is reduced, so that the compression speed of a large number of small files is improved.
[0047] Further, the reading of the files with file sizes less than the threshold value from the second sector group and the compression of the files include:
[0048] judging whether the files with file sizes less than the threshold value in the second sector group reach half of the disk capacity, if yes, after the files with file sizes greater than or equal to the threshold value in the second sector group are removed, the second sector group is sequentially read, and the read data is compressed;
[0049] otherwise, the files with file sizes less than the threshold value in the second sector group are read one by one, and file compression is performed.
[0050] From the above description, if the small files stored in the second sector group reach half of the disk capacity, the large files in the second sector group are removed for sequential reading, otherwise, the small files in the second sector group are read one by one, in this way, the small file reading method of the second sector group can be selected according to the file storage condition in the second sector group, and the flexibility and efficiency of data reading are improved.
[0051] Further, the obtaining of the file compression request before includes:
[0052] obtaining a file storage request, judging whether the first sector group reaches the storage upper limit, if yes, storing the files in the file storage request to the second sector group;
[0053] otherwise, judging whether the file sizes in the file storage request are less than the threshold value, if yes, storing the files in the file storage request to the first sector group, otherwise, storing the files in the file storage request to the second sector group.
[0054] From the above description, if the first sector group is full, the small files are stored in the second sector group, so that the flexible storage of files can be performed according to the actual storage condition.
[0055] Further, the obtaining of the file compression request comprises:
[0056] reading a master file table of the disk, loading path information and size information of all files in the first sector group.
[0057] As can be seen from the above description, reading the master file table of the disk can facilitate the statistics of the occupied size of the files according to the file path information and the size information in the master file table.
[0058] Further, the unified storage of the compressed files comprises:
[0059] unified compression of the compressed files to obtain a compressed file;
[0060] attaching the master file table of the disk to the compressed file.
[0061] As can be seen from the above description, attaching the master file table of the disk to the compressed file can facilitate the decompression of the compressed file in combination with the master file table of the disk.
[0062] The above-mentioned disk file compression method and terminal are suitable for the compression of a large number of small files on a disk, and improve the efficiency of the compression of a large number of small files. The following will be described in detail through specific embodiments.
[0063] Embodiment One
[0064] Please refer to Figure 1 A disk file compression method comprises the following steps:
[0065] S1, obtaining a file compression request, reading all data in a first sector group of a disk in sequence, and compressing the read data, wherein the file size of the files in the first sector group is less than a threshold value.
[0066] In this embodiment, the file compression request is a compression request of all small files in the disk.
[0067] S11, obtaining a file storage request, determining whether the first sector group reaches a storage upper limit, if yes, storing the files in the file storage request to a second sector group.
[0068] Specifically, obtaining a file storage request, determining whether the first sector group reaches a storage upper limit, if yes, storing the files to be stored in the file storage request to a second sector group.
[0069] S12, otherwise, determining whether the file size in the file storage request is less than a threshold value, if yes, storing the files in the file storage request to the first sector group, otherwise, storing the files in the file storage request to the second sector group.
[0070] In this embodiment, the threshold value is 100M. Specifically, if the size of the file to be stored in the file storage request is less than 100M, the file is stored in the first sector group; if the size of the file to be stored in the file storage request is greater than or equal to 100M, the file is stored in the second sector group.
[0071] Therefore, the first sector group is used to store small files, and when the first sector group is full, the file is stored in the second sector group.
[0072] S13, read the master file table of the disk, load the path information and size information of all files in the first sector group.
[0073] Specifically, by reading the disk MFT (Master File Table), all file paths, sizes and other attribute information in the small file grouping sector of the disk are quickly loaded; all data in the first sector group of the disk is read in sequence in combination with the MFT.
[0074] S2, judge whether the data stored in the first sector group of the disk reaches the storage upper limit of the first sector group, if yes, read the file whose size is less than the threshold value from the second sector group, and compress the file.
[0075] S21, judge whether the file whose size is less than the threshold value in the second sector group reaches half of the capacity of the disk, if yes, after removing the file whose size is greater than or equal to the threshold value in the second sector group, the second sector group is read in sequence, and the read data is compressed.
[0076] Specifically, the disk sectors are directly read, and the last sector of the first sector group is sequentially read from the 0th sector; in the second sector group, the files greater than 100M and their sector positions are quickly filtered out in the memory, the sectors covered by the files greater than 100M are skipped, and the sectors without files are skipped, and then the files in the remaining sectors are sequentially read after quick removal.
[0077] S22, otherwise, read the file whose size is less than the threshold value in the second sector group one by one, and compress the file.
[0078] Specifically, the small files in the second sector group are read one by one, and the read file content is compressed.
[0079] S3, uniformly store all compressed files.
[0080] Specifically, step S3 is:
[0081] All compressed files are uniformly compressed to obtain a compressed file.
[0082] The master file table of the disk is attached to the compressed file.
[0083] Specifically, all the read sector files are compressed and stored, and the MFT file list is also appended to the file for use during decompression.
[0084] In the prior art, please refer to Figure 3 Since a file is stored in sectors in a very scattered manner, file 1 can be stored in sectors 1 and 2, file 2 can be stored in sectors 3 and 14, file 3 can be stored in sectors 4 and 9, and file 4 can be stored in sector 5. Therefore, compressing the file one by one requires repeatedly scanning the entire disk multiple times.
[0085] In this embodiment, Figure 3 Sectors 1 to 10 are divided into the first sector group, and sectors 11 to 20 are divided into the second sector group. The file compression method in this embodiment does not require reading and compressing each file individually, which greatly improves the file compression efficiency.
[0086] Example 2
[0087] Please refer to Figure 2 A disk file compression terminal 1 includes a memory 2, a processor 3, and a computer program stored on the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, it implements the various steps of a disk file compression method according to Embodiment 1.
[0088] In summary, the disk file compression method and terminal provided by this invention, upon receiving a file compression request, ensures that the file sizes of all files in the first sector group are less than a threshold. Therefore, all data in the first sector group is read sequentially. Furthermore, it determines whether the amount of data stored in the first sector group has reached its upper limit. If so, files smaller than the threshold are read from the second sector group, thus enabling the compression of smaller files in the second sector group. In this way, when compressing a large number of small files, it is not necessary to read each file individually. Instead, by dividing the disk into a dedicated first sector group for storing small files, the first sector group is read sequentially, allowing for rapid access to small files. Therefore, repeated disk reads are unnecessary, significantly reducing disk data reading time and the compression time for each file, thereby improving the compression speed of a large number of small files.
[0089] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.
Claims
1. A method of disk file compression, characterized by, The method comprises the steps of: obtaining a file compression request, sequentially reading all data in a first sector group of a disk, and compressing the read data, wherein file sizes of files in the first sector group are all less than a threshold value; determining whether a data amount stored in the first sector group of the disk reaches a storage upper limit of the first sector group, if yes, reading files with file sizes less than the threshold value from a second sector group, and compressing the files; unified storing of all compressed files; the reading of the files with file sizes less than the threshold value from the second sector group and the compression of the files comprise: determining whether the files with file sizes less than the threshold value in the second sector group reach half of a disk capacity, if yes, sequentially reading the second sector group after removing files with file sizes greater than or equal to the threshold value in the second sector group, and compressing the read data; otherwise, reading the files with file sizes less than the threshold value in the second sector group one by one, and compressing the files; the obtaining of the file compression request comprises: obtaining a file storage request, determining whether the first sector group reaches the storage upper limit, if yes, storing files in the file storage request to the second sector group; otherwise, determining whether file sizes in the file storage request are less than the threshold value, if yes, storing the files in the file storage request to the first sector group, otherwise, storing the files in the file storage request to the second sector group.
2. The method of claim 1, wherein, the obtaining of the file compression request comprises: reading a master file table of the disk, and loading path information and size information of all files in the first sector group.
3. The method of claim 2, wherein, the unified storing of all compressed files comprises: unified compressing of all compressed files to obtain a compressed file; attaching the master file table of the disk to the compressed file.
4. A disk file compression terminal comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the following steps: obtaining a file compression request, sequentially reading all data in a first sector group of a disk, and compressing the read data, wherein file sizes of files in the first sector group are all less than a threshold value; determining whether a data amount stored in the first sector group of the disk reaches a storage upper limit of the first sector group, if yes, reading files with file sizes less than the threshold value from a second sector group, and compressing the files; unified storing of all compressed files; the reading of the files with file sizes less than the threshold value from the second sector group and the compression of the files comprise: determining whether the files with file sizes less than the threshold value in the second sector group reach half of a disk capacity, if yes, sequentially reading the second sector group after removing files with file sizes greater than or equal to the threshold value in the second sector group, and compressing the read data; otherwise, reading the files with file sizes less than the threshold value in the second sector group one by one, and compressing the files; the obtaining of the file compression request comprises: obtaining a file storage request, determining whether the first sector group reaches the storage upper limit, if yes, storing files in the file storage request to the second sector group; Otherwise, it is judged whether the file size in the file storage request is less than a threshold value, if yes, the file in the file storage request is stored to the first sector group, otherwise, the file in the file storage request is stored to the second sector group.
5. A disk file compression terminal according to claim 4, wherein The obtaining of the file compression request comprises: Reading a main file table of the disk, and loading path information and size information of all files in the first sector group.
6. A disk file compression terminal according to claim 5, wherein The unified storage of the compressed files comprises: Compressing all the compressed files to obtain a compressed file; The main file table of the disk is attached to the compressed file.
Citation Information
Patent Citations
Small file optimization storage method based on HDFS
CN103577123A
Data global merging method under mass data scene
CN113177024A