Compression Method, Device, Equipment, Storage Medium, and Program Product of File System

Abstract

A compression method, device, equipment, storage medium, and program product for a file system provided by this application relate to computer data compression technology, including: obtaining data to be written and reading compression information corresponding to the data to be written; dividing the data to be written into multiple data blocks according to the compression information, and compressing each data block to obtain compressed blocks; filling preset data in each compressed block, where the compressed block and the corresponding data block occupy the same logical space of the file system; compressing each compressed block and writing the compressed data into a storage device. In this solution, preset data is supplemented in the compressed block after the data block to be written is compressed, so that the compressed block filled with the preset data occupies the same logical space of the file system as the corresponding data block to be written, and the compression ability of the storage device itself is combined. The file system does not need to add a functional module for managing storage space, and achieves the purpose of transparent compression on the premise of ensuring performance.

Description

Technical Field

[0001] The present disclosure relates to computer data compression technology, and in particular, to a compression method, device, equipment, storage medium, and program product for a file system. Background Art

[0002] With the popularization of the Internet and the advent of the Internet of Things era, the speed and volume of data production have witnessed an explosive growth, correspondingly bringing huge pressure to data storage. Compressing data before storage can effectively remove redundancy in the data, thereby reducing the volume of data and the consumption of storage space.

[0003] In the prior art, before data storage, the file system can call a compression module to compress the data and write the compressed data into the storage device. Among them, the compression module can be a software compression module based on a Central Processing Unit (CPU) or a hardware compression module based on a compression acceleration card. This method makes the compression transparent to users and can be called transparent compression, such as ZFS and BtrFS.

[0004] However, this file system that realizes transparent compression often has lower performance than file systems without transparent compression (such as the Fourth Extended File System (Ext4) and XFS). The file system often needs to sacrifice performance to achieve the function of transparent compression. Therefore, how to achieve the purpose of transparent compression on the premise of ensuring the performance of the file system is an urgent problem to be solved. Summary of the Invention

[0005] The present application provides a compression method, device, equipment, storage medium, and program product for a file system to solve the problem in the prior art that transparent compression cannot be achieved on the premise of ensuring the performance of the file system.

[0006] According to a first aspect of the present application, there is provided a compression method for a file system, including:

[0007] Obtain data to be written and read preset compression information corresponding to the data to be written;

[0008] Divide the data to be written into multiple data blocks according to the compression information, and compress each data block to obtain a compressed block corresponding to each data block;

[0009] Fill preset data in each compressed block, and the compressed block filled with the preset data occupies the same logical space of the file system as the corresponding data block;

[0010] Compress each compressed block filled with preset data, and write the compressed data to a storage device.

[0011] According to a second aspect of the present application, there is provided a compression method for a file system, including:

[0012] Obtain the compression information of the data to be read, divide the data to be read according to the compression information to obtain a plurality of data segments, and obtain the corresponding compressed data for each data segment according to the compression information;

[0013] Decompress each of the compressed data to obtain each compressed block filled with preset data;

[0014] Delete the preset data in each compressed block filled with preset data, decompress each compressed block after deleting the preset data to obtain each decompressed data, and determine each read data in each decompressed data.

[0015] According to a third aspect of the present application, there is provided a compression device for a file system, including:

[0016] An acquisition unit, configured to acquire data to be written and read preset compression information corresponding to the data to be written;

[0017] A compression unit, configured to divide the data to be written into a plurality of data blocks according to the compression information, and compress each data block to obtain a corresponding compressed block for each data block;

[0018] A filling unit, configured to fill each compressed block with preset data, and the compressed block filled with preset data occupies the same logical space of the file system as the corresponding data block;

[0019] A writing unit, configured to compress each compressed block filled with preset data, and write the compressed data to a storage device.

[0020] According to a fourth aspect of the present application, there is provided a compression device for a file system, including:

[0021] An acquisition unit, configured to acquire the compression information of the data to be read, divide the data to be read according to the compression information to obtain a plurality of data segments, and obtain the corresponding compressed data for each data segment according to the compression information;

[0022] A decompression unit, configured to decompress each of the compressed data to obtain each compressed block filled with preset data;

[0023] A determination unit is configured to delete the preset data in each compressed block filled with the preset data, perform decompression processing on each compressed block after deleting the preset data to obtain each decompressed data, and determine each read data from each decompressed data.

[0024] According to a fifth aspect of the present application, an electronic device is provided, including a memory and a processor; wherein,

[0025] The memory is configured to store a computer program;

[0026] The processor is configured to read the computer program stored in the memory and execute a compression method of a file system as described in the first aspect or the second aspect according to the computer program in the memory.

[0027] According to a sixth aspect of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, a compression method of a file system as described in the first aspect or the second aspect is implemented.

[0028] A compression method, device, device, storage medium, and program product of a file system provided by the present application include: obtaining data to be written, and reading preset compression information corresponding to the data to be written; dividing the data to be written into multiple data blocks according to the compression information, and compressing each data block to obtain a compressed block corresponding to each data block; filling preset data in each compressed block, and the compressed block filled with the preset data occupies the same logical space of the file system as the corresponding data block; compressing each compressed block filled with the preset data, and writing the compressed data into a storage device. In the compression method, device, device, storage medium, and program product of a file system provided by this solution, preset data is supplemented in the compressed block after the data block to be written is compressed, so that the compressed block filled with the preset data occupies the same logical space of the file system as the corresponding data block to be written, and the compression ability of the storage device itself with a compression function is combined. The data before and after compression occupies the same logical space in the file system, so that the file system does not need to add functional modules for managing storage space, ensuring the performance of the file system. And the compression of the cooperation between the file system and the storage device realizes the transparent compression function of the file system. Therefore, the solution provided by the present application achieves the purpose of transparent compression while ensuring the performance of the file system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the writing process of the compression method of the file system shown in an exemplary embodiment of the present application;

[0030] Figure 2Schematic diagram of the writing process of the compression method of the file system shown in an exemplary embodiment of the present application;

[0031] Figure 3 Schematic diagram of the writing process of the compression method of the file system shown in another exemplary embodiment of the present application;

[0032] Figure 4 Schematic diagram of the writing process of the compression method of the file system shown in another exemplary embodiment of the present application;

[0033] Figure 5 Schematic diagram of the writing process of the compression method of the file system shown in yet another exemplary embodiment of the present application;

[0034] Figure 6 Schematic diagram of the reading process of the compression method of the file system shown in an exemplary embodiment of the present application;

[0035] Figure 7 Schematic diagram of the reading process of the compression method of the file system shown in another exemplary embodiment of the present application;

[0036] Figure 8 Schematic diagram of the reading process of the compression method of the file system shown in an exemplary embodiment of the present application;

[0037] Figure 9 Schematic diagram of the reading process of the compression method of the file system shown in another exemplary embodiment of the present application;

[0038] Figure 10 Structure diagram of the compression device of the file system shown in an exemplary embodiment of the present application;

[0039] Figure 11 Structure diagram of the compression device of the file system shown in another exemplary embodiment of the present application;

[0040] Figure 12 Structure diagram of the compression device of the file system shown in yet another exemplary embodiment of the present application;

[0041] Figure 13 Structure diagram of the compression device of the file system shown in still another exemplary embodiment of the present application;

[0042] Figure 14 Structure diagram of the electronic device shown in an exemplary embodiment of the present application. Detailed implementation manners

[0043] With the popularization of the Internet and the advent of the Internet of Things era, the speed and volume of data production have increased explosively, correspondingly bringing huge pressure to data storage. Compressing data before storage can effectively remove redundancy in the data, thereby reducing the volume of data and the consumption of storage space. In the prior art, the file system can call a compression module to compress the data before storing it and write the compressed data to the storage device. Among them, the compression module can be a software compression module based on the CPU or a hardware compression module based on a compression acceleration card. This method makes the compression transparent to the user and can be called transparent compression. For example, the common ZFS and BTRF implement the above-mentioned transparent compression.

[0044] Specifically, in this compression method, the space occupied by the compressed block after compressing the data block to be written is smaller than the space occupied by the data block to be written. Therefore, the file system needs to manage the space before and after compression. This will inevitably lead to the addition of a functional module for managing storage space in the file system. Because there is a functional module for managing storage space in the file system, it will affect the latency of the file system and the throughput of files within a certain period of time, that is, it will affect the performance of the file system. Therefore, this file system that implements transparent compression often has lower performance than a file system without transparent compression (such as Ext4 and XFS). The file system often needs to sacrifice performance to achieve the function of transparent compression. Therefore, how to achieve the purpose of transparent compression on the premise of ensuring the performance of the file system is an urgent problem to be solved.

[0045] To solve the above technical problems, in the solution provided by this application, it includes a compression method for a file system. Preset data is supplemented in the compressed block after compressing the data block to be written, so that the compressed block filled with the preset data occupies the same logical space in the file system as the corresponding data block to be written, and combines the compression ability of the storage device itself with compression function. The data before and after compression occupies the same logical space in the file system, so that the file system does not need to add a functional module for managing storage space, ensuring the performance of the file system. And the compression of the cooperation between the file system and the storage device realizes the transparent compression function of the file system. Therefore, the solution provided by this application also achieves the purpose of transparent compression on the premise of ensuring the performance of the file system.

[0046] Figure 1 It is a schematic diagram of the writing process of the compression method for a file system shown in an exemplary embodiment of this application. As Figure 1 shown, the compression method for the file system provided in this embodiment includes:

[0047] Step 101, obtain the data to be written and read the preset compression information corresponding to the data to be written.

[0048] Among them, the method provided by this application can be executed by a file system with computing capabilities. The file system can obtain the data to be written and read the compression information corresponding to the data to be written set in advance.

[0049] Specifically, the software institution responsible for managing and storing file information in the operating system is called the file management system, abbreviated as the file system. The file system consists of three parts: the interface of the file system, the software set for manipulating and managing objects, and the objects and their attributes. From a system perspective, the file system organizes and allocates the space of the file storage device, is responsible for file storage, and protects and retrieves the stored files. Specifically, it is responsible for creating files for users, storing, reading, modifying, and dumping files, controlling file access, and deleting files when users no longer use them, etc.

[0050] Among them, the data to be written refers to the data to be stored in the file system, which can include: newly created files in the file system, files modified in the file system, and the data contained in the files dumped from other file systems into this file system.

[0051] Specifically, the compression information can be set in advance. Different compression methods or different compression block granularities can be set in advance according to needs, or no compression can be set according to needs.

[0052] Among them, the compression information can include the compression method and the compression block granularity.

[0053] Among them, the compression method can be called the compression algorithm. The compression algorithm refers to the algorithm for data compression, including two steps: compression and decompression.

[0054] Among them, when the file system compresses the data to be written, it compresses according to a fixed data block size. This fixed data block size can be called the compression block granularity. For example, the compression block granularity can be 16 kilobytes (KB).

[0055] Step 102: Divide the data to be written into multiple data blocks according to the compression information, and compress each data block to obtain a compression block corresponding to each data block.

[0056] Among them, the compression information can include the compression method and the compression block granularity.

[0057] Optionally, the file system can divide the data to be written into multiple data blocks according to the compression block granularity, and compress each data block according to the compression method to obtain a compression block corresponding to each data block.

[0058] Step 103: Fill the preset data in each compression block, and the logical space of the file system occupied by the compression block filled with the preset data is the same as that of the corresponding data block.

[0059] Among them, the preset data can be highly compressible invalid data.

[0060] Specifically, the space occupied by each compressed block obtained after compressing each data block will become smaller, resulting in the logical space occupied by each compressed block being different from the logical space occupied by the original data blocks. In the solution provided by this application, filling highly compressible invalid data in each compressed block can make the logical space occupied by the filled compressed block in the file system the same as that of the corresponding data block.

[0061] Specifically, if the compressed block granularity is 16KB, the logical space sizes occupied by the filled compressed block and the corresponding data block are both 16KB.

[0062] Step 104: Compress each compressed block filled with the preset data, and write the compressed data into the storage device.

[0063] Among them, the storage device can be a storage device with a compression function. For example, the storage device can be a hard disk or a disk with a compression function.

[0064] Specifically, storage device compression information will be preset in the storage device. The storage device compression information can include the storage device compression method and the storage device compressed block granularity.

[0065] Specifically, the storage device can divide each compressed block into multiple data blocks according to the storage device compressed block granularity. Then, the storage device can compress each data block according to the storage device compression method and write the compressed data into the storage device. Specifically, the storage device writes the compressed data into the storage medium in the storage device.

[0066] Among them, the storage device compressed block granularity can be set to 4KB.

[0067] Specifically, the filled preset data can be highly compressible invalid data. The storage device can highly compress the filled highly compressible invalid data.

[0068] A compression method for a file system provided by this application includes: obtaining data to be written, and reading preset compression information corresponding to the data to be written; dividing the data to be written into multiple data blocks according to the compression information, and compressing each data block to obtain a compressed block corresponding to each data block; filling preset data in each compressed block, and the logical space of the file system occupied by the compressed block filled with the preset data is the same as that of the corresponding data block; compressing each compressed block filled with the preset data, and writing the compressed data into a storage device. A compression method for a file system provided by this application supplements preset data in the compressed block after the data block to be written is compressed, so that the logical space of the file system occupied by the compressed block filled with the preset data is the same as that of the corresponding data block to be written, and combines the compression ability of the storage device itself with compression function. The data before and after compression occupies the same logical space in the file system, so that the file system does not need to add a functional module for managing storage space, ensuring the performance of the file system. And the compression of the cooperation between the file system and the storage device realizes the transparent compression function of the file system. Therefore, the solution provided by this application achieves the purpose of transparent compression on the premise of ensuring the performance of the file system.

[0069] Figure 2 It is a schematic diagram of the writing process of the compression method of the file system shown in an exemplary embodiment of this application.

[0070] As Figure 2 shown, the file system divides the data to be written into multiple data blocks 21 according to the compression block granularity in the compression information. Then, the file system compresses each data block 21 according to the compression method in the compression information to obtain each compressed block 22. After the data is compressed, the occupied space becomes smaller. Therefore, preset data is filled in each compressed block 22 so that the logical space of the file system occupied by the compressed block 22 filled with the preset data is the same as that of the corresponding data block 21. Among them, the preset data can be highly compressible invalid data. The storage device can perform secondary compression on each compressed block 22 filled with the preset data, and store the data 23 after the secondary compression in the storage medium in the storage device. Specifically, after the data is subjected to secondary compression, the preset data in each compressed block 22 will be highly compressed.

[0071] Figure 3 It is a schematic diagram of the writing process of the compression method of the file system shown in another exemplary embodiment of this application.

[0072] As Figure 3 shown, a compression method for a file system provided in this embodiment includes:

[0073] Step 301, obtain data to be written, and read preset compression information corresponding to the data to be written.

[0074] The implementation method and principle of step 301 are similar to those of step 101, and will not be elaborated here.

[0075] Step 302: Divide the data to be written into multiple data blocks according to the compression block granularity, where the length of each data block is less than or equal to the value of the compression block granularity; among them, the compression information includes the compression block granularity.

[0076] Specifically, the size of the storage address allocated for each file in the file system is 4KB and an integer multiple of 4KB.

[0077] Specifically, assume that the starting address of the logical address in the file system is 1KB. When the file system allocates storage addresses for each file, it allocates them in sequence starting from the starting address. For example, the first storage address ranges from 1KB to 4KB; the second storage address ranges from 5KB to 8KB, and so on.

[0078] Specifically, assume that the compression block granularity is 16KB. The file system will divide the data to be written into multiple data blocks according to the compression block granularity. Specifically, when the file system divides the data to be written, it divides them in sequence starting from the starting address. For example, from 1KB to 16KB is one data block, from 17KB to 32KB is one data block, and so on.

[0079] Specifically, if the logical address of the data to be written ranges from 5KB to 36KB. Then the file system can divide the file to be written into 3 data blocks according to the compression block granularity of 16KB. The logical addresses of the three data blocks are respectively from 5KB to 16KB; from 17KB to 32KB; from 33KB to 36KB. The lengths of these three data blocks are 12KB, 16KB, and 4KB respectively, all of which are less than or equal to the value of the 16KB compression block granularity.

[0080] Step 303: Compress the written data based on the compression method to obtain a compression block corresponding to the data block; among them, the compression information also includes the compression method.

[0081] Specifically, the file system can obtain the written data in different ways according to the length of each data block.

[0082] Furthermore, the file system can compress the written data based on the compression method in the compression information to obtain a compression block corresponding to the data block.

[0083] Optionally, if the length of the data block is equal to the value of the compression block granularity, then update the written data in the data block in response to the write operation, and compress the updated written data to obtain a compression block corresponding to the data block.

[0084] Optionally, if the length of the data block is equal to the value of the compression block granularity, it indicates that when storing in the storage device, the data block can be compressed into a single compression block. Therefore, the data block can be directly compressed.

[0085] Optionally, if the length of the data block is less than the value of the compression block granularity, read the stored data corresponding to the data block from the storage device; where the length of the stored data is the value of the compression block granularity, and the stored data includes the written data corresponding to the logical address of the data block and other data.

[0086] If the length of the data block is less than the value of the compression block granularity, it indicates that when storing in the storage device, the data block needs to be compressed together with other data to obtain a compression block with a size equal to the compression block granularity. Therefore, the data in this compression block includes not only the data to be written but also other data.

[0087] Specifically, if the length of a certain data block is less than the value of the compression block granularity, for example, the logical address of the data block ranges from 5KB to 16KB and the value of the compression block granularity is 16KB, then the logical address of the stored data corresponding to this data block in the storage device should range from 1KB to 16KB. The file system reads the stored data with a logical address ranging from 1KB to 16KB corresponding to this data block from the storage device.

[0088] Specifically, the stored data includes the written data with a logical space corresponding to the data block ranging from 5KB to 16KB, and also includes other data with a logical space ranging from 1KB to 4KB.

[0089] The file system compresses the other data with a logical space ranging from 1KB to 4KB and the modified written data with a logical space ranging from 5KB to 16KB together according to the compression method to obtain a compression block. This compression block is the compression block corresponding to the data block with a logical space ranging from 5KB to 16KB.

[0090] Step 304, fill the preset data in each compression block. The compression block filled with the preset data occupies the same logical space of the file system as the corresponding data block.

[0091] The implementation method and principle of Step 304 are similar to those of Step 103 and will not be elaborated here.

[0092] Step 305, perform secondary compression on the compression block according to the preset storage device compression method and storage device compression block granularity, and write the compressed data into the storage medium; where the storage device compression block granularity is less than the compression block granularity.

[0093] Among them, the storage device can be a storage device with a compression function. For example, the storage device can be a hard disk or a disk with a compression function.

[0094] Specifically, storage device compression information is preset in the storage device. The storage device compression information may include a storage device compression method and a storage device compression block granularity.

[0095] Among them, the storage device compression block granularity is smaller than the file system compression block granularity. For example, the file system compression block granularity can be set to 16 KB, and the storage device compression block granularity can be set to 4 KB.

[0096] Among them, the storage device compression method and the compression method in the file system may be different.

[0097] Specifically, the storage device can divide each compression block into multiple data blocks according to the storage device compression block granularity. Then, the storage device can perform secondary compression on each data block according to the storage device compression method and write the compressed data into the storage medium.

[0098] Specifically, the preset data filled can be highly compressible invalid data. The storage device can highly compress the filled highly compressible invalid data.

[0099] Specifically, the compression function of the file system, in cooperation with the storage device with a compression function, selects different compression block granularities to achieve the purpose of transparent compression while ensuring the performance of the file system.

[0100] Figure 4 It is a schematic diagram of the writing process of the compression method of the file system shown in another exemplary embodiment of the present application.

[0101] As Figure 4 shown, the file system can divide the data to be written into multiple data blocks 41 according to the compression block granularity.

[0102] If the length of the data block 41 is equal to the value of the compression block granularity, the file system can directly compress the data block to obtain a compression block 45 corresponding to the data block 41. Preset data is filled in the obtained compression block 45 to obtain a compression block 45 filled with preset data. Among them, the compression block 45 filled with preset data and the corresponding data block 41 occupy the same logical space of the file system.

[0103] If the length of the data block 41 is less than the value of the compression block granularity, the file system reads the stored data 42 corresponding to the data block 41 from the storage device. Among them, the length of the stored data 42 is the value of the compression block granularity, and the stored data 42 includes the written data 43 corresponding to the data block 41 and other data. The stored data 42 is decompressed according to the compression method, and the written data 43 corresponding to the data block is updated. Then, the data 44 composed of the updated written data 43 and other data is compressed according to the compression method to obtain a compression block 45 corresponding to the data block 41.

[0104] Among them, the compression method used to decompress the stored data 42 is the same as the compression method used during its compression. That is, after the data in the file system is compressed using a certain compression method, if the compressed data is decompressed, the compression method used during decompression is the same as the compression method used during compression.

[0105] Fill the obtained compression block 45 with preset data to obtain a compression block 45 filled with preset data. The compression block 45 filled with preset data can be used to write into a storage device with compression function.

[0106] Figure 5 It is a schematic diagram of the writing process of the compression method of the file system shown in another exemplary embodiment of this application.

[0107] Such as Figure 5 shown, after the operating system loads the file system, the file system can respond to a write request. The file system can read the compression information used by the file system from an external configuration, where the compression information can include the compression method and the compression block granularity.

[0108] The file system can divide the data to be written into multiple data blocks according to the compression block granularity.

[0109] If the length of a certain data block is less than the value of the compression block granularity, the file system can read the stored data corresponding to the data block from the storage device. Among them, the length of the stored data is the value of the compression block granularity, and the stored data includes the write data corresponding to the logical address of the data block and other data. The file system can perform decompression processing on the stored data according to the compression method to obtain the decompressed other data. The file system can update the write data corresponding to the data block, and perform compression processing on the data composed of the updated write data and the decompressed other data according to the compression method to obtain a compression block corresponding to the data block.

[0110] Among them, the compression method used to decompress the stored data is the same as the compression method used during its compression. That is, after the data in the file system is compressed using a certain compression method, if the compressed data is decompressed, the compression method used during decompression is the same as the compression method used during compression.

[0111] If the length of a certain data block is equal to the value of the compression block granularity, the file system can directly perform compression processing on the data block to obtain a compression block corresponding to the data block.

[0112] After that, the file system fills the compressed blocks with preset data so that the compressed blocks filled with the preset data occupy the same logical space in the file system as the corresponding data blocks. Finally, each compressed block filled with the preset data is secondarily compressed according to the compression method of the storage device, and the compressed data is written into the storage medium.

[0113] Figure 6 It is a schematic diagram of the reading process of the compression method of the file system shown in an exemplary embodiment of the present application.

[0114] As Figure 6 shown, the compression method of the file system provided in this embodiment includes:

[0115] Step 601, obtain the compression information of the data to be read.

[0116] Among them, the compression information refers to the compression information used for the data to be read in the file system. Among them, the compression information may include the compression method and the compression block granularity. Optionally, there may be a flag bit in the data to be read to flag the compression information used for the data to be read. The file system can obtain the corresponding compression information according to the compression information flagged in the data to be read.

[0117] Specifically, the compression information of the data to be read refers to the compression information used when the data to be read is compressed. That is, after the data in the file system is compressed with a certain compression information, if the compressed data is decompressed, the compression information used during decompression is the same as the compression information used during compression.

[0118] Step 602, perform a partitioning process on the data to be read according to the compression information to obtain multiple data segments, and obtain the corresponding compressed data for each data segment according to the compression information.

[0119] Specifically, the file system can perform a partitioning process on the data to be read stored in the storage device according to the value of the compression block granularity in the compression information to obtain multiple data segments.

[0120] Specifically, when the file system partitions the data to be read, it partitions in sequence starting from the starting address of the logical address. Assume that the starting address of the logical address in the file system is 1KB and the compression block granularity is 16KB. Then, from 1KB to 16KB is a data segment, from 17KB to 32KB is a data segment, and so on.

[0121] Furthermore, if the logical address of the data to be read is from 5KB to 36KB, then the logical address from 5KB to 16KB is a data segment, the logical address from 17KB to 32KB is a data segment, and the logical address from 33KB to 36KB is a data segment.

[0122] Further, the file system may obtain the respective compressed data corresponding to the respective data segments according to the value of the compressed block granularity in the compression information. For example, if the logical address of the data to be read ranges from 5 KB to 36 KB, the logical address of the compressed data corresponding to the data segment with a logical address ranging from 5 KB to 16 KB ranges from 1 KB to 16 KB; the logical address of the compressed data corresponding to the data segment with a logical address ranging from 17 KB to 32 KB ranges from 17 KB to 32 KB; the logical address of the compressed data corresponding to the data segment with a logical address ranging from 33 KB to 36 KB ranges from 33 KB to 48 KB.

[0123] Step 603: Decompress each piece of compressed data to obtain each compressed block filled with preset data.

[0124] Specifically, the storage device decompresses the compressed data according to the storage device compression method and the storage device compressed block granularity pre-stored in the storage device.

[0125] Among them, the storage device compressed block granularity may be 4 KB.

[0126] Among them, the compression method pre-stored in the storage device may be different from the compression method in the compression information in the file system.

[0127] Step 604: Delete the preset data in each compressed block filled with the preset data, and decompress each compressed block after deleting the preset data to obtain the decompressed respective data, and determine the respective read data from the decompressed respective data.

[0128] Specifically, the file system may delete the preset data filled in each compressed block.

[0129] Specifically, the file system may decompress each compressed block after deleting the preset data according to the compression method in the compression information.

[0130] Specifically, for example, the logical address of the file data ranges from 5 KB to 36 KB. The logical address of each compressed block corresponding to the file data ranges from 1 KB to 48 KB. After the file system decompresses each compressed block, the logical address of the decompressed respective data ranges from 1 KB to 48 KB. Then the file system may determine the respective read data from the decompressed data.

[0131] Figure 7 It is a schematic diagram of the reading process of the compression method of the file system shown in another exemplary embodiment of the present application.

[0132] As Figure 7 shown, the compression method of the file system provided in this embodiment includes:

[0133] Step 701: Obtain the compression information of the data to be read.

[0134] The implementation method and principle of step 701 are similar to those of step 601, and will not be elaborated here.

[0135] Step 702: Divide the data to be read according to the compression block granularity to obtain multiple data segments, and obtain the corresponding compressed data for each data segment according to the compression block granularity.

[0136] Specifically, the file system can divide the data to be read stored in the storage device according to the value of the compression block granularity in the compression information (for example, 16KB) to obtain multiple data segments.

[0137] Specifically, when the file system divides the data to be read, it starts from the starting address of the logical address and divides it in sequence. Assume that the starting address of the logical address in the file system is 1KB. Then, from 1KB to 16KB is a data segment, from 17KB to 32KB is a data segment, and so on.

[0138] Furthermore, if the logical address of the data to be read is from 5KB to 36KB, then the logical address from 5KB to 16KB is a data segment, the logical address from 17KB to 32KB is a data segment, and the logical address from 33KB to 36KB is a data segment.

[0139] Furthermore, the file system can obtain the corresponding compressed data for each data segment according to the value of the compression block granularity in the compression information. For example, if the logical address of the data to be read is from 5KB to 36KB. Then, the logical address of the compressed data corresponding to the data segment with the logical address from 5KB to 16KB is from 1KB to 16KB; the logical address of the compressed data corresponding to the data segment with the logical address from 17KB to 32KB is from 17KB to 32KB; the logical address of the compressed data corresponding to the data segment with the logical address from 33KB to 36KB is from 33KB to 48KB.

[0140] Step 703: The storage device decompresses each compressed data according to the preset storage device compression method to obtain each compressed block filled with preset data.

[0141] Specifically, the storage device can be a storage device with compression function, such as a hard disk or a disk with compression function, etc. Among them, the storage device compression information can be preset in the storage device. The storage device compression information can include the storage device compression method and the storage device compression block granularity.

[0142] Among them, the storage device compression method can be different from the compression method in the compression information in the file system.

[0143] Among them, the compression block granularity of the storage device can be different from the compression block granularity in the compression information in the file system. For example, the compression block granularity of the storage device can be set to 4KB; the compression block granularity in the file system can be set to 16KB.

[0144] Step 704, the file system deletes the preset data in each compression block filled with the preset data, and decompresses the compression block after deleting the preset data according to the compression method to obtain the decompressed data.

[0145] Specifically, the file system can delete the preset data filled in each compression block.

[0146] Among them, the compression method is the compression method included in the compression information in the file system.

[0147] Specifically, the file system can decompress the compression block after deleting the preset data according to the compression method in the compression information to obtain the decompressed data.

[0148] Step 705A, if the length of the data segment is equal to the value of the compression block granularity, the decompressed data corresponding to the data segment is determined as the read data.

[0149] Exemplarily, after step 704, if the length of the data segment of a certain decompressed data is equal to the value of the compression block granularity, the decompressed data corresponding to the data segment is determined as the read data.

[0150] Among them, the compression block granularity is the compression block granularity included in the compression information in the file system. For example, the compression granularity is 16KB.

[0151] Specifically, if the logical address of the data to be read is from 5KB to 36KB. Among them, the logical address of a certain data segment is from 17KB to 32KB, and the length of this data segment is 16KB, which is equal to the value of the compression block granularity, then the decompressed data corresponding to this data segment is determined as the data to be read.

[0152] Step 705B, if the length of the data segment is less than the value of the compression block granularity, the read data is determined from the decompressed data corresponding to the data segment according to the logical address corresponding to the data segment.

[0153] Exemplarily, after step 704, if the length of the data segment of a certain decompressed data is less than the value of the compression block granularity, the read data is determined from the decompressed data corresponding to the data segment.

[0154] Specifically, if the logical address for reading data ranges from 5KB to 36KB. Among them, the logical address of a certain data segment ranges from 5KB to 16KB, and the length of this data segment is 12KB, which is less than the value of the compression block granularity. The logical address of the compression block corresponding to this data segment ranges from 1KB to 16KB, and the logical address of the decompressed data corresponding to the compression block is from 1KB to 16KB. Then, the decompressed data corresponding to this data segment is intercepted and determined as the read data, that is, the decompressed data with the logical address ranging from 5KB to 16KB is intercepted as the read data.

[0155] Specifically, after step 705A or 705B, the file system feeds back the data to be read according to the read data obtained from each data segment.

[0156] Figure 8 It is a schematic diagram of the reading process of the compression method of the file system shown in an exemplary embodiment of the present application.

[0157] As Figure 8 shown, the file system can obtain the compression information of the data to be read, and divide the data to be read into multiple data segments 81 according to the value of the compression block granularity included in the compression information. The logical addresses corresponding to the respective compressed data 82 corresponding to the respective data segments are obtained according to the compression block granularity.

[0158] The storage device performs decompression processing on the respective compressed data segments corresponding to the respective compressed data 82 according to the preset storage device compression method to obtain the respective compression blocks 82 filled with preset data.

[0159] The file system deletes the preset data filled in each compression block 82, and performs decompression processing on the compression block 82 after deleting the preset data according to the compression method in the compression information to obtain the decompressed respective data. If the length of a certain data segment 81 is equal to the value of the compression block granularity, the decompressed data corresponding to this data segment 81 is determined as the read data 83. If the length of a certain data segment 81 is less than the value of the compression block granularity, the read data 83 is determined from the decompressed data corresponding to the data segment 81 according to the logical address corresponding to the data segment 81.

[0160] Figure 9 It is a schematic diagram of the reading process of the compression method of the file system shown in another exemplary embodiment of the present application.

[0161] As Figure 9 shown, after the operating system loads the file system, the file system can respond to the read request. The file system can obtain the compression information of the data to be read, where the compression information may include the compression method and the compression block granularity.

[0162] The file system can divide the data to be read into multiple data segments according to the compression block granularity, and obtain the corresponding compressed data for each data segment according to the compression block granularity.

[0163] The storage device decompresses each compressed data according to a preset storage device compression method to obtain each compressed block filled with preset data.

[0164] After the file system deletes the preset data filled in each compressed block, it decompresses each compressed block according to the compression method in the compression information to obtain the decompressed data.

[0165] If the length of a certain data segment is equal to the value of the compression block granularity, the decompressed data corresponding to the data segment is determined as the read data. If the length of a certain data segment is less than the value of the compression block granularity, the read data is determined from the decompressed data corresponding to the data segment according to the logical address corresponding to the data segment.

[0166] Figure 10 It is a structural diagram of a compression device of a file system shown in an exemplary embodiment of the present application.

[0167] As Figure 10 shown, the compression device 1000 of the file system provided by the present application includes:

[0168] An acquisition unit 1010, configured to acquire data to be written and read preset compression information corresponding to the data to be written;

[0169] A compression unit 1020, configured to divide the data to be written into multiple data blocks according to the compression information, and compress each data block to obtain a compressed block corresponding to each data block;

[0170] A filling unit 1030, configured to fill preset data in each compressed block, and the compressed block filled with the preset data occupies the same logical space of the file system as the corresponding data block;

[0171] A writing unit 1040, configured to compress each compressed block filled with preset data and write the compressed data into a storage device.

[0172] Figure 11 It is a structural diagram of a compression device of a file system shown in another exemplary embodiment of the present application.

[0173] As Figure 11 shown, on the basis of the above embodiment, in a compression device 1100 of a file system provided by the present application, the compression unit 1020 is specifically configured to divide the data to be written into multiple data blocks according to the compression block granularity, and the length of each data block is less than or equal to the value of the compression block granularity; wherein, the compression information includes the compression block granularity.

[0174] The compression unit 1020 is further configured to perform compression processing on the written data based on the compression method to obtain a compression block corresponding to the data block; wherein, the compression information further includes the compression method.

[0175] Specifically, the compression unit 1020 is configured to, if the length of the data block is equal to the value of the compression block granularity, update the written data in the data block in response to the write operation, and perform compression processing on the updated written data to obtain a compression block corresponding to the data block.

[0176] Specifically, the compression unit 1020 is configured to, if the length of the data block is less than the value of the compression block granularity, read the stored data corresponding to the data block from the storage device; wherein, the length of the stored data is the value of the compression block granularity, and the stored data includes the written data corresponding to the data block and further includes other data;

[0177] Update the written data in response to the write operation, and perform compression processing on the updated written data and the other data to obtain a compression block corresponding to the data block.

[0178] The writing unit 1040 includes:

[0179] The compression module 1041 is configured to perform secondary compression on the compression block according to the preset storage device compression method and the storage device compression block granularity; wherein, the storage device compression block granularity is less than the compression block granularity.

[0180] Figure 12 This is the structural diagram of the compression device of the file system shown in another exemplary embodiment of the present application.

[0181] As Figure 12 shown, the compression device 1200 of the file system provided by the present application includes:

[0182] The acquisition unit 1210 is configured to acquire the compression information of the data to be read, perform division processing on the data to be read according to the compression information to obtain a plurality of data segments, and acquire the respective compressed data corresponding to the respective data segments according to the compression information;

[0183] The decompression unit 1220 is configured to perform decompression processing on the respective compressed data to obtain respective compression blocks filled with preset data;

[0184] The determination unit 1230 is configured to delete the preset data in the respective compression blocks filled with the preset data, perform decompression processing on the respective compression blocks after deleting the preset data to obtain the respective decompressed data, and determine the respective read data in the respective decompressed data.

[0185] Figure 13 This is the structural diagram of the compression device of the file system shown in another exemplary embodiment of the present application.

[0186] AsFigure 13 As shown, based on the above embodiments, in a compression device 1300 of a file system provided by the present application, an obtaining unit 1210 is specifically configured to divide data to be read according to a compression block granularity to obtain a plurality of data segments, and obtain respective compressed data corresponding to the data segments according to the compression block granularity.

[0187] A decompression unit 1220 is specifically configured to perform decompression processing on each compressed data according to a preset storage device compression method.

[0188] A determination unit 1230 includes:

[0189] A decompression module 1231, configured to perform decompression processing on a compressed block after deleting preset data according to a compression method.

[0190] The determination unit 1230 is specifically configured to, if the length of a data segment is equal to the value of the compression block granularity, determine the decompressed data corresponding to the data segment as the read data.

[0191] The determination unit 1230 is further configured to, if the length of a data segment is less than the value of the compression block granularity, determine the read data from the decompressed data corresponding to the data segment according to the logical address corresponding to the data segment.

[0192] Figure 14 It is a structural diagram of an electronic device shown in an exemplary embodiment of the present application.

[0193] As Figure 14 shown, the electronic device provided in this embodiment includes:

[0194] A memory 1401;

[0195] A processor 1402; and

[0196] A computer program;

[0197] Wherein, the computer program is stored in the memory 1401 and is configured to be executed by the processor 1402 to implement any one of the above file system compression methods.

[0198] This embodiment further provides a computer-readable storage medium, on which a computer program is stored,

[0199] The computer program is executed by a processor to implement any one of the above file system compression methods.

[0200] This embodiment further provides a computer program product, including a computer program, which when executed by a processor, implements any one of the above file system compression methods.

[0201] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including those of the above method embodiments; and the aforementioned storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0202] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A compression method for a file system, characterized in that, it includes: Obtain data to be written, and read preset compression information corresponding to the data to be written; The compression information includes a compression block granularity and a compression method; Divide the data to be written into multiple data blocks according to the compression information, and compress each data block to obtain a compression block corresponding to each data block; Fill preset data in each compression block, and the logical space of the file system occupied by the compression block filled with the preset data is the same as that of the corresponding data block; Compress each compression block filled with the preset data, and write the compressed data into a storage device; The compressing each compression block filled with the preset data includes: Perform secondary compression on the compression block according to a preset storage device compression method and a storage device compression block granularity; wherein, the storage device compression block granularity is smaller than the compression block granularity; The filled preset data is invalid data with high compressibility; the storage device can highly compress the filled invalid data with high compressibility; The compressing each data block to obtain a compression block corresponding to each data block includes: Perform compression processing on the written data based on the compression method to obtain a compression block corresponding to the data block; The performing compression processing on the written data based on the compression method to obtain a compression block corresponding to the data block includes: If the length of the data block is less than the value of the compression block granularity, read the stored data corresponding to the data block from the storage device; wherein, the length of the stored data is the value of the compression block granularity, and the stored data includes the written data corresponding to the data block and also includes other data; Respond to the write operation to update the written data, and perform compression processing on the updated written data and the other data to obtain a compression block corresponding to the data block.

2. The method according to claim 1, characterized in that, dividing the data to be written into multiple data blocks according to the compression information includes: Dividing the data to be written into multiple data blocks according to the compression block granularity, and the length of each data block is less than or equal to the value of the compression block granularity.

3. The method according to claim 1, characterized in that, the performing compression processing on the written data based on the compression method to obtain a compression block corresponding to the data block includes: If the length of the data block is equal to the value of the compression block granularity, respond to the write operation to update the written data in the data block, and perform compression processing on the updated written data to obtain a compression block corresponding to the data block.

4. A compression method for a file system, characterized in that, it includes: Obtain the compression information of the data to be read, perform division processing on the data to be read according to the compression information to obtain multiple data segments, and obtain the respective compressed data corresponding to the respective data segments according to the compression information; the compression information includes a compression block granularity and a compression method; Decompress each of the compressed data to obtain each compressed block filled with preset data; the preset data is highly compressible invalid data; the highly compressible invalid data can be highly compressed by the storage device; the compressed data is obtained by secondarily compressing the compressed block according to a preset storage device compression method and a storage device compression block granularity; wherein, the storage device compression block granularity is smaller than the compressed block granularity; Delete the preset data in each compressed block filled with preset data, and decompress each compressed block after deleting the preset data to obtain each decompressed data, and determine each read data in each decompressed data; The decompressing the compressed block after deleting the preset data includes: Decompressing the compressed block after deleting the preset data according to the compression method; The determining each read data in the decompressed data includes: If the length of the data segment is less than the value of the compressed block granularity, determine the read data in the decompressed data corresponding to the data segment according to the logical address corresponding to the data segment.

5. The method according to claim 4, wherein, the compression information includes a compressed block granularity; The dividing the data to be read into multiple data segments according to the compression information, and obtaining each compressed data corresponding to each data segment according to the compression information includes: Dividing the data to be read into multiple data segments according to the compressed block granularity, and obtaining each compressed data corresponding to each data segment according to the compressed block granularity.

6. The method according to claim 4, wherein, the decompressing each of the compressed data includes: Decompressing each of the compressed data according to a preset storage device compression method.

7. The method according to claim 4, wherein, the determining each read data in the decompressed data includes: If the length of the data segment is equal to the value of the compressed block granularity, determine the decompressed data corresponding to the data segment as the read data.

8. A compression device for a file system, wherein, comprising: An acquisition unit, configured to acquire data to be written and read preset compression information corresponding to the data to be written; the compression information includes a compressed block granularity and a compression method; A compression unit, configured to divide the data to be written into multiple data blocks according to the compression information, and compress each data block to obtain a compressed block corresponding to each data block; A filling unit, configured to fill each compressed block with preset data, and the compressed block filled with preset data occupies the same logical space of the file system as the corresponding data block; A writing unit, configured to compress each compressed block filled with preset data and write the compressed data into a storage device; A writing unit, further configured to perform secondary compression on the compression block according to a preset storage device compression method and a storage device compression block granularity; wherein, the storage device compression block granularity is smaller than the compression block granularity; the preset data filled is highly compressible invalid data; the storage device can highly compress the filled highly compressible invalid data; A compression unit, specifically configured to perform compression processing on the written data based on the compression method to obtain a compression block corresponding to the data block; The compression unit is specifically configured to, if the length of the data block is smaller than the value of the compression block granularity, read the stored data corresponding to the data block from the storage device; wherein, the length of the stored data is the value of the compression block granularity, the stored data includes the written data corresponding to the data block, and also includes other data; update the written data in response to a write operation, and perform compression processing on the updated written data and the other data to obtain a compression block corresponding to the data block.

9. A compression device for a file system, characterized in that, it includes: An acquisition unit, configured to acquire compression information of data to be read, perform division processing on the data to be read according to the compression information to obtain a plurality of data segments, and acquire respective compressed data corresponding to the data segments according to the compression information; the compression information includes a compression block granularity and a compression method; A decompression unit, configured to perform decompression processing on each of the compressed data to obtain each compression block filled with preset data; the preset data is highly compressible invalid data; the highly compressible invalid data can be highly compressed by a storage device; the compressed data is obtained by performing secondary compression on the compression block according to a preset storage device compression method and a storage device compression block granularity; wherein, the storage device compression block granularity is smaller than the compression block granularity; A determination unit, configured to delete the preset data in each of the compression blocks filled with the preset data, perform decompression processing on the compression blocks after deleting the preset data to obtain each decompressed data, and determine each read data in each decompressed data; The determination unit is specifically configured to perform decompression processing on the compression block after deleting the preset data according to the compression method; The determination unit is specifically configured to, if the length of the data segment is smaller than the value of the compression block granularity, determine the read data in the decompressed data corresponding to the data segment according to the logical address corresponding to the data segment.

10. An electronic device, characterized in that, it includes a memory and a processor; wherein, the memory is used to store a computer program; the processor is configured to read the computer program stored in the memory and execute the method according to any one of claims 1-3 or 4-7 above according to the computer program in the memory.

11. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1-3 or 4-7 above is implemented.

12. A computer program product, including a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-3 or 4-7 above.

Images