Data compression method and apparatus

Abstract

This application discloses a data compression method and apparatus. The method includes: obtaining m data blocks in a data area of ​​a read-write file system; compressing the m data blocks using a preset compression algorithm to sequentially obtain n compressed data blocks, where the first capacity of the compressed data blocks is the same, and the first capacity represents the number of bytes of compressed data that may be included in the compressed data block; and establishing a first index for each of j data blocks corresponding to the i-th compressed data block of the n compressed data, and recording a mapping relationship between the first index and the j data blocks. The first index is used to identify the storage location in the storage medium of each data block included in the j data blocks, and the attribute information included in each of the j data blocks. When the data blocks are read, the read efficiency can be effectively improved, and the data can be guaranteed to be read in a random read scenario with a small read amplification factor.

Description

[Technical Field]

[0001] This application claims priority to Chinese Patent Application No. 202110667882.7, entitled "DATA COMPRESSION METHOD AND APPARATUS," filed with the State Intellectual Property Office of China on June 16, 2021, which is incorporated herein by reference in its entirety.

[0002] This application relates to the field of data compression technology, and in particular to data compression methods and apparatus. [Background technology]

[0003] To improve the overall input / output (IO) read / write performance of a storage system, it is necessary to compress files in memory. Currently, Linux read / write file systems such as F2FS, journaling Flash file system version 2 (JFFS2), and B-tree file system (BTRFS) are supported, as are Windows read / write file systems such as NTFS. Because the metadata area occupies a small proportion of the overall file system, the data area typically occupies a large portion of the device's storage capacity. Therefore, compressing data in the data area reduces the size of IOs and improves IO read / write performance.

[0004] In existing data compression methods, the original file data (also called source data) that requires compression is generally compressed based on a fixed-size minimum compression unit. The compressed file data (also called compressed data) may include header data and compressed data. The header data is used to represent the attribute information of the file data, and the compressed data is used to represent the content of the file data. The compressed file data is then saved to a storage medium. However, existing read / write file system compression solutions suffer from random read amplification issues and low read efficiency. Summary of the Invention

[0005] The embodiments of this application provide a data compression method and apparatus that solves the problem of random read amplification in a read / write file system and improves read efficiency.

[0006] According to a first aspect, an embodiment of this application provides a data compression method. The method may be executed by an electronic device or by a component (e.g., a chip, a chip system, or a processor) located within the electronic device. Hereinafter, an explanation will be provided using an example in which the method is executed by an electronic device. The method includes: the electronic device obtaining m data blocks in a data area of ​​a read / write file system, where m is a positive integer greater than or equal to 1; the electronic device compresses the m data blocks using a preset compression algorithm to sequentially obtain n compressed data blocks, where the compressed data blocks all have the same first capacity, and the first capacity represents the number of bytes of compressed data that may be included in the compressed data block, where n is a positive integer greater than or equal to 1; the electronic device: n compressed data blocksA first index is established for each of j data blocks corresponding to the i-th compressed data block, and a mapping relationship between the first index and the j data blocks is recorded, where i is a positive integer between 1 and n, and j is a positive integer between 1 and m. The first index is used to identify a storage location in a storage medium of each data block included in the j data blocks, and attribute information included in each of the j data blocks.

[0007] The data compression method provided in the embodiment of this application can effectively improve read efficiency when a data block is read, and can ensure that data can be read in a random read scenario with a small read amplification factor. Additionally, attributes included in the index of the data block may be modified, and thus the compressed file on the storage device may be modified. It can be seen that the embodiment of this application solves the random read amplification problem of compression solutions in existing read / write file systems and solves the problem that existing file systems with fixed output compression methods cannot support data and metadata updates.

[0008] In a specific possible implementation, compressing m data blocks using a preset compression algorithm to sequentially obtain n compressed data blocks specifically involves sequentially allocating all data blocks in the m data blocks to a first set in a preset order. When the data capacity of j data blocks in the first set is equal to the rated capacity of the first set, a compression operation is performed on the j data blocks based on a specified compression threshold to obtain the ith compressed data block.

[0009] n compressed data blocksSpecifically, establishing the first index for each of the j data blocks corresponding to the i-th compressed data block is establishing the first index for each of the j data blocks when the sum of the total data length of the header data and compressed data of the i-th compressed data block and a specified compression threshold is less than or equal to the total data length of the j data blocks.

[0010] In a specific possible implementation, the attribute information includes a first attribute representing whether a storage location of the compressed data block where the data block is compressed is pre-allocated; a second attribute representing whether a data page of the data block is valid; a third attribute representing whether the data page of the data block is the first compressed page of the compressed data block of the data block; a fourth attribute representing whether the data page of the data block is included in the compressed data page of the two compressed blocks; and a fourth attribute representing whether the data page of the data block is the compressed page of the compressed data block obtained by compressing the data block. a sixth attribute used to represent an index address of a compressed data block in which a data page of the data block is located; and a seventh attribute used to represent, when the data page of the data block belongs to the first compressed page of the compressed data block of the data block, an attribute value of the seventh attribute being an offset of the data block in the set corresponding to the compressed data block, and, when the data page of the data block does not belong to the first compressed page of the compressed data block of the data block, an attribute value of the seventh attribute being a distance between the data page of the data block and the first compressed page of the compressed data block.

[0011] In a specific possible implementation, the attribute information includes a third attribute: n compressed data blocksEstablishing the first index of each of the j data blocks corresponding to the ith compressed data block specifically involves assigning 1 to the attribute value of the third attribute when the data page of each of the j data blocks is the first compressed page of the ith compressed data block, or assigning 0 to the attribute value of the third attribute when the data page of each of the j data blocks is not the first compressed page of the ith compressed data block.

[0012] In a specific possible implementation, the attribute information includes a seventh attribute, and the method further includes updating the attribute value of the seventh attribute to the offset of the data block in the set corresponding to the compressed data block when the attribute value of the third attribute is 1, or updating the attribute value of the seventh attribute to the distance between the data page of the data block and the first compressed page of the compressed data block when the attribute value of the third attribute is 0.

[0013] In a specific possible implementation, the attribute information includes a fourth attribute: n compressed data blocks Establishing the first index of each of the j data blocks corresponding to the ith compressed data block specifically involves assigning 1 to the attribute value of the fourth attribute when the data page of each of the j data blocks is included in the compressed data pages of the two compressed blocks, or assigning 0 to the attribute value of the fourth attribute when the data page of each of the j data blocks is not included in the compressed data pages of the two compressed blocks.

[0014] In a specific possible implementation, the attribute information includes a second attribute, n compressed data blocks Establishing the first index of each of the j data blocks corresponding to the ith compressed data block specifically involves assigning 1 to the attribute value of the second attribute when the data page of each of the j data blocks is valid, or assigning 0 to the attribute value of the second attribute when the data page of each of the j data blocks is invalid.

[0015] In some possible implementations, the method includes compressing the m data blocks using a preset compression algorithm to obtain a second set of data to be overwritten before sequentially obtaining n compressed data blocks, the second set including p compressed data blocks, where p is a positive integer greater than or equal to 1; First target compressed data block and q data blocks corresponding to the compressed page of the first target compressed data block, where q is a positive integer greater than or equal to 1; determining, in the q data blocks, a location offset of the first target data block within the q data blocks; and determining that the data page of the first target data block is the data page of the data to be overwritten.

[0016] In a specific possible implementation, the first index is used to identify the storage location of the ith compressed data block on the storage medium, and the attribute information contained in each of the j data blocks.

[0017] In some possible implementations, the method further includes: reading a first index of the first data block to obtain an index address of a first compressed data block corresponding to the first data block, where the first index includes attribute information of the first data block; reading an index of the first compressed data block corresponding to the first data block; decompressing the first compressed data block based on the index of the first compressed data block to obtain a plurality of data blocks corresponding to the first compressed data block, where the plurality of data blocks includes the first data block; determining an offset of the first data block in the plurality of decompressed data blocks; and obtaining data of the first data block based on the offset of the first data block in the plurality of decompressed data blocks.

[0018] According to a second aspect, an embodiment of the present application provides a data compression device, comprising: a first acquisition unit configured to acquire m data blocks in a data area of ​​a read-write file system, where m is a positive integer greater than or equal to 1; and a compression unit configured to compress the m data blocks using a preset compression algorithm to sequentially acquire n compressed data blocks, where all of the compressed data blocks have the same first capacity, the first capacity representing the number of bytes of compressed data that may be included in the compressed data block, where n is a positive integer greater than or equal to 1; n compressed data blocks and an updating unit configured to establish a first index for each of j data blocks corresponding to the i-th compressed data block and record a mapping relationship between the first index and the j data blocks. i is a positive integer between 1 and n, and j is a positive integer between 1 and m. The first index is used to identify the storage location in the storage medium of each data block included in the j data blocks, and the attribute information included in each of the j data blocks.

[0019] The data compression method provided in the embodiment of this application can effectively improve read efficiency when a data block is read, and can ensure that data can be read in a random read scenario with a small read amplification factor. Additionally, attributes included in the index of the data block may be modified, and thus the compressed file on the storage device may be modified. It can be seen that the embodiment of this application solves the random read amplification problem of compression solutions in existing read / write file systems and solves the problem that existing file systems with fixed output compression methods cannot support data and metadata updates.

[0020] In a specific possible implementation, the compression unit is configured to sequentially allocate all data blocks of the m data blocks to the first set in a predetermined order, and when the data capacity of the j data blocks in the first set is equal to the rated capacity of the first set, a compression operation is performed on the j data blocks based on a specified compression threshold to obtain the i-th compressed data block.

[0021] In a specific possible implementation, the update unit is configured to establish a first index for each of the j data blocks when the sum of the total data length of the header data and compressed data of the i-th compressed data block and a specified compression threshold is less than or equal to the total data length of the j data blocks.

[0022] In a specific possible implementation, the attribute information includes at least one of: a first attribute representing whether a storage location of the compressed data block where the data block is compressed is pre-allocated; a second attribute used to represent whether a data page of the data block is valid; a third attribute used to represent whether the data page of the data block is the first compressed page of the compressed data block of the data block; a fourth attribute used to represent whether the data page of the data block is included in the compressed data page of two compressed blocks; a fifth attribute indicating whether the data page of the data block is the compressed page of a compressed data block obtained by compressing the data block; a sixth attribute used to represent an index address of the compressed data block where the data page of the data block is located; and a seventh attribute used to represent that when the data page of the data block belongs to the first compressed page of the compressed data block of the data block, the attribute value of the seventh attribute is the offset of the data block in the set corresponding to the compressed data block. When the data page of the data block does not belong to the first compressed page of the compressed data block of the data block, the attribute value of the seventh attribute is the distance between the data page of the data block and the first compressed page of the compressed data block.

[0023] In a specific possible implementation, the attribute information includes a third attribute, and the update unit is further configured to assign 1 to an attribute value of the third attribute when a data page of each of the j data blocks is a first compressed page of the i-th compressed data block, or to assign 0 to an attribute value of the third attribute when a data page of each of the j data blocks is not a first compressed page of the i-th compressed data block.

[0024] In a specific possible implementation, the attribute information includes a seventh attribute, and the updating unit is further configured to: update an attribute value of the seventh attribute to an offset of a data block in a set corresponding to the compressed data block when the attribute value of the third attribute is 1; or update an attribute value of the seventh attribute to a distance between a data page of the data block and a first compressed page of the compressed data block when the attribute value of the third attribute is 0.

[0025] In a specific possible implementation, the attribute information includes a fourth attribute, and the update unit is further configured to assign 1 to an attribute value of the fourth attribute when a data page of each of the j data blocks is included in the compressed data pages of the two compressed blocks, or to assign 0 to an attribute value of the fourth attribute when a data page of each of the j data blocks is not included in the compressed data pages of the two compressed blocks.

[0026] In a specific possible implementation, the attribute information includes a second attribute, and the update unit is further configured to assign 1 to an attribute value of the second attribute when a data page of each of the j data blocks is valid, or to assign 0 to an attribute value of the second attribute when a data page of each of the j data blocks is invalid.

[0027] In some possible implementations, the apparatus includes a second acquisition unit configured to acquire a second set of data to be overwritten, the second set including p compressed data blocks, where p is a positive integer greater than or equal to 1; and First target compressed data blockand q data blocks corresponding to the compressed page of the first target compressed data block, where q is a positive integer greater than or equal to 1; a first determining unit configured to determine, in the q data blocks, a location offset of the first target data block within the q data blocks; and a second determining unit configured to determine that the data page of the first target data block is a data page of the data to be overwritten.

[0028] In a specific possible implementation, the first index is used to identify the storage location of the ith compressed data block on the storage medium, and the attribute information contained in each of the j data blocks.

[0029] In some possible implementations, the apparatus further includes: a first reading unit configured to read a first index of the first data block to obtain an index address of a first compressed data block corresponding to the first data block, where the first index includes attribute information of the first data block; a second reading unit configured to read an index of the first compressed data block corresponding to the first data block; a decompression unit configured to decompress the first compressed data block based on the index of the first compressed data block to obtain a plurality of data blocks corresponding to the first compressed data block, where the plurality of data blocks includes the first data block; a third determining unit configured to determine an offset of the first data block in the plurality of decompressed data blocks; and a third obtaining unit configured to obtain data of the first data block based on the offset of the first data block in the plurality of decompressed data blocks.

[0030] According to a third aspect, an embodiment of the present application provides a device configured to perform the method of the first aspect.

[0031] According to a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer instructions that, when executed on an electronic device, enable the electronic device to perform the data compression method of the first aspect.

[0032] According to a fifth aspect, an embodiment of the present application provides a computer program product, wherein when the program is invoked by a processor, the data compression method of the first aspect is executed.

[0033] According to a sixth aspect, an embodiment of the present application provides a chip system including one or more processors, wherein when the one or more processors execute instructions, the one or more processors perform the data compression method of the first aspect.

[0034] It should be understood that the description of technical features, technical solutions, beneficial effects, or similar terms in this application does not imply that all features and advantages can be implemented in any individual embodiment. Conversely, the description of a feature or beneficial effect should be understood to mean that at least one embodiment includes the particular technical feature, technical solution, or beneficial effect. Thus, the description of a technical feature, technical solution, or beneficial effect in this specification does not necessarily pertain to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in embodiments may be combined in any suitable manner. Those skilled in the art may understand that an embodiment may be implemented in a particular embodiment without one or more particular technical features, technical solutions, or beneficial effects. In other embodiments, additional technical features and beneficial effects may be identified in a particular embodiment, which may not reflect all embodiments. [Brief explanation of the drawings]

[0035] [Figure 1a] FIG. 1 is a block diagram of an operating system according to an embodiment of the present application.

[0036] [Figure 1b] 1 is a schematic diagram of the structure of a storage system according to one embodiment of the present application.

[0037] [Figure 2] Figure 1b is a schematic diagram of the structure of a solid-state disk in the storage system.

[0038] [Figure 3] Figure 2 is a schematic diagram of the structure of a flash chip in a solid-state disk.

[0039] [Figure 4] Schematic diagram of the flash translation layer corresponding to the flash chip in Figure 3.

[0040] [Figure 5] FIG. 1 is a schematic diagram of a fixed input compression mode.

[0041] [Figure 6] FIG. 1 is a schematic diagram of a fixed output compression mode according to an embodiment of the present application.

[0042] [Figure 7] 1 is a schematic diagram of a data block index according to an embodiment of the present application.

[0043] [Figure 8] 1 is a schematic diagram of a data block index in an existing extensible read-only file system.

[0044] [Figure 9A] 1 is a schematic flowchart of a data compression method according to an embodiment of the present application; [Figure 9B]1 is a schematic flowchart of a data compression method according to an embodiment of the present application;

[0045] [Figure 10A] 1 is a schematic flowchart of updating a data block index according to an embodiment of the present application; [Figure 10B] 1 is a schematic flowchart of updating a data block index according to an embodiment of the present application;

[0046] [Figure 11] FIG. 2 is a schematic diagram of data block index relationships during data compression according to an embodiment of the present application.

[0047] [Figure 12] 4 is a schematic flowchart of another data compression method according to an embodiment of the present application;

[0048] [Figure 13] 1 is a schematic diagram of data block index relationships in an overwrite or read procedure according to an embodiment of the present application;

[0049] [Figure 14] 1 is a schematic flowchart of a data reading procedure according to an embodiment of the present application;

[0050] [Figure 15] 1 is a schematic diagram of the structure of a data compression device according to an embodiment of this application; DETAILED DESCRIPTION OF THE INVENTION

[0051] The terms "comprises," "having," and any other variations thereof referred to in the description of this application are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes other steps or units that are not listed, or that optionally further include other steps or units that are inherent to the process, method, product, or device.

[0052] It should be noted that in the embodiments of this application, the words "example" or "for example" are used to denote providing an example, illustration, or explanation. Any embodiment or design scheme described in this application as an "example" or "for example" should not be described as being preferred or having more advantages than other embodiments or design schemes. Rather, words such as "example," "for example," etc. are intended to present the related concept in a particular manner.

[0053] In the description of embodiments of this application, unless otherwise specified, "plurality" means two or more. The term "and / or" in this specification describes only the association relationship between related objects and indicates that there are three possible relationships. For example, A and / or B may represent the following three cases: only A exists, both A and B exist, and only B exists.

[0054] For ease of understanding, relevant terms and concepts that may be used in the embodiments of this application are first described.

[0055] Figure 1a is a block diagram of the operating system.

[0056] An operating system (OS) is a computer program that manages the hardware and software resources of a computer, such as Unix, Windows, and Linux. An operating system must handle basic transactions, such as managing and configuring memory, prioritizing supply and demand for system resources, controlling input / output devices, operating the network, and managing the file system. An operating system also provides an operational interface through which users can interact with the system.

[0057] The operating system kernel is the core of most operating systems. It includes the parts used to manage storage devices, files, peripherals, and system resources within an operating system. It manages the system's processes, memory, device drivers, file and network systems, and determines the performance and stability of the operating system. The operating system kernel is system software that provides functions such as a hardware abstraction layer, disk and file system control, and multitasking. The operating system kernel provides secure access to computer hardware for many applications and determines when and how long applications should perform operations on parts of computer hardware. Because operating directly on computer hardware is very complex, the operating system kernel can provide a set of hardware abstraction methods to complete these operations.

[0058] The file system is the core module or main component of an operating system kernel. It organizes files on storage devices, manages and stores file information, and is primarily a way for users to create files, store, read, modify, and dump files, control file access, and cancel files when the user is no longer using them.

[0059] A file system provides an abstract representation of files in the kernel, maps files to physical storage devices (disks, hard drives, etc.), maps the physical addresses of files on storage devices to user-visible paths and file names, and facilitates the rapid retrieval, modification, and persistence of file data.

[0060] File systems include read / write file systems and read-only file systems. A read / write file system is one that can write files to or read files from a storage device, such as the file allocation table (FAT), high performance file system (HPFS), new technology file system (NTFS), fourth extended file system (EXT4), and flash friendly file system (F2FS). A read-only file system is one that can only read files from a storage device, but cannot write files to a storage device, such as the extendable read-only file system (EROFS).

[0061] To make this application clearer, we first describe the application scenario of this application.

[0062] Figure 1b is a schematic diagram of the storage system structure.

[0063] In the application scenario shown in FIG. 1b, users access data using applications. The computers running these applications are called "application servers." Application server 100 may be a physical machine or a virtual machine. Physical application servers include, but are not limited to, desktop computers, servers, notebook computers, and mobile devices. To access data, the application server uses a Fibre Channel switch 110 to access the storage system. However, switch 110 is optional; application server 100 may also communicate directly with storage system 120 using a network. Alternatively, Fibre Channel switch 110 may be replaced with an Ethernet switch, an InfiniBand switch, a RoCE (RDMA over Converged Ethernet) switch, or the like.

[0064] The storage system 120 shown in FIG. 1b is a centralized storage system. A centralized storage system is a central node that includes one or more main devices. Data is stored in the central node, and all data processing services for the entire system are deployed to the central node. In other words, in a centralized storage system, terminals or clients are only responsible for data input and output, and all data storage and control processing is completed by the central node. A centralized storage system features an integrated portal, and all data from external devices passes through this portal. The portal is the engine 121 of the centralized storage system. The engine 121 is the core component of the centralized storage system, and many of the storage system's advanced functions are implemented in the engine 121.

[0065] As shown in FIG. 1B, the engine 121 has one or more controllers. In FIG. 1B, an example in which the engine includes two controllers is used for explanation. A mirror channel exists between controller 0 and controller 1. Therefore, after controller 0 writes data to its memory 124, controller 0 may send a copy of the data to controller 1 via the mirror channel, and controller 1 stores the copy in its local memory 124. Therefore, controller 0 and controller 1 back up each other. When controller 0 fails, controller 1 may take over the services of controller 0. When controller 1 fails, controller 0 may take over the services of controller 1, preventing the entire storage system 120 from becoming unavailable due to a hardware failure. When four controllers are arranged in the engine 121, a mirror channel exists between any two controllers, and therefore, any two controllers back up each other.

[0066] The engine 121 further includes a front-end interface 125 and a back-end interface 126, where the front-end interface 125 is configured to communicate with the application server 100 to provide storage services to the application server 100. The back-end interface 126 is configured to communicate with hard disks 134 to expand the capacity of the storage system. The engine 121 may connect to more hard disks 134 via the back-end interface 126 to form a larger storage resource pool.

[0067] Based on the type of communication protocol between the engine 121 and the disk enclosure 130, the disk enclosure 130 may be a SAS disk enclosure, an NVMe disk enclosure, an IP disk enclosure, or another type of disk enclosure. SAS disk enclosures use the SAS 3.0 protocol, and each enclosure supports 25 SAS hard disks. The engine 121 connects to the disk enclosure 130 via an onboard SAS interface or a SAS interface module. The NVMe disk enclosure is like a complete computer system. NVMe hard disks are inserted into the NVMe disk enclosure. Additionally, the NVMe disk enclosure connects to the engine 121 via an RDMA port.

[0068] In terms of hardware, as shown in Figure 1b, controller 0 includes at least a processor 123 and a memory 124. Processor 123 is a central processing unit (CPU) configured to process data access requests from outside the storage system (a server or other storage system) and to process requests generated within the storage device. For example, processor 123 may be a front-end processor. Interface When a data write request sent from the application server 100 is received, the processor 123 temporarily stores the data of the data write request in the memory 124. When the total amount of data in the memory 124 reaches a certain threshold, the processor 123 starts the back-end Interface is used to transmit data stored in memory 124 to hard disk 134 for permanent storage.

[0069] The memory 124 is an internal memory that directly exchanges data with the processor. Data can be read and written to the memory at any time, and the memory serves as temporary data storage for the operating system or other running programs. The memory includes at least two types of memory. For example, the memory may be random access memory (RAM). For example, the random access memory may be dynamic random access memory (DRAM) or storage class memory (SCM). DRAM is a semiconductor memory and, like most random access memories (RAMs), is a volatile memory device. SCM uses a hybrid storage technology that combines the characteristics of both traditional storage devices and memory. Storage class memory can provide faster reads and writes than a hard disk, but is slower than DRAM in terms of access speed and cheaper than DRAM. However, DRAM and SCM are merely examples for illustrative purposes in the embodiments. The memory may further include another random access memory, such as a static random access memory (SRAM). Additionally, the memory 124 may be a dual in-line memory module (DIMM), i.e., a module configured with dynamic random access memory (DRAM), or a solid state disk (SSD). In a practical application, the controller 0 may be configured with multiple memories 124 and different types of memories 124. In this embodiment, the number and type of memories 124 are not limited. Additionally, the memory 124 may be configured with power failure protection. Power failure protection means that data stored in the memory 124 will not be lost even if the system is powered on again after a power failure.Memory that has power failure protection is called non-volatile memory.

[0070] For example, both the memory 124 and the hard disk 134 may be solid-state drives (SSDs), which are storage devices that primarily use NAND flash as nonvolatile memory. As shown in FIG. 2, the SSD 200 includes a NAND flash and a primary controller (PC) 201. The NAND flash includes multiple flash chips 205 configured to store data. The PC 201 is the brain of the SSD and is responsible for several complex tasks, such as managing data storage and maintaining the SSD's performance and service life. The PC 201 is an embedded microchip and includes a processor 202, which functions like a command center for sending all SSD operational requests. For example, the processor 202 may use firmware stored in a buffer to perform functions such as reading and writing data, garbage collection, and wear leveling.

[0071] The SSD PC201 further includes a host interface 204 and several channel controllers. The host interface 204 is configured to communicate with a host. In this specification, a host refers to any device, such as a server, a personal computer, or an array controller. The PC201 may use multiple channel controllers to operate multiple flash chips 205 in parallel to improve the bandwidth at the lowest level. For example, there are eight channels between the PC201 and the flash chips, and the PC201 reads and writes data in parallel to the eight flash chips 205 through the eight channels.

[0072] As shown in Figure 3, a die is a package containing one or more flash chips. A single die may contain multiple panels, making multi-plane NAND a design that can effectively improve performance. As shown in Figure 3, a die is divided into two planes, and the block numbers of the two planes are single- and dual-crossed. Therefore, single- and dual-cross operations can be performed during operation to improve performance. A panel contains multiple blocks. A block contains multiple pages. A 16GB flash chip is used as an example. Each 4314*8=34512 cells logically forms a page. Each page can store 4KB of content and 218-B of ECC parity data. A page is also the smallest unit of IO operations. A block is made up of 128 pages, and a panel is made up of 2048 blocks. A flash chip contains two panels. One panel stores odd-numbered blocks, and the other panel stores even-numbered blocks. The two planes can be operated simultaneously. This is just an example. The page size, block capacity, and flash chip capacity have different uses. This is not a limitation in this application.

[0073] The host writes data to a block. When a block is full, the SSD PC 201 may select the next block to continue writing. A page is the smallest unit of data to be written. In other words, the PC 201 writes data to a block with a page as the granularity. A block is the smallest unit for erasing data. The PC can only erase an entire block at a time.

[0074] The host accesses the SSD using a logical block address (LBA). Each LBA represents a sector (for example, 512B). In an SSD, the PC accesses the SSD in pages (for example, 4KB). Therefore, every time the application server writes data, the SSD PC searches for the page to write the data to. The page address is called the physical block address (PBA). The mapping from LBA to PBA is recorded in the SSD. With this mapping, the next time the host needs to read data from the LBA, the SSD knows where in the flash chip to read the data. Figure 4 is a schematic diagram of the flash translation layer (FTL). The FTL is located in the firmware of the processor 202. As shown in Figure 4, every time the host writes new data, a new mapping relationship is generated and added to the FTL (first write) or modified (override) the FTL. When reading data, the SSD first searches for the PBA corresponding to the LBA of the data in the FTL, and then reads the corresponding data based on the PBA.

[0075] Flash chips cannot support overwriting. This means that when the host modifies data on an LBA, it cannot directly modify the data on the PBA corresponding to that LBA. The data must be written to a new PBA and a mapping must be added to the FTL. For example, there is a mapping relationship between LBA D and PBA D in the FTL. When the host sends an IO request requesting to modify data on LBA D, the SSD searches for a new location (PBA E) to write the data and adds the mapping relationship between LBA D and PBA E to the FTL. As a result, the data on PBA D becomes invalid. Invalid data (also known as junk data) is data that is not pointed to by any mapping relationship. Because the data is replaced by the new mapping relationship, users may not be able to access the data's flash space. When data is written to the host continuously As a result, the flash storage space gradually decreases and eventually runs out. If the junk data is not cleared in a timely manner, the data cannot be written to the host. All SSDs have a garbage collection mechanism. The basic principle is to move valid data in some blocks to new blocks and erase the blocks. In this way, new available blocks are created.

[0076] Additionally, the memory 124 may further store software programs, and the processor 123 may manage the hard disks by executing the software programs in the memory 124. For example, the hard disks may be abstracted into a storage resource pool and then divided into LUNs for use by the servers. The LUNs are actually hard disks located on the servers. Indeed, some centralized storage systems may also be file servers, providing shared file services to the servers.

[0077] Data stored in memory 124 may be represented using a file system. A file system is a structured data file storage and organization format. As is well known, all data in a computer consists of 0s and 1s, and it is impossible to distinguish and manage a series of 0s and 1s stored on hardware media. Therefore, computers organize data using the concept of "files." Computers organize data used for the same purpose into different types of files based on the structure required by different applications. Different suffixes are typically used to refer to different types, and computers give each file a name that is easy to understand and remember. When there are many files, the files are grouped in a specific way. Each group of files is stored in the same directory (or folder). In addition to files, a catalog may contain lower-level catalogs (called subcatalogs or subfolders). All files and catalogs form a tree structure. This tree structure has a special name, the file system. There are many types of file systems, including FAT, FAT32, and NTFS in Windows, and EXT2, EXT3, EXT4, XFS, and BtrFS in Linux. To facilitate searching, the names of the catalogs, subcatalogs, and files, descending by level from the root node to the file, are combined with special characters (for example, "\" in Windows or DOS, or " / " in Unix-like systems). Such a string is called a file path, which in Linux is " / etc / systemd / system.conf" and in Windows is "C:\Windows\System32\taskmgr.exe". A path is a unique identifier for accessing a specific file. For example, D:\data\file.exe in Windows is a file path and refers to the file.exe file in the data catalog on partition D.

[0078] File systems are built on block devices. They record not only file paths but also the blocks that form files and the blocks that store catalog and subcatalog information. Different file systems have different organizational structures. For ease of management, block devices such as hard disks are typically divided into multiple logical block devices, i.e., hard disk partitions. Conversely, the capacity and performance of a single medium are limited. Multiple physical block devices can be combined into a logical block device using various levels of technology, such as RAID and JBOD. Alternatively, file systems can be built on these logical block devices. In either case, applications on application servers do not need to consider the specific location of the file they are accessing on the underlying block device; they only need to send the file's filename / ID to the file system. The file system obtains the file path through queries based on the filename / ID.

[0079] Relatively common file access protocols include NFS, CIFS, SMB, etc., but this is not limited to this embodiment.

[0080] The file system in this application is a read / write file system, which is a file system that can write files to or read files from a storage device, such as FAT, HPFS, NTFS, EXT4, and F2FS.

[0081] A file system generally includes a metadata area and a data area. The metadata area includes a superblock and an inode area. The superblock of the metadata area may contain file system control information, data structures, and other content. The inode area of ​​the metadata area may contain file description information, such as the file length and file type. Examples of file types include regular inode, directory inode, symbol link inode, and special inode. Data stored in the data area may be data obtained by performing file-level compression based on lossless compression technology. Data in the data area is stored in the physical storage space of a storage medium (e.g., disk or flash) based on a set of disk blocks. Data for the same file may be stored in contiguous disk blocks or in discontiguous disk blocks in a cross-sectional manner.

[0082] It will be understood that the introduction of the concept of disk blocks in this application does not limit the storage medium to disks, and that disk blocks may be used to represent smaller physical storage spaces by dividing the physical storage space of a storage medium.

[0083] Of course, the storage system of the present application may also include a scale-out storage system. A scale-out storage system is a system in which data is stored in multiple independent storage nodes. Traditional network storage systems use a centralized storage array to store all data. The performance of the storage array is not only a bottleneck for system performance but also a focus for reliability and security, making it difficult to meet the requirements of large-scale storage applications.

[0084] Above, we have briefly explained the application scenario of this application.

[0085] In the above storage system, the components are sorted by their read / write speed, in descending order: central processing unit (CPU) > double data rate synchronous dynamic random access memory (DDR SDRAM) > flash chips. in It turns out that the bottleneck in data access in storage systems is the time overhead of data input / output (IO) between memory and flash.

[0086] To improve the overall IO read / write performance of a storage system, it is necessary to compress files in memory. Because metadata typically occupies a small percentage of the overall file system, it typically occupies a large amount of storage space on the device. Therefore, compressing data before writing it to flash memory reduces the storage space required and extends the life of flash memory.

[0087] Currently, data compression methods are sometimes used for Linux read / write file systems such as F2FS, journalling Flash file system version 2 (JFFS2), and B-tree file system (BTRFS), as well as for Windows read / write file systems such as NTFS.

[0088] The original file data (also called source data) that requires compression is compressed based on a fixed-size minimum compression unit (cluster). The compressed file data (also called compressed data) consists of header data and compressed data. The header data is used to represent the attribute information of the file data, and the compressed data is used to represent the content of the file data. The compressed file data is stored in flash memory and is adjusted to a size of 4KB.

[0089] For example, as shown in the schematic diagram of fixed input compression mode in Figure 5, four data blocks with consecutive addresses are compressed as cluster 0, and compressed file data including header data and compressed data is obtained. If the size of the compressed file data is less than 4 KB, the compressed file data is stored in flash memory with a size of 4 KB.

[0090] Assume that the size of the original file data (also called source data) shown in Figure 5 is 4 blocks, each block is 4KB, and 1 block is 1 logical page. The logical pages of the original file data are numbered 0, 1, 2, and 3. The original file data is compressed to compressed file data with a compression rate of 75%, and the size of the compressed file data is 12KB. Therefore, the size of the compressed file data is 3 blocks. Therefore, the actual pages of the compressed file data are 3 pages, and the size of the actual pages of flash that need to be read to read 1 logical page is shown in Table 1. [Table 1]

[0091] After the compressed file data is saved to flash, if the target logical page of the original file data needs to be read from flash, three pages of compressed file data must be read, and the target logical page can only be read after the compressed file data is decompressed. For example, in a random read scenario, if the original file data of logical page 0 needs to be read from flash, three pages of compressed file data must be read and decompressed before the original file data of logical page 0 can be successfully read. Therefore, the data read efficiency is as follows:

number

[0092] In a random read scenario, it can be seen that the read efficiency of the compressed file data obtained with the data compression method shown in Figure 5 is relatively low.

[0093] To solve the above-mentioned problems, one embodiment of this application provides a data compression method. In this method, m data blocks in a data area of ​​a read-write file system are obtained. The m data blocks are compressed using a preset compression algorithm to sequentially obtain n compressed data blocks, where the compressed data blocks all have the same first capacity and the first capacity represents the number of bytes of compressed data that may be included in the compressed data block. Both m and n are positive integers greater than or equal to 1.

[0094] The preset compression algorithm may be a compression algorithm corresponding to a fixed output compression mode, such as a Lempel-Ziv 4 (LZ4) compression algorithm. Of course, the preset compression algorithm may also be another compression algorithm, which is not specifically limited in this embodiment of the present application.

[0095] For example, assume that in an application scenario, the size of the source data is 16 KB, the data block is 4 KB, and logical pages are used as an example, as shown in Figure 6. The logical pages of the source data are numbered 0, 1, 2, and 3, as shown in the first row of Table 2.

[0096] Assume that 16KB of contiguous source data in a logical page is divided into three parts of 6KB, 7KB, and 5KB, respectively, and then the three pieces of data are compressed using a pre-defined compression algorithm (e.g., LZ4) until the size of each compressed piece of data in the compressed data block is 4KB.

[0097] The compressed data block has three data pages numbered as compressed page 4, compressed page 5, and compressed page 6, respectively, shown in Figure 6. [Table 2]

[0098] In compressed page 4, we can see that all of the source data for logical page 0 has been compressed. Therefore, logical page 0 is compressed to one page. Some of the source data for logical page 1 has been compressed to compressed page 4, and other source data for logical page 1 has been compressed to compressed page 5. Therefore, logical page 1 has been compressed to two pages. Some of the source data for logical page 2 has been compressed to compressed page 5, and other source data for logical page 2 has been compressed to compressed page 6. Therefore, logical page 2 has been compressed to two pages. Compressed page 6 has all of the source data for logical page 3 compressed. Therefore, logical page 3 has been compressed to one page.

[0099] Therefore, in a random read scenario, any one or more logical pages may be read. For example, when logical page 0 is read, only one compressed page needs to be read, as shown in the second row and second column of Table 2. After decompression, all data in logical page 0 may be retrieved.

[0100] In this case, the read efficiency may be calculated according to Equation 2 below.

number

[0101] The read efficiency of logical page 3 is the same as the read efficiency of logical page 0.

[0102] For example, when logical page 1 is read, three compressed pages must be read, as shown in the second and third rows of Table 2. All data in logical page 1 can be obtained only after the data in compressed page 4 and the data in compressed page 5 are decompressed.

[0103] In this case, the read efficiency may be calculated according to Equation 3 below.

number

[0104] The read efficiency of logical page 2 is the same as the read efficiency of logical page 1.

[0105] Additionally, the average read efficiency of the four logical pages may be calculated according to Equation 4 below.

number

[0106] From the read efficiency obtained by calculating Equation 2, Equation 3, and Equation 4, it can be seen that in a random read scenario, the read efficiency of the data compression scheme shown in FIG. 6 is much higher than the read efficiency of the data compression scheme shown in FIG.

[0107] In this embodiment of the present application, a compression algorithm corresponding to a fixed output compression mode is used to compress m data blocks in the data area of ​​a read-write file system, sequentially obtaining n compressed data blocks of the same byte size, so that each output compressed data block has a fixed size. This can effectively improve the read efficiency when the data blocks are read, and ensure that data can be read in a random read scenario with a small read amplification factor.

[0108] Additionally, Figure 8 shows the data block indexing method in an existing extensible read-only file system (erofs). In the data block address array data_addr, the block address is accessed and points to the address of the actual data block. When erofs compresses data using the method shown in Figure 5 to create a mirror, file modification is not supported because the structure of the storage device (e.g., disk) and file contents are fixed. However, in a user's actual operating scenario, many compressed files on the storage device may need to be frequently modified. erofs does not support this requirement.

[0109] The corresponding data block can be found based on the data block index, which is also an inode, i.e., metadata. An inode is an area used to store metadata, i.e., attribute information about a file, such as the file's creator, creation date, size, and location of data blocks. Each inode has a number, and the operating system uses different inode numbers to identify different files. For example, a user ostensibly opens a file using its file name. In reality, the operating system first finds the corresponding inode number based on the file name, obtains the inode information based on the inode number, and then finds the address of the data block based on the inode information and reads the data.

[0110] That is, an inode records the file's attributes and the file's actual storage location, i.e., block number. Each block (typically 4KB in size) can be searched and located using an inode. An inode is a Linux term, and is called a vnode in Unix. Essentially, an inode contains at least the following information: (1) file type, (2) file access permissions, (3) file owner and group, (4) file size, (5) number of links, i.e., the total number of file names pointing to the inode, (6) file status change time (ctime), last access time (atime), and last modification time (mtime), (7) special file attributes including SUID, SGID, and SBIT, and (8) a pointer to the actual file contents.

[0111] Figure 8 shows the existing data block index format. The data block index format does not support scalability, such as appending, block reservation, or truncating. Appending refers to adding new content to an original file without deleting the content of the original file. Block reservation refers to the file system considering the space available to allocate disk blocks and reserving disk blocks if the file size increases. Truncate refers to modifying the file, such as deleting or adding files.

[0112] For example, as shown in FIG. 8, a data block index is represented by a blk entry, which is simply referred to as blk for convenience of explanation. blk 1 is the index of compressed data block 1, and the address of compressed data block 1 in the storage device is stored in blk 1. blk 2 is the index of compressed data block 2, and the address of compressed data block 2 in the storage device is stored in blk 2. blk 3 is the index of compressed data block 3, and the address of compressed data block 3 in the storage device is stored in blk 3. blk 4 is the index of compressed data block 4, and the address of compressed data block 4 in the storage device is stored in blk 4. Therefore, the location of the compressed data block in the storage device can be determined based on the address stored in blk.

[0113] In order to enable a read / write file system to support writing, overwriting, pre-allocation, truncation, etc., the data compression method provided in the embodiments of this application: n compressed data blocksThe method further includes establishing a first index for each of j data blocks corresponding to the i-th compressed data block, and recording a mapping relationship between the first index and the j data blocks, where i is a positive integer greater than or equal to 1 and less than or equal to n. i is a positive integer greater than or equal to 1 and less than or equal to m. The first index is used to identify a storage location in the storage medium of each data block included in the j data blocks, and attribute information included in each of the j data blocks. The first index is used to identify a storage location in the storage medium of the i-th compressed data block, and attribute information included in each of the j data blocks.

[0114] The attribute information is a first attribute indicating whether the compressed data block storage location where the data block is compressed is pre-allocated; a second attribute used to indicate whether a data page of a data block is valid, i.e., whether the data page is a normal data page or an empty data page, where an empty data page can be understood as a blank data page; a third attribute used to indicate whether the data page of the data block is the first compressed page of the compressed data block of the data block; a fourth attribute used to indicate whether the data page of the data block is included in the compressed data page of the two compressed blocks; a fifth attribute indicating whether the data page of the data block is a compressed page of a compressed data block obtained by compressing the data block; a sixth attribute used to represent the index address of the compressed data block where the data page of the data block is located; a seventh attribute used to represent, when the data page of the data block belongs to the first compressed page of the compressed data block of the data block, the attribute value of the seventh attribute being an offset of the data block in the set corresponding to the compressed data block; and, when the data page of the data block does not belong to the first compressed page of the compressed data block of the data block, the attribute value of the seventh attribute being a distance between the data page of the data block and the first compressed page of the compressed data block.

[0115] For example, as shown in Figure 7, the first index of a data block includes a blk entry that stores the address of the data block or compressed data block, and an extent entry that stores extended attribute information. Each extent entry has a one-to-one correspondence with a blk entry, and each data page has a corresponding extent entry and a corresponding blk entry.

[0116] The set of extent entry members is represented in the following manner:

[0117] For example, the members contained in the data block index may be shown in set A as shown in set A. Note that each data page has a corresponding set A.

number

[0118] It will be appreciated that when compressing data in a read / write file system using the method shown in Figure 6, the attributes included in the index of data blocks shown in Figure 7 may be modified, just as the compressed file on the storage device may be modified.

[0119] The data compression method provided in the embodiments of this application will now be described with reference to a specific example.

[0120] 9A and 9B are schematic flowcharts of a data compression method according to an embodiment of this application. As shown in 9A and 9B, the method includes the following steps:

[0121] S901: m data blocks in a data area of ​​a read-write file system are obtained, where m is a positive integer equal to or greater than 1.

[0122] The m data blocks may be understood as data blocks that need to be written back. Write back may mean that during a write operation, the data is first written to memory for caching but is not immediately written to a storage device (e.g., a disk). The data cached in memory is written to a storage device only under some specific conditions or operations (e.g., a refresh mechanism or a sync operation).

[0123] S902: Compress m data blocks using a preset compression algorithm to sequentially obtain n compressed data blocks, where the compressed data blocks all have the same first capacity, and the first capacity represents the number of bytes of compressed data that may be included in the compressed data block, where n is a positive integer greater than or equal to 1.

[0124] The preset compression algorithm may be the LZ4 compression algorithm, or of course, may be another compression algorithm with a fixed output, which is not specifically limited in this embodiment of the present application.

[0125] m is any positive integer, for example, m is 4, m is 10, or m is 20.

[0126] Specifically, S902 may be implemented as follows.

[0127] S9021: Allocate all of the m data blocks to the first set in a predetermined order.

[0128] The preset order may be a contiguous storage address order, i.e., a contiguous order of the m data blocks.

[0129] The first set may be called the smallest compressible unit (cluster). In other words, the first set is the smallest compressible data block set, such as the 6 KB data block set, the 7 KB data block set, and the 5 KB data block set shown in Figure 6.

[0130] For example, m data blocks are mapped to segments of consecutive addresses on the storage medium. Starting from a data block, a fixed-size data set is divided sequentially according to the address order of the data blocks mapped on the storage medium. As shown in Figure 6, data block 0 and half the data of data block 1 form a 6KB data set, half the data of data block 1, three-quarters the data of data block 2, and a blank data page form a 7KB data set, and one-quarter the data of data block 2 and data block 3 form a 5KB data set.

[0131] S9022: Determine whether the data capacity of the j data blocks of the first set is equal to the rated capacity of the first set, where j is a positive integer greater than or equal to 1 and less than or equal to m. If the data capacity of the j data blocks is not equal to the rated capacity of the first set, S9021 is executed, or if the data capacity of the j data blocks is equal to the rated capacity of the first set, S9023 is executed.

[0132] S9023: Perform a fixed compression operation on the j data blocks of the first set based on a specified compression threshold to obtain the i-th compressed data block.

[0133] The specified compression threshold is used to represent the compression ratio. For example, the formula for the specified compression threshold may be: specified compression threshold = total data length - total data length * compression ratio.

[0134] S9024: Determine whether the total data length of the j data blocks is greater than the sum of the total data length of the header data and compressed data of the i-th compressed data block and the specified compression threshold. If the total data length of the j data blocks is greater than the sum of the total data length of the header data and compressed data of the i-th compressed data block and the specified compression threshold, S903 is executed. If the total data length of the j data blocks is not greater than the sum of the total data length of the header data and compressed data of the i-th compressed data block and the specified compression threshold, the source data page is submitted to flush.

[0135] S903: n compressed data blocks Establish a first index for each of j data blocks corresponding to the i-th compressed data block, and record a mapping relationship between the first index and the j data blocks, where i is a positive integer greater than or equal to 1 and less than or equal to n, and j is a positive integer greater than or equal to 1 and less than or equal to m.

[0136] When a compressed data block is compressed, an index is established for each of the data blocks that correspond to that compressed data block.

[0137] The first index is used to identify the storage location in the storage medium of each data block included in the j data blocks, and the attribute information included in each of the j data blocks.

[0138] For example, using the Linux f2fs read / write file system as an example, the first index / format of a data block in f2fs may be as follows:

[0139] The data structure of the attribute information included in the first index may be as follows. For example, the entry data structure may be:

number

[0140] 10A and 10B are schematic flowcharts illustrating a method for updating a data block index according to an embodiment of the present application. As shown in FIGS. 10A and 10B, the attribute information may include a third attribute (first_page) and a seventh attribute (ofs). S903 may be specifically implemented as follows:

[0141] S1031: Determine whether each data page of the j data blocks is the first compressed page of the i-th compressed data block. If each data page of the j data blocks is the first compressed page of the i-th compressed data block, assign 1 to the attribute value of the third attribute, or if each data page of the j data blocks is not the first compressed page of the i-th compressed data block, assign 0 to the attribute value of the third attribute.

[0142] S1032: When the attribute value of the third attribute is 1, update the attribute value of the seventh attribute to the offset in the first set corresponding to the compressed data block.

[0143] S1033: When the attribute value of the third attribute is 0, update the attribute value of the seventh attribute to the distance between the data page of the data block and the first compressed page of the i-th compressed data block.

[0144] Of course, the attribute information may further include a fourth attribute (cross_block), and S103 may be specifically implemented as follows.

[0145] S1034: Determine whether each data page of the j data blocks is included in the compressed data pages of the two compressed blocks. If each data page of the j data blocks is included in the compressed data pages of the two compressed blocks, assign 1 to the attribute value of the fourth attribute, or if each data page of the j data blocks is not included in the compressed data pages of the two compressed blocks, assign 0 to the attribute value of the fourth attribute.

[0146] Of course, the attribute information may further include a second attribute (is_valid), and S103 may be specifically implemented as follows:

[0147] S1035: Determine whether each data page of the j data blocks is valid. If each data page of the j data blocks is valid, assign 1 to the attribute value of the second attribute. If each data page of the j data blocks is invalid, assign 0 to the attribute value of the second attribute.

[0148] Of course, the attribute information may further include a sixth attribute (blkidx), and S103 may be specifically implemented as follows.

[0149] S1036: Determine the index address of the compressed data block where the data page of each of the m data blocks is located.

[0150] The compression is performed according to the sequence of the memory locations of the m data blocks in the memory, using the size of a minimum fixed compression unit (for example, the first set). When the first compression is completed (i.e., the first compressed data block is obtained), the data pages of the completed data block corresponding to the first compressed data block are all compressed by the first compression. data· The data pages are located at the index position of the block. For example, the data blocks corresponding to the first compressed data block include part of the data of data block 0, data block 1, data block 2, and data block 3. The complete data blocks corresponding to the first compressed data block are data block 0, data block 1, and data block 2. Therefore, the data pages of data block 0, data block 1, and data block 2 are located at the index position of the first compressed data block.

[0151] It should be noted that in addition to the seventh attribute that needs to be added to the third attribute, the data block index update procedures corresponding to the other attributes are independent of each other, and the sequence of the data block index update procedures corresponding to the first attribute, second attribute, fourth attribute, fifth attribute, and sixth attribute is not particularly limited in this embodiment of this application.

[0152] For example, as shown in FIG. 11, it is assumed that the m data blocks include data block 0 (i.e., block 0), data block 1 (i.e., block 1), data block 2 (i.e., block 2), and data block 3 (i.e., block 3). A segment of consecutive addresses is mapped to memory using block 0, block 1, block 2, and block 3. During compression, when compression is performed using the size of a minimum fixed compression unit (e.g., the first set) according to the sequence of the segment of consecutive addresses mapped to memory using block 0, block 1, block 2, and block 3 (e.g., the compression direction from left to right in FIG. 11), When part of block 0 and block 1 reach the minimum fixed compression unit (e.g., 4KB), the first compression is performed to obtain the first compressed data block (compress blk 0). In this case, the data block index of block 0 is established as shown in Table 3. [Table 3]

[0153] Referring to Figures 10A and 10B, it can be seen that the data page of block 0 is located in the first compressed page of the first compressed data block. Therefore, 1 is assigned to first_page. The data page of block 0 is located only in the first compressed page of the first compressed data block. Therefore, 0 is assigned to cross_block. The index address within the first compressed data block of the data page of block 0 is the number of the first compressed data block (compress blk 0). Therefore, 0 is assigned to blkidx. The data page of block 0 is located in the first compressed page of the first compressed data block, and the offset of block 0 in the first set corresponding to block 0 is 0. Therefore, 0 is assigned to ofs. The data page of block 0 is a valid data page. Therefore, 1 is assigned to is_valid.

[0154] When the remaining part of block 1, part of block 2, and block 3 reach the minimum fixed compression unit (e.g., 4KB), a second compression is performed to obtain the second compressed data block (compress blk 1). In this case, the data block indexes of blocks 1 and 2 are established as shown in Table 4. [Table 4]

[0155] Referring to Figures 10A and 10B, it can be seen that the data page of block 1 is located on the first compressed page of the second compressed data block. Therefore, 1 is assigned to first_page. The data page of block 1 is located on the compressed page of the first compressed data block and the compressed page of the second compressed data block. Therefore, 1 is assigned to cross_block. The index address of block 1's data page within the second compressed data block is the number of the second compressed data block (compress blk 1). Therefore, 1 is assigned to blkidx. The data page of block 1 is located on the first compressed page of the second compressed data block, and the offset of block 1 in the set of data blocks is Ofs1. Therefore, Ofs1 is assigned to ofs1. The data page of block 1 is a valid data page. Therefore, 1 is assigned to is_valid.

[0156] Similarly, the data page of block 2 is not on the first compressed page of the second compressed data block. Therefore, 0 is assigned to first_page. The data page of block 2 is only on the compressed page of the second compressed data block. Therefore, 0 is assigned to cross_block. The index address of the data page of block 2 within the second compressed data block is the number of the second compressed data block (compress blk 1). Therefore, 1 is assigned to blkidx. The data page of block 2 is not on the first compressed page of the second compressed data block, and the distance between the data page of block 2 and the first compressed page of the first compressed data block is 1. Therefore, 1 is assigned to ofs. The data page of block 2 is a valid data page. Therefore, 1 is assigned to is_valid.

[0157] When the remaining portion of block 3 reaches the minimum fixed compression unit (e.g., 4KB), a third compression is performed to obtain the third compressed data block (compress blk 2). In this case, the data block index of block 3 is established as shown in Table 5. [Table 5]

[0158] Referring to Figures 10A and 10B, we can see that the data page of block 3 is located on the first compressed page of the third compressed data block. Therefore, 1 is assigned to first_page. The data page of block 1 is located on the compressed page of the second compressed data block and the compressed page of the third compressed data block. Therefore, 1 is assigned to cross_block. The index address of the data page of block 1 within the third compressed data block is the number of the second compressed data block (compress blk 2). Therefore, 2 is assigned to blkidx. The data page of block 3 is located on the first compressed page of the third compressed data block, and the offset of block 3 in the set of data blocks is Ofs2. Therefore, Ofs2 is assigned to ofs1. The data page of block 3 is a valid data page. Therefore, 1 is assigned to is_valid.

[0159] S904: Determine whether the compression of m data is complete. When the compression is complete, the compressed page of the compressed data block is submitted to the device. If the compression is not complete, execute S902.

[0160] In some embodiments, Figure 12 is a schematic flowchart of a data compression method according to an embodiment of this application. As shown in Figure 12, before S902 is performed, the data compression method provided in this embodiment of this application includes the following steps:

[0161] S905: Obtain a second set of data to be overwritten.

[0162] S906: Determine whether the second set of data to be overwritten includes compressed data blocks. If the second set of data to be overwritten includes compressed data blocks, S907 is executed; if the second set of data to be overwritten does not include compressed data blocks, an overwrite process for the existing data is executed.

[0163] The second set may include p compressed data blocks, where p is a positive integer greater than or equal to 1.

[0164] S907: The first target compressed data in the p compressed data blocks ·block and q data blocks corresponding to the compressed page of the first target compressed data block, where q is a positive integer equal to or greater than 1.

[0165] S908: In the q data blocks, a location offset of the first target data block within the q data blocks is determined.

[0166] Specifically, the index address of each compressed data block in the second set is read, and the compressed data block is decompressed to obtain a data block corresponding to the compressed data block. Then, the location offset of each of the q data blocks is determined.

[0167] S909: Obtain the data page of the first data block based on the first compressed page and the offset position of the first data block in the compressed page.

[0168] S910: Determine that the data page of the first target data block is the data page of the data to be overwritten.

[0169] S911: Overwrite the data page of the first data block with the second data block.

[0170] S912: Allocate a second data block to the first set.

[0171] In summary, in the read / write file system f2fs, by compressing specified so, vdex, and odex files using the fixed output compression mode provided in this embodiment of the present application, a beneficial effect can be achieved, for example, in the process of installing 40 applications on an electronic device, where each application can achieve an average time benefit of 12%. During application installation, the so file undergoes an append procedure, and both the vdex and odex files undergo an overwrite procedure. The average boot gain for the 40 applications is 8% higher than the boot gain for compressed data in the fixed input compression mode.

[0172] According to the data compression method provided in this embodiment of the present application, after the data is compressed using the aforementioned data compression method, the data needs to be read. Figure 14 is a schematic flowchart of the data reading procedure according to one embodiment of the present application. As shown in Figure 14, the data reading procedure is as follows:

[0173] S141: Read a first index including attribute information of a first data block to obtain an index address of a first compressed data block corresponding to the first data block, where the first index includes the attribute information of the first data block.

[0174] In the above embodiment, the attribute information of the first data block may include at least one of the first attribute to the seventh attribute, for example, the attribute information of the first data block may include the third attribute (first_page), the fourth attribute (cross_block), the sixth attribute (blkidx), and the seventh attribute (ofs).

[0175] In the overwrite and read-only scenarios, assume that the first data block is data block 2 (block 2) shown in FIG. 13. The attribute information of block 2, such as ofs, cross_block, and blkidx, is read to obtain the index address of the first compressed data block corresponding to block 2. Specifically, Table 4 is still used. When the value assigned to ofs of block 2 is read as 1, it may be determined that the data page of block 2 is not in the first compressed page of the second compressed data block, and the distance between the data page of block 2 and the first compressed page of the first compressed data block may be obtained as 1. Then, when the value assigned to cross_block of block 2 is read as 0, it may be determined that the data page of block 2 is only in the compressed page of the first compressed data block, and not in the compressed page of another compressed data block. Then, when the value assigned to blkidx of block 2 is read as 1, it may be determined that the index address in the first compressed data block of the data page of block 2 is the number of the first compressed data block, i.e., the index address of the first compressed data block may be obtained as 1.

[0176] S142: The index of the first compressed data block corresponding to the first data block is read.

[0177] S143: Decompress the first compressed data block based on the index of the first compressed data block to obtain a plurality of data blocks corresponding to the first compressed data block, where the plurality of data blocks includes the first data block.

[0178] Specifically, a first compressed data block is found on the device based on the index of the first compressed data block. After the first compressed data block is found, the first compressed data block is parsed to obtain a plurality of parsed data blocks. For example, as shown in FIG. 13, the first compressed data block is designated as compress blk 1. After parsing compress blk 1, the following data is obtained: a portion of data in data block 1 (block 1), data block 2 (block 2), and a portion of data in data block 3 (block 3).

[0179] S144: Determine the offset of the first data block of the plurality of decompressed data blocks.

[0180] Specifically, based on the attribute information of the first data block, it can be determined that the first data block is data block 2 (block 2) shown in Figure 13. As shown in Figure 13, the offset (dstofs) of block 2 in the multiple data blocks analyzed from the compressed data blk 1 is expressed as follows:

number

[0181] S145: Obtain data of the first data block based on the offset of the first data block among the plurality of decompressed data blocks.

[0182] The previous example is still used. The first data block is block 2. The data for block 2 may be obtained as shown in Figure 13 and Table 4.

[0183] Specifically, the communication system in this possible design is configured to perform the functions of each device using the data compression method shown in Figure 9, so that the same effect as the data compression method described above can be achieved.

[0184] 15 shows a data compression device according to an embodiment of the present application. The data compression device 1500 includes a first acquisition unit 1501 configured to acquire m data blocks in a data area of ​​a read-write file system, where m is a positive integer greater than or equal to 1, and a compression unit 1502 configured to compress the m data blocks using a preset compression algorithm to sequentially acquire n compressed data blocks, where the compressed data blocks all have the same first capacity, the first capacity represents the number of bytes of compressed data that may be included in the compressed data block, where n is a positive integer greater than or equal to 1. n compressed data blocks and an updating unit 1503 configured to establish a first index for each of j data blocks corresponding to the i-th compressed data block and record a mapping relationship between the first index and the j data blocks, where i is a positive integer between 1 and n, and j is a positive integer between 1 and m. The first index is used to identify a storage location in a storage medium of each data block included in the j data blocks and attribute information included in each of the j data blocks.

[0185] In a specific possible implementation, the compression unit 1502 is configured to sequentially allocate all data blocks of the m data blocks to the first set in a predetermined order, and when the data capacity of the j data blocks in the first set is equal to the rated capacity of the first set, a compression operation is performed on the j data blocks based on a specified compression threshold to obtain the i-th compressed data block.

[0186] In a specific possible implementation, the update unit 1503 is configured to establish a first index for each of the j data blocks when the sum of the total data length of the header data and compressed data of the i-th compressed data block and the specified compression threshold is less than or equal to the total data length of the j data blocks.

[0187] In a specific possible implementation, the attribute information includes a first attribute representing whether a storage location of the compressed data block where the data block is compressed is pre-allocated; a second attribute representing whether a data page of the data block is valid; a third attribute representing whether the data page of the data block is the first compressed page of the compressed data block of the data block; a fourth attribute representing whether the data page of the data block is included in the compressed data page of the two compressed blocks; and a fourth attribute representing whether the data page of the data block is the compressed page of the compressed data block obtained by compressing the data block. a sixth attribute used to represent an index address of a compressed data block in which a data page of the data block is located; and a seventh attribute used to represent, when the data page of the data block belongs to the first compressed page of the compressed data block of the data block, an attribute value of the seventh attribute being an offset of the data block in the set corresponding to the compressed data block, and, when the data page of the data block does not belong to the first compressed page of the compressed data block of the data block, an attribute value of the seventh attribute being a distance between the data page of the data block and the first compressed page of the compressed data block.

[0188] In a specific possible implementation, the attribute information includes a third attribute, and the updating unit 1503 is further configured to assign 1 to an attribute value of the third attribute when a data page of each of the j data blocks is the first compressed page of the i-th compressed data block, or to assign 0 to the attribute value of the third attribute when a data page of each of the j data blocks is not the first compressed page of the i-th compressed data block.

[0189] In a specific possible implementation, the attribute information includes a seventh attribute, and the updating unit 1503 is further configured to: update the attribute value of the seventh attribute to an offset of the data block in the set corresponding to the compressed data block when the attribute value of the third attribute is 1; or update the attribute value of the seventh attribute to a distance between a data page of the data block and a first compressed page of the compressed data block when the attribute value of the third attribute is 0.

[0190] In a specific possible implementation, the attribute information includes a fourth attribute, and the updating unit 1503 is further configured to assign 1 to an attribute value of the fourth attribute when a data page of each of the j data blocks is included in the compressed data pages of the two compressed blocks, or to assign 0 to an attribute value of the fourth attribute when a data page of each of the j data blocks is not included in the compressed data pages of the two compressed blocks.

[0191] In a specific possible implementation, the attribute information includes a second attribute, and the update unit is further configured to assign 1 to an attribute value of the second attribute when a data page of each of the j data blocks is valid, or to assign 0 to an attribute value of the second attribute when a data page of each of the j data blocks is invalid.

[0192] In some possible implementations, the apparatus includes a second acquisition unit 1504 configured to acquire a second set of data to be overwritten, the second set including p compressed data blocks, where p is a positive integer greater than or equal to 1; and a first target compressed data block in the p compressed data blocks. ·block and q data blocks corresponding to the compressed page of the first target compressed data block, where q is a positive integer greater than or equal to 1; a first determining unit 1506 configured to determine, in the q data blocks, a location offset of the first target data block within the q data blocks; and a second determining unit 1507 configured to determine that the data page of the first target data block is a data page of the data to be overwritten.

[0193] In some possible implementations, the apparatus further includes: a first reading unit configured to read a first index of the first data block to obtain an index address of a first compressed data block corresponding to the first data block, where the first index includes attribute information of the first data block; a second reading unit configured to read an index of the first compressed data block corresponding to the first data block; a decompression unit configured to decompress the first compressed data block based on the index of the first compressed data block to obtain a plurality of data blocks corresponding to the first compressed data block, where the plurality of data blocks includes the first data block; a third determining unit configured to determine an offset of the first data block in the plurality of decompressed data blocks; and a third obtaining unit configured to obtain data of the first data block based on the offset of the first data block in the plurality of decompressed data blocks.

[0194] In a specific possible implementation, the first index is used to identify the storage location of the ith compressed data block on the storage medium, and the attribute information contained in each of the j data blocks.

[0195] The data compression method provided in the embodiment of this application can effectively improve read efficiency when a data block is read, and can ensure that data can be read in a random read scenario with a small read amplification factor. Additionally, attributes included in the index of the data block may be modified, and thus the compressed file on the storage device may be modified. It can be seen that the embodiment of this application solves the random read amplification problem of compression solutions in existing read / write file systems and solves the problem that existing file systems with fixed output compression methods cannot support data and metadata updates.

[0196] An embodiment of the present application further provides a device, the device including a unit configured to perform steps according to any one of the aforementioned implementations or a unit configured to perform steps according to any one of the aforementioned implementations.

[0197] An embodiment of the present application further provides a computer-readable storage medium containing instructions, which when executed on a computer, enable the computer to perform any one of the methods described above.

[0198] One embodiment of this application provides a computer program product comprising instructions that, when run on a computer, enable the computer to perform any one of the methods described above.

[0199] An embodiment of the present application further provides a chip, the chip including a processor and an interface circuit, the interface circuit coupled to the processor, the processor configured to execute computer programs or instructions to implement the aforementioned method, and the interface circuit configured to communicate with another module outside the chip.

[0200] In the description of this application, unless otherwise specified, " / " means "or." For example, A / B may represent A or B. The term "and / or" in this specification only describes an association relationship between related objects and indicates that there are three possible relationships. For example, A and / or B may represent the following three cases: only A exists, both A and B exist, and only B exists. Additionally, "at least one" means one or more, and "multiple" means two or more. Terms such as "first," "second," etc. do not limit the number or execution sequence, and terms such as "first," "second," etc. do not indicate a clear distinction.

[0201] In the description of this application, words such as "example," "for example," and the like are used to denote providing an example, illustration, or description. Any embodiment or design scheme described in this application as an "example" or "for example" should not be described as preferred or having more advantages than other embodiments or design schemes. Rather, words such as "example," "for example," and the like are intended to present the related concept in a particular manner.

[0202] The above description of the implementation allows those skilled in the art to clearly understand that the division into the above functional modules is used as an example for explanation for the purpose of convenience and concise description. In actual applications, the above functions can be allocated to different functional modules for implementation as needed. In other words, the internal structure of the device is divided into different functional modules to implement all or part of the above functions.

[0203] In some embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other manners. For example, the described device embodiments are merely examples. For example, module or unit divisions are merely logical function divisions, and other divisions may be used in actual implementations. For example, multiple units or components may be combined or integrated into another device, or some features may be omitted or not implemented. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be implemented using some interfaces. Indirect couplings or communication connections between devices or units may be implemented in electronic, mechanical, or other forms.

[0204] Units described as separate parts may or may not be physically separated, and parts shown as units may be one or more physical units, located in one place, or distributed across multiple places. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0205] Additionally, the functional units in the embodiments of the present application may be integrated into one processing unit, and each unit may exist physically alone, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0206] Although this application has been described with reference to particular features and embodiments thereof, RangeIt is apparent that various modifications and combinations may be made thereto without departing from the scope thereof. Accordingly, the specification and the accompanying drawings are merely exemplary descriptions of this application as defined by the appended claims, and any or all of the modifications, variations, combinations, or equivalents covering the scope of this application are considered. It is apparent that a person skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. This application intends to cover these modifications and variations of this application, provided that they fall within the scope of protection defined by the following claims and their equivalent technologies.

[0207] The above description is merely a specific implementation of this application, but is not intended to limit the scope of protection of this application. Any modifications or substitutions that can be easily understood by those skilled in the art within the technical scope disclosed in this application shall be included in the scope of protection of this application. Therefore, the scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. 1. A data compression method performed by an electronic device, comprising: Obtaining m data blocks in a data area of ​​a read-write file system, where m is a positive integer greater than or equal to 3; Dividing the m data blocks into n data blocks, and compressing each of the n data blocks using a preset compression algorithm to sequentially obtain n compressed data blocks, wherein the compressed data blocks all have the same first capacity, the first capacity represents the number of bytes of compressed data that can be included in the compressed data block, and n is a positive integer equal to or greater than 2 and smaller than m; establishing a first index for each of j data blocks corresponding to the ith compressed data block of the n compressed data blocks, and recording a mapping relationship between the first index and the j data blocks, where i is a positive integer between 1 and n, and j is a positive integer between 1 and m, inclusive; A data compression method, wherein the first index is used to identify a storage location in a storage medium of each data block included in the j data blocks and attribute information included in each of the j data blocks.

2. Dividing the m data blocks into n data blocks and compressing each of the n data blocks using a predetermined compression algorithm to sequentially obtain n compressed data blocks includes: sequentially assigning all of the m data blocks to a first set in a predetermined order; 2. The data compression method of claim 1, further comprising: when a data capacity of the j data blocks of the first set is equal to a rated capacity of the first set, performing a compression operation on the j data blocks based on a specified compression threshold to obtain the i-th compressed data block.

3. Establishing a first index for each of j data blocks corresponding to an ith compressed data block of the n compressed data blocks includes:

3. The data compression method of claim 2, further comprising: establishing a first index for each of the j data blocks when a sum of a total data length of header data and compressed data of the i-th compressed data block and the specified compression threshold is less than or equal to a total data length of the j data blocks.

4. The attribute information is a first attribute indicating whether the compressed data block storage location into which the data block is compressed is pre-allocated; a second attribute used to indicate whether a data page of the data block is valid; and a third attribute used to indicate whether a data page of a data block is a first compressed page of a compressed data block of said data block; a fourth attribute used to indicate whether the data page of the data block is included in the compressed data pages of the two compressed blocks; a fifth attribute indicating whether a data page of a data block is a compressed page of a compressed data block obtained by compressing the data block; a sixth attribute used to represent an index address of a compressed data block in which a data page of the data block is located; and 4. The data compression method according to claim 1, wherein the seventh attribute is used to represent at least one of: when a data page of a data block belongs to a first compressed page of a compressed data block of the data block, an attribute value of the seventh attribute is an offset of the data block in the set corresponding to the compressed data block; and when the data page of the data block does not belong to the first compressed page of the compressed data block of the data block, the attribute value of the seventh attribute is used to represent a distance between the data page of the data block and the first compressed page of the compressed data block.

5. the attribute information includes the third attribute, and establishing a first index for each of j data blocks corresponding to an ith compressed data block of the n compressed data blocks includes: assigning 1 to an attribute value of the third attribute when the data page of each of the j data blocks is the first compressed page of the i-th compressed data block; or 5. The data compression method of claim 4, further comprising: substituting 0 for an attribute value of the third attribute when a data page of each of the j data blocks is not a first compressed page of the i-th compressed data block.

6. the attribute information includes the seventh attribute, and the data compression method includes: updating the attribute value of the seventh attribute to the offset of the data block in the set corresponding to the compressed data block when the attribute value of the third attribute is 1; or 5. The data compression method of claim 4, further comprising updating the attribute value of the seventh attribute to the distance between the data page of the data block and the first compressed page of the compressed data block when the attribute value of the third attribute is 0.

7. the attribute information includes the fourth attribute, and establishing a first index for each of j data blocks corresponding to an ith compressed data block of the n compressed data blocks comprises: assigning 1 to the attribute value of the fourth attribute when each data page of the j data blocks is included in the compressed data pages of two compressed blocks; or 5. The data compression method of claim 4, further comprising: substituting 0 for an attribute value of the fourth attribute when a data page of each of the j data blocks is not included in the compressed data pages of two compressed blocks.

8. the attribute information includes the second attribute, and establishing a first index for each of j data blocks corresponding to an ith compressed data block of the n compressed data blocks includes: assigning 1 to an attribute value of the second attribute when a data page of each of the j data blocks is valid; or 5. The data compression method according to claim 4, further comprising substituting 0 for an attribute value of said second attribute when a data page of each of said j data blocks is invalid.

9. Before dividing the m data blocks into n data blocks and compressing each of the n data blocks using a predetermined compression algorithm to sequentially obtain n compressed data blocks, obtaining a second set of data to be overwritten, the second set including p compressed data blocks, where p is a positive integer greater than or equal to 1; obtaining a compressed page of a first target compressed data block in the p compressed data blocks and q data blocks corresponding to the compressed page of the first target compressed data block, where q is a positive integer greater than or equal to 1; determining a location offset of a first target data block within the q data blocks; 4. The data compression method according to claim 1, further comprising: determining that a data page of the first target data block is a data page of the data to be overwritten.

10. 4. The data compression method according to claim 1, wherein the first index is used to identify a storage location of the i-th compressed data block on the storage medium and attribute information included in each of the j data blocks.

11. 1. A data compression device, comprising: a first acquisition unit configured to acquire m data blocks in a data area of ​​a read-write file system, where m is a positive integer greater than or equal to 3; a compression unit configured to divide the m data blocks into n data blocks and compress each of the n data blocks using a preset compression algorithm to sequentially obtain n compressed data blocks, wherein the compressed data blocks all have the same first capacity, the first capacity representing the number of bytes of compressed data that may be included in the compressed data block, and n is a positive integer equal to or greater than 2 and smaller than m; an updating unit configured to establish a first index for each of j data blocks corresponding to the ith compressed data block of the n compressed data blocks, and record a mapping relationship between the first index and the j data blocks, where i is a positive integer between 1 and n, and j is a positive integer between 1 and m, A data compression apparatus, wherein the first index is used to identify a storage location in a storage medium of each data block included in the j data blocks and attribute information included in each of the j data blocks.

12. The compression unit sequentially assigning all of the m data blocks to a first set in a predetermined order; 12. The data compression device according to claim 11, wherein when a data capacity of the j data blocks of the first set is equal to a rated capacity of the first set, a compression operation is performed on the j data blocks based on a specified compression threshold to obtain the i-th compressed data block.

13. The update unit:

13. The data compression apparatus of claim 12, further configured to establish a first index for each of the j data blocks when a sum of a total data length of header data and compressed data of the i-th compressed data block and the specified compression threshold is less than or equal to a total data length of the j data blocks.

14. The attribute information is a first attribute indicating whether the compressed data block storage location into which the data block is compressed is pre-allocated; a second attribute used to indicate whether a data page of the data block is valid; and a third attribute used to indicate whether a data page of a data block is a first compressed page of a compressed data block of said data block; a fourth attribute used to indicate whether the data page of the data block is included in the compressed data pages of the two compressed blocks; a fifth attribute indicating whether a data page of a data block is a compressed page of a compressed data block obtained by compressing the data block; a sixth attribute used to represent an index address of a compressed data block in which a data page of the data block is located; and 14. The data compression device according to claim 11, wherein the seventh attribute is used to represent at least one of: when a data page of a data block belongs to a first compressed page of a compressed data block of the data block, an attribute value of the seventh attribute is an offset of the data block in the set corresponding to the compressed data block; and when the data page of the data block does not belong to the first compressed page of the compressed data block of the data block, the attribute value of the seventh attribute is used to represent a distance between the data page of the data block and the first compressed page of the compressed data block.

15. The attribute information includes the third attribute, and the update unit assigning 1 to an attribute value of the third attribute when the data page of each of the j data blocks is the first compressed page of the i-th compressed data block; or 15. The data compression device of claim 14, further configured to assign 0 to an attribute value of the third attribute when a data page of each of the j data blocks is not a first compressed page of the i-th compressed data block.

16. The attribute information includes the seventh attribute, and the update unit updating the attribute value of the seventh attribute to the offset of the data block in the set corresponding to the compressed data block when the attribute value of the third attribute is 1; or 15. The data compression apparatus of claim 14, further configured to: when the attribute value of the third attribute is 0, update the attribute value of the seventh attribute to the distance between the data page of the data block and the first compressed page of the compressed data block.

17. The attribute information includes the fourth attribute, and the update unit assigning 1 to the attribute value of the fourth attribute when each data page of the j data blocks is included in the compressed data pages of two compressed blocks; or 15. The data compression device of claim 14, further configured to assign 0 to an attribute value of the fourth attribute when a data page of each of the j data blocks is not included in the compressed data pages of two compressed blocks.

18. The attribute information includes the second attribute, and the update unit assigning 1 to an attribute value of the second attribute when a data page of each of the j data blocks is valid; or 15. The data compression device according to claim 14, further configured to assign 0 to an attribute value of the second attribute when a data page of each of the j data blocks is invalid.

19. a second acquisition unit configured to acquire a second set of data to be overwritten, the second set including p compressed data blocks, where p is a positive integer greater than or equal to 1; a third acquisition unit configured to acquire a compressed page of a first target compressed data block in the p compressed data blocks and q data blocks corresponding to the compressed page of the first target compressed data block, where q is a positive integer greater than or equal to 1; a first determining unit configured to determine, in the q data blocks, a location offset of a first target data block within the q data blocks; and a second determination unit configured to determine that a data page of the first target data block is a data page of the data to be overwritten.

20. 14. The data compression device according to claim 11, wherein the first index is used to identify a storage location of the i-th compressed data block on the storage medium and attribute information included in each of the j data blocks.

21. A device configured to perform the data compression method according to any one of claims 1 to 3.

22. A computer-readable storage medium storing a computer program for causing a computer in an electronic device to execute the data compression method according to any one of claims 1 to 3.

23. A computer program for causing a processor to execute the data compression method according to any one of claims 1 to 3.

24. A chip system including one or more processors, wherein the one or more processors execute a computer program to perform the data compression method according to any one of claims 1 to 3.

Images