Data Compression Method and Flash Memory Device

Abstract

The embodiment of the present application relates to the field of storage device applications, and discloses a data compression method and a flash memory device. The data compression method obtains a write command sent by a host through a hardware compression module, divides the host data corresponding to the write command into a plurality of first data blocks of the same space, compresses each first data block to obtain a data chain composed of a plurality of second data blocks, generates a first command, and sends the first command to a firmware module. The firmware module writes the data chain included in the first command into a cache module, combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of a preset space; the firmware module combines a plurality of minimum units to obtain a plurality of write-down units, and writes the write-down units into the flash memory medium of the flash memory device. The present application can perform data compression on the flash memory device side, thereby reducing write amplification and improving the service life of the flash memory device.

Description

Technical Field

[0001] This application relates to the field of storage device applications, and particularly to a data compression method and a flash memory device. Background Art

[0002] A flash memory device, such as a Solid State Drive (SSD), is a hard disk made of a solid-state electronic storage chip array. The solid-state drive includes a control unit and a storage unit (FLASH storage chip or DRAM storage chip).

[0003] There are mainly two key indicators for a solid-state drive: lifespan and performance. The lifespan of a solid-state drive is mainly affected by the number of Nand erase / write cycles. How many times the Nand in a disk is erased / written is directly related to the total amount of written data. If the amount of data written in the same time is less, the Nand can be used for a longer time, and the disk can be used for a longer time. After the GC is started, the performance of the solid-state drive is mainly affected by the Write Amplifier (WA). The larger the write amplification, the worse the host write performance. The write amplification is directly affected by the size of the OverProvision (OP) in the disk. Therefore, it is usually necessary to compress the original data to reduce the amount of data written to the solid-state drive.

[0004] Currently, the data to be written is usually compressed by the hardware of the host and the compressed data is written into the solid-state drive. However, this compression method is completed on the host side and cannot substantially reduce the amount of data written to the solid-state drive, so the write performance of the solid-state drive cannot be improved.

[0005] Based on this, the existing technology urgently needs to be improved. Summary of the Invention

[0006] The embodiments of this application provide a data compression method and a flash memory device, which can perform data compression on the flash memory device side, thereby reducing the write amplification and increasing the service life of the flash memory device.

[0007] To solve the above technical problems, the embodiments of this application provide the following technical solutions:

[0008] In a first aspect, the embodiments of this application provide a data compression method, which is applied to a flash memory device. The flash memory device includes a hardware compression module, a firmware module, a cache module, and a flash memory medium. The method includes:

[0009] The hardware compression module obtains a write command sent by the host, where the write command corresponds to a piece of host data;

[0010] The hardware compression module divides the host data into a plurality of first data blocks with the same space;

[0011] The hardware compression module compresses each first data block to obtain a data chain composed of a number of second data blocks, generates a first command, and sends the first command to the firmware module, where the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block;

[0012] The firmware module writes the data chain into the cache module, and combines a number of second data blocks in the data chain in the cache module to generate a number of minimum units of a preset space;

[0013] The firmware module combines a number of minimum units to obtain a number of write-down units, and writes the write-down units into the flash memory medium of the flash memory device.

[0014] In some embodiments, combining a number of second data blocks in the data chain to generate a number of minimum units of a preset space includes:

[0015] Obtain the space corresponding to the current second data block;

[0016] If the space of the current second data block is equal to the space of the minimum unit, directly use the current second data block as the minimum unit;

[0017] If the space of the current second data block is smaller than the space of the minimum unit, search for the next second data block, and further determine whether the sum of the spaces of the current second data block and the next second data block is smaller than the space of the minimum unit. If so, merge the two as the current second data block, and so on until all second data blocks are traversed;

[0018] After traversing all second data blocks, if the sum of the spaces of a number of second data blocks is equal to the space of the minimum unit, use the combined number of second data blocks as the minimum unit; if there is no sum of the spaces of a number of second data blocks equal to the space of the minimum unit, supplement invalid data so that the sum of the spaces of a number of second data blocks and the space of the invalid data is equal to the space of the minimum unit.

[0019] In some embodiments, the method further includes:

[0020] If the space of the current second data block is smaller than the space of the minimum unit, determine whether the current search length exceeds a preset search length threshold, where the search length is the distance between the current second data block and the second data block where the search starts first;

[0021] If so, supplement invalid data so that the sum of the space of the current second data block and the space of the invalid data is equal to the space of the minimum unit;

[0022] If not, search for the next second data block.

[0023] In some embodiments, the method further includes:

[0024] When starting garbage collection for the flash device, the firmware module obtains source physical blocks in the flash medium of the flash device;

[0025] Reads a number of minimum units from the source physical blocks into the cache module;

[0026] Determines the positions and lengths of valid data in the number of minimum units according to the metadata of the flash device;

[0027] Merges the valid data in the number of minimum units to obtain the merged valid data;

[0028] Writes the merged valid data to the target physical block determined by garbage collection until the garbage collection exit condition is met.

[0029] In some embodiments, merging the valid data in the number of minimum units to obtain the merged valid data includes:

[0030] Obtains the space of the valid data of the current minimum unit;

[0031] Searches for the next minimum unit, and determines whether the sum of the space of the valid data of the current minimum unit and the space of the valid data of the next minimum unit is equal to a preset space, where the preset space is the space of one minimum unit;

[0032] If so, merges the valid data of the current minimum unit and the valid data of the next minimum unit to obtain a new minimum unit;

[0033] If not, merges the valid data of the current minimum unit and the valid data of the next minimum unit, and further searches for the next minimum unit, and so on, until all minimum units are traversed;

[0034] After traversing all minimum units, if the sum of the spaces of a number of minimum units is equal to the preset space, then the merged number of minimum units is used as a new minimum unit; if there is no sum of the spaces of a number of minimum units equal to the preset space, then invalid data is supplemented so that the sum of the spaces of the number of minimum units and the sum of the spaces of the invalid data is equal to the preset space.

[0035] In some embodiments, the method further includes:

[0036] After writing the compressed host data to the cache module, the cache module returns a write success message to the host.

[0037] In some embodiments, the method further includes:

[0038] After the lower brush unit writes to the flash memory medium, the firmware module deletes the smallest unit corresponding to the lower brush unit from the cache module.

[0039] In some embodiments, the method further includes:

[0040] When receiving a read command sent by the host, if the data corresponding to the read command is stored in the cache module, the firmware module reads the data corresponding to the read command from the cache module;

[0041] If the data corresponding to the read command is stored in the flash memory medium, the firmware module reads the data corresponding to the read command from the flash memory medium.

[0042] In some embodiments, the space of the smallest unit is in a multiple relationship with the space of the lower brush unit, the space of the first data block is equal to the mapping management granularity of the firmware of the flash device; the space of the lower brush unit is related to the size of the data page of the flash memory medium of the flash device.

[0043] In a second aspect, an embodiment of the present application provides a flash device, including: a hardware compression module, a firmware module, a cache module, and a flash memory medium, wherein,

[0044] The hardware compression module is configured to obtain a write command sent by the host, where the write command corresponds to a host data; and divide the host data into several first data blocks of the same space, compress each first data block to obtain a data chain composed of several second data blocks, so as to generate a first command, and send the first command to the firmware module, where the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block;

[0045] The firmware module is configured to write the data chain into the cache module, combine several second data blocks in the data chain in the cache module to generate multiple smallest units of a preset space; and combine multiple smallest units to obtain several lower brush units, and write the lower brush units into the flash memory medium of the flash device.

[0046] In a third aspect, an embodiment of the present application further provides a non-volatile computer-readable storage medium, and the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable the flash device to execute the command processing method as described in the first aspect.

[0047] The beneficial effects of the embodiments of this application are as follows: Different from the prior art, a data compression method provided by the embodiments of this application is applied to a flash memory device. The flash memory device includes a hardware compression module, a firmware module, a cache module, and a flash memory medium. The method includes: The hardware compression module obtains a write command sent by the host, where the write command corresponds to a host data; the hardware compression module divides the host data into several first data blocks of the same space; the hardware compression module compresses each first data block to obtain a data chain composed of several second data blocks, so as to generate a first command and send the first command to the firmware module, where the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block; the firmware module writes the data chain into the cache module, combines several second data blocks in the data chain in the cache module to generate multiple minimum units of a preset space; the firmware module combines multiple minimum units to obtain several write-down units and writes the write-down units into the flash memory medium of the flash memory device.

[0048] By the hardware compression module obtaining the write command sent by the host, dividing the host data corresponding to the write command into several first data blocks of the same space, compressing each first data block to obtain a data chain composed of several second data blocks, so as to generate a first command and sending the first command to the firmware module, and the firmware module writing the data chain included in the first command into the cache module, combining several second data blocks in the data chain in the cache module to generate multiple minimum units of a preset space; the firmware module combines multiple minimum units to obtain several write-down units and writes the write-down units into the flash memory medium of the flash memory device, this application can perform data compression on the flash memory device side, thereby reducing write amplification and improving the service life of the flash memory device. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] One or more embodiments are illustrated by way of example in the accompanying drawings, and these illustrative descriptions do not limit the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, unless otherwise stated, and the drawings in the drawings do not constitute a proportional limitation.

[0050] Figure 1 is a schematic structural diagram of a flash memory device provided by the embodiments of this application;

[0051] Figure 2 is a schematic flowchart of a data compression method provided by the embodiments of this application;

[0052] Figure 3 is a schematic diagram of processing a write command of a host provided by the embodiments of this application;

[0053] Figure 4It is a schematic diagram of data compression provided by an embodiment of the present application;

[0054] Figure 5 It is a schematic structural diagram of a firmware module provided by an embodiment of the present application;

[0055] Figure 6 It is a schematic flowchart of generating a minimum unit provided by an embodiment of the present application;

[0056] Figure 7 It is a schematic diagram of combining a second data block to obtain a minimum unit provided by an embodiment of the present application;

[0057] Figure 8 It is a schematic diagram of supplementing invalid data provided by an embodiment of the present application;

[0058] Figure 9 It is a schematic diagram of combining minimum units into a download unit provided by an embodiment of the present application;

[0059] Figure 10 It is a schematic diagram of garbage collection provided by an embodiment of the present application;

[0060] Figure 11 It is a schematic flowchart of garbage collection provided by an embodiment of the present application;

[0061] Figure 12 It is a schematic diagram of garbage collection provided by an embodiment of the present application;

[0062] Figure 13 It is a schematic diagram of data integration during garbage collection provided by an embodiment of the present application;

[0063] Figure 14 It is a schematic overall diagram of a data compression method provided by an embodiment of the present application;

[0064] Figure 15 It is a timing diagram of a data compression method provided by an embodiment of the present application;

[0065] Figure 16 It is a schematic structural diagram of a flash memory device provided by an embodiment of the present application. Detailed implementation manners

[0066] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts belong to the scope of protection of the present application.

[0067] It should be noted that if there is no conflict, the various features in the embodiments of the present application can be combined with each other, and all are within the protection scope of the present application. In addition, although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the flowchart. Furthermore, the terms "first", "second", "third", etc. adopted in the present application do not limit the data and the execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0068] Before the present application is described in detail, the nouns and terms involved in the embodiments of the present application are described. The nouns and terms involved in the embodiments of the present application are applicable to the following explanations:

[0069] Write Amplification (WA) refers to the ratio of the amount of data written to the flash memory to the amount of data written by the user. Due to the existence of garbage collection (GC), when the user wants to write a certain amount of data, the flash memory device needs to move some additional data to make room for writing this data, that is, additional writes, resulting in more data being written to the flash memory medium by the flash memory device than the amount of data actually written by the user to the flash memory device, leading to an increase in write amplification.

[0070] The technical solution of the present application will be specifically described below in conjunction with the accompanying drawings of the specification:

[0071] Please refer to Figure 1 , Figure 1 which is a schematic structural diagram of a flash memory device provided by an embodiment of the present application;

[0072] As Figure 1 shown, the flash memory device 100 includes a flash memory medium 110 and a controller 120 connected to the flash memory medium 110. Among them, the flash memory device 100 is communicatively connected to the host 200 in a wired or wireless manner to achieve data interaction.

[0073] The flash memory medium 110, as the storage medium of the flash memory device 100, is also called flash memory, Flash, Flash memory, or Flash chip. It belongs to a type of storage device and is a non-volatile memory that can permanently store data without current supply. Its storage characteristics are equivalent to those of a hard disk, making the flash memory medium 110 the basis for the storage medium of various portable digital devices.

[0074] The controller 120 includes a data converter 121, a processor 122, a buffer 123, a flash memory controller 124, and an interface 125.

[0075] A data converter 121 is respectively connected to a processor 122 and a flash memory controller 124. The data converter 121 is used to convert binary data into hexadecimal data and convert hexadecimal data into binary data. Specifically, when the flash memory controller 124 writes data to the flash memory medium 110, the binary data to be written is converted into hexadecimal data through the data converter 121 and then written to the flash memory medium 110. When the flash memory controller 124 reads data from the flash memory medium 110, the hexadecimal data stored in the flash memory medium 110 is converted into binary data through the data converter 121, and then the converted data is read from the binary data page register. Among them, the data converter 121 may include a binary data register and a hexadecimal data register. The binary data register can be used to store the data after being converted from hexadecimal to binary, and the hexadecimal data register can be used to store the data after being converted from binary to hexadecimal.

[0076] The processor 122 is respectively connected to the data converter 121, the buffer 123, the flash memory controller 124, and the interface 125. Among them, the processor 122 can be connected to the data converter 121, the buffer 123, the flash memory controller 124, and the interface 125 through a bus or other means. The processor is used to run the non-volatile software programs, instructions, and modules stored in the buffer 123, so as to implement any method embodiment of the present application.

[0077] The buffer 123 is mainly used to cache the read / write instructions sent by the host 200 and the read data or write data obtained from the flash memory medium 110 according to the read / write instructions sent by the host 200.

[0078] The flash memory controller 124 is connected to the flash memory medium 110, the data converter 121, the processor 122, and the buffer 123. It is used to access the backend flash memory medium 110 and manage various parameters and data I / O of the flash memory medium 110; or, it is used to provide access interfaces and protocols to implement the corresponding SAS / SATA target protocol end or NVMe protocol end, obtain the I / O instructions issued by the host 200 and decode and generate internal private data results waiting to be executed; or, it is used to be responsible for the core processing of the flash translation layer (FTL).

[0079] Interface 125 is connected to host 200, data converter 121, processor 122, and buffer 123, and is used to receive data sent by host 200 or receive data sent by processor 122, so as to implement data transmission between host 200 and processor 122. Interface 125 can be a SATA-2 interface, SATA-3 interface, SAS interface, MSATA interface, PCI-E interface, NGFF interface, CFast interface, SFF-8639 interface, and M.2 NVME / SATA protocol.

[0080] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a data compression method provided by an embodiment of the present application;

[0081] Among them, this data compression method is applied to a flash memory device, which includes a hardware compression module, a firmware module, a cache module, and a flash memory medium.

[0082] As Figure 2 shown, this data compression method includes:

[0083] Step S201: The hardware compression module obtains a write command sent by the host, where the write command corresponds to a host data;

[0084] Specifically, please refer to Figure 3 , Figure 3 which is a schematic diagram of processing a write command of the host provided by an embodiment of the present application;

[0085] As Figure 3 shown, when the write command of the host passes through the NVMe module, it is converted by the NVMe module into an NVMe command that conforms to the NVMe protocol, and then the NVMe command is forwarded to the hardware compression module.

[0086] Step S202: The hardware compression module divides the host data into several first data blocks of the same space;

[0087] As Figure 3 shown, the hardware compression module includes a hardware cache unit, a data splitting unit, a data compression unit, and a data output unit. Among them, the hardware cache unit is connected to the data splitting unit, the data splitting unit is connected to the data compression unit, and the data compression unit is connected to the data output unit.

[0088] Among them, the hardware cache unit is connected to the NVMe module and is used to cache the NVMe command sent by the NVMe module, where the NVMe command includes host data. When the hardware compression module receives the NVMe command, the host data included in the NVMe command is stored by the hardware cache unit.

[0089] Among them, a data splitting unit is connected to the hardware cache unit. The data splitting unit is used to split the host data cached in the hardware cache unit. The data splitting unit divides the host data into several first data blocks of the same space, that is, the space size of each first data block is equal. For example, the space size of each first data block is 4KB, 8KB, or 16KB. Preferably, the space size of each first data block in the embodiment of the present application is 4KB.

[0090] Among them, a data compression unit is connected to the data splitting unit. The data compression unit is used to compress the first data blocks obtained after the data splitting unit performs splitting to obtain compressed data.

[0091] Among them, a data output unit is connected to the data compression unit. The data output unit is used to output the data compressed by the data compression unit. Specifically, it combines the compressed data information, host data, and the original NVMe command to generate a first command and sends the first command to the firmware module.

[0092] In the embodiment of the present application, the hardware cache unit includes a memory, such as a Dynamic Random Access Memory (DRAM); the data splitting unit includes a processor, such as a microprocessor; the data compression unit includes a hardware compression engine; and the data output unit includes an IO interface.

[0093] Step S203: The hardware compression module compresses each first data block to obtain a data chain composed of several second data blocks, generates a first command, and sends the first command to the firmware module, where the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block;

[0094] Specifically, the data compression unit in the hardware compression module compresses each first data block to obtain several second data blocks, and combines the several second data blocks to obtain a data chain composed of several second data blocks.

[0095] Please refer to Figure 4 , Figure 4 which is a schematic diagram of data compression provided by the embodiment of the present application;

[0096] As Figure 4As shown, assume that the size of the host data corresponding to the host write command is 16 KB. The 16 KB of host data is divided into 4 first data blocks of 4 KB each, and each 4 KB first data block is compressed to obtain a second data block corresponding to each first data block. That is, each first data block corresponds to a second data block one by one. For example, the 4 first data blocks of 4 KB are respectively compressed into second data blocks of 1 KB, 3 KB, 2 KB, and 3 KB, and the 4 second data blocks are combined to generate a data chain, and a first command is generated from the data chain. The first command is a new write command, and the first command includes the data chain. It can be understood that the length of the compressed data is determined by the compression algorithm. Different compression algorithms or different data patterns can result in different lengths of the compressed data. Due to compression, the length of each second data block is less than the length of the first data block, that is, the space of each second data block is less than the space of the first data block.

[0097] In the embodiment of the present application, the first command includes a set of host data, the original NVMe command, and the compressed data information. The original NVMe command is the NVMe command generated after the host write command passes through the NVMe module. The NVMe command includes the starting logical address (startlba) and the data length (len); the compressed data information includes the length of the compressed data for each 4K data unit.

[0098] Step S204: The firmware module writes the data chain into the cache module, and combines several second data blocks in the data chain in the cache module to generate multiple minimum units of the preset space;

[0099] Specifically, after the hardware compression module generates the first command, it sends the first command to the firmware module, and the firmware module writes the data chain into the cache module. The cache module includes the cache (Cache) of the flash device.

[0100] Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of a firmware module provided by an embodiment of the present application;

[0101] Among them, the firmware module is used to connect the host (HOST) and the flash array (NAND) to implement the processing of data IO.

[0102] As Figure 5 shown, the firmware module 50, which is connected to the host 200 and the hardware module 60, includes:

[0103] The front-end module 501, namely the (Front End, FE), is used to obtain host commands to generate IO operations. Among them, the front-end module is also responsible for communication protocols with the host, parsing host commands, solid-state drive commands, and other operations;

[0104] The flash algorithm module 502, namely the flash translation layer (FTL), is connected to the data processing module and is used to perform mapping processing on IO operations to determine the flash array to be issued. Among them, the flash algorithm module (Flash Translation Layer, FTL) sends IO operations to the back-end module (Back End, BE) of the solid-state drive controller, so that the back-end module of the solid-state drive controller receives the IO operations sent by the flash algorithm module;

[0105] The back-end module 503, namely the (Back End, BE), is connected to the flash algorithm module and is used to receive the IO operations sent by the flash algorithm module to control the hardware module to perform read / write / erase operations on the flash array;

[0106] In the embodiment of this application, the hardware module 60, namely (HW Op Nand Mode), refers to the module that operates the FLASH memory. It is connected to the back-end module 503 and is controlled by the back-end module 503 to operate the FLASH memory. For example: according to the IO operation, it performs operations on the corresponding flash array or flash medium, that is, completes the operation processing of data to the FLASH memory, and the operations include read operation, write operation, or erase operation.

[0107] Among them, after the front-end module obtains the host command, it processes it to generate an IO operation, and sequentially passes through the flash algorithm module, the back-end module, and the hardware module to operate on the flash array. For example: when the host reads data, the host (Host) sends a host command to the flash device (device), and the front-end module (FE) of the flash device receives the host command, processes it, and distributes it to the flash algorithm module (FTL). After receiving it, the flash algorithm module performs logical-to-physical conversion processing, and then sends a request for NAND read operation to the back-end module (BE). After receiving the read operation, the back-end module sends a hardware instruction to the hardware module (HW Op Nand Mode), and the hardware module performs NAND processing in parallel.

[0108] In the embodiment of the present application, the first command sent by the hardware compression module is received by the front-end module. The front-end module forwards the first command to the flash algorithm module. The flash algorithm module parses the data chain in the first command and writes the data chain into the cache module. The flash algorithm module combines several second data blocks in the cache module to generate the minimum units of multiple preset spaces.

[0109] Specifically, combining several second data blocks in the data chain to generate the minimum units of several preset spaces includes:

[0110] Obtain the space corresponding to the current second data block;

[0111] If the space of the current second data block is equal to the space of the minimum unit, directly use the current second data block as the minimum unit;

[0112] If the space of the current second data block is less than the space of the minimum unit, search for the next second data block, and further determine whether the sum of the spaces of the current second data block and the next second data block is less than the space of the minimum unit. If so, merge the two as the current second data block, and so on until all the second data blocks are traversed;

[0113] After traversing all the second data blocks, if the sum of the spaces of several second data blocks is equal to the space of the minimum unit, use the merged several second data blocks as the minimum unit; if there is no sum of the spaces of several second data blocks equal to the space of the minimum unit, supplement invalid data so that the sum of the spaces of several second data blocks and the space of the invalid data is equal to the space of the minimum unit.

[0114] Please refer to Figure 6 , Figure 6 which is a schematic flowchart of generating the minimum unit provided by the embodiment of the present application;

[0115] As Figure 6 shown, the process of generating the minimum unit includes:

[0116] Step S601: Obtain the first second data block and the space size S1 of the second data block;

[0117] Step S602: Determine whether S1 is equal to the space size of the minimum unit;

[0118] Specifically, if S1 = the space size of the minimum unit, that is, the space of the current second data block is equal to the space of the minimum unit, directly use the current second data block as the minimum unit and end the search;

[0119] If not, that is, if the space of the current second data block is smaller than the space of the minimum unit, then go to step S803;

[0120] Step S603: Determine whether the current search length exceeds a preset search length threshold;

[0121] Specifically, if the space of the current second data block is smaller than the space of the minimum unit, determine whether the current search length exceeds a preset search length threshold, where the search length is the distance between the current second data block and the second data block where the search starts first;

[0122] If so, supplement invalid data so that the sum of the space of the current second data block and the space of the invalid data is equal to the space of the minimum unit;

[0123] If not, then search for the next second data block.

[0124] Step S604: Traverse other second data blocks in sequence and obtain the size Sx of the currently found second data block;

[0125] Step S605: Determine whether S1 + Sx ≤ the space size of the minimum unit;

[0126] Specifically, search for the next second data block, and further determine whether the sum of the spaces of the current second data block and the next second data block is smaller than the space of the minimum unit. If so, go to step 606; if not, return to step S603;

[0127] Step S606: Combine S1 and Sx into a new data block, where S1 = S1 + Sx;

[0128] If S1 + Sx ≤ the space size of the minimum unit, then combine the two as the current second data block, and so on until all second data blocks are traversed;

[0129] Step S607: Supplement invalid data to form the space size of the minimum unit;

[0130] Specifically, if the current search length exceeds the preset search length or search distance, supplement invalid data to form the space size of the minimum unit, and use the data block after supplementing the invalid data as a minimum unit.

[0131] Among them, the invalid data, that is, Dummy data, is the invalid data supplemented by the firmware. For example: all 0x0 or all 0xF data. For example, if nand requires 16KB for one write, the firmware must transmit 16KB of data length. Supplementing the invalid data Dummy is to apply for a section of data initialized to all 0x0 or 0xF to supplement the pieced - together data.

[0132] After traversing all the second data blocks, if the sum of the spaces of several second data blocks is equal to the space of the minimum unit, then the combined several second data blocks are used as the minimum unit; if there are no several second data blocks whose space sum is equal to the space of the minimum unit, then invalid data is supplemented so that the sum of the spaces of the several second data blocks and the space of the invalid data is equal to the space of the minimum unit.

[0133] End.

[0134] Please refer to Figure 7 , Figure 7 which is a schematic diagram of combining second data blocks to obtain a minimum unit provided by an embodiment of the present application;

[0135] As Figure 7 shown, the host data corresponding to the write commands of two hosts are 16KB and 8KB respectively, which are respectively split into 3 first data blocks of 4KB and 2 first data blocks of 4KB, and each first data block is compressed. The sizes of the second data blocks obtained by compression are 1KB, 3KB, 2KB, 3KB, 2KB, and 1KB respectively. At this time, several second data blocks are combined to obtain a minimum unit, where the space size of the minimum unit is preset, and the space size of the minimum unit is related to the mapping management granularity in the firmware, that is, the space size of the minimum unit is equal to the mapping management granularity in the firmware. For example: if the mapping management granularity is 4KB, 8KB, or 16KB, then the preset space of the minimum unit is 4KB, 8KB, or 16KB, which is not limited here. Preferably, the preset space of the minimum unit in the embodiment of the present application is 4KB.

[0136] Please refer to Figure 8 , Figure 8 which is a schematic diagram of supplementing invalid data provided by an embodiment of the present application;

[0137] As Figure 8 shown, when several second data blocks cannot be combined to obtain the space size of the minimum unit, that is, during the data piecing process, it may also occur that a 4K cannot be pieced together. At this time, invalid data (dummy) needs to be supplemented. For example, after all data is compressed, it is all 3KB in the Cache, and no arbitrary number of them can be combined into a 4KB. Then the final data needs to be supplemented with invalid data to make up a size of 4KB.

[0138] Step S205: The firmware module combines several minimum units to obtain several download units, and writes the download units into the flash medium of the flash device.

[0139] Specifically, the firmware module combines several minimum units in the cache module to obtain several download units, and writes the several download units into the flash medium of the flash device.

[0140] In the embodiment of the present application, the space size of the lower brush unit is greater than or equal to the space size of the minimum unit, and the space size of the lower brush unit is an integer multiple of the space size of the minimum unit. For example: if the space size of the lower brush unit is equal to the space size of the minimum unit, each minimum unit is directly written into the flash memory medium as a lower brush unit; if the space size of the lower brush unit is greater than the space size of the minimum unit, and the space size of the lower brush unit is an integer multiple of the space size of the minimum unit, for example: the space size of the lower brush unit = proportional coefficient * the space size of the minimum unit, then the proportional coefficient of minimum units are combined to obtain a lower brush unit, and so on, to obtain several lower brush units, and the lower brush units are written into the flash memory medium.

[0141] Please refer to Figure 9 , Figure 9 which is a schematic diagram of combining minimum units into lower brush units provided by the embodiment of the present application;

[0142] As Figure 9 shown, both of the two minimum units are 4KB, the size of the data chain composed of the two minimum units is 8KB, and the size of one lower brush unit is 8KB, so the two minimum units are combined to form a lower brush unit.

[0143] In the embodiment of the present application, the space size of the lower brush unit is related to the size of the data page of the flash memory medium.

[0144] It can be understood that in a solid-state drive, due to the read / write principle of NAND FLASH, the data written cannot be directly rewritten at its storage physical location. It is necessary to first perform an erase operation in units of physical blocks (Blocks), and then write operations are allowed on the data pages (Pages) included in the block. This makes the SSD have a key weakness of very inefficient in-place update operations. For this reason, SSD systems usually adopt out-of-place update operations, that is, the data is copied to a buffer for update, and after the update, it is stored at a new location, and the old location containing the original data is marked as "garbage", and all physical pages (Pages) included in a physical block (Block) must be re-converted into available space through garbage collection (GC).

[0145] Garbage Collection (GC) is to copy the data of valid pages in a physical block in NAND FLASH to another free block (unused physical block), and then completely erase this physical block. Among them, when there is data to be modified in any physical page in NAND FLASH, this physical page will be marked as an invalid page, and the data in this invalid page will become invalid data, and page space will be applied on other physical blocks for data writing. And the physical pages that are not marked as invalid pages and have been used are called valid pages. Among them, the Program / Erase Cycles (PE) of Solid State Drives (SSD), simply referred to as the number of erasures, determines the lifespan of the solid state drive.

[0146] Please refer to Figure 10 , Figure 10 which is a schematic diagram of garbage collection provided by an embodiment of the present application;

[0147] As Figure 10 shown, the process of this garbage collection is to copy the data of valid pages in a physical block to another free block (unused physical block), and then completely erase this physical block. For example: transfer the valid data of physical block A (BlockA) to physical block B (BlockB). It can be understood that there may be invalid data and valid data on a physical block (Block). By reading out the valid data in a physical block (Block) and writing it to another physical block (Block), the purpose of releasing the space of the entire source physical block can be achieved.

[0148] Please refer to again Figure 11 , Figure 11 which is a schematic flowchart of garbage collection provided by an embodiment of the present application;

[0149] As Figure 11 shown, the process of this garbage collection includes:

[0150] Step S111: Start garbage collection;

[0151] Step S112: Select the source physical block that needs garbage collection;

[0152] Specifically, after garbage collection is performed, the garbage collection module of the flash memory device selects the source physical block that needs garbage collection according to the start condition of garbage collection. Among them, the start condition includes that the proportion of valid data is less than a preset proportion threshold.

[0153] In the embodiment of the present application, the garbage collection module belongs to the hardware or firmware of the flash memory device, which is not limited herein.

[0154] Step S113: Read several minimum units from the flash medium into the cache module;

[0155] Step S114: Determine the positions and lengths of the valid data in the several minimum units according to the metadata of the flash device;

[0156] Step S115: Form a linked list with the addresses and lengths of the valid data in the several minimum units;

[0157] It can be understood that in a minimum unit stored on Nand, for example, the minimum unit is 4KB. In the 4KB space of the minimum unit, it may be the result of compressing the 4K data of multiple original hosts. So actually, the 4K data of multiple hosts corresponds to one 4K data on Nand. For example, LBA0, LBA1, LBA2, LBA3 are finally compressed and stored in a 4KB space.

[0158] When the host writes LBA0 again, LBA0 may be compressed with LBA11, LBA12, LBA13 into another 4K data. So actually, a part of the original 4K data is invalid data. If the GC relocates the entire 4K, it also relocates a part of the invalid data, increasing the write amplification.

[0159] It should be noted that if only the 4K data of one host in each 4K data relocated by the GC on NAND is still valid, and other data is compressed with other data when the host rewrites. If the GC directly relocates the 4K data on NAND, it will gradually "degrade" the compression effect when the whole data is written. Therefore, in the process of garbage collection, this solution reorganizes the data again, that is, in each 4K data read out, only the valid data is found, and the valid data in each 4K is re - spliced into a new 4K, and then written into another physical block (Block), so as to reduce the write amplification.

[0160] Step S116: Traverse the linked list to piece together a new minimum unit;

[0161] Specifically, the valid data in the several minimum units is merged to obtain the merged valid data, where the size of the merged valid data is granularity of the space size of the minimum unit; if it cannot be pieced together into a minimum unit, invalid data is supplemented to piece together the space size of the minimum unit.

[0162] It should be noted that the firmware of the flash device includes metadata, and the metadata is used to maintain the valid data in each 4K data on NAND, that is, the metadata is used to determine which position data is valid data.

[0163] Step S117: Write the pieced-together minimum unit into the target physical block;

[0164] Specifically, writing each pieced-together minimum unit into the target physical block is equivalent to writing the valid data after merging into the target physical block determined by garbage collection.

[0165] Please refer to Figure 12 , Figure 12 which is a schematic diagram of garbage collection provided by an embodiment of the present application;

[0166] Among them, the garbage collection module includes a read data module and a write data module.

[0167] Specifically, as Figure 12 shown, the read data module is used to select the source physical block that needs garbage collection from the flash memory medium (NAND), and the write data module is used to write the pieced-together minimum unit into the target physical block in the flash memory medium (NAND).

[0168] Please refer to Figure 13 , Figure 13 which is a schematic diagram of data integration during garbage collection provided by an embodiment of the present application;

[0169] It can be understood that during the garbage collection process, only the valid data in the source physical block needs to be moved. Therefore, determine the positions and lengths of the valid data in several minimum units in the source physical block, and combine the valid data in the several minimum units to splice into a new minimum unit. As Figure 13 shown, the spatial size of the minimum unit is 4KB. Combine the valid data to obtain a new minimum unit, and the spatial size of the combined minimum unit is also 4KB. Among them, the red part in the figure is the invalid data that has been rewritten, and the green part is the valid data. The two 4K in BlockA read by GC, the total actual valid data is only 3K, then only extract these 3K data and integrate them into the buffer to be written to BlockB. Thus, the write volume of GC is reduced, and the result of compression during data writing is also preserved.

[0170] Step S118: Continue the read and write data process loop until the garbage collection exit condition is met;

[0171] Step S119: End the garbage collection;

[0172] In the embodiments of the present application, the solution of combining dedicated hardware compression with firmware logic can meet the bandwidth performance requirements of the SSD. Moreover, through the compression splicing data and GC splicing data algorithms, the write amplification of the flash device is reduced, indirectly improving the service life of the flash device. Furthermore, since the solution in the present application is a compression solution that is completely imperceptible to the host, when the host reads and writes the flash device normally, the flash device automatically completes the data decompression and compression functions, thereby improving the stability of the interaction between the flash device and the host.

[0173] Please refer to Figure 14 , Figure 14 which is an overall schematic diagram of a data compression method provided by an embodiment of the present application;

[0174] As Figure 14 shown, the host (Host) sends a host command (HostCmd), and the host command includes original data (Originaldata), that is, host data. When the write command of the host passes through the NVMe module, the NVMe module converts it into an NVMe command that conforms to the NVMe protocol, and then forwards the NVMe command to the hardware compression module (HWCompressEngine). Among them, both the NVMe module and the hardware compression module belong to the hardware modules of the flash device.

[0175] Among them, the hardware compression module divides the host data into several first data blocks of the same space, and compresses each first data block to obtain a data chain composed of several second data blocks, so as to generate a first command, and sends the first command to the firmware module, where the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block;

[0176] Among them, the firmware module writes the data chain into the cache module, combines several second data blocks in the data chain in the cache module to generate several minimum units, and then combines several minimum units to obtain several cache flush units (Cacheflushunit), so as to write the several cache flush units into the flash medium of the flash device.

[0177] In the embodiments of the present application, the host command (HostCmd) includes: the identifier of the host command (Cmd ID), the logical block address (Logical Block Address, Lba), and the address length to be read (Len). Among them, relative to the host, the logical block address is generally 512Byte, 4k, etc.

[0178] Please refer to Figure 15 , Figure 15 which is a timing diagram of a data compression method provided by an embodiment of the present application;

[0179] As Figure 15 shown, the timing of the data compression method includes:

[0180] Step S151: The host writes data to the hardware compression module;

[0181] Step S152: The hardware compression module compresses the data written by the host to obtain the compressed host data;

[0182] Step S153: The hardware compression module writes the compressed host data to the cache module;

[0183] Step S154: After the hardware compression module writes the compressed host data to the cache module, the cache module returns a write success message to the host.

[0184] Among them, the write success message is sent by the firmware module to the host through the cache module, or the write success message is directly sent by the cache module.

[0185] Step S155: The cache module combines the compressed data;

[0186] Step S156: Write to the flash memory;

[0187] Specifically, the firmware module writes the download unit obtained after combining the compressed data to the flash memory.

[0188] Step S157: The write to the flash memory is successful;

[0189] Specifically, the firmware module returns a write success message to the cache module for writing to the flash memory.

[0190] Step S158: Delete the data in the cache module;

[0191] Specifically, after the download unit is written to the flash memory medium, the firmware module deletes the smallest unit corresponding to the download unit from the cache module. Specifically, when the firmware module successfully writes the download unit to the flash memory (NAND), it notifies the cache module so that the cache module deletes the corresponding data stored in its memory, where the memory of the cache module includes a cache register.

[0192] In some embodiments, the method further includes:

[0193] When receiving a read command sent by the host, if the data corresponding to the read command is stored in the cache module, the firmware module reads the data corresponding to the read command from the cache module;

[0194] If the data corresponding to the read command is stored in the flash memory medium, the firmware module reads the data corresponding to the read command from the flash memory medium.

[0195] In some embodiments, the space of the minimum unit is in a multiple relationship with the space of the lower brush unit, and the space of the first data block is equal to the mapping management granularity of the firmware of the flash memory device; the space of the lower brush unit is related to the size of the data page of the flash memory medium of the flash memory device.

[0196] In the embodiments of the present application, a data compression method is provided, which is applied to a flash memory device. The flash memory device includes a hardware compression module, a firmware module, a cache module, and a flash memory medium. The method includes: the hardware compression module obtains a write command sent by a host, where the write command corresponds to a host data; the hardware compression module divides the host data into a plurality of first data blocks with the same space; the hardware compression module compresses each first data block to obtain a data chain composed of a plurality of second data blocks, so as to generate a first command, and sends the first command to the firmware module, where the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block; the firmware module writes the data chain into the cache module, combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units with a preset space; the firmware module combines a plurality of minimum units to obtain a plurality of lower brush units, and writes the lower brush units into the flash memory medium of the flash memory device.

[0197] By the hardware compression module obtaining the write command sent by the host, dividing the host data corresponding to the write command into a plurality of first data blocks with the same space, compressing each first data block to obtain a data chain composed of a plurality of second data blocks, generating a first command, and sending the first command to the firmware module, and the firmware module writing the data chain included in the first command into the cache module, combining a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units with a preset space; the firmware module combines a plurality of minimum units to obtain a plurality of lower brush units, and writes the lower brush units into the flash memory medium of the flash memory device, the present application can perform data compression on the flash memory device side, thereby reducing write amplification and improving the service life of the flash memory device.

[0198] Please refer to Figure 16 , Figure 16 which is a schematic structural diagram of a flash memory device provided by the embodiments of the present application;

[0199] As Figure 16 shown, the flash memory device 160 includes: a hardware compression module 161, a cache module 162, a firmware module 163, and a flash memory medium 164, where

[0200] A hardware compression module 161, configured to obtain a write command sent by a host, where the write command corresponds to a host data; and divide the host data into a plurality of first data blocks of the same space, compress each first data block to obtain a data chain composed of a plurality of second data blocks, so as to generate a first command, and send the first command to a firmware module 163, where the first command includes the data chain, and the space of each second data block is smaller than that of the first data block.

[0201] A cache module 162, configured to store the data written by the hardware compression module 161, or store the data written by the firmware module 163, and combine the data.

[0202] A firmware module 163, configured to write the data chain into the cache module, combine a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of a preset space; and combine a plurality of minimum units to obtain a plurality of write-down units, and write the write-down units into a flash memory medium 164 of a flash memory device 160.

[0203] The flash memory medium 164 is configured to store the data written by the firmware module.

[0204] In an embodiment of the present application, the hardware compression module obtains a write command sent by a host, divides the host data corresponding to the write command into a plurality of first data blocks of the same space, compresses each first data block to obtain a data chain composed of a plurality of second data blocks, so as to generate a first command, and sends the first command to the firmware module. The firmware module writes the data chain included in the first command into the cache module, combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of a preset space; the firmware module combines a plurality of minimum units to obtain a plurality of write-down units, and writes the write-down units into the flash memory medium of the flash memory device. The present application can perform data compression on the flash memory device side, thereby reducing write amplification and extending the service life of the flash memory device.

[0205] An embodiment of the present application further provides a non-volatile computer storage medium. The computer storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors. For example, the one or more processors can execute the data compression method in any of the above method embodiments, for example, execute the data compression method in any of the above method embodiments, for example, execute each of the steps described above.

[0206] The device or equipment embodiments described above are merely illustrative. The unit modules described as separate components may or may not be physically separated, and the components shown as module units may or may not be physical units, that is, they may be located in one place or distributed to multiple network module units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0207] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, and the steps can be implemented in any order, and there are many other changes in different aspects of the present application as above. For the sake of brevity, they are not provided in detail; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A data compression method, characterized in that, Applied to a flash memory device, the flash memory device includes a hardware compression module, a firmware module, a cache module, and a flash memory medium. The method includes: The hardware compression module obtains a write command sent by the host, where the write command corresponds to a host data. The hardware compression module divides the host data into a plurality of first data blocks of the same space. The hardware compression module compresses each of the first data blocks to obtain a data chain composed of a plurality of second data blocks, generates a first command, and sends the first command to the firmware module, where the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block. The firmware module writes the data chain into the cache module, and combines a plurality of second data blocks in the data chain in the cache module to generate a plurality of minimum units of a preset space. The firmware module combines a plurality of the minimum units to obtain a plurality of write-down units, and writes the write-down units into the flash memory medium of the flash memory device. The combining a plurality of second data blocks in the data chain to generate a plurality of minimum units of a preset space includes: Obtaining the space corresponding to the current second data block. If the space of the current second data block is equal to the space of the minimum unit, directly use the current second data block as the minimum unit. If the space of the current second data block is smaller than the space of the minimum unit, search for the next second data block, and further determine whether the sum of the spaces of the current second data block and the next second data block is smaller than the space of the minimum unit. If so, merge the two as the current second data block, and so on until all second data blocks are traversed. After traversing all second data blocks, if the sum of the spaces of a plurality of second data blocks is equal to the space of the minimum unit, use the merged plurality of second data blocks as the minimum unit; if there is no sum of the spaces of a plurality of second data blocks equal to the space of the minimum unit, supplement invalid data so that the sum of the spaces of a plurality of second data blocks and the space of the invalid data is equal to the space of the minimum unit.

2. The method according to claim 1, characterized in that The method further includes: If the space of the current second data block is smaller than the space of the minimum unit, determine whether the current search length exceeds a preset search length threshold, where the search length is the distance between the current second data block and the second data block where the search starts first. If so, supplement invalid data so that the sum of the space of the current second data block and the space of the invalid data is equal to the space of the minimum unit. If not, search for the next second data block.

3. The method according to claim 1, wherein The method further includes: When starting garbage collection for the flash memory device, the firmware module obtains source physical blocks in the flash memory medium of the flash memory device. Reads a plurality of minimum units from the source physical block into the cache module. Determines the positions and lengths of valid data in a plurality of the minimum units according to the metadata of the flash memory device. Combines the valid data in the plurality of minimum units to obtain the combined valid data. Write the merged valid data into the target physical block determined by garbage collection until the garbage collection exit condition is met.

4. The method according to claim 3, characterized in that The merging of the valid data in the several smallest units to obtain the merged valid data includes: Obtain the space of the valid data of the current smallest unit; Search for the next smallest unit and determine whether the sum of the space of the valid data of the current smallest unit and the space of the valid data of the next smallest unit is equal to the preset space, where the preset space is the space of one smallest unit; If so, merge the valid data of the current smallest unit and the valid data of the next smallest unit to obtain a new smallest unit; If not, merge the valid data of the current smallest unit and the valid data of the next smallest unit, and further search for the next smallest unit, and so on until all the smallest units are traversed; After traversing all the smallest units, if the sum of the spaces of several smallest units is equal to the preset space, then regard the merged several smallest units as a new smallest unit; if there is no sum of the spaces of several smallest units equal to the preset space, then supplement invalid data so that the sum of the spaces of several smallest units and the sum of the spaces of the invalid data are equal to the preset space.

5. The method according to claim 1, characterized in that, The method further includes: After writing the compressed host data into the cache module, the cache module returns a write success message to the host.

6. The method according to claim 1, characterized in that, The method further includes: After the write-down unit is written into the flash memory medium, the firmware module deletes the smallest unit corresponding to the write-down unit from the cache module.

7. The method according to claim 1, wherein The method further includes: When receiving a read command sent by the host, if the data corresponding to the read command is stored in the cache module, the firmware module reads the data corresponding to the read command from the cache module; If the data corresponding to the read command is stored in the flash memory medium, the firmware module reads the data corresponding to the read command from the flash memory medium.

8. The method according to any one of claims 1 to 7, characterized in that, The space of the smallest unit is in a multiple relationship with the space of the write-down unit, the space of the first data block is equal to the mapping management granularity of the firmware of the flash memory device; the space of the write-down unit is related to the size of the data page of the flash memory medium of the flash memory device.

9. A flash memory device, characterized in that, It includes: A hardware compression module, a firmware module, a cache module, and a flash memory medium, where The hardware compression module is used to obtain a write command sent by the host, where the write command corresponds to a host data; and divide the host data into several first data blocks with the same space, compress each of the first data blocks to obtain a data chain composed of several second data blocks, generate a first command, and send the first command to the firmware module, where the first command includes the data chain, and the space of each second data block is smaller than the space of the first data block; The firmware module is used to write the data chain into the cache module, combine several second data blocks in the data chain in the cache module to generate the minimum units of multiple preset spaces; and combine multiple such minimum units to obtain several write-down units, and write the write-down units into the flash memory medium of the flash memory device. Specifically, the firmware module is used to obtain the space corresponding to the current second data block; if the space of the current second data block is equal to the space of the minimum unit, directly use the current second data block as the minimum unit; if the space of the current second data block is less than the space of the minimum unit, search for the next second data block, and further determine whether the sum of the spaces of the current second data block and the next second data block is less than the space of the minimum unit. If so, merge the two as the current second data block, and so on until all second data blocks are traversed; after traversing all second data blocks, if the sum of the spaces of several second data blocks is equal to the space of the minimum unit, use the merged several second data blocks as the minimum unit; if there is no sum of the spaces of several second data blocks equal to the space of the minimum unit, supplement invalid data so that the sum of the spaces of several second data blocks and the space of the invalid data is equal to the space of the minimum unit.

Images