Data storage system and data storage method
By setting up compression and decompression modules in the host and flash memory device, the delay problem caused by insufficient internal computing resources during data reading and writing of flash memory devices is solved, and the storage performance is improved.
Patent Information
- Application Number
- CN202211655129.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-22
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-12-22
AI Technical Summary
Existing flash memory devices rely on internal computing resources for compression and decompression when reading and writing data, resulting in an increase in read and write delay and affecting storage performance.
The compression and decompression modules are set up in the host and the flash memory device to jointly compress and decompress data. Through the processing capabilities of the host and the flash memory device, it is determined whether to compress or decompress the data to improve storage performance.
By co-compressing and decompressing data between host and flash memory devices, the read and write delay of flash memory devices is reduced, storage performance is improved, and the computing resources of host and flash memory devices are fully utilized.
Smart Images

Figure CN116243854B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of storage device applications, and in particular to a data storage system and a data storage method. Background Art
[0002] Flash memory devices, such as solid state drives (SSDs), are hard disks made using solid-state electronic storage chip arrays. SSDs include a control unit and a storage unit (FLASH storage chip or DRAM storage chip).
[0003] Currently, the computational storage architecture supports data compression and decompression inside the SSD. Specifically, the data is compressed when it is written to the SSD. When the host application or file system reads the data, the SSD first decompresses the data internally and then returns it to the host. Since the current solution relies on the computing resources inside the flash device for compression or decompression, it is easy to increase the read and write latency of the flash device, affecting the storage performance of the flash device.
[0004] Based on this, the existing technology needs to be improved urgently. Summary of the invention
[0005] The embodiments of the present application provide a data storage system and a data storage method, which improve the storage performance of a flash memory device by coordinating data compression and decompression of a host with data compression and decompression of a flash memory device.
[0006] The embodiments of the present application provide the following technical solutions:
[0007] In a first aspect, an embodiment of the present application provides a data storage system, including: a host and a flash memory device;
[0008] The host is connected to the flash memory device, and the host includes:
[0009] A host compression module, for compressing the write data corresponding to the write request when determining that the compression decision mark corresponding to the write data corresponding to the write request is compressed, so as to write the compressed write data to the flash memory device;
[0010] A host decompression module, used for decompressing the read data when determining that the decompression decision mark corresponding to the read data of the flash memory device is decompression;
[0011] Among them, the flash memory device is connected to the host, and the flash memory device includes:
[0012] A flash memory compression module, used for compressing the write data of the host when it is determined that the compression decision mark corresponding to the write data of the host is compressed;
[0013] The flash memory decompression module is used to decompress the read data corresponding to the read request when it is determined that the decompression decision mark corresponding to the read data corresponding to the read request is decompression.
[0014] In some embodiments,
[0015] The host also includes:
[0016] A host driver module, used to determine a compression decision mark corresponding to write data corresponding to a write request, and / or to determine a decompression decision mark corresponding to read data of a flash memory device;
[0017] Flash memory devices also include:
[0018] The flash memory control module is used to determine the compression decision mark corresponding to the write data of the host, and / or determine the decompression decision mark corresponding to the read data corresponding to the read request.
[0019] In some embodiments,
[0020] The host driver module is connected to the flash memory control module, and the host driver module and the flash memory control module interact with each other in writing requests, reading requests or data.
[0021] In some embodiments, the host further comprises:
[0022] The system module is connected to the host driver module and is used to send a write request or a read request to the host driver module to receive response information corresponding to the write request or the read request returned by the host driver module, wherein the system module includes an application system and / or a file system.
[0023] In a second aspect, an embodiment of the present application provides a data storage method, which is applied to the data storage system of the first aspect, and the method includes:
[0024] After the host receives the write request, the host determines the compression decision mark corresponding to the write request according to the host compression control policy;
[0025] Compressing or not compressing the write data corresponding to the write request according to the compression decision mark, so as to send the write request to the flash memory device, wherein the write request includes the compressed write data or the original write data;
[0026] After the flash memory device receives the write request sent by the host, it compresses or uncompresses the write data in the write request according to the compression mark information in the write request, so as to write the compressed write data or the original write data to the flash memory medium of the flash memory device.
[0027] In some embodiments, determining a compression decision mark corresponding to a write request according to a host compression control policy includes:
[0028] If the utilization rate of the processor of the host is less than a preset utilization rate threshold, and the data compression rate of the write data corresponding to the write request is less than a preset compression rate threshold, then determining that the compression decision mark corresponding to the write request is compressed;
[0029] If the utilization of the host processor is not less than a preset utilization threshold, or the data compression rate of the write data corresponding to the write request is not less than a preset compression rate threshold, the compression decision mark corresponding to the write request is determined to be not compressed.
[0030] In some embodiments, the compression mark information includes compressed or uncompressed, and compressing or not compressing the write data in the write request according to the compression mark information in the write request includes:
[0031] If the compression mark information is uncompressed, whether to compress the write data in the write request is determined according to the flash memory compression control policy, wherein the flash memory compression control policy includes:
[0032] If the utilization rate of the processor of the flash memory device is less than a preset utilization rate threshold, and the data compression rate of the write data corresponding to the write request is less than a preset compression rate threshold, then determining to compress the write data corresponding to the write request;
[0033] If the utilization rate of the processor of the flash memory device is not less than a preset utilization rate threshold, or the data compression rate of the write data corresponding to the write request is not less than a preset compression rate threshold, it is determined that the write data corresponding to the write request is not compressed.
[0034] In a third aspect, an embodiment of the present application provides a data storage method, which is applied to the data storage system of the first aspect, and the method includes:
[0035] After receiving the read request, the host sends the read request to the flash memory device;
[0036] After the flash memory device receives the read request, if the read data corresponding to the read request is compressed data, a decompression decision mark corresponding to the read data is determined according to the decompression control strategy;
[0037] The flash memory device determines whether to decompress the read data according to the decompression decision mark, so as to return the decompressed read data or the undecompressed read data to the host.
[0038] In some embodiments, determining a decompression decision mark corresponding to the read data according to the decompression control strategy includes:
[0039] If the decompression queue in the flash memory device is not full, and the interface bandwidth utilization rate between the host and the flash memory device is less than a preset bandwidth utilization rate threshold, determining that the decompression decision mark corresponding to the read data is decompression;
[0040] If the decompression queue in the flash memory device is full, or the interface bandwidth utilization between the host and the flash memory device is not less than a preset bandwidth utilization threshold, the decompression decision mark corresponding to the read data is determined to be not decompressed.
[0041] In some embodiments, the method further comprises:
[0042] After the host receives the undecompressed read data sent by the flash memory device, the host decompresses the undecompressed read data.
[0043] In a fourth aspect, an embodiment of the present application further provides a non-volatile computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a data storage system to execute a data storage method such as the second aspect or the third aspect.
[0044] The beneficial effect of the embodiment of the present application is: different from the prior art, the embodiment of the present application provides a data storage system, including: a host and a flash memory device; wherein, the host is connected to the flash memory device, and the host includes: a host compression module, which is used to compress the write data corresponding to the write request when it is determined that the compression decision mark corresponding to the write data corresponding to the write request is compressed, so as to write the compressed write data to the flash memory device; a host decompression module, which is used to decompress the read data of the flash memory device when it is determined that the decompression decision mark corresponding to the read data of the flash memory device is decompressed; wherein the flash memory device is connected to the host, and the flash memory device includes: a flash memory compression module, which is used to compress the write data of the host when it is determined that the compression decision mark corresponding to the write data of the host is compressed; a flash memory decompression module, which is used to decompress the read data corresponding to the read request when it is determined that the decompression decision mark corresponding to the read data corresponding to the read request is decompressed.
[0045] By coordinating data compression and decompression of the host and data compression and decompression of the flash memory device, the present application can improve the storage performance of the flash memory device. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0047] Figure 1 It is a structural schematic diagram of a data storage system provided in an embodiment of the present application;
[0048] Figure 2 is a schematic diagram of the structure of another data storage system provided in an embodiment of the present application;
[0049] Figure 3 is a structural schematic diagram of a flash memory device provided in an embodiment of the present application;
[0050] Figure 4 It is a schematic diagram of data compression and decompression of a host provided by an embodiment of the present application;
[0051] Figure 5 is a schematic diagram of data compression and decompression of a flash memory device provided in an embodiment of the present application;
[0052] Figure 6 It is a schematic diagram of a host and a flash memory device cooperating to perform data compression and decompression provided by an embodiment of the present application;
[0053] Figure 7 It is a flowchart of a data storage method provided by an embodiment of the present application;
[0054] Figure 8 yes Figure 7 A detailed flowchart of step S701 in FIG.
[0055] Fig. 9 is a schematic diagram of a data writing operation provided in an embodiment of the present application;
[0056] Fig.10 It is a structural diagram of a firmware module of a flash memory device provided in an embodiment of the present application;
[0057] Fig.11 It is a flowchart of another data storage method provided by an embodiment of the present application;
[0058] Fig.12 yes Fig.11 A detailed flowchart of step S112 in FIG.
[0059] Fig.13 It is a schematic diagram of a data read operation provided in an embodiment of the present application. DETAILED DESCRIPTION
[0060] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0061] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.
[0062] The technical solution of this application is described in detail below with reference to the accompanying drawings:
[0063] See also Figure 1 , Figure 1 It is a structural schematic diagram of a data storage system provided in an embodiment of the present application;
[0064] like Figure 1 As shown, the data storage system 300 includes a flash memory device 100 and a host 200, wherein the flash memory device 100 is communicatively connected to the host 200. The flash memory device 100 is communicatively connected to the host 200 via a wired or wireless manner to implement data and / or command and / or request interaction.
[0065] Please refer to Figure 2 , Figure 2 is a schematic diagram of the structure of another data storage system provided in an embodiment of the present application;
[0066] like Figure 2 As shown, the data storage system 300 includes a flash memory device 100 and a host 200 .
[0067] Specifically, the flash memory device 100 includes: a flash memory medium 110 , a flash memory control module 120 , a flash memory decompression module 130 and a flash memory compression module 140 .
[0068] Flash memory medium 110, as the storage medium of flash memory device 100, is also called flash memory, Flash, Flash memory or Flash particles. It is a type of storage device and a non-volatile memory. It can store data for a long time even without current supply. Its storage characteristics are equivalent to those of a hard disk, making flash memory medium 110 the basis of storage media for various portable digital devices.
[0069] The flash memory control module 120 is connected to the flash memory medium 110, the flash memory decompression module 130 and the flash memory compression module 140, and is used to control the internal operation of the flash memory device 100, and to realize the interaction of data and / or commands and / or requests between the flash memory device 100 and the host 200, wherein the request includes a read request and / or a write request.
[0070] In the embodiment of the present application, the flash memory control module 120 is a controller of the flash memory device. Figure 3 , Figure 3 is a structural schematic diagram of a flash memory device provided in an embodiment of the present application;
[0071] like Figure 3 As shown, the flash memory control module 120 includes: a data converter 121 , a processor 122 , a buffer 123 , a flash memory controller 124 and an interface 125 .
[0072] The data converter 121 is connected to the processor 122 and the flash controller 124 respectively, and the data converter 121 is used to convert binary data into hexadecimal data, and convert hexadecimal data into binary data. Specifically, when the flash controller 124 writes data to the flash medium 110, the binary data to be written is converted into hexadecimal data through the data converter 121, and then written into the flash medium 110. When the flash controller 124 reads data from the flash medium 110, the hexadecimal data stored in the flash medium 110 is converted into binary data through the data converter 121, and then the converted data is read from the binary data page register. Among them, the data converter 121 may include a binary data register and a hexadecimal data register. The binary data register can be used to save the data converted from hexadecimal to binary, and the hexadecimal data register can be used to save the data converted from binary to hexadecimal.
[0073] The processor 122 is connected to the data converter 121, the cache 123, the flash memory controller 124 and the interface 125, respectively. The processor 122 and the data converter 121, the cache 123, the flash memory controller 124 and the interface 125 can be connected via a bus or other means. The processor is used to run the non-volatile software programs, instructions and modules stored in the cache 123, so as to implement any method embodiment of the present application.
[0074] The buffer 123 is mainly used to buffer the read / write instructions sent by the host 200 and the read data or write data obtained from the flash memory medium 110 according to the read / write instructions sent by the host 200.
[0075] The flash memory controller 124 is connected to the flash memory medium 110 , the data converter 121 , the processor 122 and the buffer 123 , and is used to access the back-end flash memory medium 110 and manage various parameters and data I / O of the flash memory medium 110 .
[0076] Interface 125 connects the host 200 and the data converter 121, the processor 122 and the buffer 123, and is used to receive data sent by the host 200, or receive data sent by the processor 122 to realize data transmission between the host 200 and the processor 122. Interface 125 can be a SATA-2 interface, a SATA-3 interface, a SAS interface, an MSATA interface, a PCI-E interface, a NGFF interface, a CFast interface, an SFF-8639 interface and an M.2NVME / SATA protocol.
[0077] The flash memory decompression module 130 is connected to the flash memory control module 120 and is used to decompress the compressed data in the flash memory medium 110 of the flash memory device 100 .
[0078] The flash memory compression module 140 is connected to the flash memory control module 120 and is used to compress the data sent by the host driver module 220 of the host 200 .
[0079] Specifically, the host 200 includes: a system module 210 , a host driver module 220 , a host compression module 230 and a host decompression module 240 .
[0080] The system module 210 is connected to the host driver module 220 and is used to send a write request or a read request to the host driver module 220 to receive the response information corresponding to the write request or the response information corresponding to the read request returned by the host driver module, wherein the write request includes the logical address information and the data to be written into the flash memory medium, the response information corresponding to the write request includes the write success information or the write failure information, the write request is received by the host driver module 220, so that the host driver module 220 executes the write request and writes the write data in the write request into the flash memory medium of the flash memory device; the read request includes the logical address information, the response information corresponding to the read request includes the data corresponding to the read request, the read request is received by the host driver module 220, so that the host driver module 220 executes the write request to read the corresponding data in the flash memory medium of the flash memory device. In the embodiment of the present application, the system module 210 includes an application system and / or a file system.
[0081] The host driver module 220 is connected to the system module 210, the host compression module 230 and the host decompression module 240. The host driver module 220 is used to control the internal operation of the host 200 and realize the interaction of data and / or commands and / or requests between the flash memory device 100 and the host 200, wherein the request includes a read request and / or a write request.
[0082] Specifically, the host driver module 220 is connected to the flash memory control module 120 , and the host driver module 220 and the flash memory control module 120 interact with each other in terms of write requests, read requests, or data.
[0083] The host compression module 230 is connected to the host driving module 220 and is used to compress the data sent by the system module 210 .
[0084] The host decompression module 240 is connected to the host driver module 220 and is used to decompress the compressed data returned by the flash memory device 100 .
[0085] In the embodiment of the present application, by setting a compression module and a decompression module in both the host and the flash memory device, the processing capabilities of the host and the flash memory device can be coordinated to improve the storage performance of the flash memory device.
[0086] The following example illustrates the data compression and decompression process:
[0087] See also Figure 4 , Figure 4 It is a schematic diagram of data compression and decompression of a host provided by an embodiment of the present application;
[0088] like Figure 4 As shown, a compression module and a decompression module are set in the host, and the host compresses the data on the host side alone, or decompresses the data on the flash memory device side. However, the compression module and decompression module of this method are usually implemented based on software, which has the problems of low efficiency and slow speed.
[0089] Please refer to Figure 5 , Figure 5 is a schematic diagram of data compression and decompression of a flash memory device provided in an embodiment of the present application;
[0090] like Figure 5 As shown, a compression module and a decompression module are set in the flash memory device, and the flash memory device compresses the data on the host side alone, or decompresses the data on the flash memory device side. However, in this way, the read performance of the flash memory device is severely restricted by the decompression operation speed of the computing engine. In addition, the idle computing resources of the host are not utilized in the process of data compression, resulting in resource waste and suboptimal overall system read and write performance.
[0091] In view of this, the embodiments of the present application can achieve collaborative processing capabilities of the host and flash memory devices by setting compression modules and decompression modules in both the host and the flash memory device, combining data compression on the host side and data compression inside the storage device, and making full use of the computing power of the computing engines of the host and the flash memory device to perform data compression and decompression in a collaborative manner to achieve optimal storage performance.
[0092] For details, please refer to Figure 6 , Figure 6 It is a schematic diagram of a host and a flash memory device cooperating to perform data compression and decompression provided by an embodiment of the present application;
[0093] like Figure 6 As shown, the host includes a host compression module and a host decompression module, and the flash memory device includes a flash memory compression module and a flash memory decompression module.
[0094] When the host writes data, the host driver module determines whether to compress the data. If so, the host compression module compresses the data and sends the compressed data to the flash memory device; if not, the uncompressed data is directly sent to the flash memory device. After the flash memory device receives the data sent by the host, it determines whether the data is compressed data. If so, the data is directly written into the flash memory, that is, into the flash memory medium; if not, the flash memory compression module compresses the data and writes the compressed data into the flash memory medium.
[0095] In the embodiment of the present application, the flash memory compression module supports both online compression in the disk and offline compression in the disk. Specifically, the flash memory compression module can first compress the data on the data path when writing data, and then write it to the flash memory medium, that is, online compression in the disk; or the flash memory compression module can first write the data to the flash memory medium, then take the data out of the flash memory medium in an offline manner, compress it, and then write the compressed data back, that is, offline compression in the disk.
[0096] When reading data from the flash memory, the computing engine determines whether the data is compressed data. If so, the data is decompressed by the flash memory decompression module and the decompressed data is sent to the host; if not, the data is directly sent to the host. After the host receives the data sent by the flash memory device, it determines whether the data is compressed data. If so, the data is decompressed by the host decompression module and the decompressed data is returned to the file system; if not, the data is directly returned to the file system.
[0097] In an embodiment of the present application, a data storage system is provided, including: a host and a flash memory device; wherein the host is connected to the flash memory device, and the host includes: a host compression module, which is used to compress the write data corresponding to the write request when it is determined that the compression decision mark corresponding to the write data corresponding to the write request is compressed, so as to write the compressed write data to the flash memory device; a host decompression module, which is used to decompress the read data of the flash memory device when it is determined that the decompression decision mark corresponding to the read data of the flash memory device is decompressed; wherein the flash memory device is connected to the host, and the flash memory device includes: a flash memory compression module, which is used to compress the write data of the host when it is determined that the compression decision mark corresponding to the write data of the host is compressed; and a flash memory decompression module, which is used to decompress the read data corresponding to the read request when it is determined that the decompression decision mark corresponding to the read data corresponding to the read request is decompressed.
[0098] By coordinating data compression and decompression of the host and data compression and decompression of the flash memory device, the present application can improve the storage performance of the flash memory device.
[0099] See also Figure 7 , Figure 7 It is a flowchart of a data storage method provided by an embodiment of the present application;
[0100] Among them, the data storage method is applied to the data storage system provided in the above embodiment.
[0101] like Figure 7 As shown, the data storage method includes:
[0102] Step S701: after the host receives a write request, the compression decision mark corresponding to the write request is determined according to the host compression control policy;
[0103] Specifically, the compression decision mark is used to determine whether to compress the write data corresponding to the write request, and the compression decision mark includes compression or non-compression.
[0104] In an embodiment of the present application, the host compression control strategy is used to determine whether to compress the write data corresponding to the write request according to the utilization rate of the host processor and the data compression rate of the write data corresponding to the write request. Specifically, according to the host compression control strategy, the compression decision mark corresponding to the write request is determined, including:
[0105] If the utilization rate of the processor of the host is less than a preset utilization rate threshold, and the data compression rate of the write data corresponding to the write request is less than a preset compression rate threshold, then determining that the compression decision mark corresponding to the write request is compressed;
[0106] If the utilization of the host processor is not less than a preset utilization threshold, or the data compression rate of the write data corresponding to the write request is not less than a preset compression rate threshold, the compression decision mark corresponding to the write request is determined to be not compressed.
[0107] For details, please refer to Figure 8 , Figure 8 yes Figure 7 A detailed flowchart of step S701 in FIG.
[0108] like Figure 8 As shown, step S701 includes:
[0109] Step S7011: Obtain the utilization rate of the processor of the host and the data compression rate of the write data corresponding to the write request;
[0110] Step S7012: determining whether the utilization of the processor of the host is less than a preset utilization threshold;
[0111] Specifically, the preset utilization threshold is set according to specific needs, for example, 70%, 80%, 90%, etc. Assuming that the utilization threshold is 80%, if the utilization of the host processor is less than 80%, the process proceeds to step S7013; if the utilization of the host processor is greater than or equal to 80%, the process proceeds to step S7015;
[0112] Step S7013: determining whether the data compression rate of the write data corresponding to the write request is less than a preset compression rate threshold;
[0113] Specifically, the preset compression rate threshold is set according to specific needs, for example: 70%, 80%, 90% and other values. Assuming that the compression rate threshold is 80%, if the data compression rate of the write data corresponding to the write request is less than 80%, step S7014 is entered; if the data compression rate of the write data corresponding to the write request is greater than or equal to 80%, step S7015 is entered.
[0114] Step S7014: Determine that the compression decision mark corresponding to the write request is compressed;
[0115] It can be understood that if it is determined that the compression decision mark corresponding to the write request is compressed, it means that the host needs to compress the write data corresponding to the write request.
[0116] Step S7015: Determine that the compression decision mark corresponding to the write request is not compressed.
[0117] It can be understood that if it is determined that the compression decision mark corresponding to the write request is not compressed, it means that the host does not need to compress the write data corresponding to the write request.
[0118] Step S702: compressing or not compressing the write data corresponding to the write request according to the compression decision mark, so as to send the write request to the flash memory device, wherein the write request includes the compressed write data or the original write data;
[0119] Specifically, if the compression decision is marked as compressed, the host compresses the write data corresponding to the write request to send the write request to the flash memory device, that is, sends the compressed write data in the write request to the flash memory device; if the compression decision is marked as not compressed, the host does not compress the write data corresponding to the write request to send the write request to the flash memory device, that is, sends the original write data in the write request to the flash memory device.
[0120] Step S703: After the flash memory device receives the write request sent by the host, it compresses or uncompresses the write data in the write request according to the compression mark information in the write request, so as to write the compressed write data or the original write data to the flash memory medium of the flash memory device.
[0121] Specifically, after determining the compression decision mark, the compression mark information is added to the write request, wherein the compression mark information includes compressed or uncompressed, that is, the write request received by the flash memory device from the host contains the mark of "compressed" or "uncompressed" data. If the data is compressed, the flash memory device writes the data to the flash memory. If the data is uncompressed, the flash memory device executes its internal compression decision logic to decide whether to compress the written data and then write it to the flash memory medium. It can be understood that the compression decision mark corresponds to the compression mark information. If the compression decision mark is compressed, the corresponding compression mark information is compressed. If the compression decision mark is not compressed, the corresponding compression mark information is uncompressed.
[0122] In an embodiment of the present application, the write request corresponds to a command, and adding compression mark information in the write request is equivalent to adding a mark bit in the write command. For example, in the NVMe basic specification v1.4, the 8th to 15th bytes of the command are reserved fields. In an embodiment of the present application, the 8th byte can be written to 1 as a mark that the written data has been compressed. If the 8th byte is not written to 1, it indicates that the written data is not compressed. It should be noted that the example here is only used as an example to illustrate the implementation of the present application and does not limit the actual implementation of the present application.
[0123] Specifically, compressing or not compressing the write data in the write request according to the compression mark information in the write request includes:
[0124] If the compression mark information in the write request is uncompressed, the flash memory device compresses the write data in the write request;
[0125] If the compression mark information in the write request indicates that the data has been compressed, the flash memory device does not compress the write data in the write request.
[0126] It can be understood that if the flash memory device compresses the write data in the write request, the compressed write data is written to the flash memory medium of the flash memory device; if the flash memory device does not compress the write data in the write request, the original write data is written to the flash memory medium of the flash memory device.
[0127] In some embodiments, the host further comprises a host compression judgment module, the host compression judgment module being connected to the host driver module, wherein the host compression judgment module is used to determine a compression decision mark corresponding to the write data corresponding to the write request, and return the compression decision mark to the host driver module. It is understandable that the host compression judgment module may include a processor or a microprocessor, which is used to judge whether the utilization rate of the processor of the host is less than a preset utilization rate threshold, and to judge whether the data compression rate of the write data corresponding to the write request is less than a preset compression rate threshold.
[0128] In some embodiments, the flash memory device further includes a flash memory compression judgment module, which is connected to the flash memory control module, wherein the flash memory compression judgment module is used to determine whether to compress the write data in the write request. It is understandable that the flash memory compression judgment module may include a processor or a microprocessor, which is used to obtain the utilization rate of the processor of the flash memory device and accurately estimate the compression rate of the write data, thereby determining whether the utilization rate of the processor of the flash memory device is less than a preset utilization rate threshold, and determining whether the data compression rate of the write data corresponding to the write request is less than a preset compression rate threshold.
[0129] Please refer to Fig. 9 , Fig. 9 is a schematic diagram of a data writing operation provided in an embodiment of the present application;
[0130] like Fig. 9 As shown, the host includes a host compression module, a host compression judgment module, a system module and a host driver module, and the flash memory device includes a flash memory compression module, a flash memory compression judgment module, a flash memory control module and a flash memory medium, wherein the process of the data write operation is specifically as follows:
[0131] (1) The system module sends a write request to the host driver module, wherein the write request includes logical address information and write data. In an embodiment of the present application, the write request includes a host write command (HostCmd), and the host write command includes: an identifier of the host write command (Cmd ID), a logical block address (Logical Block Address, Lba), and data to be written.
[0132] (2) The host driver module determines the compression decision mark corresponding to the write request according to the host compression control policy; or, the host compression judgment module determines the compression decision mark corresponding to the write request according to the host compression control policy, and returns the compression decision mark to the host driver module.
[0133] (3) The host driver module determines whether to call the host compression module to compress the write data corresponding to the write request according to the compression decision mark. If the compression decision mark is compression, the host compression module is controlled to compress the write data; if the compression decision mark is not compression, the host compression module is not controlled to compress the write data.
[0134] (4) The host driver module submits a write request to the flash memory control module. The write request includes logical address information, compressed write data or original write data, and compression mark information.
[0135] (5) After receiving the write request sent by the host driver module, the flash memory control module checks the compression mark information. If the compression mark information indicates that the data is compressed, the compressed write data is written to the flash memory medium. If the compression mark information indicates that the data is not compressed, the flash memory control module determines whether to compress the write data in the write request according to the flash memory compression control strategy. Alternatively, the flash memory control module calls the flash memory compression judgment module to determine whether to compress the read data, so that the flash memory compression judgment module sends the judgment result to the flash memory control module.
[0136] (6) The flash memory control module determines whether to call the flash memory compression module to compress the written data based on the determination result or the judgment result. If the determination result is compression, the written data is compressed; if the determination result is not compression, the written data is not compressed.
[0137] (7) The flash memory control module records the compression mark of the write address and the logical-physical address mapping information in the flash memory algorithm module in its firmware module, and writes the write data corresponding to the write request to the flash memory medium.
[0138] Please refer to Fig.10 , Fig.10 It is a structural diagram of a firmware module of a flash memory device provided in an embodiment of the present application;
[0139] Among them, the firmware module is used to connect the host (HOST) and the flash memory medium (NAND) to realize data IO processing.
[0140] like Fig.10 As shown, the firmware module 10 is connected to the host 200 and the hardware module 20, and includes:
[0141] The front end module 101, i.e., (Front End, FE), is used to obtain host commands to generate IO operations, wherein the front end module is also responsible for the communication protocol with the host (Host), the parsing of host commands and solid state drive commands, etc.;
[0142] The flash algorithm module 102, i.e., the flash translation layer (FTL), is connected to the data processing module and is used to map the IO operation to determine the flash memory medium to be sent; wherein, the flash algorithm module (Flash Translation Layer, FTL) sends the IO operation to the back end module (BE) of the solid state drive controller, so that the back end module of the solid state drive controller receives the IO operation sent by the flash algorithm module;
[0143] The back-end module 103, namely (Back End, BE), is connected to the flash algorithm module and is used to receive the IO operation sent by the flash algorithm module to control the hardware module to perform read / write / erase operations on the flash medium;
[0144] In the embodiment of the present application, the hardware module 20, i.e. (HW Op Nand Mode), refers to a module for operating the FLASH flash memory, which is connected to the back-end module 103 and controlled by the back-end module 103, and is used to operate the FLASH flash memory, for example: according to the IO operation, the operation is performed on the corresponding flash memory medium, that is, the operation processing of data to the FLASH flash memory is completed, and the operation includes a read operation, a write operation, and an erase operation, etc.
[0145] Among them, after the front-end module obtains the host command, it processes it to generate IO operations, and then passes through the flash algorithm module, the back-end module and the hardware module in turn to operate the flash medium. For example: when the host reads data, the host (Host) sends the host command to the flash device (device), the front-end module (FE) of the flash device receives the host command, processes it and distributes it to the flash algorithm module (FTL), after receiving it, the flash algorithm module converts it from logic to physical, and then sends the request NAND read operation to the back-end module (BE), after receiving the read operation, the back-end module sends the hardware instruction to the hardware module (HW Op Nand Mode), and the hardware module performs NAND processing in parallel.
[0146] (8) The flash memory control module returns the response information of the write request to the host driver module.
[0147] (9) The host driver module returns the response information corresponding to the write request to the system module, so that the system module returns the response information corresponding to the write request to the application corresponding to the write request, thereby completing the current write operation.
[0148] In the embodiment of the present application, the process of completing the data write operation is achieved through the interaction and processing between the various modules of the host and the flash memory device, which can coordinate the processing of the host and the flash memory device and improve the write performance of the flash memory device.
[0149] In an embodiment of the present application, a data storage method is provided, which is applied to the data storage system of the above embodiment, and the method includes: after the host receives a write request, according to the host compression control strategy, determining the compression decision mark corresponding to the write request; according to the compression decision mark, compressing or not compressing the write data corresponding to the write request, so as to send the write request to the flash memory device, wherein the write request includes compressed write data or original write data; after the flash memory device receives the write request sent by the host, according to the compression mark information in the write request, compressing or not compressing the write data in the write request, so as to write the compressed write data or the original write data to the flash memory medium of the flash memory device. By determining the compression decision mark corresponding to the write request, determining whether to compress or not to compress the write data according to the compression decision mark, so as to write the compressed write data or the original write data to the flash memory medium of the flash memory device, the present application can reduce the write delay, thereby improving the write performance of the flash memory device.
[0150] See also Fig.11 , Fig.11 It is a flowchart of another data storage method provided by an embodiment of the present application;
[0151] Among them, the data storage method is applied to the data storage system provided in the above embodiment.
[0152] like Fig.11 As shown, the data storage method includes:
[0153] Step S111: after receiving the read request, the host sends the read request to the flash memory device;
[0154] Specifically, after receiving the read request sent by the system module, the host driver module sends the read request to the flash memory control module of the flash memory device.
[0155] Step S112: after the flash memory device receives the read request, if the read data corresponding to the read request is compressed data, a decompression decision mark corresponding to the read data is determined according to the decompression control strategy;
[0156] In some embodiments, determining a decompression decision mark corresponding to the read data according to the decompression control strategy includes:
[0157] If the decompression queue in the flash memory device is not full, and the interface bandwidth utilization rate between the host and the flash memory device is less than a preset bandwidth utilization rate threshold, determining that the decompression decision mark corresponding to the read data is decompression;
[0158] If the decompression queue in the flash memory device is full, or the interface bandwidth utilization between the host and the flash memory device is not less than a preset bandwidth utilization threshold, the decompression decision mark corresponding to the read data is determined to be not decompressed.
[0159] In addition, the present application can also support setting a "host decompression" flag in the host's read request, which is used to notify the flash memory device that there is no need to decompress the read data, and the decompression operation will be performed on the host side after the host receives the read response. Among them, whether the data returned to the host is in compressed or uncompressed form will be indicated by a bitmap in the read response.
[0160] For details, please refer to Fig.12 , Fig.12 yes Fig.11 A detailed flowchart of step S112 in FIG.
[0161] like Fig.12 As shown, step S112 includes:
[0162] Step S1121: obtaining queue information of a decompression queue in the flash memory device and bandwidth utilization of an interface between the host and the flash memory device;
[0163] Specifically, a decompression queue is maintained in the firmware module of the flash memory device, and the queue information of the decompression queue is used to identify the current state of the decompression queue, for example: the current queue length. It is understandable that the current queue length is less than or equal to the total queue length of the decompression queue.
[0164] Step S1122: determining whether the decompression queue in the flash memory device is not full;
[0165] Specifically, determining whether the decompression queue in the flash memory device is not full includes: determining whether the current queue length of the decompression queue in the firmware module of the flash memory device is less than the total queue length of the decompression queue; if so, determining that the decompression queue in the flash memory device is not full; if not, determining that the decompression queue in the flash memory device is full.
[0166] If the decompression queue in the flash memory device is not full, proceed to step S1123;
[0167] If the decompression queue in the flash memory device is full, proceed to step S1124.
[0168] Step S1123: determining whether the interface bandwidth utilization between the host and the flash memory device is less than a preset bandwidth utilization threshold;
[0169] Specifically, the preset bandwidth utilization threshold is set according to specific needs, for example: 70%, 80%, 90% and other values. Assuming that the bandwidth utilization threshold is 80%, if the interface bandwidth utilization of the host and the flash memory device is less than 80%, proceed to step S1124; if the interface bandwidth utilization of the host and the flash memory device is greater than or equal to 80%, proceed to step S1125.
[0170] Step S1124: determining that the decompression decision mark corresponding to the read data is decompression;
[0171] Specifically, the decompression decision mark is used to determine whether the flash memory device decompresses the read data. If the decompression decision mark is decompression, the flash memory control module in the flash memory device calls the flash memory decompression module to decompress the read data; if the decompression decision mark is not decompressed, the flash memory control module in the flash memory device does not call the flash memory decompression module to decompress the read data.
[0172] Step S1125: Determine that the decompression decision mark corresponding to the read data is not to be decompressed.
[0173] Step S113: The flash memory device determines whether to decompress the read data according to the decompression decision mark, so as to return the decompressed read data or the undecompressed read data to the host.
[0174] Specifically, if the decompression decision mark is decompression, the flash memory control module calls the flash memory decompression module to decompress the read data, and returns the decompressed read data to the host driver module of the host;
[0175] If the decompression decision mark is not to decompress, the flash memory control module does not call the flash memory decompression module to decompress the read data, and returns the undecompressed read data to the host driver module of the host.
[0176] Please refer to Fig.13 , Fig.13 is a schematic diagram of a data read operation provided in an embodiment of the present application;
[0177] like Fig.13 As shown, the host includes a host decompression judgment module, a host decompression module, a system module and a host driver module, and the flash memory device includes a flash memory decompression module, a flash memory decompression judgment module, a flash memory control module and a flash memory medium, wherein the process of the data read operation is specifically as follows:
[0178] (1) The system module sends a read request to the host driver module, and the read request includes a logical address list and the length of the data expected to be read. In the embodiment of the present application, the read request includes a host read command (HostCmd), and the host read command includes: an identifier of a host write command (Cmd ID), a logical block address (Logical Block Address, Lba) and an address length to be read (Len), wherein, relative to the host, the logical block address is generally 512Byte, 4k, etc.
[0179] (2) The host driver module sends a read request to the flash memory control module.
[0180] (3) The flash memory control module queries the write address, compression mark, and logical-physical address mapping information recorded in the flash memory algorithm module in the firmware module, and reads the read data required by the read request from the flash memory medium.
[0181] (4) The flash memory control module determines whether the read data is compressed data based on the compression mark information corresponding to the read data. If so, the flash memory control module further determines whether to decompress the read data; or, the flash memory control module calls the flash memory decompression judgment module to determine whether to decompress the read data, so that the flash memory decompression judgment module sends the judgment result to the flash memory control module.
[0182] Specifically, determining whether to decompress the read data includes: determining a decompression decision mark corresponding to the read data according to a decompression control strategy, wherein determining a decompression decision mark corresponding to the read data according to the decompression control strategy includes:
[0183] If the decompression queue in the flash memory device is not full, and the interface bandwidth utilization rate between the host and the flash memory device is less than a preset bandwidth utilization rate threshold, determining that the decompression decision mark corresponding to the read data is decompression;
[0184] If the decompression queue in the flash memory device is full, or the interface bandwidth utilization between the host and the flash memory device is not less than a preset bandwidth utilization threshold, the decompression decision mark corresponding to the read data is determined to be not decompressed.
[0185] (5) The flash memory control module determines whether to decompress the read data based on the decompression decision mark. If the decompression decision mark is for decompression, the flash memory control module calls the flash memory decompression module to decompress the read data; if the decompression decision mark is for not decompressing, the flash memory control module does not call the flash memory decompression module to decompress the read data.
[0186] (6) The flash memory control module returns response information corresponding to the read request to the host driver module, wherein the response information corresponding to the read request includes the read data and a compression mark of the read data, and the compression mark is used to indicate the compression status of the read data, and the compression status includes uncompressed or compressed, that is, the compression mark includes uncompressed or compressed.
[0187] (7) The host driver module checks the response information corresponding to the read request returned by the flash memory control module. If the compression mark of the read data in the response information corresponding to the read request is uncompressed, then in the host, it is determined that the decompression decision mark corresponding to the read data is not decompressed; if the compression mark of the read data is compressed, then in the host, it is determined that the decompression decision mark corresponding to the read data is decompressed.
[0188] (8) The host driver module determines whether to call the host decompression module to decompress the read data according to the decompression decision mark. If the decompression decision mark is decompression, the read data is decompressed; if the decompression decision mark is not decompression, the write data is not decompressed. It can be understood that after the host receives the undecompressed read data sent by the flash memory device, the host decompresses the undecompressed read data.
[0189] (9) The host driver module returns the response information corresponding to the read request to the system module, so that the system module returns the response information corresponding to the read request to the application corresponding to the read request, thereby completing the current read operation.
[0190] In the embodiment of the present application, the process of completing the data read operation is achieved through the interaction and processing between the various modules of the host and the flash memory device, which can coordinate the processing of the host and the flash memory device and improve the read performance of the flash memory device.
[0191] Furthermore, through a unified protocol between the flash memory control module and the firmware module of the flash memory device, the traditional logical page data structure is not changed, and data status information is added to indicate whether the logical page is in a compressed state or an uncompressed state. For example, a bitmap is added to the read response data containing multiple pages, and each bit in the bitmap is used to mark whether the corresponding page is in a compressed state or an uncompressed state. In addition, the solid-state disk drive on the host side and the firmware inside the solid-state disk can perform corresponding read and write processing on the written data page according to the compressed state or the uncompressed state when the system is running, thereby improving the overall performance of the flash memory device.
[0192] In addition, by utilizing the host's idle computing resources to increase the IOPS and bandwidth that the flash memory device can achieve, the read and write latency can be reduced.
[0193] In an embodiment of the present application, a data storage method is provided, which is applied to the data storage system of the above embodiment, and the method includes: after the host receives the read request, the read request is sent to the flash memory device; after the flash memory device receives the read request, if the read data corresponding to the read request is compressed data, the decompression decision mark corresponding to the read data is determined according to the decompression control strategy; the flash memory device determines whether to decompress the read data according to the decompression decision mark, so as to return the decompressed read data or the undecompressed read data to the host. By determining the decompression decision mark corresponding to the read data through the decompression control strategy, and determining whether to decompress the read data according to the decompression decision mark, so as to return the decompressed read data or the undecompressed read data to the host, the present application can reduce the read latency of the flash memory device, thereby improving the read performance of the flash memory device.
[0194] An embodiment of the present application also provides a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are executed by one or more processors. For example, the one or more processors can execute the data storage method in any of the above method embodiments, for example, execute the data storage method in any of the above method embodiments, for example, execute the various steps described above.
[0195] The above-described device or equipment embodiments are merely illustrative, wherein the unit modules described as separate components may or may not be physically separated, and the components displayed as module units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.
[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution can be essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute various embodiments or certain parts of the embodiments.
[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A data storage system, characterized in that: include: Host and Flash Devices; Wherein, the host is connected to the flash memory device, and the host is used to determine a compression decision mark corresponding to a write request according to a host compression control strategy; according to the compression decision mark, compress or not compress the write data corresponding to the write request to send the write request to the flash memory device, wherein the write request includes compressed write data or original write data, and the compression decision mark is used to determine whether to compress the write data corresponding to the write request, and the compression decision mark includes compression or not compression; the host compression control strategy is used to determine whether to compress the write data corresponding to the write request according to the utilization rate of the host processor and the data compression rate of the write data corresponding to the write request; The host comprises: a host compression module, configured to compress the write data corresponding to the write request when determining that the compression decision mark corresponding to the write data corresponding to the write request is compressed, so as to write the compressed write data to the flash memory device; A host decompression module, configured to decompress the read data when determining that the decompression decision mark corresponding to the read data of the flash memory device is decompression; The flash memory device is connected to the host, and is used to compress or not compress the write data in the write request according to the compression mark information in the write request, so as to write the compressed write data or the original write data to the flash memory medium of the flash memory device, wherein the compression mark information includes compressed or not compressed; The flash memory device comprises: A flash memory compression module, configured to compress the write data of the host when it is determined that the compression decision mark corresponding to the write data of the host is compressed; The flash memory decompression module is used to decompress the read data corresponding to the read request when it is determined that the decompression decision mark corresponding to the read data corresponding to the read request is decompression.
2. The system according to claim 1, characterized in that The host also includes: A host driver module, used to determine a compression decision mark corresponding to write data corresponding to a write request, and / or to determine a decompression decision mark corresponding to read data of a flash memory device; The flash memory device also includes: The flash memory control module is used to determine the compression decision mark corresponding to the write data of the host, and / or determine the decompression decision mark corresponding to the read data corresponding to the read request.
3. The system according to claim 2, characterized in that The host driver module is connected to the flash memory control module, and the host driver module and the flash memory control module interact with each other in writing requests, reading requests or data.
4. The system according to claim 2 or 3, characterized in that: The host also includes: A system module, connected to the host driver module, is used to send a write request or a read request to the host driver module to receive response information corresponding to the write request or response information corresponding to the read request returned by the host driver module, wherein the system module includes an application system and / or a file system.
5. A data storage method, characterized in that: Applied to the data storage system according to any one of claims 1 to 4, the method comprises: After the host receives the write request, the host determines the compression decision mark corresponding to the write request according to the host compression control policy; Compressing or not compressing the write data corresponding to the write request according to the compression decision mark, so as to send the write request to the flash memory device, wherein the write request includes the compressed write data or the original write data; After the flash memory device receives a write request sent by the host, it compresses or uncompresses the write data in the write request according to the compression mark information in the write request, so as to write the compressed write data or the original write data to the flash memory medium of the flash memory device.
6. The method according to claim 5, characterized in that Determining the compression decision mark corresponding to the write request according to the host compression control policy includes: If the utilization rate of the processor of the host is less than a preset utilization rate threshold, and the data compression rate of the write data corresponding to the write request is less than a preset compression rate threshold, determining that the compression decision mark corresponding to the write request is compressed; If the utilization rate of the processor of the host is not less than a preset utilization rate threshold, or the data compression rate of the write data corresponding to the write request is not less than a preset compression rate threshold, the compression decision mark corresponding to the write request is determined to be not compressed.
7. The method according to claim 5, characterized in that The compression mark information includes compressed or uncompressed, and the compressing or uncompressing the write data in the write request according to the compression mark information in the write request includes: If the compression mark information is uncompressed, determining whether to compress the write data in the write request according to a flash memory compression control strategy, wherein the flash memory compression control strategy includes: If the utilization rate of the processor of the flash memory device is less than a preset utilization rate threshold, and the data compression rate of the write data corresponding to the write request is less than a preset compression rate threshold, then determining to compress the write data corresponding to the write request; If the utilization rate of the processor of the flash memory device is not less than a preset utilization rate threshold, or the data compression rate of the write data corresponding to the write request is not less than a preset compression rate threshold, it is determined that the write data corresponding to the write request is not compressed.
8. A data storage method, characterized in that: Applied to the data storage system according to any one of claims 1 to 4, the method comprises: After the host receives the read request, sending the read request to the flash memory device; After the flash memory device receives a read request, if the read data corresponding to the read request is compressed data, determining a decompression decision mark corresponding to the read data according to a decompression control strategy; The flash memory device determines whether to decompress the read data according to the decompression decision mark, so as to return the decompressed read data or the undecompressed read data to the host.
9. The method according to claim 8, characterized in that The step of determining the decompression decision mark corresponding to the read data according to the decompression control strategy includes: If the decompression queue in the flash memory device is not full, and the bandwidth utilization of the interface between the host and the flash memory device is less than a preset bandwidth utilization threshold, determining that the decompression decision mark corresponding to the read data is decompression; If the decompression queue in the flash memory device is full, or the interface bandwidth utilization between the host and the flash memory device is not less than a preset bandwidth utilization threshold, the decompression decision mark corresponding to the read data is determined to be not decompressed.
10. The method according to claim 8 or 9, characterized in that: The method further comprises: After the host receives the undecompressed read data sent by the flash memory device, the host decompresses the undecompressed read data.
