Method for real-time data compression and storage of SSD persistent storage area

Abstract

The invention relates to the field of storage, in particular to an SSD persistent storage real-time data compression storage method. A PMR space is divided into a dynamic compression buffer area and a metadata management area, and efficient compression storage of data in a persistent memory is realized by combining an adaptive compression algorithm and intelligent selection of a storage medium. According to the method, the effective storage capacity of the PMR is improved by 3-5 times while the DRAM-level access delay is ensured, the write-in wear of the NAND flash memory is reduced, and the application prospect of the PMR function is expanded.

Description

Technical Field

[0001] This invention relates to the field of storage, specifically a method for SSD persistent storage and real-time data compression storage. Background Technology

[0002] PMR (Persistent Memory Region) is a persistent storage region feature of NVMe SSDs. SSDs supporting this feature have an internal memory space that provides a persistent cache for the host, eliminating concerns about cached data loss. This is suitable for handling high-frequency data modification cache operations and reducing wear and tear on non-volatile media. In enterprise applications, to effectively cope with scenarios such as abnormal power outages, enterprise-grade SSDs are typically equipped with energy storage capacitors. For the same output power, a larger capacitor capacity provides a longer power supply time, but this increases cost and size. Therefore, reducing the need for energy storage capacitors has become a research direction for engineering applications. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method for real-time data compression and storage in the persistent memory (PMR) region of an SSD. By dividing the PMR space into a dynamic compression buffer and a metadata management area, and combining an adaptive compression algorithm with intelligent storage medium selection, efficient compressed storage of data in persistent memory is achieved. This method increases the effective storage capacity of PMR by 3-5 times while maintaining DRAM-level access latency, and reduces write wear on NAND flash memory, expanding the application prospects of PMR functionality.

[0004] To solve the aforementioned technical problem, the present invention adopts the following technical solution: a method for real-time data compression and storage in the persistent storage area of ​​an SSD. This method adds a data compression unit to the SSD storage system to achieve power-down compression storage and power-on decompression for use of PMR data. The power-down compression storage includes: A. The SSD controller chip detects a power-down signal interruption and triggers the data compression unit to compress the PMR data; B. After compression, output the size of the compressed data, calculate the data compression ratio, which is the ratio of the original data size to the compressed data size. Compare the data compression ratio with a set threshold L. If the data compression ratio > L, store the compressed data in the MRAM medium. If the data compression ratio < L, store the compressed data in the SLC medium. After selecting the storage medium, save the compression information in the PMR area and update the FTL mapping. Electrodecompression uses include: C. Power on the system and configure the PMR region size and compression block size; D. Locate the storage medium storing the PMR compressed data based on the power-off record information, and retrieve the PMR compressed data from the storage medium; E. Based on the extracted compressed data and compression information, decompress the original data, enable the PMR region, and store the decompressed data in the PMR region.

[0005] Furthermore, the data compression unit includes a data blocker, a prefetch buffer, and a processing unit. The data blocker is used to divide the input data to be compressed into fixed-size data blocks and supports a backpressure mechanism to avoid overflow. The prefetch buffer uses double buffering technology to ensure that the processing unit has no waiting time when the data to be compressed is continuously input. The processing unit is used to execute the compression algorithm on the divided data blocks.

[0006] Furthermore, the processing unit uses the ZSTD algorithm to compress the segmented data blocks.

[0007] Furthermore, after compression is complete, the length of the compressed data is updated in the register, and the CPU is notified via an interrupt. The CPU then queries the size of the compressed data by reading the register.

[0008] Furthermore, the PMR area is divided into a compressed data area and a metadata area. The compressed data area is the user data area, used to store the decompressed user data, while the metadata area is used to record compression information, including the location of the compressed block, logical address mapping, and verification information.

[0009] Furthermore, a 32-bit CRC is added to each compressed block for enhanced data verification during data recovery.

[0010] Furthermore, after selecting SLC as the storage medium, the SSD controller chip is configured with the optimal number of concurrent channels and processes.

[0011] Furthermore, compressed data is stored in batches to the selected storage medium via asynchronous threads.

[0012] This invention also discloses a system for real-time data compression and storage in the persistent storage area of ​​an SSD, including a PMR area and a data compression unit. The PMR area is divided into a compressed data area and a metadata area. The compressed data area is the user data area, used to store decompressed user data. The metadata area is used to record compression information, including the location of the compressed block, logical address mapping, and verification information. The data compression unit is used to compress and store the PMR data when power is off and to decompress it when power is on, realizing the separate storage of the metadata area content and the compressed data. Power-down compressed storage includes: A. The SSD controller chip detects a power-down signal interruption and triggers the data compression unit to compress the PMR data; B. After compression, output the size of the compressed data, calculate the data compression ratio, which is the ratio of the original data size to the compressed data size. Compare the data compression ratio with a set threshold L. If the data compression ratio > L, store the compressed data in the MRAM medium. If the data compression ratio < L, store the compressed data in the SLC medium. After selecting the storage medium, save the compression information in the PMR area and update the FTL mapping. Electrodecompression uses include: C. Power on the system and configure the PMR region size and compression block size; D. Locate the storage medium storing the PMR compressed data based on the power-off record information, and retrieve the PMR compressed data from the storage medium; E. Based on the extracted compressed data and compression information, decompress the original data, enable the PMR region, and store the decompressed data in the PMR region.

[0013] Furthermore, the data compression unit includes a data blocker, a prefetch buffer, and a processing unit. The data blocker is used to divide the input data to be compressed into fixed-size data blocks and supports a backpressure mechanism to avoid overflow. The prefetch buffer uses double buffering technology to ensure that the processing unit has no waiting time when the data to be compressed is continuously input. The processing unit is used to execute the compression algorithm on the divided data blocks.

[0014] The beneficial effects of this invention are as follows: This invention divides the PMR space into a dynamic compression buffer and a metadata management area, and combines an adaptive compression algorithm and intelligent selection of storage media to achieve efficient compressed storage of data in persistent memory. This method increases the effective storage capacity of PMR by 3-5 times while ensuring DRAM-level access latency, and reduces write wear on NAND flash memory, expanding the application prospects of PMR functionality. This invention embeds the data compression engine into the PMR management process, improving storage efficiency and capacity utilization, accelerating the storage of large amounts of data, reducing dependence on energy storage capacitors, lowering CPU load, expanding the application scenarios of this function, and achieving a balance between cost and functionality. Attached Figure Description

[0015] Figure 1 A flowchart for power-down compressed storage; Figure 2 Flowchart used for power-on decompression; Figure 3 This is a schematic diagram of the system described in Example 2. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0017] Example 1 This embodiment discloses a method for real-time data compression in the persistent memory (PMR) area of ​​an SSD. The method divides the PMR space into a compressed data area and a metadata area. The compressed data area is the user data area, used to store decompressed user data. The metadata area records compression information through a metadata linked list, including the location of compressed blocks, logical address mapping, and verification information. A data compression unit is used for power-down compression and power-on decompression of PMR data. When power is lost, user data is compressed and stored on the non-volatile storage medium of the SSD, while compressed information is stored in the metadata area. The metadata area supports rapid data recovery after power failure, achieving separate storage of metadata content and compressed data, reducing dependence on energy storage capacitors, and improving storage efficiency and capacity utilization. A hardware data compression unit is designed, using the ZSTD algorithm for adaptive compression of data written to the PMR. After compression, the compressed data size is output for batch flushing to the storage medium using an adaptive storage algorithm.

[0018] like Figure 1 As shown, the power-down compressed storage includes: A. The SSD controller chip detects a power-down signal interruption and triggers the data compression unit to compress the PMR data; B. After compression, output the size of the compressed data and calculate the data compression ratio, which is the ratio of the original data size to the compressed data size. Compare the data compression ratio with the set threshold L. If the data compression ratio > L, store the compressed data in the MRAM medium. If the data compression ratio < L, store the compressed data in the SLC medium. After the storage medium is selected, save the compression information in the PMR area and update the FTL mapping.

[0019] In this embodiment, the data compression unit includes a data block divider, a prefetch buffer, and a processing unit. The data block divider is used to divide the input data to be compressed into fixed-size data blocks, supporting a backpressure mechanism to prevent overflow. The backpressure mechanism means that when dividing data blocks, it is necessary to determine whether the downstream buffer can receive the data. If the downstream can receive it, the block divider receives the data sent from the upper layer; otherwise, it cannot return a data reception completion message to the upper layer. The prefetch buffer uses double buffering technology to ensure that the processing unit has no waiting time when the data to be compressed is continuously input, guaranteeing compression speed. The processing unit is used to execute the compression algorithm on the divided data blocks. In this embodiment, the ZSTD algorithm is used to compress the divided data blocks. This algorithm has advantages such as high compression ratio and speed, meeting the requirements of abnormal power failure.

[0020] After compression is complete, the length of the compressed data is updated in the register, and the CPU is notified via an interrupt. The CPU then queries the size of the compressed data by reading the register.

[0021] To ensure data accuracy, a 32-bit CRC is added to each compressed block for enhanced data verification during data recovery.

[0022] This embodiment includes an adaptive storage algorithm, which selects the operation mode, number of concurrent processes, and number of concurrent media channels for the storage medium. Specifically, it first sets the data compression ratio L. When the compressed data ratio is less than L, the adaptive selection algorithm uses pSLC mode to operate the storage medium and configures the optimal number of concurrent channels and processes to improve storage speed. Conversely, when the ratio is greater than L, the adaptive selection algorithm uses other on-chip non-volatile media (such as a small amount of MRAM) to store data, reducing the occupation of other user data storage channels on the disk. After the storage medium selection is completed, the storage location of each data block is recorded when the corresponding table is updated, facilitating data recovery upon power-up.

[0023] This embodiment uses an asynchronous thread to store compressed data in batches to the selected storage medium.

[0024] like Figure 2 As shown, the electro-decompression process includes: C. Power on the system and configure the PMR region size and compression block size; D. Locate the storage medium storing the PMR compressed data based on the power-off record information, and retrieve the PMR compressed data from the storage medium; E. Based on the extracted compressed data and compression information, decompress the original data, enable the PMR region, and store the decompressed data in the PMR region.

[0025] Example 2 This embodiment discloses a system for real-time data compression and storage of Zhang Yihong's SSD persistent storage area, such as... Figure 3 As shown, it includes a PMR region and a data compression unit ( Figure 3 The ZSTD compression module in the PMR area is divided into a compressed data area and a metadata area. The compressed data area is the user data area, which is used to store the decompressed user data. The metadata area is used to record compression information, including the location of the compressed block, logical address mapping, and verification information. The data compression unit is used to compress and store PMR data when power is off and to decompress it when power is on, so as to realize the separation of the metadata area content and the compressed data storage. Power-down compressed storage includes: A. The SSD controller chip detects a power-down signal interruption and triggers the data compression unit to compress the PMR data; B. After compression, output the size of the compressed data, calculate the data compression ratio, and compare the data compression ratio with the set threshold L. If the data compression ratio > L, store the compressed data in the MRAM medium; if the data compression ratio < L, store the compressed data in the SLC medium. After the storage medium is selected, save the compression information in the PMR area and update the FTL mapping. Electrodecompression uses include: C. Power on the system and configure the PMR region size and compression block size. The compression block size specifies the unit of data volume to be compressed. For example, if you input 4k data, you can compress it as 512 or as 4k. D. Locate the storage medium storing the PMR compressed data based on the power-off record information, and retrieve the PMR compressed data from the storage medium; E. Based on the extracted compressed data and compression information, decompress the original data, enable the PMR region, and store the decompressed data in the PMR region.

[0026] Furthermore, the data compression unit includes a data block divider, a prefetch buffer, and a processing unit. The data block divider is used to divide the input data to be compressed into fixed-size data blocks, supporting a backpressure mechanism to prevent overflow. The backpressure mechanism means that when dividing data into blocks, it needs to determine whether the downstream buffer can receive the data. If the downstream can receive it, the block divider accepts the data sent from the upper layer; otherwise, it cannot return a data acceptance completion message to the upper layer. The prefetch buffer uses double buffering technology to ensure that the processing unit has no waiting time when the data to be compressed is continuously input. The processing unit is used to execute the compression algorithm on the divided data blocks.

[0027] This invention embeds a data compression engine into the PMR management process, improving storage efficiency and capacity utilization, accelerating the storage of large amounts of data, reducing reliance on energy storage capacitors, lowering CPU load, expanding the application scenarios of this function, and achieving a balance between cost and functionality.

[0028] The above description is merely the basic principle and preferred embodiment of the present invention. Improvements and substitutions made by those skilled in the art based on the present invention are within the scope of protection of the present invention.

Claims

1. A method for real-time data compression and storage in the persistent storage area of ​​an SSD, characterized in that: This method adds a data compression unit to the SSD storage system to achieve power-down compression storage and power-on decompression of PMR data. The power-down compression storage includes: A. The SSD controller chip detects a power-down signal interruption and triggers the data compression unit to compress the PMR data; B. After compression, output the size of the compressed data, calculate the data compression ratio, which is the ratio of the original data size to the compressed data size. Compare the data compression ratio with a set threshold L. If the data compression ratio > L, store the compressed data in the MRAM medium. If the data compression ratio < L, store the compressed data in the SLC medium. After selecting the storage medium, save the compression information in the PMR area and update the FTL mapping. Electrodecompression uses include: C. Power on the system and configure the PMR region size and compression block size; D. Locate the storage medium storing the PMR compressed data based on the power-off record information, and retrieve the PMR compressed data from the storage medium; E. Based on the extracted compressed data and compression information, decompress the original data, enable the PMR region, and store the decompressed data in the PMR region.

2. The method for real-time data compression and storage in the persistent storage area of ​​an SSD according to claim 1, characterized in that: The data compression unit includes a data block divider, a prefetch buffer, and a processing unit. The data block divider is used to divide the input data to be compressed into fixed-size data blocks and supports a back pressure mechanism to avoid overflow. The prefetch buffer uses double buffering technology to ensure that the processing unit has no waiting time when the data to be compressed is continuously input. The processing unit is used to execute the compression algorithm on the divided data blocks.

3. The method for real-time data compression and storage in the persistent storage area of ​​an SSD according to claim 2, characterized in that: The processing unit uses the ZSTD algorithm to compress the segmented data blocks.

4. The method for real-time data compression and storage in the persistent storage area of ​​an SSD according to claim 1 or 2, characterized in that: After compression is complete, the length of the compressed data is updated in the register, and the CPU is notified via an interrupt. The CPU then queries the size of the compressed data by reading the register.

5. The method for real-time data compression and storage in the persistent storage area of ​​an SSD according to claim 1, characterized in that: The PMR area is divided into a compressed data area and a metadata area. The compressed data area is the user data area, which is used to store the decompressed user data. The metadata area is used to record compression information, including the location of the compressed block, logical address mapping, and verification information.

6. The method for real-time data compression and storage in the persistent storage area of ​​an SSD according to claim 1, characterized in that: Add a 32-bit CRC to each compressed block for enhanced data verification during data recovery.

7. The method for real-time data compression and storage in the persistent storage area of ​​an SSD according to claim 1, characterized in that: After selecting SLC as the storage medium, the SSD controller chip is configured with the optimal number of concurrent channels and processes.

8. The method for real-time data compression and storage in the persistent storage area of ​​an SSD according to claim 1, characterized in that: Compressed data is stored in batches to the selected storage medium via asynchronous threads.

9. A system for real-time data compression and storage in the persistent storage area of ​​an SSD, characterized in that: It includes a PMR area and a data compression unit. The PMR area is divided into a compressed data area and a metadata area. The compressed data area is the user data area, which is used to store the decompressed user data. The metadata area is used to record compression information, including the location of the compressed block, logical address mapping, and verification information. The data compression unit is used to compress and store PMR data when power is off and decompress it when power is on, realizing the separation of metadata area content from compressed data storage. Power-down compressed storage includes: A. The SSD controller chip detects a power-down signal interruption and triggers the data compression unit to compress the PMR data; B. After compression, output the size of the compressed data, calculate the data compression ratio, which is the ratio of the original data size to the compressed data size. Compare the data compression ratio with a set threshold L. If the data compression ratio > L, store the compressed data in the MRAM medium. If the data compression ratio < L, store the compressed data in the SLC medium. After selecting the storage medium, save the compression information in the PMR area and update the FTL mapping. Electrodecompression uses include: C. Power on the system and configure the PMR region size and compression block size; D. Locate the storage medium storing the PMR compressed data based on the power-off record information, and retrieve the PMR compressed data from the storage medium; E. Based on the extracted compressed data and compression information, decompress the original data, enable the PMR region, and store the decompressed data in the PMR region.

10. The system for real-time data compression and storage of SSD persistent storage area according to claim 9, characterized in that: The data compression unit includes a data block divider, a prefetch buffer, and a processing unit. The data block divider is used to divide the input data to be compressed into fixed-size data blocks and supports a back pressure mechanism to avoid overflow. The prefetch buffer uses double buffering technology to ensure that the processing unit has no waiting time when the data to be compressed is continuously input. The processing unit is used to execute the compression algorithm on the divided data blocks.

Images