Transparent compression and intelligent caching method for SSD (Solid State Disk) storage system

Abstract

The invention relates to the field of computer storage systems, in particular to a transparent compression and intelligent caching method of an SSD storage system. The method comprises the following steps of: compressing and storing one or more write-in requests of a logic page: caching one or more continuous write-in requests of the logic page by utilizing an aligned cache slot position corresponding to the logic page in a DRAM (Dynamic Random Access Memory) cache region, the aligned cache slot performs the following operations when caching the write-in request: monitoring a bitmap and a state identifier of the aligned cache slot, when the bitmap of the aligned cache slot displays that data are written into all the sub-data blocks or the difference between the first write-in timestamp or the latest write-in timestamp in the state identifier and the current time exceeds an overtime threshold, triggering aggregation submission of the aligned cache slot; the write-in data cached in the aligned cache slots are compressed, the compression operation of any aligned cache slot is triggered by the corresponding aggregation submission, and the compressed data are written into the SLC cache region.

Description

Technical Field

[0001] The present invention relates to the field of computer storage systems, and in particular to a transparent compression and intelligent caching method for an SSD storage system. Background Art

[0002] In an SSD storage system, its controller typically maintains an L2P (Logical-to-Physical) mapping table between LP (Logical Page) and PP (Physical Page) in DRAM (Dynamic Random-Access Memory), and divides the NAND (Flash Memory) into SLC or pseudo-SLC (Single-Level Cell) cache areas and TLC (Triple-Level Cell) cache areas or QLC (Quad-Level Cell) storage areas.

[0003] In high-frequency, small random write scenarios (such as database logs, file system metadata, or incremental writes to container images), the SSD controller performs inline compression on each data write, generating compressed segments that are insufficient to fill a complete physical NAND page. These segments are then cached in the SLC cache and then migrated to the TLC cache / QLC cache for storage in the background. Because these fragmented compressed segments are scattered across different storage areas, garbage collection (GC) must read and write entire blocks, resulting in a surge in write amplification. During this process, the FTL (Flash Translation Layer) must maintain an L2P mapping entry for each variable compressed segment, which maps to "logical page-offset-data length-physical page number." This significantly increases the number of L2P mapping entries in DRAM and significantly increases table lookup and update latency. Summary of the Invention

[0004] The present invention provides a transparent compression and intelligent caching method for an SSD storage system, which compresses and stores one or more write requests for a logical page through the following steps: S01. Utilize an aligned cache slot corresponding to the logical page in the DRAM cache area to cache one or multiple consecutive write requests for the logical page. The aligned cache slot performs the following operations when caching the write request: Monitor the bitmap and status identifier of the aligned cache slot, and trigger the aggregation submission of the aligned cache slot when the bitmap of the aligned cache slot shows that all sub-data blocks have been written or when the difference between the first write timestamp or the latest write timestamp in the status identifier and the current time exceeds a timeout threshold; S02. Compress the write data in the aligned cache slots. The compression operation of any aligned cache slot is triggered by its corresponding aggregate commit. When an aligned cache slot is triggered to commit, the following operations are performed: Compress the write data in this cache slot for the first time to obtain a first compressed segment; write the first compressed segment to the SLC cache area, allocate a first physical page number based on the physical position of the first compressed segment in the SLC cache area, and record a mapping entry from the logical page to the first physical page number in the L2P mapping table.

[0005] In some examples, any of the aligned cache slots in the transparent compression and intelligent caching method of the SSD storage system consists of several sub-data blocks of the same size, a bitmap corresponding to the several sub-data blocks of the same size, and a status identifier recording the aligned cache slot.

[0006] In some examples, the transparent compression and intelligent caching method for the SSD storage system further includes the following steps: Monitor the amount of data written for each write request: When the amount of written data is less than or equal to the data amount threshold, directing the write request to the corresponding aligned cache slot for temporary storage; When the write data amount exceeds a data amount threshold, the write request is directly written into the SLC cache area.

[0007] In some examples, the data volume threshold in the transparent compression and intelligent caching method of the SSD storage system is set based on the granularity of the sub-data blocks in the aligned cache slots.

[0008] In some examples, the data volume threshold in the transparent compression and intelligent caching method of the SSD storage system is the same as the size of the sub-data block in the aligned cache slot.

[0009] In some examples, the data volume threshold in the transparent compression and intelligent caching method for the SSD storage system is set by the following steps: Get the current available resources and current load of the DRAM cache; Generate the current optimal data volume threshold based on currently available resources and current load; Based on the current optimal data volume threshold, adjust the data volume threshold at the next moment.

[0010] In some examples, the optimal data volume threshold in the transparent compression and intelligent caching method of the SSD storage system is such that: When the utilization of the aligned cache slots in the DRAM cache area approaches 0, each aligned cache slot exhibits the maximum aggregate wait time; When the aligned cache slot utilization in the DRAM cache area approaches 1, the aggregate waiting time of each aligned cache slot at least meets the basic write latency performance indicator.

[0011] In some examples, the transparent compression and intelligent caching method for an SSD storage system further includes the following steps: The first compressed segment is compressed for the second time to obtain a second compressed segment, and the second compressed segment with a smaller data volume after compression is written into the SLC cache area. Two physical page numbers are allocated based on the physical position of the second compressed segment with a smaller data volume in the SLC cache area, and the corresponding mapping entry in the L2P mapping table is updated.

[0012] In some examples, the transparent compression and intelligent caching method for an SSD storage system further includes the following steps: For a number of storage data indexed by a number of physical page numbers in the SLC cache area, the access frequency of each physical page number in the SLC cache area is calculated, and the storage data corresponding to the physical pages that do not exceed the access frequency threshold are migrated to the TLC high-density area.

[0013] In some examples, the size of any of the aligned cache slots in the transparent compression and intelligent caching methods of the SSD storage system is the same as the size of a physical page.

[0014] The transparent compression and intelligent caching method for an SSD storage system provided by the present invention has benefits including but not limited to: By aggregating multiple write operations that are less than a page in size into complete original data blocks and then performing batch compression writes, this method effectively avoids the problems of reduced space utilization and redundant erasure caused by "half-page" or smaller storage fragmentation caused by small-scale writes in traditional solutions. This mechanism can significantly reduce the write amplification factor, thereby improving the durability of NAND flash memory devices. Furthermore, under the aggregate write mechanism, the present invention implements a one-to-one mapping relationship between logical page numbers (LPNs) and physical page numbers (PPNs), eliminating the need to maintain variable-length compressed segment offset and length fields. This significantly reduces the number of entries and storage capacity in the L2P mapping table, helping to improve the processing efficiency of address mapping queries and updates. Furthermore, the present invention adopts a page alignment aggregation strategy to ensure that compressed data segments in the SLC and TLC storage areas are written in units of complete pages, effectively solving the page-level and block-level storage fragmentation problems caused by the scattered storage of variable-length data segments.

[0015] Furthermore, the present invention uses a dual protection mechanism of full bitmap triggering and timeout forced submission to ensure that the optimal data aggregation rate is achieved under high write load scenarios, and that the aggregation waiting time is strictly controlled within the preset threshold range in the distributed write mode, thereby achieving a good balance between throughput and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 shows the aligned cache slots of several logical pages in one example of the present invention; Figure 2 The following shows a process for setting a data volume threshold in an example of the present invention; Figure 3 The transparent compression and intelligent caching method flow in an example of the present invention is shown. DETAILED DESCRIPTION

[0017] In the following description, for the purpose of illustration rather than limitation, specific details such as specific systems, structures, and technologies are provided to facilitate a thorough understanding of the examples of this application. It should be clear to those skilled in the art that the present application can also be implemented in other examples without these specific details.

[0018] In response to the frequent small random write operations in the above-mentioned high-frequency small random write scenario, the present invention provides a transparent compression and intelligent caching method for an SSD storage system to avoid waste of storage space and management resources.

[0019] In one example of the present invention, Figure 1 As shown, for write requests of different logical pages, an aligned cache slot is allocated from the DRAM cache area in the SSC storage system to cache one or multiple consecutive small write requests of the corresponding logical page, and the data amount of the small write request is less than or equal to a preset data amount threshold.

[0020] like Figure 1 As shown, for logical pages LP1 to LP N Any logical page LP in i , divide the aligned cache slots S from the DRAM cache area i； Any of the aligned cache slots is composed of a number of sub-data blocks of the same size, a bitmap corresponding to the number of sub-data blocks of the same size, and a status identifier recording the aligned cache slot.

[0021] Specifically, a binary bitmap with a length of K is set in any of the aligned cache slots, where K is equal to the total number of sub-data blocks divided by the aligned cache slot; the j-th bit of the bitmap corresponds to the j-th sub-data block, and when the sub-data block receives and stores data written by the host, the corresponding bit is set from "0" to "1", otherwise it remains at "0", so as to determine whether the slot has accumulated the preset data volume threshold by judging whether the bitmap is all "1".

[0022] At the same time, each aligned cache slot also carries a set of status identifiers, which include a status field and a timestamp field. The status field is used to indicate the current life cycle stage of the aligned cache slot, and its values ​​include "idle", "aggregating" and "pending submission": initially, the status field is "idle". After the first sub-data block is written, the status field is set from "idle" to "aggregating", and the current time is recorded in the timestamp field to start timeout monitoring; when the bitmap is full or the time difference between the current time and the time recorded in the timestamp field exceeds the preset timeout threshold, the status field is set to "pending submission", and the aggregated data in the aligned cache slot is triggered to enter the subsequent compression and physical writing process.

[0023] It should be noted that the data volume threshold in this example refers to the maximum aggregate data volume that a single aligned cache slot can accommodate. Typically, the data volume threshold is the same as the size of the logical page (e.g., 4kB, which is the maximum amount of data that can be written to a logical page).

[0024] In some other embodiments, the data size threshold is consistent with the sub-data block size in the aligned cache slot (e.g., 256B, 512B). Specifically, when the sub-data block size is 256B, the data size threshold can be set to 256B; in this case, any write request less than or equal to 256B will be restricted to writing to a single sub-data block, and the bitmap bit of the sub-data block will be set to 1; similarly, if the sub-data block size is 512B, the data size threshold can be set to 512B to ensure that each small write request occupies only one sub-block, thereby achieving a balance between granularity and latency.

[0025] In some other embodiments, the data volume threshold is dynamically adjusted according to the cache resources. Figure 2 Adjust the process shown: S001. Obtain the current available resources and current load of the DRAM cache area.

[0026] Specifically, the current available resources and current load of a logical page in the DRAM cache area can be represented by calculating the corresponding utilization rate by the number of all aligned cache slots corresponding to this logical page in the DRAM cache area and the number of currently used aligned cache slots.

[0027] Furthermore, the utilization rates described in these embodiments satisfy the following calculation model: , in, Represents a logical page Resource utilization in the DRAM cache area, Represents a logical page The number of aligned cache slots currently used in the DRAM cache area, Represents a logical page The total number of aligned cache slots in the DRAM cache area.

[0028] S002. Generate a current optimal data volume threshold based on currently available resources and current load.

[0029] Specifically, in these embodiments, the current optimal data volume threshold is set such that: When the utilization of the aligned cache slots in the DRAM cache area approaches 0 (i.e., U≈0), the aggregate wait time of each aligned cache slot presents the maximum aggregate wait time; when the utilization of the aligned cache slots in the DRAM cache area approaches 1 (i.e., U≈1), the aggregate wait time of each aligned cache slot at least meets the basic write latency performance indicator.

[0030] It can be understood that the maximum aggregation wait time refers to the maximum time each slot is allowed to wait (determined by the timeout threshold or the upper limit of the delay based on the first write) in order to obtain the highest aggregation rate (that is, accumulate as many small writes as possible) when the utilization approaches 0 (that is, U≈0).

[0031] It is also understandable that the basic write latency performance indicator refers to when the utilization approaches 1 (that is, U≈1). In order to prevent write requests from exceeding the tolerable delay of the business due to aggregation, it is necessary to ensure that the waiting time of each aggregation at least meets this indicator (for example, 1ms or the write latency indicator required by the hardware manufacturer, such as NVDIMM write latency).

[0032] Furthermore, the current optimal data volume threshold described in these embodiments satisfies the following calculation model: , in, Indicates the logical page at the current time t The optimal data volume threshold, Indicates the minimum aggregation threshold set based on the write latency performance indicator (for example, to ensure fast commit in latency-sensitive scenarios, it is set to 512B). Indicates the logical page at the current time t Resource utilization in the DRAM cache area, Represents a logical page size.

[0033] S003. Adjust the data volume threshold at the next moment based on the current optimal data volume threshold.

[0034] Specifically, in order to avoid large fluctuations in the data volume threshold, in these embodiments, a smoothing coefficient ranging from 0 to 1 is introduced to optimize the data volume threshold at the next moment.

[0035] Furthermore, the data volume threshold at the next moment satisfies the following calculation model: , in, Indicates the next logical page at the next moment t+1 The data volume threshold, Indicates the logical page at the current time t The data volume threshold, Represents the smoothing coefficient. The smaller the value is, the smoother the update is and the stronger the anti-noise ability is. Indicates the logical page at the current time t The optimal data volume threshold.

[0036] In one example, for at least one aligned cache slot divided from a DRAM cache area by a logical page, small random write operations that frequently occur in the above-mentioned high-frequency small random write scenario are performed by Figure 3 The transparent compression and intelligent caching method shown achieves compression and storage: S01. Utilize an aligned cache slot corresponding to the logical page in the DRAM cache area to cache one or multiple consecutive write requests for the logical page. The aligned cache slot performs the following operations when caching the write request: Monitor the bitmap and status identifier of the aligned cache slot, and trigger the aggregation submission of the aligned cache slot when the bitmap of the aligned cache slot shows that all sub-data blocks have been written with data or the difference between the first write timestamp or the latest write timestamp in the status identifier and the current time exceeds the timeout threshold.

[0037] S02. Compress the write data in the aligned cache slots. The compression operation of any aligned cache slot is triggered by its corresponding aggregate commit. When an aligned cache slot is triggered to commit, the following operations are performed: Compress the write data in this cache slot for the first time to obtain a first compressed segment; write the first compressed segment to the SLC cache area, allocate a first physical page number based on the physical position of the first compressed segment in the SLC cache area, and record a mapping entry from the logical page to the first physical page number in the L2P mapping table.

[0038] In this example, for one or multiple consecutive write requests to a logical page in a high-frequency small random write scenario, a dedicated aligned cache slot is allocated for the logical page in the DRAM cache area to accumulate multiple small write requests until the bitmap is full or a timeout is triggered. The aggregated entire page data is then uniformly compressed and written to the physical page of the SLC area at one time, thereby achieving page-level aligned aggregation and batch compressed writing of small write requests.

[0039] Compared with the existing technology, this example allocates a dedicated aligned cache slot for each logical page in the DRAM cache area, tracks multiple writes smaller than one page through a bitmap, and compresses the entire page data at one time before writing it to the SLC physical page after the page is full or timed out. This not only significantly reduces the write amplification ratio and mapping table metadata overhead, but also eliminates cache and GC fragmentation. It also achieves a balance between high throughput and controllable latency based on the dual mechanisms of aggregation triggering and timeout triggering.

[0040] Based on the transparent compression and intelligent caching method provided by the above example, in some specific embodiments, the size of the aligned cache slots divided for any logical page in a high-frequency small random write scenario is 4kB, which is the same as the size of any physical page, thereby achieving a one-to-one mapping between logical pages and physical pages, avoiding cross-page writes and fragmentation, and helping to simplify L2P management.

[0041] Furthermore, in these embodiments, each 4kB aligned cache slot is further divided into several sub-data blocks of the same size, such as 8 512B sub-data blocks or 16 256B sub-data blocks, 4 1KB sub-data blocks, etc.

[0042] Furthermore, each sub-data block corresponds to a bit in the bitmap, which is used to record whether the sub-data block has been written: "0" indicates not written, and "1" indicates written; at the same time, each aligned cache slot maintains a set of status identifiers, including a status field and a timestamp field; the status field includes "Idle", "Active" or "Ready", which respectively correspond to the slot not being accessed, at least one sub-block being written and the commit condition not being met, the bitmap being full or the timeout threshold being met and waiting for downstream processing; the timestamp field records the time of the most recent sub-data block write, which is used to compare with the preset timeout threshold (for example, 1ms) in the "aggregating" state to determine whether to set the slot state from "aggregating" to "pending submission".

[0043] Based on the transparent compression and intelligent caching method provided by the above example, in some specific embodiments, the first compression described in step S02 is implemented by the hardware-accelerated LZ4 streaming compression engine built into the controller, and the aggregated 4KB original data block is quickly compressed in a pipeline manner to generate a first compressed segment with a stable length. During the compression process, the compressed length information is written into the SLC cache area for subsequent physical page writing and decompression.

[0044] To focus on optimizing small random writes and avoid unnecessary impact of large writes on the aggregate cache, in one example of the present invention, the transparent compression and intelligent caching method further includes the following steps: monitoring the amount of written data for each write request: When the write data volume is less than or equal to the data volume threshold, the write request is directed to the corresponding aligned cache slot for temporary storage; when the write data volume exceeds the data volume threshold, the write request is directly written to the SLC cache area.

[0045] Furthermore, the data volume threshold described in this example can be set with reference to the setting method in the above example. In some specific embodiments, the data volume threshold is set based on the granularity of the sub-data block within the aligned cache slot. For example, the data volume threshold is the same as the size of the sub-data block within the aligned cache slot: when the sub-data block size is 512B, the data volume threshold is set to 512B; when the sub-data block size is 256B, the data volume threshold is set to 256B; and so on, to ensure that each write request occupies at most one sub-data block.

[0046] In some other specific embodiments, the data volume threshold is regularly updated based on the above steps S001 to S003, so that each aligned cache slot can achieve an adaptive balance between performance stability and space utilization based on real-time cache resource usage.

[0047] To further improve space utilization, in one example of the present invention, the transparent compression and intelligent caching method further includes the following steps: The first compressed segment is compressed for the second time to obtain a second compressed segment, and the second compressed segment with a smaller data volume after compression is written into the SLC cache area. Two physical page numbers are allocated based on the physical position of the second compressed segment with a smaller data volume in the SLC cache area, and the corresponding mapping entry in the L2P mapping table is updated.

[0048] It should be noted that if the data volume remains unchanged or increases abnormally after the second compression, the first compressed segment is retained and not updated. Based on further adjustments in this example, in some specific embodiments, the second compression uses Zstandard high-pressure mode compression to compress the written first compressed segment into a smaller second compressed segment for storage without affecting online I / O, thereby minimizing NAND Flash write volume and occupied space.

[0049] To further improve space utilization, in one example of the present invention, the transparent compression and intelligent caching method further includes the following steps: For a number of storage data indexed by a number of physical page numbers in the SLC cache area, the access frequency of each physical page number in the SLC cache area is calculated, and the storage data corresponding to the physical pages that do not exceed the access frequency threshold are migrated to the TLC high-density area.

[0050] Specifically, the controller in the SSD system can maintain an access counter (or sliding window LRU structure) for each physical page in the SLC area during the regular I / O monitoring process. The counter will increment with each read or write operation on the page and periodically decay based on the global clock or I / O event trigger to reflect recent heat.

[0051] For example, by applying an exponential decay function to the historical access frequency, access events are weighted in a way that the weight decreases exponentially over time, so that the heat evaluation results are more sensitive to the latest access; further, a sliding window mechanism is used to regularly update the heat value after decay to ensure that the heat evaluation always reflects the characteristics of recent access. Based on initial heat and the time difference between adjacent accesses According to the formula Calculate, where is the attenuation coefficient; and by using the preset threshold θ, pages with a heat value greater than θ are identified as hot spots for subsequent cache retention or migration strategies; pages with a heat value lower than θ are determined to be cold data and can be migrated in batches to high-density storage areas.

[0052] Through the above method: the heat value of the page that is continuously visited is maintained at a high level (i.e. H>θ), while the heat value of the page that is terminated will decrease exponentially according to the attenuation coefficient α, which is , and then by setting a reasonable heat threshold θ, accurate identification of hot data can be achieved, providing a theoretical basis for the data migration strategy of the storage system.

[0053] In the above examples, the description of each example has its own focus. For parts that are not described or recorded in detail in an example, please refer to the relevant descriptions of other examples.

[0054] It should be noted that the above examples can be freely combined as needed. The above are only preferred embodiments of the present invention; it should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A transparent compression and intelligent caching method for an SSD storage system, characterized in that: For one or more write requests to a logical page, compression and storage are achieved through the following steps: S01. Utilize an aligned cache slot corresponding to the logical page in the DRAM cache area to cache one or multiple consecutive write requests for the logical page. The aligned cache slot performs the following operations when caching the write request: Monitor the bitmap and status identifier of the aligned cache slot, and trigger the aggregation submission of the aligned cache slot when the bitmap of the aligned cache slot shows that all sub-data blocks have been written or when the difference between the first write timestamp or the latest write timestamp in the status identifier and the current time exceeds a timeout threshold; S02. Compress the write data in the aligned cache slots. The compression operation of any aligned cache slot is triggered by its corresponding aggregate commit. When an aligned cache slot is triggered to commit, the following operations are performed: Compress the write data in this cache slot for the first time to obtain a first compressed segment; write the first compressed segment to the SLC cache area, allocate a first physical page number based on the physical position of the first compressed segment in the SLC cache area, and record a mapping entry from the logical page to the first physical page number in the L2P mapping table.

2. The transparent compression and intelligent caching method for an SSD storage system according to claim 1, wherein: Any of the aligned cache slots is composed of a number of sub-data blocks of the same size, a bitmap corresponding to the number of sub-data blocks of the same size, and a status identifier recording the aligned cache slot.

3. The transparent compression and intelligent caching method for an SSD storage system according to claim 1, wherein: The following steps are also included: Monitor the amount of data written for each write request: When the amount of written data is less than or equal to the data amount threshold, directing the write request to the corresponding aligned cache slot for temporary storage; When the write data amount exceeds a data amount threshold, the write request is directly written into the SLC cache area.

4. The transparent compression and intelligent caching method for an SSD storage system according to claim 3, wherein: The data volume threshold is set based on the granularity of the sub-data blocks in the aligned cache slots.

5. The transparent compression and intelligent caching method for an SSD storage system according to claim 4, wherein: The data amount threshold is the same as the size of the sub-data block in the aligned cache slot.

6. The transparent compression and intelligent caching method for an SSD storage system according to claim 3, wherein: The data volume threshold is set by the following steps: Get the current available resources and current load of the DRAM cache; Generate the current optimal data volume threshold based on currently available resources and current load; Based on the current optimal data volume threshold, adjust the data volume threshold at the next moment.

7. The transparent compression and intelligent caching method for an SSD storage system according to claim 6, wherein: The optimal data volume threshold is such that: When the utilization of the aligned cache slots in the DRAM cache area approaches 0, each aligned cache slot exhibits the maximum aggregate wait time; When the aligned cache slot utilization in the DRAM cache area approaches 1, the aggregate waiting time of each aligned cache slot at least meets the basic write latency performance indicator.

8. The transparent compression and intelligent caching method for an SSD storage system according to claim 1, wherein: The following steps are also included: The first compressed segment is compressed for the second time to obtain a second compressed segment, and the second compressed segment with a smaller data volume after compression is written into the SLC cache area. Two physical page numbers are allocated based on the physical position of the second compressed segment with a smaller data volume in the SLC cache area, and the corresponding mapping entry in the L2P mapping table is updated.

9. The transparent compression and intelligent caching method for an SSD storage system according to claim 1, wherein: The following steps are also included: For a number of storage data indexed by a number of physical page numbers in the SLC cache area, the access frequency of each physical page number in the SLC cache area is calculated, and the storage data corresponding to the physical pages that do not exceed the access frequency threshold are migrated to the TLC high-density area.

10. The transparent compression and intelligent caching method for an SSD storage system according to any one of claims 1 to 9, characterized in that: The size of any of the aligned cache slots is the same as the size of a physical page.

Images