A hierarchical cache system, method, apparatus and storage device

Abstract

This application discloses a hierarchical caching system, method, apparatus, and storage device, relating to the field of computer technology. By setting up three layers of cache areas, namely the first layer cache area, the second layer cache area, and the third layer cache area, it realizes the accurate division and partitioning of high-hot data, regular hot data, and warm data, and realizes the migration of data in different cache areas due to changes in popularity, so as to avoid flushing the data that the host is about to access to the cache module, thereby improving the cache hit rate.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a hierarchical caching system, method, apparatus and storage device. Background Technology

[0002] With the explosive growth of data-intensive applications such as cloud computing, big data analytics, and artificial intelligence, data storage volume is increasing exponentially, posing challenges to storage system performance, such as low latency, high bandwidth, and capacity. Therefore, the Compute Express Link (CXL) protocol and Solid State Drives (SSDs) have received widespread attention as key technologies for addressing storage performance bottlenecks.

[0003] In related technologies, solid-state drives (SSDs) typically consist of a storage controller and multiple flash memory modules. The storage controller integrates a cache, which usually caches frequently used data. The host accesses the cache based on the CXL protocol to quickly retrieve frequently used data. However, because the controller's cache storage space is limited, data needs to be flushed frequently, and the data that is frequently flushed is data that the host is about to access, which reduces the cache hit rate. Summary of the Invention

[0004] This application provides a hierarchical caching system, method, apparatus, and storage device to at least address the problem of reduced cache hit rate in related technologies.

[0005] This application provides a hierarchical caching system, including: a heat monitoring module and a multi-layer caching module. The multi-layer caching module includes at least a first-layer cache area, a second-layer cache area, and a third-layer cache area. The first-layer cache area, the second-layer cache area, and the third-layer cache area are connected through a direct memory access bus. The capacity and response latency of the first-layer cache area, the second-layer cache area, and the third-layer cache area increase sequentially. The heat monitoring module is used to monitor the temperature of high-heat data cached in the first-layer buffer, regular heat data cached in the second-layer buffer, and warm data cached in the third-layer buffer, so as to obtain the temperature monitoring results of high-heat data, regular heat data, and warm data. Specifically, if the temperature monitoring results of high-temperature data, regular-temperature data, or warm-temperature data indicate that the high-temperature data, regular-temperature data, or warm-temperature data meet the preset migration conditions, based on the direct memory access bus, the high-temperature data is migrated as regular-temperature data to the second-level cache, or the regular-temperature data is migrated as high-temperature data or warm-temperature data to the first-level cache or the third-level cache, or the warm-temperature data is migrated as regular-temperature data to the second-level cache.

[0006] This application also provides a hierarchical caching method, applied to any of the above-mentioned hierarchical caching systems, the method comprising: Obtain the temperature monitoring results of high-heat data in the first-layer cache, the temperature monitoring results of regular heat data in the second-layer cache, and the temperature monitoring results of mild data in the third-layer cache in the hierarchical caching system. If the high-temperature data monitoring results, regular-temperature data monitoring results, or warm-temperature data monitoring results indicate that the high-temperature data, regular-temperature data, or warm-temperature data meet the preset migration conditions, based on the direct memory access bus, the high-temperature data is migrated as regular-temperature data to the second-level buffer, or the regular-temperature data is migrated as high-temperature data or warm-temperature data to the first-level buffer or the third-level buffer, or the warm-temperature data is migrated as regular-temperature data to the second-level buffer.

[0007] This application also provides a hierarchical caching device for use in any of the above-described hierarchical caching systems, the device comprising: The acquisition module is used to acquire the temperature monitoring results of high-heat data in the first layer of the hierarchical caching system, the temperature monitoring results of regular heat data in the second layer of the caching system, and the temperature monitoring results of mild data in the third layer of the caching system. The caching module is used to migrate high-temperature data as regular-temperature data to the second-level cache area, or regular-temperature data as high-temperature data to the first-level cache area or the third-level cache area, or warm-temperature data as regular-temperature data to the second-level cache area, based on the direct memory access bus, when the high-temperature data, regular-temperature data, or warm-temperature data monitoring results indicate that the high-temperature data, regular-temperature data, or warm-temperature data meet the preset migration conditions.

[0008] This application also provides a storage device, including: a storage controller, wherein the storage controller includes any of the above-described hierarchical caching systems; The storage controller performs data tiered caching in the tiered caching system based on any of the above-mentioned tiered caching methods.

[0009] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing any of the above-described hierarchical caching methods when executing the computer program.

[0010] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described hierarchical caching methods.

[0011] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described hierarchical caching methods.

[0012] This application achieves precise partitioning and caching of high-hot data, regular hot data, and warm data by setting up three-layer cache areas: the first layer, the second layer, and the third layer. It also enables data to migrate between different cache areas due to changes in popularity, thereby avoiding flushing data that the host is about to access to the cache module and improving the cache hit rate. Attached Figure Description

[0013] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the structure of a hierarchical caching system provided in an embodiment of this application; Figure 2 This is a schematic diagram of another hierarchical caching system provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the application process of an exemplary hierarchical caching system provided in this application embodiment; Figure 4 A flowchart illustrating the hierarchical caching method provided in this application embodiment; Figure 5 A schematic diagram of the structure of the hierarchical caching device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of a storage device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0016] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0017] With the explosive growth of data-intensive applications such as cloud computing, big data analytics, and artificial intelligence, data storage volume is increasing exponentially, posing a dual challenge to storage system performance, including low latency, high bandwidth, and capacity. In traditional storage architectures, the communication efficiency between computing units and storage devices is limited by the bandwidth and latency bottlenecks of the PCIe bus, making it difficult to meet the real-time data access requirements of high-performance computing scenarios. Against this backdrop, the CXL protocol and solid-state drives (SSDs) have become key technologies for solving storage performance bottlenecks, and their integrated application is considered a core direction for breakthroughs.

[0018] CXL, as a PCIe-based interconnect protocol, enables efficient interconnection between computing units and storage devices. It supports multiple communication modes such as CXL.io, CXL.cache, and CXL.memory, which can significantly reduce end-to-end communication latency and increase bandwidth. At the same time, it natively supports cross-device cache consistency, reducing data synchronization overhead and providing a technical foundation for the collaborative work of storage devices and host memory.

[0019] SSDs are based on NAND flash memory and consist of a controller, cache, and flash arrays such as Tri-Level Cell (TLC) or Quad-Level Cell (QLC). Due to their lack of mechanical structure, their random access latency is significantly lower than that of mechanical hard drives. However, traditional SSD caching mechanisms are mostly fixed-level (such as a level 2 cache architecture) and employ simple replacement strategies like Least Recently Used (LRU). This makes it difficult to differentiate data access frequency, resulting in low caching efficiency for high-frequency data and a waste of cache resources and bandwidth.

[0020] The integration of CXL and SSD, along with optimized hybrid SSD designs, can fully leverage the advantages of both: CXL's high bandwidth and low latency improve communication efficiency between the SSD and the host; hybrid SSDs, through the synergy of high- and low-density flash memory, such as using Single-Level Cell (SLC) as a cache for TLC, can balance performance and capacity. However, current CXL-SSD integration solutions mainly fall into two categories: one is the CXL interface and traditional SSD caching, such as using only Dynamic Random Access Memory (DRAM) as a single cache; the other is hybrid flash memory and the CXL protocol (such as a hybrid SSD with SLC caching TLC connected to the CXL bus). Both solutions suffer from limitations such as fixed cache levels and lack of differentiated data processing based on data hotness, making it difficult to meet the demands of high-performance computing scenarios.

[0021] To address the aforementioned technical problems, this application provides a hierarchical caching system, method, apparatus, and storage device. The system includes a heat monitoring module and a multi-layer caching module. The multi-layer caching module includes at least a first-layer cache, a second-layer cache, and a third-layer cache. The first-layer cache, second-layer cache, and third-layer cache are connected via a direct memory access bus. The capacity and response latency of the first-layer cache, second-layer cache, and third-layer cache increase sequentially. The heat monitoring module is used to monitor high-frequency data cached in the first-layer cache, regular high-frequency data cached in the second-layer cache, and high-frequency data cached in the third-layer cache. Temperature monitoring is performed on the cached temperature data to obtain temperature monitoring results for high-temperature data, regular-temperature data, and warm-temperature data. When the temperature monitoring results for high-temperature data, regular-temperature data, or warm-temperature data indicate that the high-temperature data, regular-temperature data, or warm-temperature data meet preset migration conditions, based on the direct memory access bus, the high-temperature data is migrated as regular-temperature data to the second-level cache, or the regular-temperature data is migrated as high-temperature data or warm-temperature data to the first-level or third-level cache, or the warm-temperature data is migrated as regular-temperature data to the second-level cache. The system provided by the above solution, by setting up three levels of cache (first-level, second-level, and third-level cache), achieves precise partitioning and caching of high-temperature data, regular-temperature data, and warm-temperature data. It also enables data migration between different caches due to changes in temperature, avoiding flushing data about to be accessed by the host to the cache module, thereby improving the cache hit rate.

[0022] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] This application provides a hierarchical caching system for partitioning and caching data on a storage controller.

[0024] like Figure 1 The diagram shown is a schematic of a hierarchical caching system provided in an embodiment of this application. The system includes a heat monitoring module and a multi-layer caching module. The multi-layer caching module includes at least a first-layer cache area, a second-layer cache area, and a third-layer cache area. The first-layer cache area, the second-layer cache area, and the third-layer cache area are connected through a direct memory access bus. The capacity and response latency of the first-layer cache area, the second-layer cache area, and the third-layer cache area increase sequentially.

[0025] The heat monitoring module is used to monitor the temperature of high-heat data cached in the first-layer cache, regular heat data cached in the second-layer cache, and warm data cached in the third-layer cache, so as to obtain the temperature monitoring results of high-heat data, regular heat data, and warm data.

[0026] Specifically, if the temperature monitoring results of high-temperature data, regular-temperature data, or warm-temperature data indicate that the high-temperature data, regular-temperature data, or warm-temperature data meet the preset migration conditions, based on the direct memory access bus, the high-temperature data is migrated as regular-temperature data to the second-level cache, or the regular-temperature data is migrated as high-temperature data or warm-temperature data to the first-level cache or the third-level cache, or the warm-temperature data is migrated as regular-temperature data to the second-level cache.

[0027] It should be noted that the first-level cache can use Static Random-Access Memory (SRAM), the second-level cache can use Dynamic Random-Access Memory (DRAM), and the third-level cache can use Non-Volatile Dynamic Random-Access Memory (NVDRAM). Through the gradient latency design of the three-level cache (SRAM, DRAM, NVDRAM), data with different frequencies automatically resides in the most appropriate cache level, preventing hot data from penetrating into the NAND flash memory module, thereby improving the overall cache utilization.

[0028] To ensure that data of varying latency always stays in the appropriate location, this embodiment of the application arranges SRAM, DRAM, NVDRAM, and NAND flash memory modules sequentially inside the SSD, forming a continuous gradient of latency from low to high. The smallest layer, closest to the host interface, is the small-capacity SRAM, which caches hot data at a 64B row-level granularity; the middle layer is the moderately sized DRAM, which caches hot data at a 256B block-level granularity; below that is the large-capacity NVDRAM, which caches warm data at a 4KB page-level granularity; and at the very end is the NAND, which stores cold data.

[0029] In the design of CXL-SSDs in related technologies, the storage controller typically uses DRAM with limited capacity as the sole cache layer and establishes a fixed interface between DRAM and flash memory. When a cache line or page accessed by the host misses, the request must pass through DRAM directly to NAND, resulting in high latency. This application proposes a tiered caching system that divides the volatile and non-volatile storage media within the SSD into three levels according to latency, from high to low. Furthermore, through the unified address space and cache coherency mechanism provided by the CXL protocol, these three levels of media can be presented to the host as a dynamically scalable cache pool.

[0030] Specifically, during the caching of high-hot data, regular hot data, and warm data in the first, second, and third layer caches respectively, the heat monitoring module samples at a fixed period (e.g., 1ms) to count the cumulative access count of each data point within a preset time window (e.g., 100ms). This cumulative access count is used as the access frequency of each data point. Taking the second layer cache as an example, if the access frequency of any regular hot data is not less than the preset high-hot threshold (e.g., not less than 100 times / 100ms), then the regular hot data is migrated to the first layer cache as high-hot data. If the access frequency of the regular hot data is less than the preset regular hot threshold, then the regular hot data is migrated to the third layer cache as warm data. To avoid invalid data migration, the data is only considered to meet the preset migration condition if the threshold is met for three consecutive statistical periods (300ms in total), thus avoiding erroneous migration caused by short-term sudden access.

[0031] In this system, the entire upgrade and migration process is completed independently by hardware (direct memory access bus). The host always sees a continuous physical address space without any software intervention or additional page table or driver updates.

[0032] Based on the above embodiments, such as Figure 2 The diagram shown illustrates the structure of another hierarchical caching system provided in this application. As an implementable approach, in one embodiment, the system further includes: The request and response module is used to obtain host requests, select target channels from the preset channel cluster based on the length of the host request and the current target bandwidth information, and send the host request to the corresponding target cache through the target channel, so as to respond to the host request based on the data cached in the target cache. The length of the host request is used to characterize the size of the data to be accessed on the host side. The preset channel cluster includes at least a memory semantic channel and a regular channel. The memory semantic channel supports a reverse invalidation mechanism, and the access granularity of the regular channel is larger than that of the memory semantic channel.

[0033] It should be noted that host requests include memory data read / write requests on the host side, which are sent through the host interface of the SSD storage controller. In this embodiment, to eliminate the obstacle of software-based data transfer between the host and the SSD storage device, the SSD is registered as a CXL Type 3 device. SRAM, DRAM, and NVDRAM are directly mapped to the host's physical address space through the HDM-DB (preset address mapping table). Therefore, the CPU and other host devices can use ordinary load / store instructions (memory data read / write requests) to read or write to any level of cache as if accessing local memory.

[0034] Specifically, the memory semantic channel refers to the CXL.mem protocol channel, and the regular channel refers to the CXL.io protocol channel. The reverse invalidation mechanism is a hardware-level cache consistency maintenance mechanism provided by the CXL.mem protocol. When the data state within the SSD changes, such as when data is moved due to migration, the SSD can proactively send a message to the host via the CXL.mem protocol channel to notify it to mark the old copy of the corresponding data in the cache as invalid. This ensures that the host can always read the latest data, and the entire process requires no software intervention.

[0035] Based on the above embodiments, as one implementable approach, in one embodiment, the request-response module includes: The address compilation unit is used to obtain host requests, parse the target physical address in the host request, and determine the target cache corresponding to the target physical address according to the preset address mapping table. The preset address mapping table records the correspondence between the first-level cache, the second-level cache, and the third-level cache and the physical address segment, so that the first-level cache, the second-level cache, and the third-level cache are mapped to the physical address space of the host through the preset address mapping table.

[0036] The channel arbitrator is used to select a target channel from the preset channel cluster based on the length of the host request and the current target bandwidth information, so as to send the host request to the corresponding target buffer through the target channel.

[0037] For example, the storage controller has an internal address decoding module used to resolve the physical address of a host request. This module determines whether the target address accessed by the host belongs to the SRAM, DRAM, NVDRAM, or NAND segment based on the address mapping table established during device registration. The NAND segment refers to the flash memory module.

[0038] The SRAM segment is located at the lowest end of the address space, with a small capacity, using a 64-byte row-level granularity, and a latency on the order of 10 nanoseconds. Next is the DRAM layer, with a capacity more than 10 times that of SRAM, using 256-byte blocks as the smallest migration unit, and a latency about 5 times that of SRAM. The NVDRAM segment has the largest capacity, using 4KB pages as the granularity, and a latency that increases by another order of magnitude.

[0039] Specifically, in one embodiment, the channel arbitrator is used to determine whether the host request is a preset target request based on the length of the host request; if the host request is determined to be a preset target request, it determines whether the bandwidth utilization of the memory semantic channel reaches a preset threshold; if the bandwidth utilization of the memory semantic channel does not reach the preset threshold, the memory semantic channel is used as the target channel.

[0040] Specifically, when a host request arrives, the channel arbiter first determines the access granularity: if the request length is ≤256B, the host request is determined to be a preset target request; if the current memory semantic channel CXL.mem bandwidth utilization is <70% of the preset threshold, the memory semantic channel is used as the target channel, and then completed through the CXL.mem 64B row-level channel. The bandwidth utilization is achieved through the controller's built-in bandwidth monitoring module, which calculates the ratio of used bandwidth to total bandwidth in the CXL.mem channel in real time (used bandwidth = amount of data transmitted in the current period / period duration), updating every 100ns to ensure timely judgment.

[0041] Accordingly, in one embodiment, the channel arbitrator is specifically used to use a regular channel as the target channel when it is determined that the host request is not a preset target request, or when the bandwidth utilization of the memory semantic channel reaches a preset threshold.

[0042] Specifically, if the request length is ≥4KB, it is determined that the host request is not the preset target request, or if the link is already saturated (CXL.mem bandwidth utilization ≥70%), it will automatically switch to the CXL.io 4KB block-level channel (normal channel) with a switching latency of less than 30ns.

[0043] For small-granularity requests (≤256B), using CXL.io (4KB block level) requires additional encapsulation / decapsulation of the 4KB block protocol, resulting in protocol conversion overhead and increased latency. CXL.mem (64B row level), however, natively supports small-granularity access, requiring no conversion and allowing direct response. For large-block requests (≥4KB), using CXL.mem (64B row level) requires multiple split transmissions (e.g., 4KB data needs to be split into 64 64B transmissions), resulting in link fragmentation overhead and reduced bandwidth utilization. CXL.io (4KB block level), on the other hand, supports single-block transmission, offering higher efficiency.

[0044] Specifically, in one embodiment, the address compilation unit is further configured to determine that the host request has not hit the cache if the target physical address does not belong to any cache physical address segment according to the address mapping table; in response to the host request, obtain the host-side data to be accessed from the flash memory module through a regular channel, and write the host-side data to be accessed as warm data into the third-level cache; and use the third-level cache as the target cache.

[0045] Specifically, if all three levels of cache fail (meaning the target physical address does not belong to any cache's physical address range), the controller initiates a read operation to the NAND flash module via CXL.io to retrieve the data to be accessed from the flash module. The returned data is first written to the third-level cache NVDRAM, and then a heat counter begins to accumulate. Once the heat threshold is crossed, the hardware state machine repeats the migration process, pushing the data to higher-level caches as needed.

[0046] Based on the above embodiments, as one implementable approach, in one embodiment, the system further includes: The replica management module is used to send a consistency broadcast to the host through the memory semantic channel when high-hot data cached in the first-level cache, regular hot data cached in the second-level cache, or warm data cached in the third-level cache are migrated, so that the old replicas corresponding to the migrated data in the host's local cache are invalidated.

[0047] It's important to note that the host has its own local cache. After accessing data in the SSD cache, the host retains a copy in the local cache to reduce latency for subsequent accesses. When data in the SSD has been migrated from the first-level cache to the second-level cache, the host may still read the old copy from the first-level cache in its local cache, leading to data inconsistency. The host might be reading the migrated old data, while the second-level cache contains the new data. The CXL.mem protocol natively supports cross-device cache consistency semantics and has a back-invalidation mechanism.

[0048] Specifically, the CXL.mem channel can transmit consistency broadcast messages, which can be directly recognized by the host-side cache controller without going through the operating system or driver intermediaries, with latency controllable to the nanosecond level. The consistency broadcast message includes the physical address and data granularity of the migrated data. Upon receiving the message, the host-side CPU cache controller retrieves the copy corresponding to that address in its local cache and changes its status from valid to invalid. When the host accesses that address again, it will find that the local copy has become invalid and automatically redirect to the new cache layer on the SSD to read the latest data, thus avoiding data inconsistency issues and ensuring consistent data access results between the host and the SSD at any time. This solves the data consistency problem while achieving zero-awareness for the host.

[0049] Specifically, in one embodiment, the copy management module is further configured to generate a new copy of the migration data on the host side according to the migration destination cache area when the high-hot data cached in the first-level cache area, the regular hot data cached in the second-level cache area, or the warm data cached in the third-level cache area completes the migration. If the host request on the host side is responded to and the host side obtains the migration data, the module is also configured to generate a new copy of the migration data on the host side according to the migration destination cache area of ​​the migration data.

[0050] Specifically, when data is migrated to the new cache layer, the next time the host accesses the data, although it won't read the old copy, it needs to read the data again from the new cache layer on the SSD. If the host will access this data frequently in the future (such as intermediate results of AI inference), reading from the SSD every time would waste CXL link bandwidth and would not take advantage of the low latency of the host's local cache. Therefore, a new copy is generated. When the host first accesses the new cache layer with migrated data, the data is automatically cached locally on the host, allowing subsequent high-frequency accesses to directly hit the host cache, further reducing latency.

[0051] Specifically, in one embodiment, the third-layer buffer has a power-loss protection function.

[0052] When the hierarchical caching system experiences an unexpected power outage, the hot data cached in the first-level cache and the regular hot data cached in the second-level cache are written to the third-level cache. When the hierarchical caching system is powered on again, the hot data in the third-level cache is written back to the first-level cache, and the regular hot data in the third-level cache is written back to the second-level cache.

[0053] Specifically, when a traditional SSD loses power, the storage controller's cache is cleared, and the data needs to be read from the NAND flash memory again upon power-up. However, the third-layer cache in this embodiment has a power-loss protection function. When the SSD loses power, each layer cache writes its cached data to the third-layer cache. When the tiered caching system is powered on again, the data can be quickly and safely written back from the third-layer cache, improving the efficiency of cached data recovery after power-up.

[0054] Specifically, in one embodiment, such as Figure 2 As shown, the system also includes a hardware compression engine for compressing and moving data that needs to be moved between buffers. The three buffer segments are connected via an 8-channel DMA bus. Each channel independently maintains a set of buffer queues and supports the hardware compression engine to compress or decompress data online during the migration process to save internal bandwidth.

[0055] In fact, the core hardware parameter configuration of the system provided in this application embodiment is shown in Table 1 below: Table 1 Core Hardware Parameter Configuration Table

[0056] Among them, such as Figure 3 The diagram shown is an exemplary application flow diagram of a hierarchical caching system provided in an embodiment of this application. The specific flow is as follows: 1) Host request processing: The host CPU accesses address 0x00123456 (which falls within the SRAM address range) through the load instruction. The CXL interface module queries the SRAM tag array in parallel. After a hit, it directly returns 64B of data. The total latency is 18ns (including 8ns of CXL protocol interaction + 10ns of SRAM access).

[0057] 2) Cache Miss Handling: If the host accesses the NAND and all three levels of cache miss: The controller sends a 4KB page read command to the NAND array via the CXL.io channel, taking 100μs; after the data is returned, it is written to NVDRAM (address 0x08800000), and the tag is updated synchronously (recording the physical address and initial heat value of 0); the heat counter counts the number of accesses every 1ms, and after 30ms, a total of 45 accesses are accumulated (falling into the hot data range), and the hardware state machine triggers DMA migration: the compression engine compresses the 4KB data to 2.5KB and moves it into DRAM (address 0x01000000) via the DMA bus; when migration starts, the CXL.mem protocol sends a reverse invalid message, invalidating the old copy in the host cache; after migration is completed, CXL.mem sends a reverse invalid message, and the host's next access directly hits DRAM, reducing the latency to 60ns (CXL interaction 10ns + DRAM access 50ns). The multi-level tags include high-hot data tags, regular hot data tags, and warm data tags.

[0058] 3) Data hotness degradation: If the number of accesses to a certain 256B data (address 0x02000000) in DRAM drops to 25 times in 100ms (warm data range), the DMA will automatically decompress it and migrate it to NVDRAM to free up DRAM space.

[0059] In the tiered caching system provided in the application embodiment, under mixed load conditions where random small I / O accounts for 40% and sequential large I / O accounts for 60%, the cache hit rate is increased to 82% (compared to 45% for traditional single-layer DRAM cache); the average access latency for high-hot data is 18ns, for regular hot data it is 60ns, and for warm data it is 520ns, with the overall average latency reduced by 65% ​​compared to traditional SSDs; the CXL.mem channel utilization is 65% (not exceeding the 70% threshold), and small-granularity requests ≤256B are all processed through CXL.mem without protocol conversion overhead.

[0060] In fact, if the system provided in this application embodiment is applied to a distributed storage scenario (such as a cloud computing cluster), it is necessary to further optimize cross-node collaboration and capacity scalability to adapt to the needs of multiple CXL-SSD shared cache pools. The core hardware parameter configuration optimization is shown in Table 2 below: Table 2 Core Hardware Parameter Configuration Optimization Table

[0061] Among them, such as Figure 2 As shown, the system also includes pre-defined functional modules, including a Flash Translation Layer (FTL), a Garbage Collection (GC) module, and a wear leveling module. The FTL, through the garbage collection and wear leveling modules, encapsulates NAND flash memory with specific physical characteristics into a storage device that allows the operating system to use it as conveniently, reliably, and with high performance as a traditional hard drive. The garbage collection module reclaims invalid blocks because flash memory has the physical characteristic of erasing before reprogramming. The wear leveling module avoids excessive local write / erase operations by migrating data across blocks.

[0062] The system provided in this application supports sharing of hot data information. In the CXL-SSD of node A, a certain 4KB data (NVDRAM address 0x10000000) is accessed 130 times within 100ms (extremely hot). The cross-node hot data synchronization unit transmits the address and hot data value of the data to the other 15 nodes in the cluster through the CXL.mem protocol. It supports migration decision-making. When the CXL-SSD of node B detects that the local host has accessed the data 20 times / 100ms, it sends a collaborative migration request to node A. It also supports consistent migration. Node A compresses the data (from 4KB to 2.2KB) and transmits it to the SRAM of node B (address 0x00200000) through the cross-node DMA channel. During the migration, the CXL.mem protocol sends a global invalid message to the node to ensure that there is no dirty data residue. After the migration is completed, node B updates its local label, and the host access latency is reduced to 20ns, avoiding cross-node remote access (the original remote access latency is ≥200 microseconds). In a distributed cluster environment, the cross-node cache hit rate is increased to 90%, avoiding duplicate caching and remote data transfer; the 8GB capacity of NVDRAM supports the storage of 5000+ 4KB warm data pages, reducing NAND access times by up to 70%; cluster-level threshold collaboration makes heat judgment more accurate, avoiding resource waste caused by single node misjudging extremely hot data.

[0063] The hierarchical caching system provided in this application includes a heat monitoring module and a multi-layer caching module. The multi-layer caching module includes at least a first-layer cache, a second-layer cache, and a third-layer cache. The first-layer cache, the second-layer cache, and the third-layer cache are connected via a direct memory access bus. The capacity and response latency of the first-layer cache, the second-layer cache, and the third-layer cache increase sequentially. The heat monitoring module is used to monitor the temperature of high-heat data cached in the first-layer cache, regular-heat data cached in the second-layer cache, and warm data cached in the third-layer cache, so as to obtain the temperature monitoring results of high-heat data, regular-heat data, and warm data. Wherein, when the temperature monitoring results of high-heat data, regular-heat data, or warm data indicate that the high-heat data, regular-heat data, or warm data meet the preset migration conditions, based on the direct memory access bus, the high-heat data is migrated as regular-heat data to the second-layer cache, or the regular-heat data is migrated as high-heat data or warm data to the first-layer cache or the third-layer cache, or the warm data is migrated as regular-heat data to the second-layer cache. The system provided by the above solution, with its three-layer cache (Level 1, Level 2, and Level 3), achieves precise partitioning and caching of high-frequency, regular-frequency, and warm-frequency data. It also enables data migration between different cache layers based on its access frequency, avoiding the flushing of data about to be accessed by the host to the cache module, thus improving cache hit rate. Furthermore, the dynamic tiered caching architecture ensures that data with different access frequencies is stored in the most appropriate cache layer, reducing the number of accesses to low-speed flash memory. High-frequency data (high-hot data) is quickly responded to in the L1 cache (Level 1 cache), while mid-frequency data (regular-frequency data) is also processed promptly in the corresponding cache layer (Level 2 cache). High-frequency data resides in SRAM, resulting in significantly lower average read / write latency than traditional single-layer DRAM caches. Dynamic migration prevents cold data from occupying the high-speed cache, improving the overall hit rate. The CXL.mem protocol ensures real-time consistency between the host and SSD caches, eliminating software synchronization overhead.

[0064] Through the above description of the embodiments, those skilled in the art can clearly understand that the system according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0065] This application provides a tiered caching method applied to the tiered caching system provided in the above embodiments, used for partitioned caching of data on the storage controller. The execution entity in this application embodiment is the storage controller in a storage device such as an SSD or other electronic device.

[0066] like Figure 4The diagram shown is a flowchart illustrating a hierarchical caching method provided in an embodiment of this application. The method includes: Step 401: Obtain the temperature monitoring results of high-heat data in the first layer of the tiered caching system, the temperature monitoring results of regular heat data in the second layer of the tiered caching system, and the temperature monitoring results of warm data in the third layer of the tiered caching system. Step 402: If the high-temperature data monitoring results, regular-temperature data monitoring results, or warm-temperature data monitoring results indicate that the high-temperature data, regular-temperature data, or warm-temperature data meet the preset migration conditions, based on the direct memory access bus, the high-temperature data is migrated as regular-temperature data to the second-level buffer, or the regular-temperature data is migrated as high-temperature data or warm-temperature data to the first-level buffer or the third-level buffer, or the warm-temperature data is migrated as regular-temperature data to the second-level buffer.

[0067] Accordingly, in one embodiment, if the temperature detection result indicates that the temperature data is cold data, the cold data is flushed from the third-layer cache to the flash memory module through a conventional channel.

[0068] The first layer, closest to the host interface, is a small-capacity, extremely low-latency SRAM (first-level cache) used to capture repeatedly accessed data within a very short time window. The middle layer uses DRAM (second-level cache) with moderate capacity and slightly lower latency to handle the caching of frequently accessed data. The outermost layer uses NVDRAM (third-level cache) with larger capacity and power-loss retention capabilities, serving as both a buffer for warm data and a safe write-back area in case of unexpected power loss. These three layers of media are not isolated but are automatically migrated up and down at the data granularity level through a high-speed DMA bus and the global consistency semantics provided by the CXL.mem protocol.

[0069] Based on the above embodiments, as an implementable approach, in one embodiment, acquiring the high-temperature data temperature monitoring results of the first-layer cache, the regular-temperature data temperature monitoring results of the second-layer cache, and the moderate-temperature data temperature monitoring results of the third-layer cache in the hierarchical caching system includes: Step 4011: For any hot data in the first-level cache, determine the temperature monitoring result of the hot data based on the local access frequency of the hot data on the local node and the remote access frequency on multiple remote nodes. Step 4012: For any regular hot data in the second-level buffer, determine the temperature monitoring result of the regular hot data based on the local access frequency of the regular hot data on the local node and the remote access frequency on multiple remote nodes. Step 4013: For any temperature data in the third-level cache, determine the temperature monitoring result of the temperature data based on the local access frequency of the temperature data on the local node and the remote access frequency on multiple remote nodes.

[0070] Specifically, in a distributed storage system, the hotness or coldness of cached data in any cache layer is determined not only by the local access frequency of the data on the local node, but also by the access frequency of the data on remote nodes other than the local node in the distributed storage system. That is, the hotness value of the cached data is determined based on both the local and remote access frequencies. The data temperature monitoring result includes at least the data's hotness value. In a distributed scenario, when the hotness value of data reaches 120 times / 100ms, the data is determined to be hot data; when the hotness value of data is below 20 times / 100ms, it is treated as cold data and flushed to the flash memory module.

[0071] Based on the above embodiments, as one implementable approach, in one embodiment, high-hot data is migrated to a second-level cache as regular hot data based on a direct memory access bus, including: Step 4021: Determine the smallest migration unit based on the target access granularity of the second-level cache. Step 4022: Based on the direct memory access bus, the high-hot data is migrated to the second-level cache as regular hot data according to the smallest migration unit.

[0072] Specifically, if the target access granularity of the second-level cache is 256B block granularity, where B represents bytes, then the 256-byte block is used as the smallest migration unit. Then, based on the direct memory access bus, high-frequency data is migrated to the second-level cache as regular high-frequency data according to the smallest migration unit.

[0073] Specifically, in one embodiment, the hot data to be migrated can be encapsulated into byte blocks according to the smallest migration unit to obtain the byte blocks to be migrated; the byte blocks to be migrated can be compressed to obtain the compressed byte blocks to be migrated; the compressed byte blocks to be migrated can be migrated to the second-level cache based on the direct memory access bus; the compressed byte blocks to be migrated can be decompressed in the second-level cache to obtain the regular hot data to be written to the second-level cache.

[0074] For example, the hot data in SRAM is packaged into the smallest migration unit, such as merging four 64B data entries into one 256B block, thus obtaining the byte block to be migrated. Then, based on compression algorithms such as LZ4, the byte block to be migrated is compressed to obtain a compressed byte block to be migrated, with a compression ratio of 1.5 to 3 times. Based on the direct memory access bus, the compressed byte block to be migrated is migrated to the second-level cache to save bandwidth resources of the memory access bus. After being migrated to the second-level cache, the compressed byte block to be migrated is decompressed to obtain the regular hot data to be written to the second-level cache, and then written to the second-level cache to complete the entire data migration process.

[0075] For a description of the features in the embodiments corresponding to the hierarchical caching method, please refer to the relevant descriptions in the embodiments corresponding to the hierarchical caching system, which will not be repeated here.

[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0077] Embodiments of this application also provide a hierarchical caching apparatus for executing the hierarchical caching method provided in the above embodiments.

[0078] like Figure 5 The diagram shown is a structural schematic of a hierarchical caching device provided in an embodiment of this application. The hierarchical caching device 50 includes an acquisition module 501 and a caching module 502.

[0079] The acquisition module is used to acquire the temperature monitoring results of high-temperature data in the first-layer cache, the temperature monitoring results of regular-temperature data in the second-layer cache, and the temperature monitoring results of warm-temperature data in the third-layer cache in the hierarchical caching system. The caching module is used to migrate high-temperature data as regular-temperature data to the second-layer cache, or migrate regular-temperature data as high-temperature data or warm-temperature data to the first-layer cache or the third-layer cache, or migrate warm-temperature data as regular-temperature data to the second-layer cache, based on the direct memory access bus, when the temperature monitoring results of high-temperature data, regular-temperature data, or warm-temperature data indicate that the high-temperature data, regular-temperature data, or warm-temperature data meet the preset migration conditions.

[0080] For a description of the features in the embodiments corresponding to the hierarchical caching device, please refer to the relevant descriptions in the embodiments corresponding to the hierarchical caching method, which will not be repeated here.

[0081] Embodiments of this application also provide a storage device, such as... Figure 6 The diagram shown is a structural schematic of a storage device provided in an embodiment of this application, including a storage controller, which includes any of the above-described hierarchical caching systems; The storage controller performs data tiered caching in the tiered caching system based on any of the above-mentioned tiered caching methods.

[0082] Embodiments of this application also provide an electronic device, such as... Figure 7 The diagram shown is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, including a processor 10 and a memory 20. The memory 20 stores a computer program, and the processor 10 is configured to run the computer program to execute the steps in any of the above-described hierarchical caching method embodiments.

[0083] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described hierarchical caching method embodiments at runtime.

[0084] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0085] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described hierarchical caching method embodiments.

[0086] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described hierarchical caching method embodiments.

[0087] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0088] The foregoing has provided a detailed description of a hierarchical caching system, method, apparatus, and storage device provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to aid in understanding the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A hierarchical cache system, characterized by, include: The system includes a heat monitoring module and a multi-layer caching module. The multi-layer caching module includes at least a first-layer cache, a second-layer cache, and a third-layer cache. The first-layer cache, the second-layer cache, and the third-layer cache are connected through a direct memory access bus. The capacity and response latency of the first-layer cache, the second-layer cache, and the third-layer cache increase sequentially. The heat monitoring module is used to monitor the temperature of high-heat data cached in the first layer cache, regular heat data cached in the second layer cache, and warm data cached in the third layer cache, so as to obtain the temperature monitoring results of high-heat data, regular heat data, and warm data. Wherein, when the high-temperature data temperature monitoring results, the regular-temperature data temperature monitoring results, or the warm-temperature data temperature monitoring results indicate that the high-temperature data, the regular-temperature data, or the warm-temperature data meet the preset migration conditions, based on the direct memory access bus, the high-temperature data is migrated as regular-temperature data to the second-level cache area, or the regular-temperature data is migrated as high-temperature data or warm-temperature data to the first-level cache area or the third-level cache area, or the warm-temperature data is migrated as regular-temperature data to the second-level cache area; The system also includes: The request-response module is used to obtain a host request, select a target channel in a preset channel cluster according to the length of the host request and the current target bandwidth information, and send the host request to the corresponding target cache through the target channel, so as to respond to the host request based on the data cached in the target cache. The length of the host request is used to characterize the size of the data to be accessed on the host side. The preset channel cluster includes at least a memory semantic channel and a regular channel. The memory semantic channel refers to the CXL.mem protocol channel, and the regular channel refers to the CXL.io protocol channel. The memory semantic channel supports a reverse invalidation mechanism, which is used to implement hardware-level cache consistency maintenance. The access granularity of the regular channel is larger than that of the memory semantic channel. The request-response module includes: The address compilation unit is used to obtain the host request, parse the target physical address in the host request, and determine the target cache corresponding to the target physical address according to a preset address mapping table; wherein, the preset address mapping table records the correspondence between the first-level cache, the second-level cache, and the third-level cache and the physical address segment, so that the first-level cache, the second-level cache, and the third-level cache are mapped to the physical address space of the host through the preset address mapping table; A channel arbitrator is used to select a target channel in a preset channel cluster based on the length of the host request and the current target bandwidth information, so as to send the host request to the corresponding target buffer through the target channel; Specifically, the channel arbitrator is used for: Based on the length of the host request, determine whether the host request is a preset target request; wherein the length of the preset target request is ≤256B; If the host request is determined to be a preset target request, it is determined whether the bandwidth utilization of the memory semantic channel has reached a preset threshold. If the bandwidth utilization of the memory semantic channel does not reach a preset threshold, the memory semantic channel will be used as the target channel. The system also includes: The replica management module is used to send a consistency broadcast to the host through the memory semantic channel when high-hot data cached in the first-level cache, regular hot data cached in the second-level cache, or warm data cached in the third-level cache is migrated, so that the old replicas corresponding to the migrated data in the host's local cache are invalidated; the consistency broadcast includes the physical address and data granularity of the migrated data.

2. The tiered cache system of claim 1, wherein, The channel arbitrator is specifically used for: If it is determined that the host request is not a preset target request, or if the bandwidth utilization of the memory semantic channel reaches a preset threshold, the regular channel will be used as the target channel.

3. The tiered cache system of claim 1, wherein, The address compilation unit is also used for: If, according to the address mapping table, it is determined that the target physical address does not belong to any cache area's physical address range, then it is determined that the host request has not currently hit the cache. In response to the host request, the host-side data to be accessed is obtained from the flash memory module through the conventional channel, and the host-side data to be accessed is written as warm data into the third-level cache. The third-level cache area is used as the target cache area.

4. The tiered caching system of claim 1, wherein, The host request includes memory data read / write requests on the host side.

5. The tiered caching system of claim 1, wherein, The copy management module is also used for: When the high-hot data cached in the first-layer cache, the regular hot data cached in the second-layer cache, or the warm data cached in the third-layer cache are migrated, if the host request on the host side is responded to and the host side obtains the migration data, then a new copy of the migration data is generated on the host side according to the migration destination cache of the migration data.

6. The tiered caching system of claim 1, wherein, The third-layer buffer has power-loss protection. When the hierarchical caching system experiences an unexpected power outage, the high-hot data cached in the first-layer cache and the regular hot data cached in the second-layer cache are written to the third-layer cache. When the hierarchical caching system is powered on again, the high-hot data in the third-layer cache is written back to the first-layer cache, and the regular hot data in the third-layer cache is written back to the second-layer cache.

7. A hierarchical caching method, characterized by, Applied to the hierarchical caching system as described in any one of claims 1 to 6, the method comprises: Obtain the temperature monitoring results of high-heat data in the first-layer cache area, the temperature monitoring results of normal-heat data in the second-layer cache area, and the temperature monitoring results of mild data in the third-layer cache area in the hierarchical caching system. If the high-temperature data monitoring results, regular-temperature data monitoring results, or warm-temperature data monitoring results indicate that the high-temperature data, regular-temperature data, or warm-temperature data meet the preset migration conditions, based on the direct memory access bus, the high-temperature data is migrated as regular-temperature data to the second-level cache area, or the regular-temperature data is migrated as high-temperature data or warm-temperature data to the first-level cache area or the third-level cache area, or the warm-temperature data is migrated as regular-temperature data to the second-level cache area.

8. The tiered caching method of claim 7, wherein, The method further includes: If the temperature detection result indicates that the temperature data is cold data, the cold data is flushed to the flash memory module from the third-layer cache through a conventional channel.

9. The tiered caching method of claim 7, wherein, The acquisition of the high-temperature data monitoring results of the first-layer cache, the regular-temperature data monitoring results of the second-layer cache, and the moderate-temperature data monitoring results of the third-layer cache in the hierarchical caching system includes: For any hot data in the first layer of the cache, the temperature monitoring result of the hot data is determined based on the local access frequency of the hot data on the local node and the remote access frequency on multiple remote nodes. For any regular hot data in the second-level cache, the temperature monitoring result of the regular hot data is determined based on the local access frequency of the regular hot data on the local node and the remote access frequency on multiple remote nodes. For any temperature data in the third-layer cache, the temperature monitoring result of the temperature data is determined based on the local access frequency of the temperature data on the local node and the remote access frequency on multiple remote nodes.

10. The tiered caching method of claim 7, wherein, The process of migrating the high-hot data as regular hot data to the second-level cache area based on the direct memory access bus includes: The smallest migration unit is determined based on the target access granularity of the second-level cache. Based on the direct memory access bus, the high-hot data is migrated to the second-level cache area as regular hot data according to the minimum migration unit.

11. The tiered caching method of claim 10, wherein, The process of migrating the high-hot data as regular hot data to the second-level cache area based on the direct memory access bus and according to the minimum migration unit includes: According to the minimum migration unit, the hot data to be migrated is encapsulated into byte blocks to obtain the byte blocks to be migrated; The byte block to be migrated is compressed to obtain a compressed byte block to be migrated; Based on the direct memory access bus, the compressed byte block to be migrated is migrated to the second-level cache area; The compressed byte block to be migrated is decompressed in the second-level cache to obtain regular hot data to be written to the second-level cache.

12. A hierarchical cache apparatus, characterized by comprising: The apparatus is applied to the hierarchical caching system as described in any one of claims 1 to 6, the apparatus comprising: The acquisition module is used to acquire the high-temperature data temperature monitoring results of the first-layer cache area, the regular-temperature data temperature monitoring results of the second-layer cache area, and the temperature monitoring results of the third-layer cache area in the hierarchical caching system. The caching module is used to migrate the high-temperature data as regular thermal data to the second-level cache area, or the regular thermal data as high-temperature data to the first-level cache area or the third-level cache area, or the warm data as regular thermal data to the second-level cache area, based on the direct memory access bus, when the high-temperature data temperature monitoring results, regular thermal data temperature monitoring results, or warm data temperature monitoring results indicate that the high-temperature data, regular thermal data, or warm data meet preset migration conditions.

13. A storage device, comprising: include: Storage controller, the storage controller comprising the tiered caching system as described in any one of claims 1 to 6; The storage controller performs data tiered caching in the tiered caching system based on the tiered caching method as described in any one of claims 7 to 11.

14. An electronic device, comprising: include: Memory, used to store computer programs; A processor, configured to implement the steps of the hierarchical caching method as described in any one of claims 7 to 11 when executing the computer program.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the hierarchical caching method as described in any one of claims 7 to 11.

16. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the hierarchical caching method as described in any one of claims 7 to 11.

Images