Adaptive cache management for storage medium systems

Abstract

The present disclosure describes an apparatus and method for adaptive cache management of a storage medium system. In aspects, an adaptive cache manager obtains telemetry information related to access to a cache memory and access to a storage medium of a storage medium system. Based on the telemetry information, the adaptive cache manager determines a caching policy for the cache memory, and applies the caching policy to the cache memory to modify a caching scheme or a prefetching scheme for data of the cache memory. In some cases, the adaptive cache manager receives cache parameters from an application or user of the host system and uses the parameters in determining the cache policy. By doing so, the adaptive cache manager may dynamically change the caching and prefetch activity of the cache memory to improve the efficiency of the cache memory.

Description

[0001] Cross - Reference to Related Applications

[0002] This disclosure claims priority to U.S. Provisional Patent Application Serial No. 63 / 601,202, filed on November 20, 2023, and U.S. Non-Provisional Patent Application Serial No. 18 / 587,362, filed on February 26, 2024, the disclosures of which are incorporated herein by reference in their entireties. BACKGROUND OF THE INVENTION

[0003] Application and cloud service providers generally implement enterprise-level data systems to collect and store large amounts of data generated from ongoing operations, research, data mining, or other data sources. To accommodate the increasing volume of data collected from these sources, enterprise-level data systems are built with increasing storage capacity to hold the ever-flowing input data. Although data system architects can scale storage capacity with larger data volumes, the low-latency working memory available for various applications to access and analyze the stored data has not scaled in a commensurate manner. When the data requested by an application is not present in the working memory, the data system fetches the requested data from the storage device, evicts other data from the working memory to make room for the fetched data, and places the requested data into the working memory for use by the application. Due to the high latency of the storage device and the time consumed by the operation of placing data, the performance of complex applications that require a large amount of working memory (e.g., artificial intelligence or deep learning systems) is often impaired due to the limited capacity and inefficient data acquisition schemes associated with conventional working memories. SUMMARY OF THE INVENTION

[0004] This Summary of the Invention is provided to introduce a subject that will be further described in the Detailed Description and the Drawings. Accordingly, this Summary of the Invention should not be considered to describe essential features nor be used to limit the scope of the claimed subject matter.

[0005] In some aspects, a method for adaptive cache management includes: receiving, from a host system and via a Compute Express Link (CXL TM ) interface, a transaction packet for accessing data in a cache memory of a storage media system. The method includes: determining first telemetry information associated with the transaction packet received from the host system for accessing the cache memory, and determining second telemetry information associated with access to a storage medium of the storage media system associated with the transaction packet. The method then determines a cache policy for the cache memory based on the first telemetry information and the second telemetry information, and applies the cache policy to the cache memory to modify a caching scheme or a prefetching scheme for data in the cache memory.

[0006] In other aspects, a device includes: a cache memory; a CXL interface configured to receive transaction packets from a host system to access the cache memory; a storage medium configured to store data of the host system; and a storage medium controller configured to enable data transfer between the cache memory and the storage medium. The device further includes a telemetry unit and an adaptive cache manager. The telemetry unit is operatively coupled to the CXL interface and the storage medium controller. The adaptive cache manager is configured to obtain from the telemetry unit first telemetry information related to transaction packets received from the host system for accessing the cache memory and second telemetry information related to data present in the cache memory and accesses to the storage medium associated with the transaction packets. The adaptive cache manager can determine a cache policy for the cache memory based on the first telemetry information and the second telemetry information, and apply the cache policy to the cache memory to modify a caching scheme or a prefetching scheme for data stored by the cache memory.

[0007] In other aspects, a system-on-chip (SoC) includes: a cache memory CXL interface configured to receive transaction packets from a host system to access the cache memory; and a storage medium controller having a storage medium interface and configured to enable data transfer between the cache memory and a storage medium coupled to the storage medium interface. The SoC further includes a telemetry unit and an adaptive cache manager. The telemetry unit is operatively coupled to the cache memory and the storage medium controller. The adaptive cache manager is configured to receive from the telemetry unit first telemetry information related to transaction packets received from the host system and second telemetry information related to accesses to the storage medium associated with the transaction packets. The adaptive cache manager can determine a cache policy for the cache memory based on the first telemetry information and the second telemetry information, and apply the cache policy to the cache memory to modify a caching scheme or a prefetching scheme for data stored by the cache memory.

[0008] Details of one or more aspects of adaptive cache management of a storage medium system are set forth in the accompanying drawings and the following description. Other features and advantages will be apparent from the specification, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Details of one or more aspects of adaptive cache management of a storage medium system are set forth in the accompanying drawings and the following detailed description. In the drawings, the leftmost digit of a reference numeral indicates the figure in which the reference numeral first appears. The use of the same reference numerals in different instances in the specification and the drawings may indicate the same element:

[0010] Figure 1Illustrates an example operating environment with a storage system in which aspects of adaptive cache management can be implemented;

[0011] Figure 2 Illustrates an example computing system in which aspects of adaptive cache management can be implemented;

[0012] Figure 3 Illustrates an example storage media controller with an adaptive cache manager implemented according to one or more aspects;

[0013] Figure 4 Illustrates an example implementation of an adaptive cache manager according to one or more aspects;

[0014] Figure 5 Depicts an example method of adaptive cache management according to one or more aspects;

[0015] Figure 6 Depicts an example method of determining a cache policy based on parameters received from a host system according to one or more aspects;

[0016] Figure 7 Depicts an example method of configuring a machine learning model based on telemetry information according to one or more aspects;

[0017] Figure 8 Illustrates an example system-on-chip (SoC) environment in which aspects of adaptive cache management can be implemented; and

[0018] Figure 9 Illustrates an example storage media controller in which an adaptive cache manager can be implemented according to one or more aspects. Detailed Description

[0019] With the development of modern applications, data and memory requirements have grown exponentially in recent years. For example, applications such as deep learning recommendation (DLRM) systems require large amounts of memory because data embeddings need to capture complex relationships and patterns required for accurate predictions. In some cases, the accelerator memory of an artificial intelligence (AI) model that implements data embeddings for complex information spans trillions of bytes of data to improve the quality of recommendations. Therefore, it has been proposed to use NAND flash memory instead of dynamic random access memory (DRAM) as the working memory for such types of applications to address the growing memory requirements. Although the storage density of NAND flash memory is greater than that of DRAM memory, the access latency associated with conventional NAND flash devices is significantly higher and is generally not suitable for use as application working memory.

[0020] To reduce this high access latency, many NAND flash drives include a smaller DRAM cache memory to enable faster retrieval of recently or frequently used data. For example, a NAND flash drive can implement a cache mechanism to prevent eviction of frequently accessed data from the cache, thereby allowing faster retrieval, and implement a prefetch mechanism to proactively fetch anticipated data before the data is requested. However, these conventional cache and prefetch mechanisms are static in nature and are preconfigured by the manufacturer of the NAND flash drive. Generally, the efficiency of the cache mechanism depends on different application characteristics (such as the size of the accessed data, the frequency of updates, the ratio of read / write operations, the data lifetime, etc.).

[0021] Since these characteristics vary between applications and the cache mechanism is static, a cache mechanism that works well for one application may result in poor performance for another application with different access requirements. Additionally, when application performance is impaired due to suboptimal cache configuration, these conventional mechanisms lack control over updating or changing the cache configuration, which prevents users from performing cache tuning that could improve cache performance. Thus, conventional cache mechanisms are generally static in configuration, provide suboptimal performance for most applications, and lack the ability to adapt to the various cache behaviors associated with different applications, which can lead to reduced application performance.

[0022] This disclosure describes apparatus and techniques for adaptive cache management in a storage medium system. Compared with the previous cache mechanisms, the described apparatus and techniques can implement aspects of adaptive cache management that capture telemetry information related to cache memory transactions and storage medium access, which can be used to determine or update the cache policy of the cache memory. Additionally, the described aspects can receive cache parameters or hints from applications or users of the host system and use these parameters or hints when determining the cache policy. Further, these aspects can use the telemetry information with machine learning (ML) techniques to obtain ML-based cache parameters for configuring or updating the cache policy of the cache memory. By doing so, the adaptive cache manager can dynamically change the cache and prefetch activities of the cache memory to improve the efficiency of the cache memory.

[0023] In various aspects, a storage medium system can be implemented with a flexible or adaptive cache system that includes an adaptive cache manager (e.g., an adaptive computing unit) that manages the cache, a telemetry unit that captures or obtains telemetry information within the storage medium system, and a host system interface through which cache parameters can be received from an application or user of the host system. The adaptive cache manager can support customizable cache and prefetching schemes and enable the update of the system's ML model to improve cache efficiency. In other words, the adaptive cache manager can provide and tailor customized cache and prefetching mechanisms for specific application requirements. The telemetry unit can capture and provide different types of telemetry information (e.g., cache misses, cache hits, request size, logical block address (LBA), data age, access frequency to a specific LBA, etc.) to the adaptive cache manager for determining or adjusting the cache policy. For example, the adaptive cache manager can utilize such information to configure a customized cache and prefetching mechanism to proactively fetch and retain data in the cache, thereby improving the overall efficiency of the cache memory. The host system interface for cache parameters generally enables the adaptive cache manager to establish a collaborative (host device) cache and prefetching mechanism. This can be important because an application or user can use the application context to provide application-specific hints that the storage medium system lacks. These are just a few example details of adaptive cache management, which can be further described in this disclosure in conjunction with other aspects.

[0024] In various aspects, a storage medium system includes a CXL interface to a host system, a cache memory, a storage medium, and an adaptive cache manager. The adaptive cache manager can obtain telemetry information related to accesses to the cache memory and accesses to the storage medium from the telemetry unit. Based on the telemetry information, the adaptive cache manager determines a cache policy for the cache memory and applies the cache policy to the cache memory to modify the cache scheme or prefetching scheme for data in the cache memory. In some cases, the adaptive cache manager receives cache parameters from an application or user of the host system and uses these parameters in determining the cache policy. Alternatively or additionally, the adaptive cache manager can provide the telemetry information to a machine learning model for processing. Then, the adaptive cache manager receives ML-based cache parameters from the machine learning model, which can also be used to determine the cache policy for the cache memory. By doing so, the adaptive cache manager can dynamically change the cache and prefetching activities of the cache memory to improve the efficiency of the cache memory.

[0025] The following discussion describes an operating environment, techniques that may be employed in the operating environment, a system-on-chip (SoC), and various storage media controllers that may include components of the operating environment. In the context of the present disclosure, the operating environment, techniques, or various components are referenced only by way of example.

[0026] Operating environment

[0027] Figure 1 FIG. illustrates an example operating environment 100 having a host system 102 (referred to as a single “host system 102”) in which adaptive cache management may be implemented in accordance with one or more aspects. Generally, a host system is capable of communicating, storing, or accessing various forms of data or information. Examples of host system 102 may include a compute cluster 104 (e.g., a compute cluster of cloud 106), server hardware of a server 108 or data center 110, or a server 112 (e.g., stand-alone), any of which may be configured as part of a storage network, data center, or cloud system. Other examples of host system 102 (not shown) may include a laptop computer, a tablet computer, a desktop computer, a set-top box, a data storage device, a wearable smart device, a television, a content streaming device, a high-definition multimedia interface (HDMI) media stick, a smart device, a home automation controller, a smart thermostat, an Internet of Things (IoT) device, a mobile Internet device (MID), a network-attached storage (NAS) drive, an aggregated storage system, an aggregated memory system, a memory expander, a gaming console, an automotive computing system, and the like. Generally, host system 102 may transfer or store data for any suitable purpose, such as enabling the functionality of a particular type of device, application, virtual machine (VM), tenant, cloud service, memory system, cache system, storage system, and the like.

[0028] The host system 102 includes a processor 114 and a computer-readable medium 116. The processor 114 can be implemented as any suitable type or number of processors (e.g., x86 or ARM), single-core or multi-core, for executing instructions or commands of an operating system, an application, a tenant, a VM, or other software executing on the host system 102. The computer-readable medium 116 (CRM 116) includes a system memory 118 from which a tenant 120, a VM, or an application (not shown) of the host system 102 can execute or be implemented. The system memory 118 of the host system 102 can include any suitable type or combination of volatile memory or non-volatile memory. For example, the volatile memory of the host system 102 can include various types of random access memory (RAM), dynamic RAM (DRAM), static RAM (SRAM), etc. The non-volatile memory can include read-only memory (ROM), electrically erasable programmable ROM (EEPROM), or flash memory (e.g., NOR flash or NAND flash). These memories can store, individually or in combination, data associated with the applications, tenants, workloads, initiators, virtual machines, and / or operating systems of the host system 102.

[0029] In this example, the host system 102 includes an interconnect 122 and a storage media system 124, and the storage media system 124 can be accessed via the interconnect 122 using any suitable protocol. In some implementations, the interconnect 122 is configured as a Compute Express Link (CXL) interconnect over a Peripheral Component Interconnect Express (PCIe) interface. Thus, the host system 102 and the storage media system 124 can communicate using transaction packets or "microslices" that conform to the CXL Input / Output (CXL.io) protocol, the CXL Memory (CXL.mem) protocol, or any other suitable CXL protocol. In various aspects, the host system 102 issues load and / or store as CXL.mem transaction packets via the interconnect 122 to access the storage media system as the working memory of a tenant, an application, or a VM executing on the host system.

[0030] The storage media system 124 can be configured as any suitable type of data storage system, such as a storage device, a storage drive, a storage array, a storage volume, a data storage center, etc. Although described with reference to the host system 102, the storage media system 124 can also be implemented separately as an independent device or as part of a larger collection of storage devices, such as a network-attached storage device, an external storage drive, a data storage center, a server farm, or a virtualized storage system (e.g., for cloud-based storage, applications, or services). Examples of the storage media system 124 include non-volatile memory express (NVMe) solid state drives 126, peripheral component interconnect express (PCIe) solid state drives 128, solid state drives 130 (SSD 130), and storage arrays 132, which can be implemented as devices that implement CXL (e.g., an SSD that implements CXL.mem) and / or any combination of storage devices or storage drives.

[0031] In this example, the storage media system 124 includes a storage media controller (not shown), a cache memory 134, an adaptive cache manager 136, a telemetry unit 138, and a storage media 140 of the storage media device 142 (e.g., NAND die or device). As described with reference to Figures 2 to 4 the storage media controller manages various operations or functions of the storage media system 124. The cache memory 134 of the storage media system 124 can include or be formed by any suitable type of volatile memory device (such as a RAM device, a DRAM device, etc.). The storage media 140 can include or be formed by a non-volatile storage device on which data or information of the host system 102 is stored. The storage media 140 can be implemented with any type of solid state storage media (such as flash memory, NAND flash memory, SRAM, etc.) or a combination thereof. For example, the storage media 140 of the storage media system 124 can include NAND flash memory, single-level cell (SLC) flash memory, multi-level cell (MLC) flash memory, three-level cell (TLC) flash memory, quad-level cell flash memory (QLC), NOR cell flash memory, or any combination thereof. These memories can store data associated with users, applications, tenants, operational loads, services, and / or the operating system of the host system 102, either individually or in combination.

[0032] In various aspects, the host system 102 or an application executing on the host system can use the storage medium system 124 as a working memory and request access to data stored in the storage medium system 124 via CXL.mem transactions. The telemetry unit 138 can be operatively coupled to the host interface, the cache memory 134, and / or the storage medium controller of the storage medium system to capture or collect telemetry information related to accesses to the storage medium system. Generally, the adaptive cache manager 136 can obtain telemetry information or statistics related to accesses to the cache memory 134 and / or the storage medium 140, indicating the efficiency or performance of the cache memory 134. For example, when data requested by a load request from the host system 102 is present in the cache memory 134, the cache memory 134 can return the data as a "cache hit", and the telemetry information from the cache memory can reflect the cache hit. Alternatively, when the data requested by the load request is not present in the cache memory 134, the storage medium controller extracts the data from the storage medium as a "cache miss", and the telemetry information can reflect the cache miss.

[0033] Based on this or other telemetry information, the adaptive cache manager 136 can select, determine, or configure the cache policy of the cache memory 134 and apply this cache policy to the cache memory 134 to modify the cache scheme or prefetch scheme for data in the cache memory. Example cache and / or prefetch policies that can be selected or configured by the adaptive cache manager 136 can include a randomization policy, a first-in-first-out (FIFO) policy, a last-in-first-out (LIFO) policy, least recently used (LRU), most recently used (MRU), least frequently used (LFU), most frequently used (MFU), size-based policies, retrieval cost-based policies, identification-based policies (e.g., hashing or Bloom filters), hop count, quality of service (QoS) priorities, etc. In some cases, the adaptive cache manager 136 receives cache parameters from an application or user of the host system 102 and uses these parameters when selecting, determining, or configuring the cache policy. Alternatively or additionally, the adaptive cache manager 136 can provide telemetry information to a machine learning (ML) model or a neural network (NN) for processing. From the ML model or NN, the adaptive cache manager 136 can receive ML-based or NN-based cache parameters, which can also be used to determine the cache policy for the cache memory. By doing so, the adaptive cache manager 136 can reduce latency, improve cache memory efficiency (e.g., increase the cache hit rate / miss rate), and improve the working memory performance of the application. These are just a few examples of adaptive cache management, which are described in detail throughout the disclosure.

[0034] Return to Figure 1 In addition, the host system 102 may also include an I / O port 144, a graphics processing unit 146 (GPU), and a data interface 148. Generally, the I / O port 144 allows the host system 102 to interact with other devices, peripherals, or users. For example, the I / O port 144 may include or be coupled to a universal serial bus, a human interface device, an audio input, an audio output, etc. The GPU 146 processes and renders graphics-related data (such as user interface elements of an operating system, applications, etc.) for the host system 102. In some cases, the GPU 146 accesses a portion of the local memory to render graphics, or includes dedicated memory (e.g., video RAM) for rendering the graphics of the host system 102.

[0035] The data interface 148 of the host system 102 provides a connection to one or more networks and other devices connected to these networks. The data interface 148 may include a wired interface (such as an Ethernet or fiber optic interface) for communication on a local network, an intranet, or the Internet. Alternatively or additionally, the data interface 148 may include a wireless interface to facilitate communication via a wireless network, such as a wireless LAN, a wide area wireless network (e.g., a cellular network), and / or a wireless personal area network (WPAN). According to one or more aspects of the present disclosure, any data communicated through the I / O port 144 or the data interface 148 may be written to or read from the storage medium system 124 of the host system 102.

[0036] Figure 2 An example computing system having a host 202 and a storage medium system 124 with an adaptive cache manager 136 implemented according to one or more aspects is illustrated at 200. The computing system 200 may represent an example configuration of the host system 102, the storage medium system 124, and the adaptive cache manager 136 described with reference to Figure 1 The host 202 and the computing resources 204 may be similar to or different from Figure 1It is implemented by the host system 102 and the processor 114, and can represent a single host executed on the processor, several hosts executed on the corresponding processors, multiple hosts executed on a processing resource pool, and so on. In this example, the storage medium system 124 is implemented as an array of multiple SSDs 130-1 to 130-m (collectively referred to as SSD 130) coupled to the host 202 through an interconnect 122 (e.g., a CXL interconnect). As shown, each SSD 130 may include instances of storage medium controllers 206-1 to 206-m, which may include a cache memory 134, an adaptive cache manager 136, and a telemetry unit 138. The SSD 130 also includes a storage medium 140 formed by multiple channels 208-1 to 208-4 of NAND memory devices.

[0037] In various aspects, the host 202 (e.g., the host system 102) may include multiple tenants 120-0 to 120-n executed on the computing resources 204 of the host. Generally, the computing resources 204 of the host 202 may include a combination of the processing resources and system memory of the host 202 for implementing tenants, applications, virtual machines, or initiators that access the memory or storage device associated with the host 202. Thus, although shown as a single host, the host 202 may represent multiple hosts, applications, virtual machines, guests, tenants, and / or initiators that may issue commands or requests to access (e.g., as working memory) data stored in the cache memory or storage medium of the SSD 130 of the storage medium system 124.

[0038] Generally, the tenant 120 or the application of the host 202 may use the cache memory 134 and / or the storage medium 140 as working memory or host-managed device storage space (HDM). In various aspects, the host 202 or the tenant 120 of the host issues transaction packets (such as load or store commands) to the corresponding storage medium controller 206 to write data to or read data from the storage device presented by the storage medium controller 206. These commands or requests may be received and processed by the storage medium controller 206 or a cache controller (not shown), and corresponding commands or requests may be issued to the cache memory 134 or the storage medium 140 to service the data load or data storage operation received from the host 202. The telemetry unit 138 may be operably coupled to the interconnect 122, the storage medium controller 206, the CXL endpoint, and / or the cache memory 134 of the SSD 130 to capture or collect telemetry information related to the access of the SSD.

[0039] In various aspects, the adaptive cache manager 136 obtains telemetry information or statistics related to accesses to the cache memory 134 and / or the storage medium 140 from the telemetry unit 138, indicating the efficiency or performance of the cache memory 134. For example, the adaptive cache manager 136 may be able to determine the hit / miss ratio, latency, hit rate, byte hit rate, miss rate, or access time for the cache memory 134. Based on the telemetry information, the adaptive cache manager 136 may determine a cache policy for the cache memory 134 and apply the cache policy to the cache memory 134 to modify the caching scheme or prefetching scheme for the data in the cache memory. By doing so, the adaptive cache manager 136 may reduce latency, improve cache memory efficiency (e.g., increase the cache hit / miss ratio), and improve the working memory performance of the application.

[0040] Figure 3 FIG. 300 illustrates an example configuration of a storage medium controller 206 with an adaptive cache manager 136 implemented in accordance with one or more aspects. In this example, the adaptive cache manager 136 and the storage medium controller 206 are illustrated in the context of a storage medium system implemented as an SSD 130. The adaptive cache manager 136 may interact with components of the host system 102 and / or the storage medium controller 206 to implement aspects of adaptive cache management of the storage medium system. In this example, the host 202 includes a CXL root port 302 that conveys transaction packets to a CXL interface 304 (e.g., a CXL endpoint or a CXL controller) of the storage medium controller 206. The host 202 also includes a host-side cache assistant 306 (cache assistant 306) and one or more tenants 120 that execute applications on the host. The cache assistant 306 may be configured as an application programming interface or a user interface through which an application or a user of the host system may provide cache parameters or cache hints to the adaptive cache manager 136. In various aspects, the cache parameters or hints may include the data access pattern or behavior of the application, the optimal type of cache policy for the application, threshold settings related to the cache parameters, and the like.

[0041] Generally, the operation of the SSD 130 is enabled or managed by an instance of the storage medium controller 206, in which the storage medium controller 206 includes a CXL interface 304 that enables communication with the host 202 and a media interface 308 that enables access to the storage medium 140. As Figure 3As shown, the storage medium 140 may include multiple NAND flash dies 310-1 through 310-n, where n is any suitable integer. In some cases, the NAND chips 310 form a NAND device that includes multiple flash memory channels of memory devices, dies, or chips that may be accessed or managed at the channel level (die groups), device level (individual dies), or block level (individual blocks or pages of storage medium units). Although described as a CXL root port 302, the host interface of the storage medium controller 206 may be configured to implement any suitable type of storage interface or protocol, such as Serial Advanced Technology Attachment (SATA), Universal Serial Bus (USB), PCIe, Advanced Host Controller Interface (AHCI), NVMe, NVM-over fabric (NVM-OF), NVM Host Controller Interface Specification (NVMHCIS), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), Secure Digital I / O (SDIO), Fibre Channel, any combination thereof (e.g., M.2 or Next Generation Form Factor (NGFF) combined interfaces), etc. Alternatively or additionally, the media interface 308 may implement any suitable type of storage medium interface, such as a flash memory interface, flash bus channel interface, NAND channel interface, Physical Page Addressing (PPA) interface, etc.

[0042] Components of the storage medium controller 206 can provide a data path between the CXL root port 302, the cache memory 134, and the media interface 308 to the storage medium 140. In other words, the storage medium controller can be configured to be capable of transferring data between the cache memory and the storage medium, such as for storing data to the storage medium when evicted from the cache memory, or fetching data from the storage medium to the cache memory to service a cache miss. In this example, the storage medium controller 206 includes a processor core 312 for executing a kernel, firmware, or driver to implement the functions of the storage medium controller 206, and the processor core 312 can include a flash translation layer (FTL) for generating media access I / O based on host access I / O for data access (e.g., in response to a cache miss). In some cases, the processor core 312 can also execute processor-executable instructions to implement the adaptive cache manager 136 of the storage medium controller 206. Alternatively or additionally, the adaptive cache manager 136 can execute or run on cache-specific hardware or a separate processor core. The static random access memory 314 (SRAM 314) of the storage medium controller 206 can store processor-executable instructions or code for the firmware or driver of the storage medium controller, which can be executed by the processor core 312. The storage medium controller 206 can also include a dynamic random access memory (DRAM) controller 316 for the cache memory 134 and an associated DRAM 318. In various aspects, when the controller moves data between the CXL interface 304, the storage medium 140, or other components of the storage medium controller, the storage medium controller 206 stores or caches the data to the DRAM 318.

[0043] As Figure 3As shown in, the structure 320 of the storage medium controller 126 that may include control and data buses is operatively coupled and enables communication between components of the storage medium controller 206. For example, the adaptive cache manager 136 or the telemetry unit 138 may communicate with the host 202, the CXL interface 304, the media interface 308, the processor core 312 (e.g., firmware), the SRAM 314, and the DRAM 318 to exchange data, information, transactions, or I / O within the storage medium controller 206. In various aspects, the telemetry unit 138 may obtain or capture telemetry information from the CXL interface 304, the media interface 308, or the storage controller firmware, the telemetry information being related to access to the cache memory 134 and / or the storage medium 140 of the application data, which may be configured to use the SSD 130 as the working memory. The adaptive cache manager 136 may receive the telemetry information, as well as ML-based cache parameters and / or host-based cache parameters, from the telemetry unit and determine a cache policy for the cache memory 134 based at least on the telemetry information.

[0044] In various aspects, the adaptive cache manager 136 may configure or adjust the cache engine 322 or the prefetch engine 324 based on the determined cache policy. In some cases, the cache policy includes a selection of a cache policy type or configuration parameters of the cache policy. Alternatively or additionally, the adaptive cache manager 136 may provide the telemetry information to an ML model or a neural network to obtain ML-based cache parameters, by which to configure or adjust the cache engine 322 or the prefetch engine 324. By doing so, the adaptive cache manager may dynamically change the caching and prefetching activities of the cache memory to improve the efficiency of the cache memory.

[0045] Figure 4 An example implementation of the adaptive cache manager according to one or more aspects is illustrated at 400. In this example, the adaptive cache manager 136 is implemented in the CXL-enabled storage medium system 124, which is operatively coupled to the host 202 via the interconnect 122. The storage medium system 124 may be configured similarly to or differently from the SSD 130 described with reference to Figure 3 The host 202 includes a tenant 120 or an application (not shown) executing on the computing resources of the host, a CXL root port 302, and a host-side cache assistant 306. As Figure 4 shown, the cache assistant 306 and the adaptive cache manager 136 may communicate via the side channel 402 or through the CXL interface 304 of the storage medium system 124.

[0046] In various aspects, the telemetry unit 138 captures telemetry information related to accesses of the cache memory 134 and / or the storage medium 140 in response to transaction packets transmitted between the host 202 and the storage medium system 124. In some cases, the telemetry unit 138 captures first telemetry information related to accesses of the storage medium device as a memory, which may include CXL.mem transaction packets or microtiles to load data or store data associated with an application executed on the host. For example, the first telemetry information may include cache hits, request sizes to the cache, cache byte addresses, cache line addresses, data ages of cache lines, or access frequencies to cache lines. The first telemetry information may particularly relate to accesses to the cache memory 134 (such as cache hits), where data is returned to the host 202 without fetching data from the storage medium 140. Alternatively or additionally, the telemetry unit 138 may capture second telemetry information related to accesses of the storage medium 140, such as cache misses, request sizes to the storage medium, logical block addresses (LBAs) of the requested data, data ages at the LBAs, or access frequencies to the LBAs. In other words, when the requested data is not present in the cache memory 134, the second telemetry information may indicate activities related to fetching data from the storage medium for cache misses. Alternatively or additionally, the telemetry unit 138 maintains or records time series data, which may include a history of data accesses to the cache memory 134 and the storage medium 140. Such time series data may be used by a machine learning model, such as an LSTM (Long Short-Term Memory (LSTM) network, a type of recurrent neural network (RNN)), to initiate prefetch or cache eviction instructions.

[0047] In various aspects, the adaptive cache manager 136 determines, selects, or modifies the cache policy 404 of the cache memory 134 based on the first telemetry information and the second telemetry information. In some implementations, the adaptive cache manager 136 determines metrics for cache performance or efficiency based on the first telemetry information and the second telemetry information. For example, using the telemetry information or information provided by the storage medium controller 206, the adaptive cache manager 136 may determine cache hit rates, cache byte hit rates, cache miss rates, cache latencies, or cache access times. In some cases, the adaptive cache manager 136 monitors or tracks the metrics over time and compares the metrics to a threshold, which may be configured to trigger an update of the cache policy 404 or retraining of the ML model when the cache efficiency is below the threshold.

[0048] As Figure 4As shown, the adaptive cache manager 136 may include or access multiple ML models 326 and / or neural networks, configured or trained to assist or optimize cache policies 404 or cache schemes applied to the cache memory 134. The ML models 326 or neural networks may be stored in the persistent storage medium of the storage system, such as the storage medium 140, the internal memory of the storage controller 206 (not shown), or the memory of the adaptive cache manager 136. In this example, the ML model 326 is implemented as part of the adaptive cache manager 136, which is also shown to implement a cache engine 322 and a prefetch engine 324. The ML model 326 may include any suitable number of ML models, AI models, or neural networks, which may be differently or similarly configured to each other.

[0049] Various aspects of adaptive cache management may be implemented by the adaptive cache manager 136, which interacts with the ML model 326, neural network, or any suitable AI engine, AI model, or AI driver associated with or related to the storage medium controller, cache memory, or data cache component. For example, the adaptive cache manager 136 may use first telemetry information and / or second telemetry information to configure the ML machine learning model, and then use the output provided by the ML model to determine, select, and / or configure the cache policy 404 of the cache memory. In some cases, the adaptive cache manager 136 configures the ML model by training or retraining the ML model based on the first telemetry information and / or the second telemetry information. Alternatively or additionally, the adaptive cache manager 136 may be configured to perform or initiate retraining of the ML model in response to the duration of time elapsed or when a metric of the cache memory is below a performance threshold of the cache memory. Regarding various features of handling cache memory activities and / or cache memory policies and settings (e.g., first telemetry information and / or second telemetry information), one or more ML models 326 may be implemented with machine learning based on one or more neural networks (e.g., pre-trained, real-time trained, or dynamically retrained) to implement the aspects or techniques described herein (such as selecting a cache memory policy, selecting a prefetch policy, or configuring the corresponding settings of either type of policy). Any ML model, neural network, AI model, ML algorithm, etc. of the adaptive cache controller 136 may include a set of connected nodes (such as neurons or perceptrons) organized into one or more layers.

[0050] Typically, an instance of the ML model 326 associated with the adaptive cache manager 136 can be implemented with a deep neural network (DNN) that includes an input layer, an output layer, and one or more hidden intermediate layers positioned between the input layer, a pre-input layer (e.g., an embedding and / or averaging network), and the output layer of the neural network. Each node of the deep neural network can in turn be fully connected or partially connected between the layers of the neural network. The ML model or neural network can be any deep neural network (DNN) (such as a convolutional neural network (CNN) including one of AlexNet, ResNet, GoogleNet, MobileNet, etc.). Alternatively or additionally, the ML model or neural network can be implemented as or include any suitable recurrent neural network (RNN) or any variant thereof. Typically, the ML model 326, neural network, ML algorithm, or AI model employed by the adaptive cache controller 136 can also include any other supervised learning, unsupervised learning, reinforcement learning algorithms, etc.

[0051] Adaptive cache management techniques

[0052] The following discussion describes adaptive cache management techniques according to various aspects. Such techniques can be implemented using any of the environments and entities described herein, such as the adaptive cache manager 136, the telemetry unit 138, the cache engine 322, the prefetch engine 324, and / or the ML model 326. These techniques include Figures 5 to 7 the various methods shown in, each method being shown as a set of operations that can be performed by one or more entities of a storage medium controller. The described operations of the method can be performed using any suitable circuitry or components, such as the adaptive cache manager 136, the telemetry unit 138, the cache engine 322, the prefetch engine 324, and / or the ML model 326, which can provide the components for implementing one or more operations described in reference to Figures 5 to 7 the method description.

[0053] These methods are not necessarily limited to the order of operations shown in the associated figures. Instead, any operation can be repeated, skipped, replaced, or reordered to implement the various aspects described herein. Additionally, these methods can be used in combination with each other, either in whole or in part, whether performed by the same entity, separate entities, or any combination thereof. For example, these methods can be combined to implement adaptive cache management to set or change the parameters of the cache policy of the cache memory based on telemetry information, user input, and / or ML-based parameters to improve the cache efficiency of the cache memory. In the following discussion sections, the operating environment 100 of Figure 1 will be referred to by way of example and Figures 2 to 4various entities, configurations, or components. Such references should not be construed as limiting the described aspects to the operating environment 100, storage controller, entities, algorithms, or configurations, but rather as illustrative of one example among many. Alternatively or additionally, the operations of the method can also be implemented or utilized by entities described with reference to Figure 8 the SoC or Figure 9 the storage media controller described to implement or utilize these entities.

[0054] Figure 5 depicts an example method 500 for adaptive cache management in accordance with one or more aspects. The operations of method 500 can be implemented by the adaptive cache manager 136, telemetry unit 138, cache engine 322, prefetch engine 324, and / or ML model 326 of the media controller.

[0055] At 502, the adaptive cache manager receives, via the CXL interface, a transaction packet for data to access the cache memory of the storage media system from the host system. The transaction packet received from the host system can be compliant with or formatted according to the CXL memory protocol or the CXL cache protocol. When the storage media system is configured as the working memory of an application of the host system, the transaction packet can include a load instruction to load data from the cache memory (or the storage media) or a store instruction to store data to the cache memory (or the storage media).

[0056] At 504, the adaptive cache manager determines first telemetry information associated with the transaction packet for accessing the cache memory of the storage media system. The first telemetry information can include one or more of cache hit, request size to the cache, cache byte address, cache line address, data age of the cache line, or access frequency to the cache line. At 506, the adaptive cache manager determines second telemetry information associated with the access to the storage media of the storage media system associated with the transaction packet. The second telemetry information can include one or more of cache miss, request size to the storage media, LBA, data age at the LBA, or access frequency to the LBA.

[0057] At 508, an adaptive cache manager determines a cache policy for a cache memory based on first telemetry information and second telemetry information. This can include selecting a cache policy or configuring a cache policy to be applied to the cache memory or a controller of the cache memory. In some cases, the adaptive cache manager determines a metric for the cache memory based on the first telemetry information and the second telemetry information, and then determines a cache policy for the cache memory based on the metric of the cache memory. These metrics can include cache hit rate, cache byte hit rate, cache miss rate, cache latency, or cache access time.

[0058] At 510, the adaptive cache manager applies the cache policy to the cache memory to modify a caching scheme or a prefetching scheme for data in the cache memory. In some cases, the cache policy applied to the cache memory changes the caching scheme or the prefetching scheme of the cache memory. Alternatively or additionally, the adaptive cache manager adjusts settings or configures a cache engine or a prefetch engine associated with the cache memory. In some implementations, the operations of method 500 can be performed iteratively (e.g., at predetermined or random intervals) to optimize the settings of the cache memory (e.g., increase the hit / miss rate). By doing so, the adaptive cache manager can dynamically change the caching and prefetching activities of the cache memory to improve the efficiency of the cache memory.

[0059] Figure 6 An example method 600 for allocating streaming transaction requests according to various aspects is depicted. The operations of method 600 can be implemented by an adaptive cache manager 136, a telemetry unit 138, a cache engine 322, a prefetch engine 324, and / or an ML model 326 of a media controller.

[0060] At 602, an adaptive cache manager of a storage media system receives cache parameters from a host system. The cache parameters can be received for a user or an application executing on the host via a host system interface. Generally, the cache parameters provided by the host can enable the adaptive cache manager to build a collaborative (host-device) caching and prefetching mechanism. This can be important because an application or a user can use application context to provide application-specific hints that are lacking in the storage media system.

[0061] At 604, the adaptive cache manager receives, via the CXL interface, a transaction packet from the host system for accessing data in the cache memory of the storage media system. The transaction packet received from the host system can be compliant with or formatted according to the CXL memory protocol or the CXL cache protocol. When the storage media system is configured as the working memory of an application of the host system, the transaction packet can include a load instruction to load data from the cache memory (or the storage media), or a store instruction to store data to the cache memory (or the storage media).

[0062] At 606, the adaptive cache manager obtains telemetry information related to the transaction packet for accessing the cache memory and the storage media of the storage media system. The telemetry information can include or indicate cache memory accesses (such as cache hits, request size to the cache, cache byte address, cache line address, data age of the cache line, or access frequency to the cache line). Alternatively or additionally, the telemetry information can indicate or be related to accesses to the storage media of the storage media system (such as cache misses, request size to the storage media, LBA, data age at the LBA, or access frequency to the LBA).

[0063] At 608, the adaptive cache manager determines a cache policy for the cache memory based on the telemetry information received from the host system and cache parameters. This can include selecting a cache policy or configuring a cache policy to be applied to the cache memory or the controller of the cache memory. In some cases, the adaptive cache manager determines metrics for the cache memory based on first telemetry information and second telemetry information, and then determines a cache policy for the cache memory based on the metrics of the cache memory. These metrics can include cache hit rate, cache byte hit rate, cache miss rate, cache latency, or cache access time.

[0064] Optionally, at 610, the adaptive cache manager applies the cache policy to the cache engine of the cache memory. The application of the cache policy can change or set the cache engine to implement different types of cache policies or the same cache policy with different settings for saving or evicting data from the cache memory. Optionally, at 612, the adaptive cache manager applies the cache policy to the prefetch engine of the cache memory. The application of the cache policy can change or set the prefetch engine to implement different types of prefetch schemes or the same prefetch scheme with different settings for prefetching cache lines for anticipated host accesses and / or from the storage media.

[0065] Figure 7 FIG. 700 depicts an example method for managing flow transactions to balance bandwidth utilization in accordance with various embodiments. Operations of method 700 may be implemented by an adaptive cache manager 136, a telemetry unit 138, a cache engine 322, a prefetch engine 324, and / or an ML model 326 of a media controller.

[0066] At 702, the adaptive cache manager receives, via a CXL interface, a transaction packet for accessing data in a cache memory of a storage media system from a host system. The transaction packet received from the host system may conform to or be formatted according to a CXL memory protocol or a CXL cache protocol. When the storage media system is configured as a working memory for an application of the host system, the transaction packet may include a load instruction to load data from the cache memory (or the storage media), or a store instruction to store data to the cache memory (or the storage media).

[0067] At 704, the adaptive cache manager obtains telemetry information associated with the transaction packet for accessing the cache memory and the storage media of the storage media system. The telemetry information may include or indicate cache memory accesses (such as cache hits, request size to the cache, cache byte address, cache line address, data age of a cache line, or access frequency to a cache line). Alternatively or additionally, the telemetry information may indicate or be associated with accesses to the storage media of the storage media system (such as cache misses, request size to the storage media, LBA, data age at the LBA, or access frequency to the LBA).

[0068] At 706, the adaptive cache manager provides the telemetry information to a machine learning model. In some cases, the adaptive cache manager adjusts settings, weights, and / or layers of a neural network through which the machine learning model is implemented. Generally, the telemetry information may relate to or indicate access characteristics associated with cache memory accesses (e.g., cache hits) and / or accesses to the storage media (e.g., cache misses) that are generated in response to a transaction packet issued by the host for loading or storing application data.

[0069] At 708, the adaptive cache manager receives ML-based cache parameters from the machine learning model based on the telemetry information. The ML-based cache parameters may indicate a selection of a cache policy or a setting of a cache policy based on the telemetry information. Generally, the ML-based parameters provided by the ML model may be optimized based on the telemetry information such that the efficiency of the cache memory may be improved when determining an updated cache policy or a cache policy setting.

[0070] At 710, the adaptive cache manager uses ML-based cache parameters received from a machine learning model to determine a cache policy for the cache. This can include selecting a cache policy or configuring a cache policy to apply to the cache memory or a controller of the cache memory. In some cases, the adaptive cache manager determines metrics for the cache memory based on first telemetry information and second telemetry information, and then determines a cache policy for the cache memory based on the metrics of the cache memory. These metrics can include cache hit rate, cache byte hit rate, cache miss rate, cache latency, or cache access time.

[0071] At 712, the adaptive cache manager applies the cache policy to the cache memory to modify the caching scheme or prefetching scheme for data in the cache memory. In some cases, the cache policy applied to the cache memory changes the caching scheme or prefetching scheme of the cache memory. Alternatively or additionally, the adaptive cache manager adjusts settings or configures a cache engine or a prefetch engine associated with the cache memory. In some implementations, the operations of method 500 can be performed iteratively (e.g., at predetermined or random intervals) to optimize the settings of the cache memory (e.g., increase the hit / miss rate). By doing so, the adaptive cache manager can dynamically change the caching and prefetching activities of the cache memory to improve the efficiency of the cache memory.

[0072] System-on-Chip and Controller

[0073] Figure 8 An example system-on-chip (SoC) 800 environment is illustrated in which various aspects of adaptive cache management of a storage medium system can be implemented. The SoC 800 can be implemented in any suitable system or device, such as a storage device, a memory device, a router, a wireless access point, a smart phone, a netbook, a tablet computer, an access point, a network-attached storage device, a camera, a smart appliance, a printer, a set-top box, a server, a data storage center, a solid state drive (SSD), a hard disk drive (HDD), an array of storage drives, a memory module, an automotive computing system, an aggregated storage controller, an aggregated memory controller, or any other suitable type of device (e.g., other devices described herein). Although described with reference to an SoC, Figure 8The entity can also be implemented as other types of integrated circuits or embedded systems, such as application-specific integrated circuits (ASICs), memory controllers, storage controllers, communication controllers, application-specific standard products (ASSPs), digital signal processors (DSPs), programmable system-on-chips (PSoCs), system-in-packages (SiPs), or field-programmable gate arrays (FPGAs).

[0074] The SoC 800 can be integrated with electronic circuitry, microprocessors, memories, input-output (I / O) control logic, media interfaces, host interfaces, firmware, and / or software for providing the functionality of a computing device, host system, memory system, or storage system, such as any device or component described herein (e.g., storage controller, memory controller, CXL cache controller). The SoC 800 can also include an integrated data bus or interconnect structure (not shown) that couples the various components of the SoC for control signaling, data communication, and / or routing between the components. The integrated data bus, interconnect structure, or other components of the SoC 800 can be exposed or accessed through an external port, and a parallel data interface, serial data interface, fabric-based interface, peripheral component interface (e.g., a PCIe-based interface), or any other suitable data interface. For example, the components of the SoC 800 can access or control an external storage medium, external memory, processing block, network interface, or neural network through an external interface or off-chip data interface.

[0075] In this example, the SoC 800 includes various components, such as input / output (I / O) control logic 802 and a hardware-based processor 804 (processor 804), such as a microprocessor, a processor core, an application processor, a DSP, an ASIC, etc. The SoC 800 also includes a memory 806, which can include any type and / or combination of RAM, SRAM, DRAM, non-volatile memory, ROM, one-time programmable (OTP) memory, multi-time programmable (MTP) memory, flash memory, and / or other suitable electronic data storage devices. In this instance, the memory 806 includes an instance of a cache memory 134, which can include a cache of volatile memory (e.g., RAM or DRAM). In some aspects, the processor 804 and the code stored on the memory 806 are implemented as a storage controller, a cache controller, or a memory controller to provide various functions associated with adaptive cache management. In the context of the present disclosure, the memory 806 stores data, code, instructions, or other information via a non-transitory signal and does not include a carrier wave or a transitory signal. Alternatively or additionally, the SoC 800 can include a data interface (not shown) for accessing additional or expandable off-chip media, such as solid-state memory (e.g., flash memory or NAND memory), storage media (e.g., DRAM modules or dies), magnetic-based storage media, or optical-based storage media).

[0076] The SoC 800 can also include firmware 808, applications, programs, software, and / or an operating system, which can be implemented as processor-executable instructions stored on the memory 806 for execution by the processor 804 to implement the functions of the SoC 800. The SoC 800 can also include other communication interfaces (such as components for controlling or communicating with a local on-chip (not shown) or off-chip communication transceiver). Thus, in some aspects, the SoC 800 can be implemented or configured as a communication transceiver capable of implementing aspects of adaptive cache management to process data received through a communication channel or network interface. Alternatively or additionally, the transceiver interface can also include or implement a signal interface to transmit radio frequency (RF), intermediate frequency (IF), or baseband frequency signals outside the chip to facilitate wired or wireless communication through transceivers, PHYs, and MACs coupled to the SoC 800. For example, the SoC 800 can include a transceiver interface configured to be capable of storing through a wired or wireless network to provide adaptive cache management for communicating data and / or storing data for a network-attached storage (NAS) volume or a storage accelerator.

[0077] The SoC 800 also includes an adaptive cache manager 136, a telemetry unit 138, a cache and prefetch engine 322 / 324, and / or an ML model 326, which may be implemented separately as shown or in combination with a media controller, a host interface, or a media interface. In accordance with various aspects of adaptive cache management, the adaptive cache manager 136 obtains telemetry information related to access to the cache memory 134 and the storage media coupled to the SoC 800 from the telemetry unit 138. Based on the telemetry information, the adaptive cache manager 136 determines a cache policy for the cache memory 134 and applies the cache policy to the cache memory 134 to modify the settings of the cache and prefetch engine 322 / 324 (or the cache scheme implemented by the engine). As described with reference to the various aspects given herein, any one of these entities may be embodied as different or combined components. For example, the adaptive cache manager 136 may be implemented as part of a storage media controller, a memory controller, or other media aggregator or accelerator. Refer to Figure 1 the operating environment 100 of Figures 2 to 4 the storage controller and configuration, and / or Figures 5 to 7 the corresponding components or entities of methods 500 to 700 of

[0078] to describe examples of these components and / or entities, or the corresponding functions. The adaptive cache manager 136 or its components, in whole or in part, may be implemented as processor-executable instructions maintained by the memory 806 and executed by the processor 804 to implement various aspects and / or features of adaptive cache management.

[0079] As another example, consider Figure 9, which illustrates an example storage medium controller 900 in accordance with one or more aspects of adaptive cache management. In various aspects, storage medium controller 900 or any combination of its components may be implemented as a storage drive controller (CXL-enabled SSD controller), a distributed storage center controller (e.g., between a host and an SSD), a storage medium controller, a NAS controller, a fabric interface, an NVMe target, or a storage aggregation controller for solid-state storage media. In some cases, storage medium controller 900 is implemented similar to or leveraging components of the SoC 800 described in reference Figure 8 . In other words, an instance of the SoC 800 may be configured as a storage controller (such as storage medium controller 900) to enable data communication, data access, or data storage with aspects of adaptive cache management.

[0080] As Figure 9 illustrates, storage medium controller 900 includes input / output (I / O) control logic 902 and a processor 904, such as a microprocessor, a processor core, an application processor, a DSP, etc. In some aspects, the processor 904 and the firmware of storage medium controller 900 may be implemented to provide various functions associated with adaptive cache management (such as those described in any of the methods of reference methods 500 to 700). Storage medium controller 900 also includes a host interface 906 (e.g., CXL, SATA, PCIe, NVMe, or a fabric interface) and a storage medium interface 908 (e.g., a NAND interface or a flash interface), which enable access to the host system and the storage medium, respectively. The storage medium controller also includes a flash translation layer 910, a cache memory 134, an adaptive cache manager 136, which may be operably coupled to the cache and prefetch engines 322 / 324, the telemetry unit 138, and / or the ML model 326 of the controller. In some aspects of adaptive cache management, the adaptive cache manager 136 and its components may interact with the host interface 906, the storage medium interface 908, and the cache memory 134 to implement adaptive cache management to select, determine, configure, and / or apply cache policies to the cache memory 134 to improve cache efficiency, reduce cache access latency, or improve host application performance.

[0081] Any one or all of these components may be implemented separately as illustrated, or in combination with the processor 904, the host interface 906, and / or the storage medium interface 908 of the storage medium controller 900. Reference Figure 1 to the corresponding components or entities of the operating environment 100 of Figures 2 to 4The operations of the storage controller and components or methods 500 to 700 are described, and examples of these components and / or entities or corresponding functions are provided. According to various aspects of adaptive cache management, the adaptive cache manager 136 of the storage medium controller 900 can obtain telemetry information related to the access of the cache memory 134 and the access of the storage medium coupled to the controller from the telemetry unit 138. Based on the telemetry information, user-provided parameters, and / or ML-based parameters, the adaptive cache manager determines a cache policy for the cache memory and applies the cache policy to the cache memory to modify the caching scheme or prefetching scheme for the data in the cache memory. By doing so, the adaptive cache manager can dynamically change the caching and prefetching activities of the cache memory to improve the efficiency of the cache memory.

[0082] Although the subject matter of the adaptive cache management of the storage system has been described in language specific to structural features and / or method operations, it should be understood that the subject matter of the appended claims need not be limited to the particular examples, features, configurations, or operations described herein, including the order in which they are performed.

Claims

1. A method for adaptive cache management implemented by a storage medium system, comprising: receiving, via a compute fast link (CXL) interface, a transaction packet for accessing data of a cache memory of the storage medium system; determining first telemetry information associated with the transaction packet received for accessing the cache memory; determining second telemetry information related to access of a storage medium of the storage medium system associated with the transaction group; determining a cache policy for the cache memory based on the first telemetry information and the second telemetry information; as well as The cache policy is applied to the cache memory to modify a caching scheme or a pre-fetching scheme for the data of the cache memory.

2. The method according to claim 1, further comprising: receiving cache parameters provided by an application or a user via the CXL interface; as well as The cache policy is determined based on the first telemetry information, the second telemetry information, and the cache parameters provided by the application or the user.

3. The method according to claim 1, further comprising: configuring a machine learning model using the first telemetry information and the second telemetry information, and wherein: The cache policy for the cache memory is determined using the machine learning model.

4. The method of claim 3, wherein configuring the machine learning model comprises: Train or retrain the machine learning model based on the first telemetry information and the second telemetry information.

5. The method of claim 4, wherein retraining the machine learning model is performed in response to: the duration of the passage of time; or A metric of the cache memory is below a performance threshold of the cache memory.

6. The method according to claim 1, further comprising: determining a metric of the cache memory based on the first telemetry information and the second telemetry information, and wherein: The cache policy for the cache memory is determined based on the metric of the cache memory.

7. The method of claim 6, wherein the metric for the cache memory comprises: Cache hit ratio, cache byte hit ratio, cache miss ratio, cache latency, or cache access time.

8. The method of claim 1, wherein: The first telemetry information includes: cache hits, request sizes to the cache, cache byte addresses, cache line addresses, data age of cache lines, or access frequency to cache lines; and The second telemetry information includes: cache miss, request size to the storage medium, logical block address LBA, data age at LBA, or access frequency to LBA. 9 . The method of claim 1 , wherein the received transaction packet complies with a CXL memory protocol or a CXL cache protocol.

10. The method of claim 1, wherein the received transaction packet comprises a load instruction to load data from the cache memory or a store instruction to store data to the cache memory.

11. The method according to claim 1, wherein: The cache memory includes a dynamic random access memory DRAM of the storage medium system; and The storage medium includes a NAND memory of the storage medium system.

12. An apparatus comprising: Cache memory; A computing fast link CXL interface configured to receive a transaction packet for accessing the cache memory; a storage medium configured to store data; a storage medium controller configured to enable transfer of the data between the cache memory and the storage medium; a telemetry unit operably coupled to the CXL interface and the storage medium controller; and Adaptive Cache Manager, configured to: obtaining, from the telemetry unit, first telemetry information associated with the transaction packet received for accessing the cache memory; obtaining, from the telemetry unit, second telemetry information related to data present in the cache memory and access of the storage medium associated with the transaction packet; determining a cache policy for the cache memory based on the first telemetry information and the second telemetry information; as well as The cache policy is applied to the cache memory to modify a caching scheme or a pre-fetching scheme for the data stored by the cache memory.

13. The apparatus of claim 12, wherein the adaptive cache manager is further configured to: receiving cache parameters provided by an application or a user via the CXL interface; and The cache policy is determined based on the first telemetry information, the second telemetry information, and the cache parameters provided by the application or the user.

14. The apparatus of claim 12, wherein the adaptive cache manager is further configured to: configuring a machine learning model using the first telemetry information and the second telemetry information, and wherein: The cache policy for the cache memory is determined using the machine learning model.

15. The apparatus of claim 12, wherein the adaptive cache manager is further configured to: determining a metric for the cache memory based on the first telemetry information and the second telemetry information, and wherein: The cache policy for the cache memory is determined based on the metric of the cache memory.

16. The apparatus of claim 15, wherein the metric for the cache memory comprises: Cache hit ratio, cache byte hit ratio, cache miss ratio, cache latency, or cache access time.

17. A system on chip, comprising: Cache memory; A computing fast link interface CXL interface configured to receive a transaction packet for accessing the cache memory; a storage medium controller having a storage medium interface and configured to cause data to be transferred between the cache memory and a storage medium coupled to the storage medium interface; a telemetry unit operably coupled to the cache memory and the storage medium controller; and Adaptive Cache Manager, configured to: receiving, from the telemetry unit, first telemetry information associated with the transaction packet for accessing the cache memory; receiving, from the telemetry unit, second telemetry information related to access of the storage medium associated with the transaction packet; determining a cache policy for the cache memory based on the first telemetry information and the second telemetry information; as well as The cache policy is applied to the cache memory to modify a caching scheme or a pre-fetching scheme for the data stored by the cache memory.

18. The system on chip of claim 17, wherein the adaptive cache manager is further configured to: receiving cache parameters provided by an application or a user via the CXL interface; and The cache policy is determined based on the first telemetry information, the second telemetry information, and the cache parameters provided by the application or the user.

19. The system on chip of claim 17, wherein the adaptive cache manager is further configured to: configuring a machine learning model using the first telemetry information and the second telemetry information, and wherein: The cache policy for the cache memory is determined using the machine learning model.

20. An apparatus comprising: Cache memory; A computing fast link CXL interface configured to receive a transaction packet for accessing the cache memory; an interface configured to enable data to be transferred to a storage medium coupled to the interface; means for transferring said data between said cache memory and said storage medium; means for obtaining first telemetry information related to the transaction packet received for accessing the cache memory and second telemetry information related to data present in the cache memory and access to the storage medium associated with the transaction packet; means for determining a cache policy for said cache memory based on said first telemetry information and said second telemetry information; as well as Means for applying the cache policy to the cache memory to modify a caching scheme or a pre-fetching scheme for the data stored by the cache memory.

Images