Memory-side page table for a storage system

Abstract

This application relates to accessing data in a storage system of an electronic system. The storage system includes a memory-side DRAM and one or more storage channels. The storage system receives a storage access request for target data stored in the storage system, and the storage access request includes a target virtual address of the target data. In response to the storage access request, the storage system searches a page table of the memory-side dynamic random access memory (DRAM) for a target physical address that maps to the target virtual address of the target data. The page table includes mappings between a plurality of virtual addresses and a plurality of physical addresses of the memory-side DRAM. Based on the search result, the storage system fetches the target data stored in the memory-side DRAM and one of the one or more storage channels according to the target physical address.

Description

Technical Field

[0001] This application generally relates to storage management, including but not limited to methods, systems, and non-transitory computer-readable media for accessing data stored in storage systems of electronic systems. Background Technology

[0002] Memory is used in computer systems to store instructions and data. Specifically, computer systems rely on non-volatile memory to retain the instructions and data stored thereon when the computer system is disconnected from power. Examples of secondary storage include, but are not limited to, hard disk drives (HDDs) and solid-state drives (SSDs). Many electronic systems (e.g., servers, jig disks (JBODs), racks, electronic devices) include a large number (e.g., 4 to 30) of SSDs. Additionally, computer systems may rely on dynamic random access memory (DRAM) for storing and retrieving data and instructions that do not require power-off persistence. Examples include application and kernel data structures that are reconstructed when the computer system boots up and loads its operating system and applications are loaded and executed. Both SSD and DRAM devices can be configured to perform different memory functions under the control of their host devices. Generally, DRAM devices support byte-addressable access known as read (read from storage device) and write (write to storage device), while SSDs support indirect command and response protocols using larger-granularity logical block addressing schemes. With the introduction of protocols such as compute fast links, byte-addressable DRAM devices can now be attached to computer systems via peripheral buses similar to the High-Speed ​​Peripheral Component Interconnect (PCIe). Processor cores use logical addressing methods that rely on translation tables to translate logical addresses into physical addresses. These tables typically reside in DRAM memory. For performance reasons, frequently used logical-to-physical associations are cached in a Translation Lookaside Buffer (TLB). For cost and capacity / density reasons, system designers have desired to place cheaper and slower memory, such as NAND flash memory, where DRAM memory has traditionally been used. However, this approach is challenging because the processor architecture of computer systems does not scale well to the highly parallel load and store operations necessary to prevent excessive processor stalls when load or store operations take a long time to complete. The processor core enters a stall, waiting for load and store operations from memory to complete, and may not be able to issue more overlapping load and store operations outside the boundaries of the reordering window. Developing an efficient and effective mechanism to manage data access requests in electronic systems would be beneficial. Summary of the Invention

[0003] Various embodiments of this application relate to methods, systems, apparatuses, and non-transitory computer-readable media for accessing data stored in a storage system of an electronic system via memory-side dynamic random access memory (DRAM). The electronic system includes a host device coupled to the storage system. In some embodiments, the host device is coupled to the storage system via a compute fast link (CXL), which is a high-speed interconnect industry-standard interface for communication between processors, accelerators, storage, memory, and other I / O devices. The storage system hosts local page tables describing mappings stored in the global address space. These page tables of the storage system reflect the hierarchical structure of storage types present in the storage system. The storage system manages the page tables, for example, by placing the most frequently and / or recently accessed pages in the fastest layer of memory (e.g., DRAM) in the storage system based on access mode self-checking and hot / cold separation. Furthermore, the storage system communicates with the host device to manage host-side cache memory to clear outdated information.

[0004] In some embodiments, the storage system includes memory-side DRAM for storing the local page table, which maps multiple virtual addresses to multiple physical addresses in the memory-side DRAM and one or more memory channels. The storage system includes the DRAM and one or more different types of memory (e.g., single-cell (SLC) memory, multi-cell (MLC) memory, three-cell (TLC) memory, four-cell (QLC) memory, five-cell (PLC) memory, or any suitable number of data bits). Data pages migrate between DRAM and one or more different types of memory based on whether the data page is frequently or recently accessed or is predicted to be accessed in the near future. In these ways, the storage system's local page table offloads the workload of central processing unit (CPU) page table lookups, thereby reducing the host CPU's involvement in some memory management unit (MMU) operations and improving overall system performance. Multiple memory types are aggregated into a unified load / store pool, allowing memory cost arbitration within the load / store pool. Potential memory accesses are hidden using existing mechanisms (page faults). Compared to the host device's processor, the memory-side dedicated hardware is configured to cache a larger number of Translation Lookaside Buffer (TLB) entries. Distributed TLBs are generated using one or more memory-side TLBs, supplementing the processor-side TLBs, and provide a larger overall cache for translations. Furthermore, the memory-side dedicated hardware is configured to query local page tables more efficiently than the host device, for example, by placing co-addressable page table entries in the same DRAM column, mitigating row activation overhead.

[0005] In one aspect, a method is implemented to access data in a storage system of an electronic system. The storage system receives a storage access request for target data stored in the storage system, and the storage access request includes a target virtual address of the target data. The storage system includes memory-side DRAM and one or more storage channels. In response to the storage access request, the storage system searches a page table in the memory-side DRAM for a target physical address mapped to the target virtual address of the target data. The page table includes mappings between multiple virtual addresses and multiple physical addresses in the memory-side DRAM. In response to the storage access request and based on the search result, the storage system retrieves the target data stored in the memory-side DRAM and one of the one or more storage channels according to the target physical address corresponding to the target virtual address.

[0006] In another aspect, some embodiments of this application provide an electronic system including one or more processors and a memory storing instructions thereon, the instructions, when executed by the one or more processors, causing the processors to perform any of the methods described above on a storage system (e.g., one or more SSDs).

[0007] In another aspect, some embodiments of this application provide a storage system including a plurality of storage devices (e.g., including one or more SSDs) and a memory storing instructions thereon, the instructions causing the processors, when executed by the one or more processors, to perform any of the methods described above on the storage system.

[0008] In another aspect, some embodiments provide a non-transitory computer-readable storage medium storing one or more programs. The one or more programs include instructions that, when executed by one or more processors, cause the processors to perform any of the methods described above on a storage system (e.g., including one or more SSDs).

[0009] These illustrative embodiments and implementations are not intended to limit or restrict the invention, but rather to provide examples to aid in understanding the invention. Additional embodiments are discussed in the detailed description and further description is provided therein. Attached Figure Description

[0010] To better understand the various implementations described, the following detailed descriptions should be taken in conjunction with the accompanying drawings, in which the same reference numerals refer to corresponding parts throughout the drawings.

[0011] Figure 1 This is a block diagram of an example system module in a typical electronic system according to some embodiments.

[0012] Figure 2This is a block diagram of a storage system of an example electronic system having one or more storage access queues according to some embodiments.

[0013] Figure 3 This is a block diagram of an example electronic system for handling TLB misses in the page table of host-side DRAM according to some embodiments.

[0014] Figure 4 This is a block diagram of another example electronic system for handling TLB misses and identifying unmapped pages in a storage system, according to some embodiments.

[0015] Figure 5 This is a block diagram of an example electronic system for handling TLB misses in the page table of a memory-side DRAM located in a storage system, according to some embodiments.

[0016] Figure 6 This is a block diagram of another example electronic system for handling TLB misses and cold page migrations in a storage system according to some embodiments.

[0017] Figure 7 This is a flowchart of an example method for accessing data in a storage system according to some embodiments.

[0018] In the various views shown in the accompanying drawings, the same reference numerals refer to corresponding parts. Detailed Implementation

[0019] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. Numerous non-limiting specific details are set forth in the following detailed description to aid in understanding the subject matter presented herein. However, it will be apparent to those skilled in the art that various alternatives may be used without departing from the scope of the claims, and that the subject matter may be practiced without these specific details. For example, it will be apparent to those skilled in the art that the subject matter presented herein can be implemented in many types of electronic systems or devices with data storage capabilities.

[0020] This application relates to methods, systems, apparatuses, and non-transitory computer-readable media for accessing data stored in a storage system within an electronic system via memory-side DRAM. The electronic system further includes a host device coupled to the storage system. The storage system hosts local page tables describing the mapping between multiple virtual addresses and multiple physical addresses of the memory-side DRAM and one or more memory channels. These page tables of the storage system reflect the hierarchical structure of storage types present in the storage system and are managed based on access pattern self-checking and hot / cold separation. In other words, data pages migrate between DRAM and one or more different types of memory based on whether the data page is determined to be frequently or recently accessed or is predicted to be accessed in the near future. In these ways, the local page tables of the storage system offload the workload of page table lookups by the central processing unit (CPU), allow storage cost arbitration in load / storage pools of different storage types, hide potential storage accesses via existing mechanisms (page faults), and cache more TLB entries than the host device, thereby providing an efficient and effective mechanism to be applied to the storage system to manage data access requests received from its host device.

[0021] Figure 1 This is a block diagram of an example system module 100 in a typical electronic system according to some embodiments. System module 100 in this electronic system includes at least: a processor module 102; a storage module 104 for storing programs, instructions, and data; an input / output (I / O) controller 106; one or more communication interfaces, such as a network interface 108; and one or more communication buses 140 for interconnecting these components. In some embodiments, the I / O controller 106 allows the processor module 102 to communicate with I / O devices (e.g., a keyboard, mouse, or trackpad) via a universal serial bus interface. In some embodiments, the network interface 108 includes one or more interfaces for Wi-Fi, Ethernet, and Bluetooth networks, each allowing the electronic system to exchange data with external sources, such as a server or another electronic system. In some embodiments, the communication bus 140 includes a circuitry (sometimes referred to as a chipset) that interconnects and controls communication between the various system components contained in system module 100.

[0022] In some embodiments, storage module 104 includes high-speed random access memory, such as DRAM, static random access memory (SRAM), dual data rate (DDR) dynamic random access memory (RAM), or other random access solid-state storage devices. In some embodiments, storage module 104 includes non-volatile memory, such as one or more disk storage devices, optical disk storage devices, flash memory storage devices, or other non-volatile solid-state storage devices. In some embodiments, storage module 104, or alternatively, the non-volatile storage device within storage module 104, includes a non-transitory computer-readable storage medium. In some embodiments, a storage slot is reserved on system module 100 for housing storage module 104. Once inserted into the storage slot, storage module 104 is integrated into system module 100.

[0023] In some embodiments, system module 100 further includes one or more components selected from storage controller 110, SSD 112, hard disk drive (HDD) 114, power management integrated circuit (PMIC) 118, graphics module 120, and audio module 122. Storage controller 110 is configured to control communication between processor module 102 in the electronic system and storage components including storage module 104. SSD 112 is configured to apply integrated circuit assemblies to store data in the electronic system and, in many embodiments, is based on NAND or NOR memory. HDD 114 is a conventional data storage device for storing and retrieving digital information based on electromechanical disks. Power connector 116 is electrically coupled to receive external power. PMIC 118 is configured to modulate the received external power to other desired DC voltage levels, such as 5 V, 3.3 V, or 1.8 V, as needed by various components or circuits within the electronic system (e.g., processor module 102). In some embodiments, the graphics module 120 is configured to generate an output image to feed to one or more display devices according to a desired image / video format of the display devices. In some embodiments, the audio module 122 is configured to facilitate the input of audio signals to and from an electronic system under the control of a computer program.

[0024] In some embodiments, system module 100 further includes an SSD 112' directly coupled to I / O controller 106. Instead, SSD 112 is coupled to communication bus 140. In one example, communication bus 140 operates according to a high-speed peripheral component interconnect (PCIe or PCI-E) standard, which is a serial extended bus standard for interconnecting processor module 102 to one or more peripheral devices and various system components including components 110 to 122, and controlling said one or more peripheral devices and various system components.

[0025] Furthermore, those skilled in the art will recognize the use of other non-transitory computer-readable storage media, as new data storage technologies have been developed for storing information in non-transitory computer-readable storage media within storage modules 104, SSDs 112 and 112', and HDD 114. These new non-transitory computer-readable storage media include, but are not limited to, media made of biomaterials, nanowires, carbon nanotubes, and individual molecules, even if the corresponding data storage technology is currently under development and not yet commercialized.

[0026] Figure 2 This is a block diagram of a storage system 200 of an example electronic system having one or more storage access queues according to some embodiments. The storage system 200 is coupled to a host device 220 (e.g., Figure 1 The processor module 102 is configured to store instructions and data for extended periods, such as when the electronic system is in sleep, hibernation, or powered off. The host device 220 is configured to access and process the instructions and data stored in the storage system 200 to run an operating system and execute user applications. The storage system 200 further includes a controller 202 and a plurality of storage channels 204. Each storage channel 204 includes a plurality of storage units. The controller 202 is configured to execute firmware-level software to bridge the plurality of storage channels 204 to the host device 220. In some embodiments, a set of storage channels 204 forms a storage device (e.g., an SSD), and the storage system 200 includes one or more storage devices.

[0027] Each storage channel 204 includes one or more storage packages 206 (e.g., two storage dies). In one example, each storage package 206 corresponds to a storage die. Each storage package 206 includes a plurality of storage planes 208, and each storage plane 208 further includes a plurality of storage pages 210. Each storage page 210 includes a set of ordered storage cells, and each storage cell is identified by a corresponding physical address. In some embodiments, the storage system 200 includes a plurality of superblocks. Each superblock includes a plurality of storage blocks, each of which further includes a plurality of storage pages 210. For each superblock, the plurality of storage blocks are configured to be written to and read from the storage system in parallel via a storage input / output (I / O) interface. Optionally, each superblock groups the storage cells distributed across the plurality of storage planes 208, the plurality of storage channels 204, and the plurality of storage dies 206. In one example, each superblock comprises at least one set of storage pages, each page being distributed across different storage dies 206, having the same die, plane, block, and page names, and accessed through different channels of different storage dies 206. In another example, each superblock comprises at least one set of storage blocks, each storage block being distributed across different storage dies 206, comprising multiple pages, having the same die, plane, and block names, and accessed through different channels of different storage dies 206. Storage system 200 stores information about the ordered list of superblocks in a cache memory of storage system 200. In some embodiments, the cache memory is managed by a host driver of host device 220 and is referred to as host managed cache memory (HMC).

[0028] In some embodiments, the storage system 200 includes single-cell (SLC) NAND flash memory chips, with each cell storing a single data bit. In some embodiments, the storage system 200 includes multi-cell (MLC) NAND flash memory chips, with each cell of the MLC NAND flash memory chip storing two data bits. In one example, each cell of a three-cell (TLC) NAND flash memory chip stores three data bits. In another example, each cell of a four-cell (QLC) NAND flash memory chip stores four data bits. In yet another example, each cell of a five-cell (PLC) NAND flash memory chip stores five data bits. In some embodiments, each cell may store any suitable number of data bits. Compared to non-SLC NAND flash memory chips (e.g., MLC SSDs, TLC SSDs, QLC SSDs, PLCSSDs), SSDs with SLC NAND flash memory chips operate at higher speeds, higher reliability, and longer lifespans; however, they have lower device density and higher prices.

[0029] Each storage channel 204 is coupled to a corresponding channel controller 214, which is configured to control internal and external requests for accessing storage cells in the corresponding storage channel 204. In some embodiments, each storage package 206 (e.g., each storage die) corresponds to a corresponding storage access request queue 216. In some embodiments, each storage channel 204 corresponds to one corresponding storage access request queue 216. Additionally, in some embodiments, each storage channel 204 corresponds to different and distinct storage access request queues 216. In some embodiments, a subset (less than all) of the plurality of storage channels 204 corresponds to different storage access request queues 216. In some embodiments, all of the plurality of storage channels 204 of the storage system 200 correspond to a single storage access request queue 216. Each storage access request may optionally be received internally from the storage system 200 to manage the corresponding storage channel 204, or externally from the host device 220 to write or read data stored in the corresponding channel 204. Specifically, each storage access request includes one of the following: a system write request received from storage system 200 to write to the corresponding storage channel 204; a system read request received from storage system 200 to read from the corresponding storage channel 204; a host write request originating from host device 220 to write to the corresponding storage channel 204; and a host read request received from host device 220 to read from the corresponding storage channel 204. It should be noted that system read requests (also referred to as background read requests or non-host read requests) and system write requests are dispatched by the storage controller to implement internal storage management functions, including but not limited to garbage collection, wear leveling, read interference mitigation, memory snapshot capture, memory mirroring, caching, and memory spare.

[0030] In some embodiments, in addition to channel controller 214, controller 202 further includes local storage processor 218, host interface controller 222, SRAM buffer 224, and DRAM controller 226. Local storage processor 218 accesses multiple storage channels 204 based on one or more storage access request queues 216. In some embodiments, local storage processor 218 writes to and reads from the multiple storage channels 204 based on storage blocks. Data from one or more storage blocks is jointly written to or read from the multiple channels. Data in the same storage block is not written in parallel by more than one operation. Each storage block optionally corresponds to one or more storage pages. In one example, each storage block to be jointly written to or read from the multiple storage channels 204 has a size of 16 KB (e.g., one storage page). In another example, each storage block to be jointly written to or read from the multiple storage channels 204 has a size of 64 KB (e.g., four storage pages). In some embodiments, each page has 16 KB of user data and 2 KB of metadata. Additionally, the number of storage blocks to be jointly accessed and the size of each storage block can be configured for each of the system read, host read, system write, and host write operations.

[0031] In some embodiments, the local storage processor 218 stores data to be written to or read from each of the plurality of storage channels 204 in an SRAM buffer 224 of the controller 202. Alternatively, in some embodiments, the local storage processor 218 stores data to be written to or read from each of the plurality of storage channels 204 in a DRAM buffer 228A, which is included in the storage system 200, for example by means of a DRAM controller 226. Alternatively, in some embodiments, the local storage processor 218 stores data to be written to or read from each of the plurality of storage channels 204 in a DRAM buffer 228B, which is a DRAM buffer provided by the processor module 102 (…). Figure 1 The main memory used. The local storage processor 218 of controller 202 accesses the DRAM buffer 228B through the host interface controller 222.

[0032] In some embodiments, data in multiple storage channels 204 is grouped into decoded blocks, and each decoded block is called a codeword. For example, each codeword contains n bits, where k bits correspond to user data, and (nk) corresponds to integrity data of the user data, where k and n are positive integers. In some embodiments, the storage system 200 includes an integrity engine 230 (e.g., an LDPC engine) and a register 232 including multiple registers or SRAM cells or flip-flops and coupled to the integrity engine 230. The integrity engine 230 is coupled to the storage channels 204 via a channel controller 214 and an SRAM buffer 224. Specifically, in some embodiments, the integrity engine 230 has a data path connection to the SRAM buffer 224, which is further connected to the channel controller 214 via a data path controlled by a local storage processor 218. The integrity engine 230 is configured to verify the data integrity of each decoded block of the storage channel 204.

[0033] In some embodiments, host device 220 is coupled to storage system 200 via compute fast links (CXL), which are industry-standard high-speed interconnect interfaces for communication between processors, accelerators, storage, memory, and other I / O devices. CXL improves efficiency by allowing composability, scalability, and flexibility in heterogeneous and distributed computing architectures. A key advantage of CXL is the storage expansion of compute nodes, filling the gaps in data-intensive applications requiring high bandwidth, capacity, and low latency. The CXL infrastructure implements multiple switching layers in front of load / store memory, thereby increasing latency.

[0034] In some embodiments, storage system 200 virtualizes various memory types (DRAM, NAND, emerging memory) with different latency characteristics into load / memory pools mapped to the address space of host device 220. In some embodiments, host device 220 uses page faults to hide the latency of load operations. Page faults and hardware context switching 402 ( Figure 4 ) is an exception condition that invokes a software exception condition handler. This exception condition handler may be designed to estimate the latency of the load operation and determine whether it is permissible for an application executing on host device 220 to wait for the load operation or to invoke kernel or hypervisor capabilities to reschedule another thread on host device 220 while the load operation is being serviced in the background. In some embodiments, one or more host page tables 302 ( Figures 3 to 6 The logical page to physical page mapping is contained and stored close to the processor DRAM 228B. Alternatively, in some embodiments, one or more supplementary page tables 502 ( Figure 5 and 6The logical page to physical page mapping is contained and stored in the memory-side DRAM 228A of the storage system 200. In some embodiments, the application accesses the logical page without caching existing logical-to-physical translations in the host device 220's TLB 306. Page table lookup 512 ( Figure 5 The page table structure stored in DRAM 228A of storage system 200 is repeatedly checked, the logical-to-physical mapping is determined, the mapping in TLB 306 is cached, and application execution is resumed. Furthermore, in some embodiments, storage system 200 manages page table entries stored in DRAM 228A to enable support for a large number of storage types with different latency (e.g., SLC, MLC, TLC, MLC, PLC, HMC). Page fault exception handler 404 ( Figure 4 This can be applied to facilitate the cancellation or rescheduling of pages that will face frequent loading and have extended latency from the first storage type to the second faster storage type, or to coordinate the cancellation or rescheduling of applications associated with pages loaded in potential incomplete operations.

[0035] Figure 3 This is a block diagram of an example electronic system 300 for handling page table 302 of host-side DRAM 228B in a TLB not in accordance with some embodiments. Electronic system 300 includes host device 220 and memory system 200 coupled to host device 220. Examples of host devices include processor module 102 and I / O controller 106. Figure 1 The host device 220 includes a processor core 304, and the processor core 304 further includes a TLB 306 that stores the latest translation of virtual addresses to physical addresses in host-side DRAM 228B or in the storage system 200. In some scenarios, the host device issues a storage access request 308 for target data stored in the electronic system 300, and the storage access request 308 includes a target virtual address 322 of the target data. For example, the target data is stored on storage page 210T, and the target virtual address 322 corresponds to the storage page 210T storing the target data. The host device 220 determines that the target virtual address 322 has not been accessed recently and is therefore not found in the TLB 306. The host device 220 has a TLB miss in response to the storage access request 308.

[0036] In some embodiments, the host-side DRAM 228B further stores one or more page tables 302 and page directories 310. The one or more page tables 302 include mappings between multiple virtual addresses and multiple physical addresses of the host-side DRAM 228B and the storage system 200. Each page table 302 includes multiple page table entries, and each page table entry points to a storage page 210 (…). Figure 2The page directory 310 differs from the page table 302 and includes multiple directory entries, each pointing to a corresponding entry in one or more page tables 302. In response to a detected TLB miss, the host device 220 uses a first subset of the bits of the target virtual address 322 (e.g., bits 31:22) to index the page directory 310 to identify the target page table 302. A second subset of the bits of the target virtual address 322 (e.g., bits 21:0) is used to index the target page table 302 in page table lookup 312 to identify the page table entry 320 that includes the target physical address 324 of the target storage page 210T storing the target data.

[0037] In response to storage access request 308, host device 220 retrieves target data from target storage page 210T based on target physical address 324. Additionally, the target virtual address 322 and target physical address 324 of the target storage page 210T storing the target data are stored in TLB 306 as recently accessed addresses to accelerate subsequent storage access requests 308.

[0038] Figure 4 This is a block diagram of another example of an electronic system 300 for handling TLB misses and identifying unmapped pages in storage system 200 according to some embodiments. Electronic system 300 includes a host device 220 and a storage system 200 coupled to the host device 220. The host device 220 includes a processor core 304, and the processor core 304 further includes a TLB 306 storing the latest translation from virtual address to physical address in host-side DRAM 228B or in storage system 200. In some scenarios, the host device issues a storage access request 308 for target data stored in electronic system 300, and the storage access request 308 includes a target virtual address 322 of the target data stored on storage page 210T. The host device 220 determines that the target virtual address 322 has not been accessed recently and is therefore not found in TLB 306. The host device 220 has a TLB miss in response to storage access request 308. In response to a TLB miss, host device 220 continues to examine page directory 310 and one or more page tables 302 stored in host-side DRAM 228B. For example, host device 220 uses a first subset of bits (e.g., bits 31:22) to index page directory 310 to identify target page table 302. A second subset of bits of target virtual address 322 (e.g., bits 21:0) is used to compile an index to target page table 302 via page table lookup 312. Host device 220 finds a page table entry that identifies the physical page address, and the page table entry contains metadata flags indicating that the physical page is in an unmapped state. In other words, target memory page 210T is an unmapped page in host-side DRAM 228B.

[0039] Based on the determination that the target memory page 210T is an unmapped page in DRAM 228B, a page fault exception and hardware context exchange 402 invokes the page fault exception handler 404 of the host device 220. The page fault exception handler 404 forwards the memory access request 308 to the memory system 200 coupled to the host device 220. In response to the memory access request 308, the memory system 200 identifies the target page 210T from the page table entries using a set unmapped flag and provides the target page 210T to the page fault exception handler of the host device 220. The target page 210T is further stored in the host-side DRAM 228B through a paging shift-in direct memory access operation. The page directory 310 and page table 302 are updated to include the mapping between the target virtual address 322 and the target physical address 324 of the target page 210T. In addition, the target virtual address 322 and target physical address 324 of the target storage page 210T storing the target data are also stored in TLB 306 as recently accessed addresses to accelerate subsequent storage access requests 308 for the target data.

[0040] Figure 5 This is a block diagram of an electronic system 300 for handling an example of a TLB miss in page table 502 of memory-side DRAM 228A located in storage system 200, according to some embodiments. Electronic system 300 includes a host device 220 and a storage system 200 coupled to the host device 220. Host device 220 includes a processor core 304, and processor core 304 further includes a TLB 306 that stores the latest translation from virtual address to physical address in host-side DRAM 228B or storage system 200. Storage system 200 further includes a storage controller 202, memory-side DRAM 228A, and one or more memories of different types. In some embodiments, storage system 200 includes only one of SLC memory 504 and X-level cell (XLC) memory, where X is greater than 1. Examples of XLC memory include MLC memory, TLC memory, QLC memory, and PLC. In some embodiments, XLC memory includes six or more levels. In some embodiments, memory system 200 includes SLC memory 504 and QLC memory 506. In some embodiments, the storage system 200 further includes a high-bandwidth memory (HBM) 508 having a high-speed computer memory interface for 3D stacked synchronous dynamic random access memory (SDRAM). One or more page tables 502 are stored in the memory-side DRAM 228A and are configured to provide, for example, a mapping between virtual and physical addresses in the event of a TLB miss. One or more page tables 302 include the memory-side DRAM 228A and one or more memory channels 204 in the storage system 200. Figure 2 The mapping between multiple virtual addresses and multiple physical addresses.

[0041] Host device 220 issues a storage access request 308 for target data stored in electronic system 300, and the storage access request 308 includes a target virtual address 322 of the target data. For example, the target data is stored on storage page 210T, and the target virtual address 322 corresponds to the storage page 210T storing the target data. Host device 220 determines that the target virtual address 322 has not been accessed recently and is therefore not found in TLB 306. Host device 220 has a TLB miss in response to storage access request 308. In some embodiments, host-side DRAM 228B further stores one or more page tables 302 and page directories 310. Page directory 310 is coupled to both page table 302 and page table 502. Each page table 302 or 502 includes multiple page table entries, and each page table entry points to storage page 210T (…). Figure 2 The physical address of the page table 210 is specified. Each page table entry in the corresponding page table 502 points to the physical address of the memory page 210 stored in the memory-side DRAM 228A or other memory (e.g., SLC 504, QLC 506, HBM 508) in the memory system 200. The page directory 310 is different from page tables 302 and 502 and includes multiple directory entries, each of which points to a corresponding page table in page tables 302 and 502.

[0042] In response to a detected TLB miss, host device 220 uses a first subset of the bits of the target virtual address 322 (e.g., bits 31:22) to index the page directory 310 to identify the target page table 302. Host device 220 further uses a second subset of the bits of the target virtual address 322 (e.g., bits 21:0) to index the target page table 502 in page table lookup 512 to identify a page table entry 520 that includes the target physical address 324 of the target storage page 210T storing the target data. The target physical address 324 of the target storage page 210T is returned to host device 220. In response to storage access request 308, host device 220 retrieves the target data from the target storage page 210T based on the target physical address 324. Additionally, the target virtual address 322 and the target physical address 324 of the target storage page 210T storing the target data are stored in TLB 306 as recently accessed addresses to facilitate the upcoming storage access request 308.

[0043] In some embodiments, asynchronous page migration operation 514 is implemented to move memory pages between different types of memory in storage system 200, independently of searching for a target physical address 324 in page table 502 stored in memory-side DRAM 228A that maps to a target virtual address 322 of the target data. Storage system 200 identifies hot memory blocks in memory 504, 506, or 508 of storage system 200, for example, those that are recently accessed or frequently accessed. Electronic system 300 further identifies victim memory blocks at updated physical addresses in memory-side DRAM 228A. The victim memory block may optionally be a least recently used or least frequently used memory block in memory-side DRAM 228A. In some embodiments, a memory heatmap is generated for memory-side DRAM 228A to identify victim memory blocks in memory-side DRAM 228A. The victim memory block is migrated to memory 504, 506, or 508 of storage system 200, and the hot memory block is migrated to an updated physical address in memory-side DRAM 228A. Page table 502 of memory-side DRAM 228A is updated to include the mapping between virtual addresses and updated physical addresses in memory-side DRAM 228A.

[0044] Alternatively, in some embodiments, an asynchronous page migration operation 516 is implemented to move memory pages between memories in storage system 200 (e.g., SLC memory 504, XLC memory 506, other memories 508) to host-side DRAM memory 228B, independently of searching for target virtual address 322 for target data in page table 502 stored in memory-side DRAM 228A. Electronic system 300 identifies hot storage blocks in storage system 200, such as those recently accessed, frequently accessed, or speculatively pre-fetched. Electronic system 300 further identifies victim storage blocks at updated physical addresses in host-side DRAM 228B. A victim storage block may optionally be a least recently used or least frequently used storage block in host-side DRAM 228B. In some embodiments, a memory heatmap is generated for host-side DRAM 228B to identify victim storage blocks in host-side DRAM 228B. The victim storage block is migrated to storage system 200, and the hot storage block is migrated to an updated physical address in host-side DRAM 228B. The page table of host-side DRAM 228B is updated to include the mapping between virtual addresses in host-side DRAM and updated physical addresses.

[0045] Figure 6This is a block diagram of another example of an electronic system 300 for handling TLB misses and cold page migrations in a storage system 200 according to some embodiments. The electronic system 300 includes a host device 220 and a storage system 200 coupled to the host device 220. The host device 220 includes a TLB 306 that stores the latest virtual-to-physical address translation in host-side DRAM 228B or storage system 200. The storage system 200 further includes a storage controller 202, memory-side DRAM 228A, and one or more memories of different types. In some embodiments, the storage system 200 includes only one of SLC memory 504 and XLC memory, where X is greater than 1. Examples of XLC memory include MLC memory, TLC memory, QLC memory, PLC memory, and higher-level cell memory. In some embodiments, the storage system 200 further includes high-bandwidth memory (HBM) 508. One or more page tables 502 are stored in the memory-side DRAM 228A and configured to provide, for example, a mapping between virtual and physical addresses in the event of a TLB miss. One or more page tables 302 include mappings between multiple virtual addresses and multiple physical addresses of memory-side DRAM 228A and one or more memory channels 204 in memory system 200.

[0046] In some embodiments, the storage system 200 receives a storage access request 308 for target data stored in the storage system 200. The storage access request 308 includes a target virtual address 322 of the target data. In response to the storage access request 308, the storage system 200 searches in the page table 502 of the memory-side DRAM 228A for a target physical address 324 mapped to the target virtual address 322 of the target data. Based on the search result, the storage system 200 retrieves the target data stored in the memory-side DRAM 228A and one or more storage channels 204 according to the target physical address 324 mapped to the target virtual address 322 in the page table 502. In some embodiments, in response to the storage access request 308 and before searching the page table 502, the electronic system 300 searches in the page table 302 of the host-side DRAM 228B for the target virtual address 322 of the target data.

[0047] In some scenarios, the host device issues a storage access request 308 for target data stored in electronic system 300, and the storage access request 308 includes a target virtual address 322 of the target data stored on storage page 210T. Host device 220 determines that the target virtual address 322 has not been accessed recently and is therefore not found in TLB 306. Host device 220 has a TLB miss in response to storage access request 308. In response to the TLB miss, host device 220 continues to check page directory 310 and page table 502 stored in memory-side DRAM 228A. Page directory 310 is a higher level of page table 502. For example, host device 220 uses a first subset of bits (e.g., bits 31:22) to index page directory 310 to identify target page table 302. A second subset of bits of target virtual address 322 (e.g., bits 21:0) is used to compile an index to target page table 502 via page table lookup 512. In some embodiments, the host-side DRAM 228B further includes a target directory entry indicating that a page table 502 mapping a target physical address 324 to a target virtual address 322 is located in the storage system 200, and the page table 502 of the memory-side DRAM 228A is searched in response to identifying the target directory entry in the page directory 310 of the host-side DRAM 228B. In some embodiments, the page directory 310 is a part of the page table 302 and has a higher hierarchy than the other page tables 302.

[0048] In some embodiments, page table 502 includes page table entries 520 indicating the target physical address 324 of the target storage page 210T in the DRAM memory 228A. In other words, the search results in page table 502 include mapped page table entries, and based on the content of the page table entries, the storage system 200 identifies the target physical address 324 corresponding to the target virtual address 322 in the memory-side DRAM 228A based on page table 502, and retrieves the target data stored in the memory-side DRAM 228A based on the target physical address 324. Alternatively, in some embodiments ( Figure 6Page table 502 includes page table entries 520 indicating the target physical address 324 of the target memory page 210T in a memory different from the memory-side DRAM 228A (e.g., SLC 504, XLC memory 506). Host device 220 identifies the target physical address 324 of the target memory page 210T and identifies the target data as a cold page by checking the page table entry flags. In other words, the search results include page tables with flags indicating that the physical page is not mapped, and storage system 200 identifies the target physical address 324 corresponding to the target virtual address 322 in one or more storage channels 204 of memory 504, 506, or 508, and retrieves the target data stored in one or more storage channels 204 based on the target physical address 324. Furthermore, in some embodiments, one or more storage channels 204 include one or more of SLC memory 504 and XLC memory, where X is greater than 1. Examples of XLC memory include MLC memory, TLC memory, QLC memory, PLC memory, and higher-level cell memory.

[0049] A page fault occurs based on the determination that the target memory page 210T corresponds to a cold page outside the DRAM 228B, thereby invoking the page fault handler 404 of the host device 220. In a page swap-in migration operation 602, the storage system 200 provides the target page 210T, stored both outside the DRAM 228B and within the storage system 200, to the page fault handler 404 of the host device 220. The target page 210T is also stored in the host-side DRAM 228B via the page swap-in storage operation 602. The page table 502 is updated to include the mapping between the target virtual address 322 and the target physical address 324 of the target page 210T. Furthermore, the target virtual address 322 and the target physical address 324 of the target memory page 210T storing the target data are also stored in the TLB 306 as recently accessed addresses to accelerate subsequent storage access requests 308 for the target data.

[0050] In some embodiments not shown, the memory-side DRAM 228A further includes a page directory, which is a higher level of the page tables. The host device 220 checks the page directory to identify page table 502. After searching page table 502, the target physical address 324 corresponding to the target virtual address 322 is identified in page table entry 520 of page table 502. Optionally, the host device 220 checks the page directory 310 in the host-side DRAM 228B before checking the page directory of the memory-side DRAM 228A. Optionally, the host device 220 directly checks the page directory of the memory-side DRAM 228A in response to a TLB miss.

[0051] refer to Figure 6In some embodiments, the page swapping-in memory operation 602 is associated with and synchronized with page table 512 in page table 502. One or more memory channels 204 of the memory system 200 include at least: a first memory (e.g., DRAM 228A), where each memory cell has a first latency; and a second memory (e.g., SLC memory 504, XLC memory 506), where each memory cell has a second latency, and the second latency is greater than the first latency. The memory system 200 determines that the second memory includes a target physical address 324 and is not mapped for target data. Based on the determination that the second memory is not mapped, the memory system 200 moves the target data from the target physical address 324 in the second memory to an updated physical address in the first memory, and updates page table 502 to store the mapping between the target virtual address 322 of the target data and the updated physical address in the first memory. Each virtual address in page table 502 points to a memory page 210 in one of the different memory types (e.g., DRAM 228A, SLC memory 504, XLC memory 506) coupled to the memory controller 202. In some embodiments, for the slower memory (e.g., SLC, XLC) in the first and second memories, page table entries 520 remain in an unmapped state corresponding to a flag, and a pointer to the target physical address 324 storing the target data exists in page table entries 520. Because page table entries 520 are unmapped, a page fault occurs in memory access in memory system 200, allowing page fault software to move the target data from the target physical address 324 in the higher-latency second memory to a lower-latency storage location (e.g., DRAM 228A). Page table 502 is updated to reflect that the target virtual address 322 of the storage page 210 containing the target data is mapped to the lower-latency storage location.

[0052] Furthermore, in some embodiments, the first memory includes a memory-side DRAM 228A and is full. The memory system 200 selects a victim memory block stored in the memory-side DRAM 228A at an updated target physical address 324. The memory system 200 moves the victim memory block to one or more memory channels 204 and stores the target memory block, not the victim memory block, corresponding to the target physical address 324 in the memory-side DRAM 228A. The page table 502 of the memory-side DRAM 228A is updated to include a mapping between the target virtual address 322 and the updated target physical address 324 in the memory-side DRAM 228A. In some embodiments, the victim memory block is the least recently used memory block or the least frequently accessed memory block. A memory heatmap is generated for the memory-side DRAM 228A to identify the victim memory block in the memory-side DRAM 228A.

[0053] In some embodiments, the host-side DRAM 228B uses one of a dual data rate (DDR) data interface, a graphics DDR (GDDR) data interface, and a low-power DDR (LPDDR) data interface. In some embodiments, the memory-side DRAM 228A uses one of a DDR data interface, a GDDR data interface, and an LPDDR data interface. Furthermore, in some embodiments, the host-side DRAM 228B and the memory-side DRAM 228A use the same type of data interface. Conversely, in some embodiments, the host-side DRAM 228B and the memory-side DRAM 228A use different types of data interfaces.

[0054] Figure 7 This is a flowchart of an example method 700 for accessing data in a storage system 200 according to some embodiments. The electronic system 300 includes a host device 220 and a storage system 200 coupled to the host device 220. The storage system 200 receives (operation 702) a storage access request 308 for target data stored in the storage system 200. The storage access request 308 includes (operation 704) a target virtual address 322 of the target data. The storage system 200 includes (operation 706) a memory-side DRAM 228A and one or more storage channels 204 (…). Figure 2 In response to storage access request 308, storage system 200 accesses DRAM 228A on the memory side (…). Figure 5 The page table 502 is searched (operation 708) for the target physical address 324 mapped to the target virtual address 322 of the target data. The page table 502 includes (operation 710) mappings between multiple virtual addresses and multiple physical addresses. Based on the search results, the storage system 200 retrieves (operation 712) the target data stored in the memory-side DRAM 228A and one or more storage channels 204 according to the target physical address 324 mapped to the target virtual address 322 in the page table 502. In some embodiments, after searching the page table 502, the page table 502 ( Figure 5 The target physical address 324 corresponding to the target virtual address 322 is identified in page table entry 520 of the target virtual address 322.

[0055] In some embodiments, the search results include a page table hit, which simply means that the page table entry of the target physical page is in a mapped state. The storage system 200 extracts the target data by: based on the page table hit, identifying the target physical address 324 corresponding to the target virtual address 322 in the memory-side DRAM 228A based on the page table 502, and extracting the target data stored in the memory-side DRAM 228A based on the target physical address 324.

[0056] In some embodiments, the search results include page table misses, which simply means that the page table entry for the target physical address is in an unmapped state. The storage device retrieves the target data by: identifying the target physical address 324 corresponding to the target virtual address 322 in one or more storage channels 204 based on the page table miss, and retrieving the target data stored in one or more storage channels 204 based on the target physical address 324. Furthermore, in some embodiments, the one or more storage channels 204 include one or more of the following: single-level cell (SLC) memory, multi-level cell (MLC) memory, three-level cell (TLC) memory, four-level cell (QLC) memory, five-level cell (PLC) memory, and higher-level cell memory, and identifying the target physical address 324 of the target data in one or more storage channels 204.

[0057] In some embodiments, one or more storage channels 204 include at least: a first memory (e.g., DRAM 228A), wherein each storage cell has a first latency; and a second memory (e.g., QLC memory 506), wherein each storage cell has a second latency. The second latency is greater than the first latency. The storage device retrieves target data by: determining that the second memory includes a target physical address 324 and is unmapped for the target data; moving the target data from the target physical address 324 in the second memory to an updated physical address in the first memory based on the determination that the second memory is unmapped in the first memory; and updating page table 502 to store the mapping between the target virtual address 322 of the target data and the updated physical address in the first memory.

[0058] In synchronous page swapping migration operation 602 ( Figure 6 In some associated embodiments, the storage device selects a victim memory block stored in the memory-side DRAM 228A at an updated target physical address 324, moves the victim memory block to one or more storage channels 204, stores a target memory block corresponding to the target physical address 324 but not the victim memory block in the memory-side DRAM 228A, and updates the page table 502 of the memory-side DRAM 228A to include the mapping between the target virtual address 322 and the updated target physical address 324 in the memory-side DRAM 228A.

[0059] In some embodiments, page table 502 of memory-side DRAM 228A includes supplementary page table 502. Storage system 200 is coupled to host device 220 having host-side DRAM 228B, which is different from memory-side DRAM 228A. Host-side DRAM 228B has host page table 302. Furthermore, in some embodiments, in response to storage access request 308 and before searching supplementary page table 502, host device 220 searches in host page table 302 stored in host-side DRAM 228B for a target physical address 324 mapped to a target virtual address 322 of the target data. Host page table 302 includes mappings between multiple virtual addresses and multiple physical addresses of host-side DRAM 228B. Supplementary page table 502 is searched based on page table misses in host page table 302.

[0060] In some embodiments, the host-side DRAM 228B further includes a page directory 310, which is a higher level of page table 502. The page directory 310 further includes a target directory entry indicating that the page table 502 mapping target physical address 324 to target virtual address 322 is located in the storage system 200. In response to identifying the target directory entry in the page directory 310 of the host-side DRAM 228B, a search is performed on the supplementary page table 502 of the memory-side DRAM 228A.

[0061] In addition, in some embodiments, the host-side DRAM 228B uses a first dual data rate (DDR) interface, and the memory-side DRAM 228A uses a second DDR interface different from the first DDR interface.

[0062] In some embodiments, storage system 200 is coupled to host device 220 having a TLB 306 that stores multiple latest translations from virtual address to physical address. After retrieving target data, host device 220 stores a first latest translation between the multiple latest translations of storage system 200 from target virtual address 322 to target physical address 324 in TLB 306.

[0063] In the asynchronous page migration operation 514 ( Figure 5 In some associated embodiments, the storage system 200 identifies hot storage blocks in one or more storage channels 204, migrates the hot storage blocks to an updated physical address in the memory-side DRAM 228A, and updates the page table 502 of the memory-side DRAM 228A to include the mapping between virtual addresses and updated physical addresses in the memory-side DRAM 228A.

[0064] In conjunction with asynchronous page migration operation 516 ( Figure 5 and 6In some associated embodiments, the storage system 200 identifies a hot storage block in the storage system 200, identifies a victim storage block at an updated physical address in a host-side DRAM 228B that is different from the storage system 200, migrates the victim storage block to the storage system 200, migrates the hot storage block to the updated physical address in the host-side DRAM 228B, and updates the page table 302 of the host-side DRAM 228B to include the mapping between virtual addresses and updated physical addresses in the host-side DRAM 228B.

[0065] In some embodiments, the storage system 200 is coupled to the host device 220 via a Computer High-Speed ​​Link (CXL) that operates in accordance with the High-Speed ​​Computer Interconnect Interface Standard.

[0066] In another aspect of this application, the storage system 200 includes a storage controller 202, a memory-side DRAM 228A, and one or more storage channels 204. The storage controller is configured to receive a storage access request 308 for target data stored in the storage system 200. The storage access request 308 further includes a target virtual address 322 of the target data. The memory-side DRAM 228A is coupled to the storage controller and includes a page table 502. The page table 502 includes a mapping between a plurality of virtual addresses and a plurality of physical addresses, and the memory-side DRAM 228A is configured to search in the page table 502 stored in the memory-side DRAM 228A for a target physical address 324 mapped to the target virtual address 322 of the target data in response to the storage access request 308. One or more storage channels 204 are coupled to the storage controller, and the memory-side DRAM 228A and the one or more storage channels 204 are configured to provide the target data based on the search results.

[0067] In another aspect of this application, an electronic system includes a host device 220 comprising one or more processors and a storage system 200. The storage system 200 is configured to receive a storage access request 308 for target data stored in the storage system 200. The storage access request 308 includes a target virtual address 322 of the target data, and the storage system 200 includes a memory-side DRAM 228A and one or more memory channels 204. The storage system 200 is further configured to search for the target virtual address 322 of the target data in a page table 502 of the memory-side DRAM 228A in response to the storage access request 308. The page table 502 includes a mapping between multiple virtual addresses and multiple physical addresses. The storage system 200 is also configured to retrieve the target data stored in one of the memory-side DRAM 228A and one or more memory channels 204 based on the search result and according to a target physical address 324 mapped to the target virtual address 322 in the page table 502.

[0068] In some embodiments, method 700 requires kernel support for page table locations and enhancements to the kernel page fault handler. Some standardized assessments of the storage system's capabilities implement these functionalities in software (this will be easily detectable as it is coordinated via the CXL bus in software).

[0069] The memory also stores instructions and data associated with method 700, and includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state storage devices; and optionally includes non-volatile memory, such as one or more disk storage devices, one or more optical disk storage devices, one or more flash memory devices, or one or more other non-volatile solid-state storage devices. The memory may optionally include one or more storage devices located remotely to one or more processing units. The memory, or alternatively, non-volatile memory within the memory, includes a non-transitory computer-readable storage medium. In some embodiments, the memory or the non-transitory computer-readable storage medium of the memory stores programs, modules, and data structures, subsets, or supersets for implementing method 600.

[0070] Each of the elements identified above may be stored in one or more of the previously mentioned storage devices and corresponds to a set of instructions for performing the functions described above. The modules or programs identified above (e.g., instruction sets) need not be implemented as separate software programs, procedures, modules, or data structures, and therefore, in various embodiments, various subsets of these modules may be combined or otherwise rearranged. In some embodiments, the memory may optionally store a subset of the modules and data structures identified above. Furthermore, the memory may optionally store additional modules and data structures not described above.

[0071] The terminology used in the descriptions of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the descriptions of the various embodiments described and the appended claims, the singular forms “a” and “described” are intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. It should be further understood that the terms “comprising” and / or “including” as used in this specification specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Furthermore, it should be understood that although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another.

[0072] As used herein, depending on the context, the term "if" may optionally be interpreted as meaning "when," "after," "in response to determination," "in response to detection," or "according to determination." Similarly, depending on the context, the phrase "if determination" or "if [the stated condition or event] is detected" may optionally be interpreted as meaning "after determination," "in response to determination," "after detection of [the stated condition or event]," "in response to detection of [the stated condition or event]," or "according to determination of detection of [the stated condition or event]."

[0073] For illustrative purposes, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise form disclosed. Many modifications and variations are possible in light of the foregoing teachings. The embodiments were chosen and described to best explain the operating principles and practical applications, thereby enabling those skilled in the art to understand.

[0074] Although the various figures illustrate several logical stages in a specific order, stages independent of this order can be reordered and other stages can be combined or decomposed. While some reorderings or other groupings are specifically mentioned, other reorderings or groupings will be apparent to those skilled in the art, and therefore the orderings and groupings presented herein are not an exhaustive list of alternatives. Furthermore, it should be understood that the stages can be implemented in hardware, firmware, software, or any combination thereof.

Claims

1. A method for accessing data in a storage system, the method comprising: Receive a storage access request for target data stored in the storage system, wherein the storage access request includes a target virtual address of the target data, and the storage system includes memory-side dynamic random access memory (DRAM) and one or more storage channels; and In response to the storage access request: The target physical address of the target virtual address mapped to the target data is searched in the page table of the memory-side DRAM, wherein the page table includes mappings between multiple virtual addresses and multiple physical addresses; as well as Based on the search results, the target data stored in the memory-side DRAM and one of the one or more memory channels is extracted according to the target physical address in the page table.

2. The method according to claim 1, wherein the search results include page table hits, and extracting the target data further includes, based on the page table hits: The target physical address corresponding to the target virtual address in the memory-side DRAM is identified based on the page table; and The target data stored in the memory-side DRAM is extracted based on the target physical address.

3. The method according to claim 1, wherein the search results include page table misses, and extracting the target data further includes, based on the page table misses: Identify the target physical address corresponding to the target virtual address in the one or more storage channels; and The target data stored in the one or more storage channels is extracted based on the target physical address.

4. The method of claim 3, wherein the one or more storage channels include one or more of the following: single-level cell (SLC) memory, multi-level cell (MLC) memory, three-level cell (TLC) memory, four-level cell (QLC) memory, five-level cell (PLC) memory, and higher-level cell memory, and the target physical address of the target data is identified in the one or more storage channels.

5. The method of claim 3, wherein the one or more storage channels comprise at least: A first memory, wherein each memory cell has a first delay; And a second memory, wherein each memory cell has a second delay, the second delay being greater than the first delay, and extracting the target data further includes: It is determined that the second memory contains the target physical address and is unmapped for the target data; as well as Based on the determination by the second memory that the first memory is not mapped, the target data is moved from the target physical address in the second memory to the updated physical address in the first memory, and the page table is updated to store the mapping between the target virtual address and the updated physical address of the target data in the first memory.

6. The method of claim 3, further comprising: Select the victim memory block stored in the updated target physical address in the memory-side DRAM; Copy the victim's storage block to the one or more storage channels; The target memory block corresponding to the target physical address, instead of the victim memory block, is stored in the memory-side DRAM; as well as The page table of the memory-side DRAM is updated to include the mapping between the target virtual address and the updated target physical address in the memory-side DRAM.

7. The method of claim 1, wherein after searching the page table, the target physical address corresponding to the target virtual address is identified in a page table entry of the page table.

8. The method according to claim 1, wherein: The page table of the memory-side DRAM includes a supplementary page table; The storage system is coupled to a host device having host-side DRAM, and the host-side DRAM is different from the memory-side DRAM; and The host-side DRAM has a host page table.

9. The method of claim 8, further comprising, in response to the storage access request and before searching the supplementary page table: The host page table stored in the host-side DRAM is used to search for the target physical address of the target virtual address mapped to the target data, wherein the host page table includes a mapping between multiple virtual addresses and multiple physical addresses in the host-side DRAM; The supplementary page table is searched based on page table misses in the host page table.

10. The method according to claim 8, wherein: The host-side DRAM further includes a higher-level page directory for the page table; The page directory further includes a target directory entry, the target directory entry indicating that the page table mapping the target physical address to the target virtual address is located in the storage system; and In response to identifying the target directory entry in the page directory of the host-side DRAM, the supplementary page table of the memory-side DRAM is searched.

11. The method of claim 8, wherein the host-side DRAM uses a first dual data rate (DDR) interface, and the memory-side DRAM uses a second DDR interface different from the first DDR interface.

12. The method of claim 1, wherein the storage system is coupled to a host device having a translation back buffer (TLB) that stores multiple latest translations from virtual addresses to physical addresses, and the host device is configured to: After extracting the target data, the first latest translation from the target virtual address to the target physical address in the storage system between the plurality of latest translations is stored in the TLB.

13. The method of claim 1, further comprising: Identify hot storage blocks in the one or more storage channels; The hot storage block is migrated to the updated physical address of the memory-side DRAM; as well as The page table of the memory-side DRAM is updated to include the mapping between virtual addresses in the memory-side DRAM and the updated physical addresses.

14. The method of claim 1, wherein the search results include page table hits, and extracting the target data further includes, based on the page table hits: The target physical address corresponding to the target virtual address in the memory-side DRAM is identified based on the page table; and The target data stored in the memory-side DRAM is extracted based on the target physical address.

15. The method of claim 1, wherein the search results include page table misses, and extracting the target data further includes, based on the page table misses: Identify the target physical address corresponding to the target virtual address in the one or more storage channels; and The target data stored in the one or more storage channels is extracted based on the target physical address.

16. The method of claim 1, further comprising: Identify the hot storage blocks in the storage system; Identify the victim memory block at the updated physical address in a host-side DRAM that is different from the storage system; Migrate the victim's storage block to the storage system; The hot storage block is migrated to the updated physical address of the host-side DRAM; as well as Update the page table of the host-side DRAM to include the mapping between the virtual addresses in the host-side DRAM and the updated physical addresses.

17. The method of claim 1, wherein the storage system is coupled to the host device via a Computer High-Speed ​​Link (CXL) operating in accordance with the High-Speed ​​Computer Interconnect Interface Standard.

18. A storage system comprising: Storage controller; DRAM on the memory side, which is coupled to the memory controller; as well as One or more storage channels coupled to the storage controller, wherein the storage system is configured to perform the method according to any one of claims 1 to 17.

19. An electronic system comprising: Host device, which includes one or more processors; as well as A storage system configured to perform the method according to any one of claims 1 to 17.

20. A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors, the one or more programs further comprising instructions for performing the method according to any one of claims 1 to 17.

Images