Page request interface support when handling pointer extraction translates data with cache host memory addresses

Abstract

Examples of the invention relate to page request interface support when handling pointer extraction translates data with cache host memory addresses. A method performed by pointer fetch circuitry includes buffering, in a pointer buffer of host interface circuitry, a pointer associated with a chopping command of a logical block address read command residing in a host system commit queue. The method includes sending an address translation request for a respective translation unit of a respective chopping command to an address translation circuit, each translation unit including a subset of pointers. The method includes detecting, at a cache of an address translation circuit, an address translation request miss of a translation unit for a chopping command. The method includes sending a translation miss message to a page request interface (PRI) handler. The translation miss message contains a virtual address of the translation unit and a restart point of the chopping command, and the translation miss message is used to trigger the PRI handler to send the page miss request to a translation agent of the host system.

Description

Technical Field

[0001] The present disclosure relates generally to a memory system, and more particularly to handling a host submission queue associated with caching host memory address translation data in a memory subsystem and page request interface support in completing automation. Background Art

[0002] The memory subsystem may include one or more memory components that store data. The memory components may be, for example, nonvolatile memory components and volatile memory components. In general, the host system may utilize the memory subsystem to store data at the memory components and retrieve data from the memory components. Summary of the invention

[0003] In an example, a memory subsystem controller is described. The memory subsystem controller includes a host interface circuit system for interacting with a host system. The host interface circuit system includes: a pointer extraction interface circuit, which is coupled to a pointer buffer; and an address translation circuit, which is coupled to the pointer extraction interface circuit. The address translation circuit is used to handle address translation requests for the host system and includes a cache for storing address translations. The pointer extraction interface circuit is used to: buffer multiple pointers associated with memory commands in the pointer buffer; send address translation requests of corresponding translation units for the memory commands to the address translation circuit; when the address translation request of the translation unit for the memory command misses at the cache, trigger a page request interface (PRI) handler to send a page miss request to a translation agent of the host system; and discard the multiple pointers from the pointer buffer. The memory command is one of a write command or a non-logical block address (non-LBA) read command. Each translation unit includes a subset of the multiple pointers. The page miss request includes a virtual address of the translation unit.

[0004] In an example, a memory subsystem controller is described. The memory subsystem controller includes a host interface circuit system for interacting with a host system. The host interface circuit system includes: a pointer extraction interface circuit, which is coupled to a pointer buffer; and an address translation circuit, which is coupled to the pointer extraction interface circuit. The address translation circuit is used to handle address translation requests for the host system and includes a cache for storing address translations. The pointer extraction interface circuit is used to: buffer in the pointer buffer a plurality of pointers associated with a plurality of chopping commands in a logical block address (LBA) read command residing in a submission queue of the host system; send address translation requests for corresponding translation units of corresponding chopping commands to the address translation circuit; detect at the cache a miss of the address translation request for the translation unit of a chopping command in the plurality of chopping commands; and cause a translation miss message to be sent to a page request interface (PRI) handler. Each translation unit includes a subset of the plurality of pointers. The translation miss message contains a virtual address of the translation unit and a restart point of the chop command, and the translation miss message is used to trigger the PRI handler to send a page miss request to a translation agent of the host system.

[0005] In an example, a method is described. The method includes: buffering a plurality of pointers associated with a plurality of chop commands of a logical block address (LBA) read command residing in a submission queue of a host system in a pointer buffer of a host interface circuit system of a processing device; sending an address translation request of a corresponding translation unit for a corresponding chop command to an address translation circuit; detecting an address translation request miss of a translation unit for a chop command in the plurality of chop commands at a cache of the address translation circuit; and sending a translation miss message to a page request interface (PRI) handler. Each translation unit includes a subset of the plurality of pointers. The translation miss message contains a virtual address of the translation unit and a restart point of the chop command, and the translation miss message is used to trigger the PRI handler to send a page miss request to a translation agent of the host system. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The present disclosure will be more fully understood from the detailed description given below and from the accompanying drawings of various embodiments of the present disclosure.

[0007] Figure 1 An example computing environment is described for page request interface support when caching host memory address translation data in a memory subsystem according to some embodiments of the present disclosure.

[0008] Figure 2is a schematic block diagram of a system (or device) implementing Peripheral Component Interface Express (PCIe) and Non-Volatile Memory Express (NVMe) functionality according to some embodiments, and a disclosed page request interface supports operation within the system (or device).

[0009] Figure 3 is a schematic diagram illustrating an embodiment of a controller memory buffer (CMB) manager for a storage device.

[0010] Figure 4 is an example physical host interface between a host system and a memory subsystem that implements caching host memory address translation data according to some embodiments.

[0011] Figure 5 A system is provided in which a memory subsystem controller contains an address translation cache (ATC) according to some embodiments.

[0012] Figure 6 A memory subsystem for page request interface support when caching host memory address translation data for multiple host interface circuits according to some embodiments.

[0013] Figure 7 is an example memory subsystem controller including address translation circuitry that implements caching of host memory address translation data in accordance with some embodiments.

[0014] Figure 8 is a flow chart of an example method of caching host memory address translation data in a memory subsystem according to some embodiments.

[0015] Fig. 9 is a block diagram of host interface circuitry (including host queue interface circuitry) for interacting with a page request interface handler according to some embodiments.

[0016] Fig.10 is a schematic block diagram of a system including a processing bridge circuit for implementing page request interface support when caching host memory address translation data in a memory subsystem, according to some embodiments.

[0017] Fig.11 is a flow chart of an example method for page request interface support when caching host memory address translation data in a memory subsystem according to some embodiments.

[0018] Fig.12 is a flow chart of an example method of handling page request interface support in a host submission queue while caching host memory address translation data in a memory subsystem in accordance with some embodiments.

[0019] Fig.13is a schematic block diagram of a system including completion queue interface circuitry for implementing page request interface support when caching host memory address translation data in a memory subsystem in accordance with some embodiments.

[0020] Fig.14 is a flow chart of an example method for handling page request interface support in completion automation when caching host memory address translation data in a memory subsystem in accordance with some embodiments.

[0021] Fig.15 is a block diagram of host interface circuitry including a pointer extraction interface circuit (PFIC) for retrieving pointers to support data transfer operations in accordance with some embodiments.

[0022] Fig.16A is a diagram of an example command (eg, a write command or a non-logical block address (LBA) read command) in response to a translation miss during a pointer fetch, according to some embodiments.

[0023] Fig. 16B is divided into different actions of the chop command and the PFIC (or other host interface circuitry) in response to a translation miss during pointer fetch according to some embodiments. Fig.16A Figure 1 shows an example command for .

[0024] Fig.17 is a flow chart of an example method for handling translation misses during pointer fetches for memory commands, according to some embodiments.

[0025] Fig.18 is a diagram of an example LBA read command that has been divided into a chop command and different actions of the PFIC (or other host interface circuitry) in response to a translation miss during a pointer fetch, in accordance with some embodiments.

[0026] Fig.19 is a flow chart of an example method for handling translation misses during pointer fetch for an LBA read command, according to some embodiments.

[0027] Fig. 20 is a block diagram of an example computer system in which embodiments of the present disclosure may operate. DETAILED DESCRIPTION

[0028] Aspects of the present disclosure relate to page request interface support when caching host memory address translation data in a memory subsystem. The memory subsystem may be a storage device, a memory module, or a hybrid of a storage device and a memory module. Figure 1Examples of storage devices and memory modules are described. In general, a host system may utilize a memory subsystem that includes one or more memory components (hereinafter also referred to as a "memory device"). The host system may provide data to be stored at the memory subsystem and may request data to be retrieved from the memory subsystem.

[0029] When requesting to write data to or read data from a memory device, the host system typically generates a memory command (e.g., an erase (or unmap) command, a write command, or a read command) that is sent to a memory subsystem controller (e.g., a processing device or "controller"). The controller then executes these memory commands to perform the erase (or unmap) operation, write operation, or read operation at the memory device. Because the host operates in logical addresses (which are referred to as virtual addresses (or guest physical addresses) in the context of virtual machines (VMs) running on the host system), the host system includes a root complex that serves as a connection between the physical and virtual components of the host system and a Peripheral Controller Interconnect Express (PCIe) bus. This PCIe root complex can generate transaction requests (to include address translation requests) on behalf of entities of the host system (e.g., a virtual processing device in one of the VMs).

[0030] The host system typically further includes a translation agent (TA) that performs virtual address to physical address translation on behalf of the controller. To this end, the TA is configured to communicate with the translation request / response through the PCIe root complex. In some systems, the TA is also referred to as an input / output memory management unit (IOMMU) executed by a hypervisor or virtual machine manager running on the host system. Thus, the TA can be a hardware component or software (IOMMU) with a dedicated driver.

[0031] The controllers in these systems may be configured to include address translation circuitry, more specifically referred to as an address translation service (ATS), that requests the TA to perform certain address translations from virtual (or logical) addresses of the memory device to available (or assigned) physical addresses. In this manner, the address translation circuitry (or ATS) dynamically determines address translations based on virtual addresses in corresponding memory commands queued within the host memory. The various aspects of the ATS eliminate the need to pin large amounts of memory associated with applications run by the host system.

[0032] Especially when supporting multiple Non-Volatile Memory Express (NVMe) devices, the need to continuously request the TA to perform address translations is a bottleneck and impacts performance in terms of speed, latency, and quality of service when fulfilling memory commands. As the submission, completion, I / O, and management queues located within the host memory grow larger and the media speeds of the memory devices increase, performance may be increasingly impacted. For example, the number of address translation requests and responses for command queues and direct memory access (DMA) addresses may be slowed down by having to move back and forth across the PCIe bus, which also generates additional I / O traffic that slows down the entire memory subsystem.

[0033] Aspects of the present disclosure address the above and other deficiencies by implementing an address translation cache (ATC) within the address translation circuitry of a host interface circuitry within a controller, the ATC storing address translations corresponding to incoming address translation requests from a host interface (HIF) circuit of the host interface circuitry. The ATC can store address translations associated with address translation requests for future access by the host interface circuitry. For example, these address translation requests can be related to the processing of memory commands and the handling of DMA operations or commands. In this way, when a cached address translation matches a subsequent (or later) address translation request from the HIF circuitry (e.g., hits at the cache), the address translation circuitry can retrieve the cached address translation and pass it back to the HIF circuitry without having to request the TA to perform the translation on behalf of the controller.

[0034] In some embodiments, for each memory command within a submission queue of the host memory, the address translation circuitry may store in the ATC a first address translation corresponding to a current page identified by the memory command (referenced in the address translation request) and store in the ATC a second address translation for a subsequent page that sequentially follows the current page according to the virtual address number. This look-ahead buffering of address translations for a predetermined number of submission queues in the ATC greatly reduces the number of misses at the ATC while keeping the size of the ATC reasonable given the expense of cache memory (e.g., static random access memory (SRAM)), availability at the controller. When the command (and other) queues in the host memory are arranged to store memory commands sequentially according to virtual addresses, the hit rate at the cache may be further improved by this approach.

[0035] In most devices or systems, DMA operations cannot handle page faults (e.g., cannot resolve page faults (e.g., via paging)) to successfully execute a given DMA command. To prevent DMA operations from page faults, the host system pins pages in host memory (either the source or destination of the DMA) before the driver submits I / O data to the device. Pinning pages within the host address space of the memory guarantees that DMA operations are executed immediately when triggered by an interrupt. If the I / O device supports a large number of pre-allocated queues and data buffers, a large portion of the host memory is reserved in advance, even if the I / O device does not use all of these pinned memories at the same time. For most applications and workload situations, all of these pinned memories are usually idle.

[0036] Thus, while the address translation circuitry will appear to enable the ability to abandon pinned memory (by accessing the ATC), there are instances of misses at the ATC (or the cache of the ATS) which will create a need for paged memory. However, because DMA operations do not support page faults, the address translation circuitry may still be an incomplete solution for DMA operations. To address this problem, in some embodiments, an additional page interface request (PRI) handler is added to the host interface design which, from the perspective of the memory controller, automatically sends PRI-related page miss requests to the host system to request that the host system re-pin the physical pages of memory to the virtual address of each corresponding page miss request, thereby making these pages available again in the host memory.

[0037] For example, in some embodiments, the PRI handler tracks translation miss messages received from the host interface circuit, each translation miss message including a virtual address of a miss at the cache. The PRI handler may further remove duplicate translation miss messages having the same virtual address, for example due to overlap of virtual addresses from multiple host interface circuits concurrently. The PRI handler may further create page miss requests from non-duplicate translation miss messages that are classified into page request groups. For example, a page request group corresponds to one of the host interface circuits.

[0038] In these embodiments, the PRI handler further queues the group of page requests to be sent to the translation agent of the host system, for example to handle the page miss request and send back an acknowledgement in the form of a page request response. In some embodiments, the page request response triggers the PRI handler to send a restart message back to the host interface circuit and identify which hardware pipeline portion or queue can be restarted because the page is available in memory. In these embodiments, the address translation to the repinned page can be further stored in the ATC for future access by any of the HIF circuits.

[0039] Therefore, the advantages of the systems and methods implemented according to some embodiments of the present disclosure include, but are not limited to, improving the performance of the memory subsystem in terms of speed, latency, and throughput of handling memory commands. Part of the reason for the improved performance is the reduction of I / O traffic on the PCIe bus of the memory subsystem and at the host TA. The disclosed address translation circuit can also reduce the possibility that a previously cached translation will become invalid and must be re-fetched from the TA of the host system. Other advantages of the PRI handler participating in caching host memory address translation data in the memory subsystem include avoiding the need to pin large amounts of host memory to support DMA operations or the like by requesting pages to be re-pinned when needed. Other advantages will be apparent to those skilled in the art of address translation within the memory subsystem, which will be discussed later. The following will be about Figures 1 to 19 Provide additional details on these techniques.

[0040] Figure 1 An example computing environment 100 is illustrated that includes a memory subsystem 110 according to some embodiments of the present disclosure. Memory subsystem 110 may include media such as one or more volatile memory devices (e.g., memory device 140), one or more non-volatile memory devices (e.g., memory device 130), or a combination thereof.

[0041] The memory subsystem 110 may be a storage device, a memory module, or a hybrid of a storage device and a memory module. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage (UFS) drives, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and non-volatile dual in-line memory modules (NVDIMMs).

[0042] The computing environment 100 may include a host system 120 coupled to one or more memory subsystems 110. In some embodiments, the host system 120 is coupled to memory subsystems 110 of different types. Figure 1 An example of a host system 120 coupled to one memory subsystem 110 is illustrated. The host system 120 uses the memory subsystem 110, for example, to write data to the memory subsystem 110 and read data from the memory subsystem 110. As used herein, "coupled to" generally refers to a connection between components, which can be an indirect communication connection or a direct communication connection (e.g., without intervening components), whether wired or wireless, including connections such as electrical, optical, magnetic, etc.

[0043] The host system 120 may be a computing device such as a desktop computer, a laptop computer, a network server, a mobile device, an embedded computer (e.g., a computer included in a vehicle, industrial equipment, or a networked commercial device), or such a computing device including a memory and a processing device. The host system 120 may be coupled to the memory subsystem 110 via a physical host interface. Examples of the physical host interface include, but are not limited to, a Serial Advanced Technology Attachment (SATA) interface, a Compute Express Link (CXL) interface, a Peripheral Component Interconnect Express (PCIe) interface, a Universal Serial Bus (USB) interface, Fibre Channel, Serial Attached SCSI (SAS), etc. The physical host interface may be used to transfer data between the host system 120 and the memory subsystem 110. When the memory subsystem 110 is coupled to the host system 120 through a physical host interface (e.g., a PCIe or CXL bus), the host system 120 may further utilize an NVM Express (NVMe) interface to access memory components (e.g., memory device 130). The physical host interface may provide an interface for passing control, address, data, and other signals between the memory subsystem 110 and the host system 120 .

[0044] The memory devices may include any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 140) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0045] Some examples of non-volatile memory devices (e.g., memory device 130) include NAND-type flash memory and write-in-place memory, such as three-dimensional cross-point ("3D cross-point") memory. A 3D cross-point memory device is a cross-point array of non-volatile memory cells that can perform bit storage based on changes in body resistance in conjunction with a stackable cross-grid data access array. In addition, in contrast to many flash-based memories, cross-point non-volatile memories can perform write-in-place operations, where non-volatile memory cells can be programmed without first erasing the non-volatile memory cells.

[0046] Each of the memory devices 130 may include one or more arrays of memory cells, such as single-level cells (SLC), multi-level cells (MLC), three-level cells (TLC), or four-level cells (QLC). In some embodiments, a particular memory component may include an SLC portion, an MLC portion, a TLC portion, or a QLC portion of memory cells. Each of the memory cells may store one or more data bits used by the host system 120. In addition, the memory cells of the memory devices 130 may be grouped to form pages, which may refer to a unit of a memory component used to store data. For some types of memory, such as NAND, pages may be grouped to form blocks. Some types of memory, such as 3D crosspoint, may group pages across dies and channels to form management units (MUs).

[0047] Although non-volatile memory components such as NAND-type flash memory and 3D crosspoint are described, the memory device 130 may be based on any other type of non-volatile memory such as read-only memory (ROM), phase-change memory (PCM), magnetic random access memory (MRAM), or non-NOR flash memory, electrically erasable programmable read-only memory (EEPROM).

[0048] The memory subsystem controller 115 may communicate with the memory device 130 to perform operations such as reading data, writing data, or erasing data and other such operations at the memory device 130. The memory subsystem controller 115 may include hardware, such as one or more integrated circuits and / or discrete components, buffer memory, or a combination thereof. The hardware may include digital circuitry with dedicated (i.e., hard-coded) logic to perform the operations described herein. The memory subsystem controller 115 may be a microcontroller, dedicated logic circuitry (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), or other suitable processor.

[0049] The memory subsystem controller 115 may include a processor (e.g., a processing device) 117 configured to execute instructions stored in a local memory 119. In the illustrated example, the local memory 119 of the memory subsystem controller 115 includes an embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines that control the operation of the memory subsystem 110, including handling communications between the memory subsystem 110 and the host system 120.

[0050] In some embodiments, local memory 119 may include memory registers that store memory pointers, fetched data, etc. Local memory 119 may also include read-only memory (ROM) for storing microcode. Figure 1The example memory subsystem 110 in FIG. 1 has been described as including a memory subsystem controller 115, but in another embodiment of the present disclosure, the memory subsystem 110 may not include the memory subsystem controller 115, but may rely on external control (e.g., provided by an external host, or provided by a processor or controller separate from the memory subsystem).

[0051] In general, the memory subsystem controller 115 may receive commands or operations from the host system 120 and may convert the commands or operations into instructions or appropriate commands to achieve the desired access to the memory device 130. The memory subsystem controller 115 may be responsible for other operations associated with the memory device 130, such as wear leveling operations, garbage collection operations, error detection and error correction code (ECC) operations, encryption operations, cache operations, and address translation between logical block addresses and physical block addresses. The memory subsystem controller 115 may further include a host interface circuit system for communicating with the host system 120 via a physical host interface. The host interface circuit system may convert commands received from the host system into command instructions for accessing the memory device 130, and convert responses associated with the memory device 130 into information for the host system 120.

[0052] The memory subsystem 110 may also include additional circuitry or components not illustrated. In some embodiments, the memory subsystem 110 may include a cache or buffer (e.g., DRAM) and address circuitry (e.g., row decoders and column decoders) that may receive addresses from the memory subsystem controller 115 and decode the addresses to access the memory device 130.

[0053] In some embodiments, the memory device 130 includes a local media controller 135 that operates in conjunction with the memory subsystem controller 115 to perform operations on one or more memory cells of the memory device 130. In some embodiments, the memory device 130 is a managed memory device, which is a raw memory device combined with a local controller for memory management within the same memory device package, such as the local media controller 135. An example of a managed memory device is a managed NAND (MNAND) device.

[0054] The memory subsystem 110 includes an address translation circuit 113 and an address translation cache (or ATC 116) that may be used to perform caching of host memory address translation data for queues, physical region pages (PRPs) scatter gather lists (SGLs), and data transfers in the memory subsystem 110. For example, the address translation circuit 113 may receive an address translation request from a HIF circuit handling a memory command, requesting that the TA perform a translation of a virtual address located within the memory command, and upon receiving a physical address, store a mapping between the virtual address and the physical address (also referred to as an L2P mapping) in the ATC 116. Upon receiving a subsequent address translation request containing the same virtual address, the address translation circuit 113 may verify that the virtual address hits at the ATC 116 and directly copy the corresponding address translation from the ATC to the pipeline of the address translation circuit 113, which passes the corresponding address translation back to the requesting HIF circuit. Similar caching of address translations may also be performed for DMA operations, which will be discussed in more detail. Further details regarding the operation of address translation circuit 113 and ATC 116 are described below.

[0055] Figure 2 2 is a schematic block diagram of a system 200 (or device) implementing Peripheral Component Interface Express (PCIe) and Non-Volatile Memory Express (NVMe) functionality, with the disclosed page request interface supporting operations within the system 200 (or device) according to some embodiments. In various embodiments, the system 200 includes a host system 220 (e.g., host system 120), a memory subsystem 210 (e.g., memory subsystem 110), which in turn includes a controller 215 (e.g., controller 115), one or more memory devices 130, and DRAM 222. In some embodiments, aspects of the controller 215 (including hardware and / or firmware functionality) are included in the local media controller 135.

[0056] In an embodiment, the host system 220 includes a central processing unit (CPU) 209 connected to a host memory 212 (e.g., DRAM or other main memory). Applications may be stored in the memory space 212 for execution by components of the host system 220. The host system 220 includes a bus 207 (e.g., a memory device interface of the memory device 130), which interacts with a host interface 208 (which may include a media access control (MAC) and a physical layer (PHY) component) for directing communications from the host system 220 to the memory device 130 and exporting communications from the memory device 130 to the host system 220. The bus 207 and the host interface 208 operate under a communication protocol (e.g., a peripheral component interface express (PCIe) serial communication protocol or other suitable communication protocol). Other suitable communication protocols include Ethernet, serial attached SCSI (SAS), serial AT attachment (SATA), any protocol related to remote direct memory access (RDMA) (e.g., Infiniband, iWARP, or RDMA over Ethernet Convergence (RoCE)), and other suitable serial communication protocols.

[0057] The memory device 130 may also be connected to a host system 220 through a switch or bridge. A single host system 220 is shown connected to the memory device 130 and supports the PCI-SIG Single Root I / O Virtualization and Sharing Specification (SR-IOV) single host virtualization protocol as discussed in more detail below, where the memory device 130 may be shared by multiple hosts, where the multiple hosts may be a physical function 211 (PF) and one or more virtual functions 205 (VF) of a single physical host system that is virtualized. In other embodiments, it is contemplated that the SR-IOV standard for virtualizing multiple physical hosts may be implemented using features of the disclosed systems and methods.

[0058] In an embodiment, the non-volatile memory array of memory device 130 (or NVM 206) may be configured to store information long-term as a non-volatile storage space and retain the information after power on / off cycles. Figure 1 In the same way as described, Figure 2 The NVM 206 in may include reference Figure 1 One or more dies of NAND-type flash memory or other memory is discussed.

[0059] The memory subsystem 210 includes a controller 215 (e.g., a processing device) that manages the operation of the memory device 130, such as writing to and reading from the NVM 206. The controller 215 may include one or more processors 217, which may be multi-core processors. The processor 217 may typically handle or interact with the components of the memory device 130 through firmware code.

[0060] The controller 215 may operate under the NVM Express (NVMe) protocol, but other protocols are also suitable. The NVMe protocol is a communication interface / protocol developed for SSDs to operate on a host and memory device linked by a PCIe interface. The NVMe protocol provides a command queue and completion path for accessing data stored in the memory device 130 by the host system 220.

[0061] Controller 215 also includes a controller memory buffer (CMB) manager 202. CMB manager 202 may be connected to DRAM 222, static random access memory (SRAM) 224, and read-only memory (ROM) 226. CMB manager 202 may also communicate with NVM 206 through media interface module 228. DRAM 222 and SRAM 224 are volatile memories or cache buffers used for short-term storage or temporary storage during operation of memory device 130. In some embodiments, SRAM 224 also includes tightly coupled memory. Volatile memory does not retain stored data if power is lost. DRAM typically requires periodic refresh of stored data, while SRAM does not require refresh. Although SRAM typically provides faster data access than DRAM, it may also be more expensive.

[0062] The controller 215 executes computer-readable program code (e.g., software or firmware) executable instructions (referred to herein as "instructions"), which may be executed by various components of the controller 215 (e.g., processor 218, logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, embedded microcontrollers, and other components of the controller 215).

[0063] Instructions that can be executed by the controller 215 for carrying out the embodiments described herein are stored in a non-transitory computer-readable storage medium. In some embodiments, the instructions are stored in a non-transitory computer-readable storage medium of the memory device 130, such as in a read-only memory (ROM) or NVM 206. The instructions stored in the memory device 130 can be executed without input or instructions added from the host system 220. In other embodiments, the instructions are transmitted from the host system 220. The controller 215 is configured with hardware and instructions to perform various functions described herein and shown in the figures.

[0064] The controller 215 may also include other components, such as an NVMe controller 203, a media interface module 228 coupled between the NVMe controller 203 and the memory device 130, and an error correction module 234. In an embodiment, the NVMe controller 203 includes an SRAM 204, an address translation circuit 213 (ATS) having an address translation cache 216, a direct memory access (DMA) module 230, a host data path automation (HDPA) circuit 232, a command parser 236, a command executor 238, and a control path 240. In various embodiments, the address translation circuit 213 is the same as the address translation circuit 113 and the address translation cache 216 is the same as the address translation cache 116, all of which will be discussed in more detail below. The SRAM 204 may be an internal SRAM of the NVMe controller 203 that is separate from the SRAM 224. The CMB manager 202 may be directly coupled to the NVMe controller 203 so that the NVMe controller 203 may interact with the CMB manager 202 to access the DRAM 222 and the SRAM 224 .

[0065] In an embodiment, the media interface module 228 interacts with the NVM 206 for read and write operations. The DMA module 230 performs data transfers between the host system 220 and the memory device 130 without the involvement of the CPU 209. The HDPA circuit 232 controls the data transfer while activating the control path 240 for extracting PRP / SGL, issuing completions and interrupts, and activating the DMA 230 for the actual data transfer between the host system 220 and the memory device 130. The error correction module 234 corrects the data extracted from the memory array in the NVM 206. The command parser 236 parses the command to the command executor 238 for execution on the media interface module 228.

[0066] In an embodiment, the controller 215 further includes a page request interface (PRI) handler 219 coupled to or integrated with the processor 217 and the host interface 208, as will be discussed in greater detail. The PRI handler 219 may be processing logic that may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), firmware (e.g., instructions running or executed on the processor 217), or a combination thereof.

[0067] In various embodiments, from the perspective of the host interface 208 of the memory controller 215, the PRI handler 219 automatically sends page miss requests to the host system 120 and detects and handles page request responses returned from the host system 120. The page miss requests request the translation agent of the host system 120 to repin the physical pages of the host memory 212 to the virtual addresses of each respective page miss request, thus making those pages available again in the host system 220. In some cases, the repinning of pages in the host memory 212 involves finding a valid virtual address to map to an existing physical address (where the data is still stored) and completing the mapping, for which an address translation may be returned to the controller 215. Each page request response may include a confirmation that the respective page has been repinned in memory and the address translation of the pinned page.

[0068] Figure 3 Yes Description Figure 2 2 is a schematic diagram of an embodiment of a CMB manager 202 of a system 200, but other systems are possible. The CMB manager 202 manages data transactions between a host system 220 and a memory device 130 having a controller memory buffer (CMB) 300. The CMB 300 is a controller memory space that may span one or more of the DRAM 222, SRAM 224, and / or NVM 206. The contents in the CMB 300 typically do not persist across power cycles, so the CMB manager 202 may rebuild the CMB 300 after the system 200 is powered on.

[0069] One or more types of data structures defined by the NVMe protocol may be stored in CMB 300 by CMB manager 202 or may be stored in host memory 212 ( Figure 2 ). As described in more detail below, the host system 220 may initialize the CMB 300 before the CMB manager 202 stores the NVMe data structures to the host system 220. During the initialization phase, the memory device 130 may advertise the capabilities and size of the CMB 300 to the host system 220 and may advertise which NVMe data structures may be stored in the CMB 300. For example, the memory device 130 may store one or more of the NVMe data structures in the CMB 300, including NVMe queues 304, such as submission queues (SQs) 306, completion queues (CQs) 308, PRP lists 312, SGL segments 314, write data 320, read data 318, and combinations thereof.

[0070] The NVMe protocol standard is based on a paired submission and completion queue mechanism. Commands are placed in a submission queue (SQ) by the host software. Completions are placed in an associated completion queue (CQ) by the controller 215. The host system 220 (or device) may have multiple pairs of submission and completion queues for different types of commands. In response to a notification from the host system 220, the controller 215 extracts a command from a submission queue (e.g., one of the SQ 306). Thereafter, the controller 215 processes the command (e.g., performs internal command selection), executes the command (e.g., performs write or read), and the like. After processing the command, the controller 215 places an entry in a completion queue, wherein the entry indicates that execution of the command has been completed. The controller 215 then generates an interrupt to the host device indicating that the entry has been placed in the completion queue. The host system 220 reviews the entry of the completion queue and then notifies the controller 215 that the entry of the completion queue has been reviewed. As will be discussed in more detail, the address translation circuit 213 can help perform these functions of the controller 215.

[0071] Generally, submission and completion queues are allocated within host memory 212, where each queue may be physically located contiguously or discontinuously in host memory. However, the CMB feature supported in the NVMe standard, for example, enables the host system 220 to place submission queues, completion queues, physical page region (PRP) lists, scatter gather list (SGL) segments, and data buffers in controller memory instead of host memory 212.

[0072] Controller 215( Figure 2 ) also generates an internal mapping table 221 for use by CMB manager 202 to map PF and VF data to the correct CMB location in controller memory DRAM 222 or SRAM 224. Mapping table 221 itself is typically stored in flip-flops or DRAM 222 to reduce or eliminate any latency issues. For example, in one embodiment, the mapping table may have an entry for the PF and each VF.

[0073] The NVMe standard supports NVMe virtualization environments. A virtualization environment may use an NVM system with multiple controllers to provide direct input / output (I / O) access to a virtual or physical host (also referred to herein as a virtual or physical function). The NVM system includes (several) primary controllers and (several) secondary controllers, wherein the (several) secondary controllers are dependent on the (several) primary controllers for dynamically assigned resources. The host may issue an identification command specifying a list of secondary controllers to the primary controller to discover the secondary controllers associated with that primary controller. SR-IOV defines PCI Express to allow multiple system images (SIs) (e.g., virtual machines running on a hypervisor) to share PCI hardware resources (see Figure 4) extension.

[0074] A physical function (PF) is a PCIe function that supports SR-IOV capabilities, which in turn allows it to support one or more associated virtual functions (VFs). These PFs and VFs can support an NVMe controller that shares an underlying NVM subsystem with multipath I / O and namespace sharing capabilities. In this virtualized environment, a physical function (sometimes referred to as a primary function) and each virtual function are assigned their own CMB, which is part of the total controller memory available to the CMB. As used herein, the term physical function refers to a PCIe function that supports SR-IOV capabilities, where a single physical host is divided into a physical function and multiple virtual functions that each communicate with a controller of a memory device. The terms physical function and primary function are used interchangeably herein.

[0075] In an embodiment, the controller 215 advertises CMB 300 availability only to physical functions (PFs) of a virtualized host system, such as the host system 220, where the virtualized host system has a single physical function 211 and one or more virtual functions 205 (or VFs). Moreover, the advertised CMB 300 availability may be in the form of the total CMB size available to all functions (physical and any virtual functions), so that the physical function 211 may selectively assign any desired portion of the advertised available total CMB size to itself and all other virtual functions 205.

[0076] The controller 215 may then store the physical function selected portions of the available CMB 300 in NVMe registers dedicated to each physical function 211 and virtual function, respectively. The virtual function may store different relative portion sizes of the advertised CMB size in each NVMe register to account for the different requirements of the physical function 211 itself and each virtual function. Once the physical function 211 assigns different numbers and regions of the advertised CMB available for host access during the initiation phase (e.g., for direct access by primary and virtual functions), these settings may be managed by the controller 215 to provide access to the corresponding primary or virtual function during operation of the memory device.

[0077] The controller buffer manager 202 may include a transaction classifier module 322 to classify received host write transactions to the CMB 300. Host write transactions to the CMB 300 may be associated with host write commands and host read commands. In certain embodiments, the transaction classifier module 322 may classify host write transactions into one of three NVM data structure groups of NVMe queues 304, pointers 310, and data buffers 316. The NVMe queues 304 include (several) host submission queues (SQs) 306 and (several) host completion queues (CQs) 308. The pointers 310 may include a physical region page (PRP) list 312 and a scatter gather list (SGL) segment 314. The PRP list 312 contains pointers indicating physical memory pages that are filled with user data or are about to be filled with user data (e.g., for a read or write command in the NVMe queue 304). SGL segment 314 includes pointers indicating physical addresses of host memory 212 where data should be transferred from host memory 212 for write commands and to host memory 212 for read commands. Data buffer 316 may contain write data 320 associated with a write command to be written to NVM 206 and / or read data 318 associated with a read command to be read from memory device 130.

[0078] In certain embodiments, the NVMe queues 304, pointers 310, and data buffers 316 associated with a particular command may be stored in the CMB 300 by the CMB manager 202 to reduce command execution latency for the memory device 130. For example, host command entries written to the SQ-implemented CMB 300 avoid fetching the host command entries through the PCIe fabric, which may include multiple switches if the SQ is located in the host memory 212. The PRP list 312 and SGL segment 314 written to the CMB 300 of the memory device 130 avoid fetching the PRP list 312 and SGL segment 314 separately through the PCIe fabric if the PRP list and SGL segment are located in the host memory space. The write data 320 written to the CMB 300 of the memory device 130 avoids the memory device 130 from fetching the write data from the host memory 212.

[0079] The address translation circuit 213 may communicate with the components of the host system 220 and the memory subsystem 210 through the host interface 208. The address translation circuit 213 may also be at least partially incorporated into the host interface 208, as will be discussed in greater detail. The address translation circuit 213 may also retrieve commands from the SQ 306, handle commands to include retrieving address translations from the ATC 216 (if present), and submit completion notifications to the CQ 308 of the host system 220. Thus, in at least some embodiments, the address translation circuit 213 may include or be integrated with the command parser 236, the command executor 238, and the DMA 230.

[0080] Figure 4 is an example physical host interface between the host system 220 and the memory subsystem according to some embodiments that implements caching host memory address translation data. In at least some embodiments, the example physical host interface also implements NVMe direct virtualization, as shown in reference Figures 2 to 3 In one embodiment, the controller 415 of the memory subsystem is coupled to the host system 220 through a physical host interface (e.g., PCIe bus 411A). In one embodiment, the NVMe control module 405 running on the controller 415 generates and manages a number of virtual NVMe controllers 402 to 408 within the controller 415. The virtual NVMe controllers 402 to 408 are virtual entities that appear to other devices (e.g., the host system 220) as physical controllers connected to the PCIe bus 411A by means of physical functions 412 to 418 associated with each virtual NVMe controller 402 to 408. Figure 4 Three virtual NVMe controllers 402 to 408 and three corresponding physical functions 412 to 418 are illustrated. However, in other embodiments, there may be any other number of NVMe controllers, each with a corresponding physical function.

[0081] Each of the virtual NVMe controllers 402 to 408 manages storage access operations of the underlying memory device 130. For example, the virtual NVMe controller 402 may receive a data access request from the host system 220 via the PCIe bus 411A, including a request to read, write, or erase data. In response to the request, the virtual NVMe controller 402 may identify a physical memory address in the memory device 130 associated with the virtual memory address in the request, perform the requested memory access operation on the data stored at the physical address, and return the requested data and / or confirmation or error message to the host system 220 as appropriate. The virtual NVMe controllers 404 to 408 may operate in the same or similar manner with respect to data access requests for one or more memory devices 130.

[0082] In an embodiment, the NVMe control module 405 associates one of the physical functions 412 to 418 with each of the virtual NVMe controllers 402 to 408 so as to allow each virtual NVMe controller 402 to 408 to appear as a physical controller on the PCIe bus 411A. For example, the physical function 412 may correspond to the virtual NVMe controller 402, the physical function 414 may correspond to the virtual NVMe controller 404, and the physical function 418 may correspond to the virtual NVMe controller 408. The physical functions 412 to 418 are fully functional PCIe functions that can be discovered, managed, and manipulated like any other PCIe device and can therefore be used to configure and control PCIe devices (e.g., virtual NVMe controllers 402 to 408). Each physical function 412 to 418 may have a certain number of virtual functions (VFs) associated with it. A VF is a lightweight PCIe function that shares one or more resources with a physical function and a virtual function associated with that physical function. Each virtual function has a PCI memory space that is used to map its register set. The virtual function device driver operates on the register set to enable its functionality and the virtual function appears as an actual PCIe device accessible by the host system 220 through the PCIe bus 411A.

[0083] In at least some embodiments, the controller 415 is further configured to control the execution of memory operations associated with memory commands from the host system 220 at one or more memory devices 130 and one or more network interface cards (NICs) 450 (which are actual physical memory devices). In these embodiments, the controller 415 communicates with the memory devices 130 through the second PCIe bus 411B and communicates with the NICs 450 through the third PCIe bus 411C. Each memory device 130 may support one or more physical functions 432 and each NIC 450 may support one or more physical functions 436. Each physical function 432-436 may also have a certain number of virtual functions (VFs) associated with it.

[0084] In these embodiments, each physical function 412-418 and 432-436 may be assigned to any of the virtual machines VM(0) to VM(n) in the host system 220. When I / O data is received from the virtual machine at the virtual NVMe controllers 402-408, the virtual machine driver (e.g., NVMe driver) provides a guest physical address for the corresponding read / write command. The NVMe control module 405 may translate the physical function number into a bus, device, and function (BDF) number, and then add the command to a direct memory access (DMA) operation to perform a DMA operation on the guest physical address. In one embodiment, the controller 115 further translates the guest physical address into a system physical address of the memory subsystem 110.

[0085] In addition, each physical function 412 to 418 and 432 to 436 can be implemented in either privileged mode or normal mode. When implemented in privileged mode, the physical function has a single management point that can control resource manipulation and storage provisioning of other functions implemented in normal mode. In addition, the physical function in privileged mode can perform management options, including, for example, enabling / disabling multiple physical functions, storage and quality of service (QoS) provisioning, firmware and controller updates, vendor unique statistics and events, diagnostics, secure erase / encryption, etc. Typically, the first physical function can implement privileged mode and the rest of the physical function can implement normal mode. However, in other embodiments, any of the physical functions can be configured to operate in privileged mode. Therefore, there can be one or more functions running in privileged mode.

[0086] The host system 220 can run multiple virtual machines VM(0) to VM(n) by executing a software layer (commonly referred to as a hypervisor 424) above the hardware and below the virtual machines, such as Figure 4 Schematically shown in . In one illustrative example, the hypervisor 424 may be a component of a host operating system 422 executed by the host system 220. Alternatively, the hypervisor 424 may be provided by an application running under the host operating system 422, or may run directly on the host system 220 without an operating system underneath the hypervisor 424. The hypervisor 424 may abstract the physical layer (including processors, memory, and I / O devices) and present this abstraction to virtual machines VM(0) to VM(n) (including virtual processors, virtual memory, and virtual I / O devices) as virtual devices. The virtual machines VM(0) to VM(n) may each execute a client operating system that may utilize the underlying virtual devices, which may, for example, be mapped to a memory device 130 or a NIC 450 managed by one of the virtual NVMe controllers 402 to 408 in the memory subsystem. One or more applications may run on each VM under the client operating system.

[0087] In various embodiments, each virtual machine VM(0) to VM(n) may include one or more virtual processors and / or drivers. Processor virtualization may be implemented by hypervisor 424 scheduling time slots on one or more physical processors so that those time slots are scheduled on virtual processors from the perspective of the guest operating system. Memory virtualization may be implemented by page tables (PTs), which are memory structures that translate guest memory addresses into physical memory addresses. Hypervisor 424 may run at a higher privilege level than the guest operating system, and the guest operating system may run at a higher privilege level than the guest application.

[0088] In one embodiment, there may be multiple partitions on the host system 220 representing virtual machines VM(0) to VM(n). The parent partition corresponding to the virtual machine VM(0) is a root partition (i.e., root ring 0), which has additional privileges to control the life cycle of other child partitions (i.e., regular ring 0) corresponding to virtual machines VM(1) and VM(n), for example. Each partition has a corresponding virtual memory, and the child partitions see the physical devices assigned to them instead of presenting virtual devices. When the host system 220 is initially started, the parent partition can directly see all physical devices. A pass-through mechanism (e.g., PCIe pass-through or direct device assignment) allows the parent partition to assign an NVMe device (e.g., one of the virtual NVMe controllers 402 to 408) to a child partition. The associated virtual NVMe controllers 402 to 408 can appear as virtual storage resources for each of the virtual machines VM(0), VM(1), and VM(n), where a running client operating system or client application can access the virtual storage resources. For example, in one embodiment, virtual machine VM(0) is associated with virtual NVMe controller 402, virtual machine VM(1) is associated with virtual NVMe controller 404, and virtual machine VM(n) is associated with virtual NVMe controller 408. In other embodiments, one virtual machine may be associated with two or more virtual NVMe controllers. Virtual machines VM(0) to VM(n) may identify associated virtual NVMe controllers using corresponding bus, device, and function (BDF) numbers, as will be described in more detail below.

[0089] In some embodiments, the hypervisor 424 further includes a storage emulation program 426 coupled to the NVMe drivers on the virtual machines VM(0) to VM(n) and coupled to the physical function NVMe driver 432 of the hypervisor 424. The physical function NVMe driver 432 can drive the physical functions 412 to 418 through the PCIe bus 411A with the help of the virtual NVMe controllers 402 to 408, and also drive the physical functions 432 available on the memory device 130 through the PCIe bus 411A and the second PCIe bus 411B. In addition, the hypervisor may include a NIC emulation program 428 coupled to the NIC drivers on the virtual machines VM(0) to VM(n) and coupled to the physical function NIC driver 436 of the hypervisor 424. In various embodiments, the physical function NIC driver 434 controls the PF 436 of the NIC 450 through the PCIe bus 411A and the third PCIe bus 411C.

[0090] In at least some embodiments, the host system 220 submits memory commands (e.g., erase (or unmap), write, read) to a set of submission queues 442, input / output (I / O) commands to a set of I / O queues 446, and management ("admin") commands to a set of management queues 448, which are stored in one of the host memory 212 or CMB 300 of the host system 220 ( Figure 3 ). The controller 415 can retrieve these memory commands through the PCIe bus 411A and generally handle each memory command in turn according to priority (e.g., handling reads before writes). When the controller 415 has completed handling the memory commands residing in the set of submission queues 442, the controller 415 returns an acknowledgment of the completion of the memory command by submitting a completion entry in a corresponding completion queue in a set of completion queues 444, which are also stored in the host memory 212. In some embodiments, the host memory 212 is composed of DRAM or other main memory type memory. In various embodiments, the number of queues 442, 444, 446, 448 can be hundreds (or thousands) and are ordered sequentially (e.g., continuously) according to the virtual addresses of the memory commands. In other words, the queuing of memory commands within these queues is ordered sequentially based on the virtual addresses in those memory commands within the virtual memory space.

[0091] In the disclosed embodiment, the controller 415 further includes an address translation circuit 413, which includes a reference Figures 1 to 3ATC 416 (or "cache") is similarly introduced. ATC 416 can be a static random access memory (SRAM), a tightly coupled memory (TCM), or other fast access memory suitable for use as a cache. Address translation circuit 413 can be coupled to NVMe control module 405 and is typically coupled to a PCIe protocol component of controller 415 (e.g., a virtual NVMe controller). In this way, address translation circuit 413 provides a host interface circuit system that facilitates obtaining address translations and other processing of memory commands retrieved from queues 442, 444, 448. The address translation service (ATS) of address translation circuit 413 further enables direct connection between one or more memory devices 130 and virtual machines VM(0) to VM(n) to achieve data communication performance at near PCIe line rates, which bypasses hypervisor 424.

[0092] As explained, the address translation circuit 413 can receive (or retrieve) an address translation request from the HIF circuit handling the memory command, requesting the TA to perform the translation of the virtual address located within the memory command, and upon receiving the physical address, storing the mapping between the virtual address and the physical address (also referred to as the L2P mapping) in the ATC 416. Upon receiving a subsequent address translation request containing the same virtual address, the address translation circuit 413 can verify that the virtual address hits at the ATC 416 and directly copy the corresponding address translation from the ATC to the pipeline of the address translation circuit 413, which passes the corresponding address translation back to the requesting HIF. Similar caching of address translations can also be performed for DMA operations, which will be discussed in more detail. Therefore, for many address translation requests, the functionality of the address translation circuit 413 can achieve any need to interact with the virtual machine hypervisor 424 as software, which is a bottleneck and slows down the execution of obtaining address translations without caching such address translations. The resulting speed, latency, and throughput performance can be significantly improved by the ATS functionality.

[0093] Figure 5 5 is a system 500 in accordance with some embodiments in which a memory subsystem controller (e.g., a PCIe controller 515) contains an address translation cache (ATC) 516. Within the system 500, a host system 520 (or any host system discussed herein) includes a translation agent (TA) 517 and an address translation and protection table (ATPT) 519, which together can be used by the host system 520 to provide address translations to the PCIe controller 515 in accordance with the PCIe protocol. Specifically, the TA 517 can return a physical page address in response to a submitted virtual address in an address translation request, or return "no translation" if the corresponding physical page has been swapped out and therefore there is no current translation for the virtual address.

[0094] These address translations may be associated with memory commands resident in the host memory 512 (and / or CMB 300) handled by the PCIe controller 515 (or by other PCIe devices or virtual PCIe devices). To provide the translations, the TA 517 is configured to communicate with the address translation request / response through the PCIe root complex 523. In some systems, the TA 517 is also referred to as an input / output memory management unit (IOMMU) executed by the hypervisor 424 or virtual machine manager running on the host system 520. Thus, the TA 517 may be a hardware component or software (IOMMU) with a dedicated driver.

[0095] In various embodiments, to avoid this increased I / O traffic between the host system 520 and the PCIe controller 515 over the PCIe bus 511A, address translations provided to the PCIe controller 515 may be cached in the ATC 516 and accessed to fulfill later (or subsequent) ATS-generated address translation requests without having to go back to the TA 517 with a re-request for each required translation. If an entry in the ATC 516 fails due to a change in assignment between virtual and physical addresses within the TA 517 and ATPT 519, the PCIe controller 515 (e.g., an ATS in the PCIe controller 515) may clear the corresponding entry in the ATC 516 and other ATS queues in the host interface circuitry.

[0096] Figure 6 is a memory subsystem 610 for page request interface support when caching host memory address translation data for multiple host interface circuits according to some embodiments. In various embodiments, the memory subsystem 610 may be connected to Figure 1 The memory subsystem 110 or Figure 2 The memory subsystem 210 or Figure 5 In these embodiments, the memory subsystem 610 includes a controller memory buffer (CMB) 630 and a PRI handler 219 (see Figure 2 In some embodiments, controller 615 is controller 115, 215, or 415, local media controller 135, or a combination thereof (see Figure 1 ). In at least one embodiment, CMB 630 is a reference Figure 3 The CMB 300 discussed, and therefore may also include the NVM of the memory device 130 .

[0097] In various embodiments, CMB 630 includes a host Advanced eXtensible Interface (AXI) interface 631 that supports the functionality of controller 615, (several) CMB control / status registers 632, a host read command SRAM / FIFO 634, a host data SRAM 636, a CMB host write buffer 638, a controller write buffer 640, a host read buffer 644, a controller data SRAM 646, a controller read buffer 648, and a controller read command SRAM / FIFO 650, where FIFO stands for First In First Out buffer.

[0098] In some embodiments, the controller 615 includes a PCIe system on a chip (SoC) 601 and a host interface circuitry 603. The PCIe SoC 601 may include a PCIe IP that facilitates communication by the host interface circuitry 603 with a host system 620 (including a TA 617) using the PCIe protocol. Thus, the PCIe SoC 601 may include capabilities and control registers that exist for each physical function and each virtual function of the memory subsystem 610. The PCIe SoC 601 may include an integrated development environment (IDE) link encryption circuit 605 that encrypts translation layer packets (TLPs) passed to the host system 620 over the PCIe bus 611A and decrypts TLPs received from the host system 620.

[0099] In various embodiments, the host interface circuit system 603 includes a local memory 619 (SRAM, TCM, and the like), a number of host interface circuits 608A, 608B, 608C, ... 608N (or collectively referred to as host interface circuits 608), an address translation circuit 613 (or ATS) including an ATC 616 and a translation logic 602, and an AXI matrix 623 that can be used to send interrupts to a host system 620 in the interaction of reference handling memory commands and DMA. The host interface circuits 608A to 608N (also referred to herein as HIF circuits 608) include hardware components that help extract and process commands (for host queues) and data (for DMA) to perform specific functions at the time of command and DMA processing. Individual ones of the host interface circuits 608A to 608N sometimes request address translations of virtual addresses or guest physical addresses (e.g., associated with commands or DMA). In an embodiment, for this purpose, individual HIF circuits 608 request address translation circuits 613 to provide the translations. In this manner, address translation circuitry 613 interfaces on behalf of respective host interface circuitry 608A- 608N and supports both address translation generation and invalidation, as will be discussed in greater detail below.

[0100] Furthermore, in at least one embodiment, the host interface circuit 608N may be a processing bridge circuit 608N that provides a hardware interface between the PRI handler 219 and other host interface circuits 608A, 608B, 608C, etc. The processing bridge circuit 608N may include a reference Fig.10 The various buffers and logic for handling page miss requests, page request responses, and other commands and messages discussed in greater detail herein. Additionally, in at least one embodiment, the host interface circuitry 608B is a host queue interface circuitry 608B that handles host command fetch automation (e.g., on the submission queue 306 of the host system 120). Reference will be made to Fig. 9 and Fig.12 The host queue interface circuit 608B is discussed in more detail. In addition, in some embodiments, the host interface circuit 608C is a completion queue interface circuit 608C that handles command completion automation (e.g., completion queue 308 posting) to indicate to the host system 120 when the controller 115 has completed processing a particular command. Figures 13 to 14 Completion queue interface circuit 608C is discussed in more detail.

[0101] The address translation circuit 613 may employ a number of fields and parameters, such as the minimum translation unit (STU) of data (defined by a specific size that is typically smaller than a host tag) and the invalidation queue depth beyond which queue entries are to be cleared. The translation logic 602 may use PCIe memory read TLPs to request translations for given untranslated addresses from the TA 617 (or IOMMU). For example, these address translation requests may carry a separate PCIe tag and identifier. The PCIe tag may include, for example, "PCIe memory read complete" indicating the translated address and translation size, an "S" bit to notify the size of the translation (e.g., the translated address may represent a contiguous physical address space of 128KB or similar size), "R" and "W" bits to provide read / write permission for pages in the physical address space, and a "U" bit to tell the device to use only untranslated addresses for DMA. The U bit may be helpful when the buffer is a one-time use and the TA 617 does not have to send invalidations to the controller 615.

[0102] In an embodiment, when the translation changes in TA 617, the ATC 616 (one or more caches) in the memory subsystem 610 should clear the old entries corresponding to the virtual addresses that have been invalidated. An invalidation request can be sent by TA 617 to the address translation circuit 613 using a PCIe message TLP, which can be directed to the HIF circuit that handles some functions of memory command processing. The address translation circuit 613 can instruct the HIF circuit to clear all DMAs for the invalidated address range, remove such entries from the ATC 616, and send an invalidation completion (e.g., another PCIe message TLP) to the host system 620 when the HIF circuit confirms the invalidation.

[0103] In some embodiments, the address translation circuit 613 stores the address translation returned from the TA 617 in response to the address translation request in the ATC 616. In various embodiments, this address translation may include an I / O submission queue (SQ) base address, a PRP / SGL of outstanding commands in one or more HIF circuits, and / or an I / O completion queue (CQ) base address. The ATC 616 may be configured to handle finding an empty slot in the ATC 616 and storing a new translation in the empty slot, looking up when a data TLP occurs, and clearing entries during a function level reset (FLR), failure, and other resets. The ATC 616 may be further configured to age older entries to make room when the cache is full.

[0104] In various embodiments, the queue portion 416A of the ATC 616 cache may be sized to include at least one or two entries per queue of the SQ 442, CQ 444, I / O queue 446, and management queue 448, but more entries may be envisioned as cache memory device size and cost decrease. With enough space for two entries, the ATC 616 may store the address translation of the current page associated with the queue and the next page (e.g., which sequentially follows the virtual address of the current page). This look-ahead buffering implementation of address translations for a predetermined number of queues in the ATC 616 greatly reduces the number of misses at the ATC while keeping the size of the ATC reasonable given the cache memory cost. In addition, the DMA portion 416B of the ATC 616 (for storing DMA data) may be expanded to a certain integer multiple (e.g., 2 to 4 times) of the size of the queue portion of the ATC 616.

[0105] In at least some embodiments, the translation logic assigns a queue identifier to each of the SQ 442, CQ 444, I / O queue 446, and management queue 448 ("queues"). This may be as simple as sequentially incrementing a count number for each subsequent sequentially ordered queue in a set of queues. The translation logic 602 may then index the queue portion 416A of the ATC 616 cache according to the queue identifier of the corresponding address translation stored in the ATC 616 cache and use such queue identifiers internally (within the address translation circuitry 613) to track address translation requests and responses. In addition, the translation logic 602 may index the DMA portion 416B of the cache according to the host tag value and use the hot tag value to internally track DMA-related address translation requests and responses.

[0106] In these embodiments, each DMA command has a host tag ("htag") with a virtual address. The translation logic 602 may be configured to store one translation of each htag of the data in the ATC 616 when the data is greater than or equal to the pre-configured data size. The translation logic 602 may be further configured to store one translation of each htag of the metadata (without indicating whether the size limit of the cache) in the ATC 616. During further operation of the controller 615, address translation is used as long as subsequent TUs (within the htag) use the same translated physical address range. The cached translation is replaced when the DMA command starts to be transferred to or from different memory ranges. The cached address translation is also replaced when the htag is assigned to, for example, different commands using different memory locations.

[0107] Figure 7 is an example memory subsystem controller 715 including address translation circuitry 713 (or ATS) for implementing cache host memory address translation data according to some embodiments. The address translation circuitry 713 may be referenced to Figure 1 , Figure 2 , Figure 4 and Figure 6 A more detailed illustration of the address translation circuits 113, 213, 413 and / or 613 discussed.

[0108] In various embodiments, similar to that previously introduced and discussed, the address translation circuitry 713 includes a pipeline of queues, buffers, and multiplexers that facilitate the flow of address translation requests and responses and that interfaces with an address translation cache (ATC) 716. The multiplexers are typically included within the translation logic 602 and are therefore not always individually numbered or discussed separately from the translation logic 602.

[0109] In an embodiment, this pipeline begins with an address translation request from the host interface (HIF) circuit 608 (see Figure 6 ) flows into a set of request staging queues 702. These address translation requests are typically received as a series of address translation requests from any given HIF circuit 608, so each queue in the set of request staging queues 702 contains multiple available entries for multiple virtual addresses associated with a series of incoming address translation requests, which may be received as a group, for example, from a given HIF circuit 608. The set of request staging queues 702 buffers the address translation requests received from the host interface circuit, each address translation request including a virtual address. The multiplexer 703 then pushes each address translation request into a set of reorder buffers 714 and the initial reference Figure 6 Within the set of reorder buffers 714, an address translation request will provide an initial entry that will later be completed by an address translation provided in response to a corresponding request HIF circuit 608. In an embodiment, the translation logic 602 is configured to store incoming address translations, such as from the set of reorder buffers 714, into the ATC 716.

[0110] In an embodiment, the translation logic 602 is coupled to the set of request staging queues 702, the set of reorder buffers 714, and the ATC 716, among other components. The translation logic 602 may determine the queue identifiers of the respective queues in the plurality of submission queues (SQ) and the plurality of completion queues (CQ) discussed previously and identify the queue identifier of the respective address translation request. The translation logic 602 may further index the queue portion 416A of the cache according to the queue identifier of the respective address translation request.

[0111] In at least some embodiments, the translation logic 602 is configured to store, for each address translation request in the set of request hierarchical queues, a first address translation corresponding to a current page associated with the address translation request in the ATC 716, and a second address translation corresponding to a subsequent page that sequentially follows the current page according to the virtual address number in the ATC 716. This look-ahead buffering of address translations for a predetermined number of queues in the ATC 716 greatly reduces the number of misses at the ATC 716 while keeping the size of the ATC reasonable given the cost of cache memory. As cache memory costs decrease, it is contemplated that one or more additional address translations that sequentially follow the first address translation (according to the virtual address) may also be stored in the ATC 716.

[0112] In some embodiments, for incoming DMA requests, the translation logic 602 may identify, within at least some of the address translation requests, a first host tag associated with the data of the DMA. The translation logic may further identify, within at least some of the address translation requests, a second host tag associated with the metadata of the data. The translation logic may then index the cached DMA portion 416B to include values ​​corresponding to each first host tag and each second host tag according to the host tag values. In an embodiment, the translation unit (TU) in response to a subsequent address translation request serves as a cached host tag within a translated memory range, and the translation logic uses the address translation of the cached host tag in the ATC 716 to satisfy the subsequent address translation request. Furthermore, in response to a subsequent address translation request targeting a memory range different from the translated memory range, the translation logic may evict the address translation from the ATC 716. In response to the cached host tag being assigned to a different command, the translation logic 602 may also evict the address translation from the cache (ATC 716).

[0113] In some embodiments, for each incoming address translation request, the translation logic 602 further determines whether the virtual address (or queue identifier) ​​within the address translation request is a hit or miss at the cache (ATC 716). If the virtual address has hit within the cache, the translation logic 602 may reinsert the corresponding address translation (including the mapped physical address) into the reorder buffer 714 to provide to the requesting HIF circuit 608. Thus, address translations that hit at the ATC 716 may be provided back to the requesting HIF circuit 608 at close to line rate without any further delay in requesting the TA / IOMMU at the host system 620 to execute the translation.

[0114] In an embodiment, the address translation circuitry 713 further includes a set of outbound request queues 704 to buffer one or more of the address translation requests that miss at the cache (ATC 716), for example, as an additional pipeline stage for staging these address translation requests to be sent to the host system 620. As these address translation requests are forwarded onto the host system 620, the translation logic 602 pushes each address translation request into a set of pending response queues 708. In an embodiment, the set of pending response queues 708 is configured to buffer respective address translation requests awaiting address translation from the host system 620 while maintaining the order as received within the set of request staging queues 702. Maintaining this order within the set of pending response queues 708 helps the set of reorder buffers 714 to properly reorder address translations returning from the host system 620 (e.g., TA 617 or IOMMU) despite being pushed out of the set of request staging queues 702 while waiting for the TA 617.

[0115] In these embodiments, the address translation request that misses at the ATC 716 is forwarded to the AXI translator 722, which issues the address translation request to the host interface 208 through a set of address channels 732 for reading and leading to the TA 617. In this manner, the address translation circuit 713 can request a translation agent (e.g., TA 617) of the host system to provide a physical address that maps to a first virtual address (for the current page) of the first address translation and a second virtual address (for a subsequent page that sequentially follows the current page according to the virtual address number) of the second address translation. In response to the TA 617 providing the address translation, the address translation is received through a set of data channels 734 for reading and through the AXI translator 724.

[0116] In the disclosed embodiment, the translation logic 602 pops the incoming address translations into the set of pending response queues 708 while also storing the address translations (e.g., the first address translation and the second address translation) in the set of reorder buffers 714. The set of reorder buffers 714 also receives from the set of pending response queues 708 the proper order of the address translations in relation to the order in which the address translation requests in the set of request staging queues 702 were received. As mentioned, this may include a series of address translation requests from the same HIF circuit 608, and thus the set of reorder buffers 714 may be configured to reorder each pair of address translations for each of these address translation requests to match the order in which the address translation requests were buffered into the set of pending response queues 708. Thus, the set of reorder buffers 714 may reorder address translations in this manner and provide the first address translation (for each address translation request) to the request HIF circuit 608 in the same order as received, except for address translations that hit at the ATC 716 and have been supplied to the request HIF circuit 608. In these embodiments, the translation logic 602 detects incoming address translations and copies them from the set of reorder buffers into the ATC 716, which may be accessed later for matching with subsequent address translation requests.

[0117] In some embodiments, the translation completion response may be returned from the host system 620 as a "miss" because the page corresponding to the virtual address has been unmapped. The translation logic 602 may then reply to the requesting HIF circuit 608 that the virtual address for the requested specific address translation has been missed. In response to receiving this miss TLP, the HIF circuit 608 may be triggered to generate a page request interface (PRI) message to the host system 620 to remap the virtual address to a physical page with a physical address.

[0118] In at least some embodiments, the address translation circuit 713 further includes a set of invalidation queues 718 to buffer invalidation requests received from the translation agent of the host system. More specifically, the invalidation requests may be received through a set of data channels 736 for writing into the AXI converter 726 that receives the invalidation requests from the host system 620. The translation logic 602 may then place each incoming invalidation request into the set of invalidation queues 718 for handling in the order received.

[0119] In these embodiments, the address translation circuit 713 further includes an invalidation handler logic 750, which is coupled to the set of invalidation queues 718, the host interface circuit 608, the set of request staging queues 702, the set of pending response queues 708, and the set of reorder buffers 714. In an embodiment, the invalidation handler logic 750 detects an invalidation request within the set of invalidation queues 718, the invalidation request corresponding to a virtual address in the virtual address. The invalidation handler logic 750 may further cause the address translation associated with the virtual address to be marked as invalid within the set of request staging queues 702, the set of pending response queues 704, the set of reorder buffers 714, and the cache (ATC 716). In an embodiment, the set of outbound request queues 704 triggers (e.g., via the translation logic 602) the set of reorder buffers 714 to mark entries therein as invalid in response to a invalidation signal received from the invalidation handler logic 750. The invalidation handler logic 750 may further send the associated invalidation request to the corresponding host interface circuit 608 and also expect the host interface circuit 608 to clear the invalid entry.

[0120] In some embodiments, the address translation circuit 713 (e.g., corresponding queues and buffers) removes address translations marked as invalid within the set of request staging queues 702, the set of pending response queues 704, the set of reorder buffers 714, and the cache (ATC 716). Each HIF circuit 608 may send an invalidation completion message to the address translation circuit 713, to which the invalidation handler logic 750 acknowledges (ACKs) the invalidation completion message in reply. In this manner, the invalidation handler logic 750 may acknowledge the removal of the address translation by the host interface circuit 608, and then send an invalidation completion response to the translation agent (TA 617). The ACK response may be used by the individual HIF circuit 608 to unfreeze any frozen functional arbitration.

[0121] More specifically, when the invalidation handler logic 750 has confirmed that the address translation circuitry 713 and the HIF circuitry 608 have cleared the queues, buffers, and cache entries of the invalid address translations associated with the invalidation request, the invalidation handler logic 750 may send an invalidation response to the AXI translator 728. The AXI translator 728 may put out the invalidation response to the TA 617 at the host system 620 on a set of data lanes 738 for writes.

[0122] Figure 8 Flowchart illustrating an example method 800 for caching host memory address translation data in a memory subsystem according to some embodiments of the present disclosure. The method 800 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 800 is respectively performed by Figure 1 , Figure 2 , Figure 4 , Figure 6 and Figure 7 The address translation circuits 113, 213, 413, 613, 713 of the embodiment are executed. Although shown in a specific order or sequence, the order of the process can be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in different orders, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0123] At operation 810 , processing logic buffers address translation requests received from host interface circuitry and each including a virtual address within a set of request staging queues of address translation circuitry.

[0124] At operation 820, processing logic stores respective address translation requests awaiting address translation from the host system while maintaining the order as received within the set of request hierarchical queues within a set of pending response queues.

[0125] At operation 830, processing logic reorders address translations within a set of reorder buffers according to the order maintained within the set of pending response queues, wherein each address translation includes a physical address mapped to a corresponding virtual address.

[0126] At operation 840, processing logic sends the address translation from the reorder buffer to a corresponding one of the host interface circuits that sent the corresponding address translation request.

[0127] At operation 850, processing logic stores a plurality of address translations associated with the address translation request in a cache coupled to the set of request staging queues and the set of reorder buffers for future access by the host interface circuitry. Also, at operation 850, processing logic may further store a plurality of pointers (e.g., SGS / SGL pointers) for outstanding direct memory address (DMA) commands within one or more host interface circuits.

[0128] At operation 860 , when applicable, processing logic reinserts the first address translation from the cache into the set of reorder buffers for subsequent requests by the host interface circuitry for the first address translation.

[0129] Fig. 9 9 is a block diagram of a system 900 of host interface circuitry (including host queue interface circuitry 608B) that interacts with PRI handler 219 according to some embodiments. In some embodiments, system 900 includes address translation circuitry 613 (see Figure 6 ), host interface circuit 608 (for example, reference Figure 6 Any of the host interface circuits 608A, 608B, 608C, ... 608N discussed above), processing bridge circuit 608N (see Fig.10 ), PRI handler 219 and controller memory 912. In an embodiment, the host interface circuit 608 is a host queue interface circuit 608B, which is configured to extract commands generated by the host system and stored in one or more submission queues 306 for processing by the controller 215 or 615, as shown in FIG. Figure 3 Discuss in more detail.

[0130] In various embodiments, the host memory manager (e.g., TA 617) swaps out pages of host memory 212 for a variety of reasons, which can be done without notifying any of the user processes or virtual machines VM(0) through VM(n). As discussed, a device including a controller 615 employing address translation circuitry 613 cannot assume that the pages required for DMA exist. Therefore, reference will be made to Fig.10 The page request interface discussed in more detail in conjunction with the PRI handler 219 may provide the host interface circuitry 603 with the ability to cause the translation agent 617 (eg, a host memory manager) to repin a memory page, return confirmation that the page has been repinned, and provide a corresponding address translation.

[0131] In these embodiments, the host queue interface circuit 608B includes a translation request queue 904, a translation response queue 908, a hardware pipeline 907 having a number of hardware stages (HW_0 ... HW_N), a command packet queue 905 that buffers incoming command packets to be processed by the hardware pipeline 907, and a set of control registers 911. The set of control registers 911 may exist in association with a physical function (PF), where related or other virtual functions (VFs) may also use this set of control registers 911. The values ​​within the set of control registers 911 may define the allocation of each PF, how many PRI-related requests may be outstanding for a given PF, and other settings or parameters of the hardware functionality of the PF.

[0132] In at least some embodiments, the host queue interface circuit 608B is configured to suspend command fetch arbitration on a submission queue 606 of the host system, the submission queue 606 being targeted by an address translation request that misses at a cache (e.g., ATC 616). The host queue interface circuit 608B may further trigger the PRI handler 219 to send a page miss request to a translation agent (TA) of the host system, the page miss request including a virtual address of the address translation request. The host queue interface circuit 608B may further receive a restart message from the PRI handler 219 when the PRI handler receives a page miss response from the translation agent. The host queue interface circuit 608B may further restart command arbitration on the suspended submission queue in response to the restart message.

[0133] In various embodiments, while a command packet enters or is in one of the hardware stages of the hardware pipeline 907, the host queue interface circuit 608B submits the virtual address of the command packet to the translation request queue 904 to request the address translation circuit 613 to provide the address translation required for the hardware pipeline 907 to complete processing the command packet. If the address translation circuit 613 returns an address translation previously stored in the ATC 616, for example, via the translation response queue 908, then the host queue interface circuit 608B continues to process the command packet normally without pausing.

[0134] In at least some embodiments, host queue interface circuitry 608B receives a message from address translation circuitry 613, e.g., via translation response queue 908, that an address translation request missed at cache or ATC 616 and, therefore, host interface circuitry 608 was affected by the miss at cache. In these embodiments, the translation miss message includes a virtual address and a queue number associated with a submission queue of the host system that contains the command for which the address translation request was sent.

[0135] In an embodiment, this translation miss message, in addition to pausing command fetch arbitration on the submission queue 606, triggers the host queue interface circuit 608B (e.g., by referencing the values ​​within the set of control registers 911) to remove the command packet from the hardware pipeline 907. The command packet may be associated with a memory read operation that relies on an address translation corresponding to an address translation request that missed at the cache. The host interface circuit may further send the translation miss message to the PRI handler 219 (e.g., via the controller memory 912), wherein the translation miss message includes the virtual address in the original address translation request and a queue number associated with the submission queue. For example, the queue number may be a slot in the SQ 306. This translation miss message may be queued within the controller memory 912 before reaching the PRI handler 219, such as by referencing the values ​​within the set of control registers 911. Fig.10 Discussed.

[0136] In various embodiments, the translation miss message includes information identifying where and / or how to reintroduce the command packet into the hardware pipeline. For example, when an address translation request is triggered at hardware stage HW_2 and the command packet is removed at this third hardware stage of the hardware pipeline 907, the information included in the translation miss message to the PRI handler 219 may be an identifier of this third hardware stage (HW_2). In an embodiment, the set of control registers 911 stores hardware stage identifiers mapped to corresponding hardware stages of the hardware pipeline 907. In response to a restart message received for the PRI handler 219 corresponding to a translation miss message previously sent to the PRI handler 219, the host interface circuit 608 may reintroduce the command packet into the hardware pipeline 907 from which the command packet was originally removed. In this way, DMA can continue (even if there is a slight delay during PRI-related processing) and the host interface circuit can be enabled to more efficiently continue processing commands and other actions that interact with the host memory 212.

[0137] In additional embodiments, the PRI handler 219 is coupled to or included within the host queue interface circuit 608B. In these embodiments, the PRI handler 219 is configured to send a page miss request to a translation agent (TA) of the host system in response to receiving a translation miss message, the page miss request including the virtual address and queue number of the SQ 306. The PRI handler 219 may be further configured to send a restart message to the host queue interface circuit 608B in response to receiving a page miss response, wherein the restart message also contains the queue number associated with the submission queue of the host system.

[0138] Fig.101 is a schematic block diagram of a system 1000 (e.g., a processing bridge system) including a processing bridge circuit 608N for implementing page request interface support when caching host memory address translation data in a memory subsystem according to some embodiments. In these embodiments, the system 1000 includes the processing bridge circuit 608N, a number of queues stored in a controller memory 1012 (e.g., of the controller 615), a PRI handler 219, and an AXI matrix 1023. In some embodiments, the controller memory 1012 is DRAM, but may also be a combination of SRAM or volatile memory. The controller memory 1012 may be coupled to the controller memory 912 ( Fig. 9 ) are the same. In these embodiments, the processing bridge circuit 608N is coupled between the controller memory 1012 and the AXI matrix 1023, which provides data and address communications to the host system 620, and may also access the address translation circuit 613 previously discussed herein. Reference to the host system 620 may be understood to include references to any other host system herein (including Figure 2 A reference to a host system 220).

[0139] In various embodiments, the PRI handler 219 is coupled to the controller memory 1012 and is optionally integrated with the processing bridge circuit 608N. As discussed, the PRI handler 219 may be processing logic that may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, an integrated circuit, etc.), firmware (e.g., instructions running or executed on the processor 217), or a combination thereof. The controller memory 1012 may store a number of different queues to facilitate communication between the PRI handler 219 and the processing bridge circuit 608N.

[0140] In an embodiment, for example, the controller memory 1012 stores a set of translation miss queues 1070 coupled to respective HIF circuits 608. For example, host interface circuit 608A may be coupled to HIF1 circuit translation miss queue 1070A, host interface circuit 608B may be coupled to HIF2 circuit translation miss queue 1070B, host interface circuit 608C may be coupled to HIF3 circuit translation miss queue 1070C, and further host interface circuits may be coupled to HIF4 circuit translation miss queue 1070D, where additional or fewer miss request queues are contemplated. Each queue in the set of translation miss queues 1070 receives a page miss message from the respective HIF circuit 608, where each page miss message indicates that a host address translation misses at an address translation cache (ATC) 616. The controller memory 1012 may further store an outbound queue 1072, a host request queue 1074, and a bridge translation miss queue 1075, which will be discussed in greater detail.

[0141] In various embodiments, the PRI handler 219 tracks activities associated with each page miss message received from a respective one of the host interface circuits 608 at the set of translation miss queues 1070. In these embodiments, the PRI handler 219 tracks translation miss messages received from the host interface circuit 608, each translation miss message including a virtual address of a miss at the cache (or ATC 616). The PRI handler 219 further removes duplicate translation miss messages having the same virtual address, for example, to avoid unnecessary sending of multiple duplicate page miss requests to the host system 620. The PRI handler 219 further creates a set of page miss requests from non-duplicate translation miss messages categorized into page request groups. Thus, the page miss request is associated with the page miss message received from the respective host interface circuit. The PRI handler 219 further queues the page request groups of the translation agent to be sent to the host system, for example, within the host request queue 1074.

[0142] In these embodiments, each page request group corresponds to, for example, a host interface circuit in host interface circuit 608. Each page miss request causes a translation agent (e.g., TA 617) to repin a physical page of memory to the virtual address of the corresponding page miss request. In at least some embodiments, a page miss request is a PCIe message TLP containing the virtual address of the page and optionally a process address space identifier (PASID). The PASID may be included when the host system 620 is executing a process (e.g., virtual machines VM(0) to VM(n)) that shares controller 615 or a memory device controlled by controller 615.

[0143] In some embodiments, each page requires its own page miss request, and for large I / O loads, there may be thousands of page miss requests, such as may be required to interact with the NIC. Each page miss request may invoke page handler software at the host system 620, which batches multiple page requests. To simplify the description, TA617 (which may be an IOMMU) may be understood to include this page handler software. In an embodiment, sending the same page address in multiple page miss requests may not trigger an error, but doing so may consume host CPU cycles.

[0144] In these embodiments, the PRI handler 219 further classifies the page miss requests into groups of page requests, for example, based on an incoming queue in the set of translation miss queues 1070 or another identifier, such as a hardware identifier of the source of the page miss request. The PRI handler 219 may further limit the page miss requests outstanding by a particular host interface circuit 608 to a threshold number of page miss requests. This may be arranged in firmware or as an actual limit based on the allocation of controller memory 1012 to the respective translation miss queues assigned to the particular host interface circuit 608.

[0145] In various embodiments, the outbound queue 1072 can buffer memory commands (e.g., host memory reads or host memory writes) for the host memory 212 of the host system 620 and other messages for the host system 620. The processing bridge circuit 608N can include a multiplexer 1011 that variably transfers page miss requests from the host request queue 1074 (e.g., page response groups) and one or more memory commands or other messages from the outbound queue 1072 to the outbound buffer 1014 of the processing bridge circuit 608N. The outbound buffer 1014 is thus coupled to the controller memory 1012 and is configured to buffer the page request groups and memory commands. In some embodiments, the multiplexer 1011 is part of the outbound queue handler logic 1002 of the processing bridge circuit 608N, which decodes packets from the outbound queue 1072 and the host request queue 1074.

[0146] In an embodiment, the queue handling logic 1002 further includes a multiplexer 1016 that determines what the outbound buffer 1014 contains and directs that content to the correct destination within the processing bridge circuit 608N. For example, if the outbound buffer 1014 contains a first page request group, the multiplexer 1016 causes page miss requests for the first page request group to be sent to a translation agent (e.g., TA 617) of the host system 620, which causes the translation agent to repin a physical page of memory to each respective virtual address of a respective page miss request.

[0147] 1008 , the queue handling logic 1002 sends the memory command with the address translation to the host system 620. However, if the queue handling logic 1002 determines that the outbound buffer 1014 contains a memory command (e.g., a read or write to the host memory 212), the multiplexer 1016 directs the memory command to the additional outbound queue handling logic 1002. In various embodiments, the queue handling logic 1002 requests an address translation for the virtual address of the memory command from the address translation circuit 613 (or ATS). For example, the address translation request may be buffered in the translation request queue 1004. In response to receiving the address translation from the address translation circuit 613, for example, via the translation response queue 1008, the queue handling logic 1002 sends the memory command with the address translation to the host system 620 as usual. However, in response to receiving a miss response from the address translation circuit 613, the queue handling logic 1002 discards the memory command and submits a page miss message to the bridge translation miss queue 1075. The PRI handler 219 may then process this page miss message along with page miss messages received through the set of translation miss queues 1070 and submit a corresponding page miss request (or page request group) for the translation miss into the host request queue 1074. Once the address translation returns from the TA 617 along with the page request response, such as via the inbound buffer 1054 (which will be discussed in more depth), the PRI handler 219 may resubmit the command packet (now including the address translation for the virtual address) to an outbound queue 1072, such as an outbound command queue.

[0148] In various embodiments, the queue translation logic 1002 buffers the read commands into the pending completion tracking queue 1060, which are also sent to the host system 620. The write commands and page miss requests may be sent to the host system 620 via the AXI translator 1028. The AXI translator 1028 may put out the write commands and page miss requests to the TA 617 at the host system 620 on a set of data lanes 1038 for writing.

[0149] In these embodiments, the read command may be sent to the host system 620 via the AXI converter 1022, which sends the read command to the TA 617 of the host system 620 through a set of address channels 1032 for reading. Further, in these embodiments, the pending completion tracking queue 1060 tracks incoming data received through a set of data channels 1034 for reading and through the AXI converter 1024 that passes incoming data for the read command. In an embodiment, the pending completion tracking queue 1060 detects data received for a particular read command being tracked and, once the data is completed, sends the data to the inbound completion buffer 1064 of the processing bridge circuit 608N. In at least one embodiment, the inbound completion buffer 1964 receives the data for a particular read command from the pending completion tracking queue 1060 and provides the data to the inbound completion queue 1080 stored in the controller memory 1012.

[0150] In various embodiments, the page request response returned from the TA 617 of the host system 620 is received into the AXI converter 1026 through a set of data channels 1036 for writing. The AXI converter 1026 passes the page request response (and other memory commands and messages, including invalidation requests) received from the host system 620. The processing bridge circuit 608N further includes an inbound buffer 1054 to receive the page request response and other memory commands and messages from the host system 620. In an embodiment, the inbound queue handling logic 1052 is coupled between the inbound buffer 1054 and the controller memory 1012.

[0151] The inbound queue handling logic 1052 may include a multiplexer 1056 to determine whether the inbound buffer 1054 includes a page request response, a memory command (for the controller memory 1012), a message, or an invalidation request. If it is an invalidation request, the multiplexer 1056 sends the invalidation request to the address translation circuit 613. The multiplexer 1056 sends the message, memory command, or page request response to the second multiplexer 1058, which further directs this content. The multiplexer 1058 may send the message and memory command to the inbound queue 1082 and send the page request response to the host response queue 1082 of the memory controller 1012. The host response queue 1084 may buffer the page request response received from the inbound buffer 1054. The inbound queue 1082 may buffer the incoming memory commands and other messages received from the inbound buffer 1054.

[0152] In at least some embodiments, the PRI handler 219 receives these page request responses corresponding to the page miss requests via the host response queue 1084. In some embodiments, each page request response indicates that the TA 617 has repinned a physical page of memory to the virtual address of the corresponding page miss request. The page request response may further contain an address translation for the virtual address in the original page miss request (or page request group), which may be provided to the address translation circuit 613 for storage in the ATC 616.

[0153] In various embodiments, each page request response may be a TLP message sent by the host system 620 to indicate a failure or success in repinning a group of pages or some individual pages. It is not an error for the TA 617 of the host system 620 to make a subset of pages corresponding to a page miss request resident instead of all pages. As a result of the functionality of the PRI handler 219 in conjunction with processing bridge circuit 608N, the PRI handler 219 may not be allowed to send an "untranslated address" message even after a successful page request response.

[0154] In the disclosed embodiment, the controller memory 1012 includes a set of restart queues 1090 that buffer restart messages from the PRI controller 219 corresponding to the page request responses buffered in the host response queue 1084. In an embodiment, each restart queue 1090 is coupled to a corresponding host interface circuit 608 to provide a corresponding restart message to the corresponding host interface circuit 608. More specifically, the host interface circuit 608A may be coupled to the HIF1 circuit restart queue 1090A, the host interface circuit 608B may be coupled to the HIF2 circuit restart queue 1090B, the host interface circuit 608C may be coupled to the HIF3 circuit restart queue 1090C, and further host interface circuits may be coupled to the HIF4 circuit restart queue 1090D. Each restart message placed in one of the set of restart queues 1090 by the PRI handler 219 may be provided from the controller memory 1012 to the corresponding host interface circuit 608. As shown in FIG. Fig. 9 As discussed, the restart message causes the host interface circuit 608 to reinsert the command packet into the hardware pipeline 907 from which the command packet had been originally removed.

[0155] Fig.111 is a flow chart of an example method 1100 for page request interface support when caching host memory address translation data in a memory subsystem according to some embodiments. The method 1100 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 1100 is performed by Figure 2 , Figure 6 and Fig. 9 PRI handler 219. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0156] At operation 1110, processing logic stores a plurality of address translations associated with address translation requests from a host interface circuit in a cache. Each address translation request includes a virtual address that needs to be translated.

[0157] At operation 1120, processing logic tracks translation miss messages received from the host interface circuitry, each translation miss message including a virtual address of a miss at the cache.

[0158] At operation 1130, processing logic removes duplicate translation miss messages having the same virtual address.

[0159] At operation 1140 , processing logic creates a plurality of page miss requests from the non-duplicate translation miss messages that are classified into page request groups, each page request group corresponding to a host interface circuit of the plurality of host interface circuits.

[0160] At operation 1150, processing logic queues the group of page requests for the translation agent to be sent to the host system. In an embodiment, the processing bridge circuit 608N then sends the queued group of page requests to the host system for processing.

[0161] Fig.121 is a flow chart of an example method 1200 for handling page request interface support in a host submission queue when caching host memory address translation data in a memory subsystem according to some embodiments. The method 1200 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 1200 is performed by Figure 6 and Fig. 9 The processes are performed by the host queue interface circuit 608B of the embodiment of the present invention. Although shown in a specific order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0162] At operation 1210, processing logic stores address translations with address translation requests from a host queue interface circuit in a cache (eg, ATC), each address translation including a virtual address.

[0163] At operation 1220, processing logic suspends command fetch arbitration on a submission queue of the host system that is a target of an address translation request that missed at the cache.

[0164] At operation 1230, processing logic triggers a page request interface (PRI) handler to send a page miss request to a translation agent of the host system, the page miss request including the virtual address of the address translation request. In an embodiment, the PRI handler is the PRI handler 219 discussed throughout this disclosure.

[0165] At operation 1240 , when the PRI handler receives the page miss response from the translation agent, processing logic receives a restart message from the PRI handler.

[0166] At operation 1250 , processing logic restarts command arbitration on the suspended submission queue in response to the restart message.

[0167] Fig.13 1 is a schematic block diagram of a system 1300 including a completion queue interface circuit 608C for implementing page request interface support when caching host memory address translation data in a memory subsystem according to some embodiments. In various embodiments, the system 1300 includes an ATS 613, a completion queue interface circuit 608C, and a host command automation (HCA) circuit 1301. As shown in FIG. Figure 6As discussed and illustrated, the ATS 613 may include an ATC 616 and translation logic 602 for determining address translations for host virtual addresses.

[0168] In various embodiments particularly relevant to handling NVMe by a host system, a completion queue (CQ) 308 ( Figure 3 ) is a circular buffer with a fixed slot size used to publish the status of completed commands. Completed commands can be uniquely identified by the combination of the associated SQ identifier and the command identifier assigned by the host software. Multiple submission queues (SQs) can be associated with a single CQ. In the case where a single worker thread processes all command completions via one CQ, this feature can be used even when those commands originate from multiple SQs. The CQ head pointer can be updated by the host software after it has processed the completion entry indicating the last free CQ entry.

[0169] In some embodiments, a phase (P) bit may be defined in the completion entry of CQ 308 to indicate whether the entry is newly posted without consulting the register. This enables host software to determine whether the new entry was posted as part of the previous or current round of completion notifications. Specifically, the controller 215 or 615 inverts the phase bit each time the round passes through the CQ position.

[0170] In at least some embodiments, the completion queue interface circuit 608C is configured to write a completion message (or notification) to the host system's completion queue (CQ) 308 after detecting that the controller 215 or 615 has completed handling of the corresponding host command. For example, as part of the NVMe command completion (e.g., the controller completes handling of a write command or a read command), the completion queue interface circuit 608C may send these completion messages or notifications to the host system in response to a CQ doorbell memory request. In these embodiments, even if the completion queue interface circuit 608C handles the report of command completion, even at this stage, the address translation request may still miss at the cache (e.g., ATC 616) of the ATS 613.

[0171] In these embodiments, HCA circuit 1301 arbitrates the completion of commands and coordinates sending CQ completion messages to the host system. To this end, HCA circuit 1301 may create a CQ completion message placed in CMB 300 (see Figure 3 ) or host memory 212 (see Figure 2 ) of the completion queue (CQ) 308. The HCA circuit 1301 may generate a message signal interrupt (e.g., MSI-X in NVMe) to remind the host system to process the entries in the CQ 308.

[0172] In various embodiments, the HCA circuit 1301 examines the operation code (opcode) of the incoming memory command to determine whether to use firmware or hardware automation for completion handling. If hardware automation, the HCA circuit 1301 can be configured to directly extract the command, perform the data transfer associated with the command, and insert the command identification metadata associated with the command into one of a set of completion request queues 1304 (respectively labeled P0_Wr, P0_Rd, P1_Wr, P1_Rd) to initiate completion. However, if the HCA circuit 1301 is to use firmware for handling, the HCA circuit 1301 can trigger the PRI handler 219 to provide (and / or generate) command identification metadata, which is inserted into a firmware completion request queue (labeled FW_Compl) in the set of completion request queues 1304 to initiate completion.

[0173] In at least some embodiments, the command metadata may include, for example, a command identifier (ID) and a host tag (htag). In these embodiments, the host system assigns a specific command identifier (ID) to a command buffered in an entry of the submission queue (SQ 306) and may also be used to identify a corresponding entry in the completion queue 308. In addition, the HCA circuit 1301 (or other host command circuitry operating similarly) may generate a host tag (or htag) for tracking completion requests within the completion queue interface circuit 308C. For ease of explanation, once queued in the set of completion request queues 1304, the command identification metadata will be referred to as a "completion request."

[0174] In these embodiments, multiplexer 1308 selects a pending completion request through hardware and logic of completion queue interface circuit 608C. At operation 1310, a command table read is performed to determine one or more parameters associated with a CQ entry (corresponding to a completion request) in order to generate a completion message or notification to the host system. At operation 1312, a pacing check is performed to see if pacing is enabled, e.g., meaning that entries into CQ 308 are to be paced through the completion queue at a particular rate or in a particular manner. For example, if CQ 308 is full, pacing may be triggered, which may be detected at operation 1316. If CQ 308 is full, the completion request within the hardware pipeline may be reinserted back into the original completion request queue in which the completion request was initiated. In some embodiments, an additional multiplexer may be employed that is triggered by a retry enable signal that is asserted if pacing is enabled.

[0175] After accounting for any pacing, at operation 1320, the completion queue interface circuit 608C may identify the current CQ tail (within the circular buffer) and increment the position associated with the memory command for which the arbitration is being completed. At operation 1322, the translation request circuitry 1302 of the completion queue interface circuit 608C issues a translation request to the ATS 613 via the translation request queue 904. The ATS 613 may provide a response to the translation request via the translation response queue 908. Figure 6 This address translation process is discussed in more detail.

[0176] In various embodiments, at operation 1324, the translation request circuitry 1302 may then detect an address translation request for the completion request that missed at the cache. The completion request is cached within a particular completion request queue in the set of completion request queues 1304 and is associated with an entry in the completion queue (CQ 308) of the host system. If the translation request misses at operation 1328, then at operation 1330, the translation request circuitry 1302 pauses the particular completion request queue in response to the detected miss, for example, via control of the multiplexer 1308. Additionally, at operation 1332, the translation request circuitry 1302 may further reinsert the completion request into the set of completion request queues 1304 in response to the detected miss, similar to what is performed if the CQ 308 is full.

[0177] In an embodiment, the translation request circuitry 1302 also triggers a page request interface (PRI) handler (e.g., PRI handler 219) to send a page miss request to a translation agent (TA) of the host system. The page miss request includes a virtual address of the address translation request. To trigger the PRI handler 219, the translation request circuitry 1302 may send a translation miss message to the PRI handler 219, the translation miss message including the virtual address and a command identifier associated with an entry in the completion queue. In these embodiments, when the PRI handler receives a page miss response from the translation agent (TA), the translation request circuitry 1302 receives a restart message from the PRI handler 219. At operation 1330, the translation request circuitry 1302 may restart a specific completion request queue in response to the restart message, for example, via control of the multiplexer 1308.

[0178] In some embodiments, the PRI handler 219 is one coupled to or included within the completion queue interface circuit 608C. The PRI handler 219 may be configured to send a page miss request to a translation agent (TA) of the host system in response to receiving a translation miss message, the page miss request including a virtual address and a command identifier associated with an entry in the completion queue. The PRI handler 219 may further send a restart message to the completion queue interface circuit 608C in response to receiving a page miss response, the restart message containing a host tag associated with a particular completion queue.

[0179] In an embodiment, the translation request circuitry 1302 detects a completion request recycled from the set of completion request queues and requests the address translation circuitry 613 to obtain an address translation corresponding to the virtual address from the host system. The translation request circuitry 1302 may further receive the address translation from the address translation circuitry 613 and use the address translation to generate a completion queue message associated with the completion request and targeted to an entry in the completion queue, as will be explained in additional detail.

[0180] In various embodiments, at operation 1350, the completion queue interface circuit 608C performs a tail pointer and phase update. For example, at operation 1340, the completion queue interface circuit 608C may read the incremental CQ tail buffer updated at operation 1320. In addition, at operation 1345, the completion queue interface circuit 608C may write the updated tail value to the tail pointer memory using the CQ command ID. The phase bit (discussed previously) may be updated at operation 1350 in order to update the phase. In some embodiments, the completion queue interface circuit 608C may update a latency first-in-first-out (FIFO) buffer 1352 based on tracking this phase information in a series of completion requests. Updating the FIFO buffer 1352 may be performed to assign each command for which completion is requested to be completed to a time bucket depending on the latency required to complete the command (from fetching the command to ringing when a completion message or notification is submitted). In an embodiment, the FIFO buffer 1352, the tail pointer buffer and / or memory, and the associated pointers may be stored in a register generally represented as control register 911 (see also Fig. 9 )'s local memory and / or registers or associated with local memory and / or registers.

[0181] In these embodiments, at operation 1360, the completion queue interface circuit 608C generates a completion queue message that is written into the completion queue (CQ 308). For example, the completion queue interface circuit 608C may determine a submission queue identifier that identifies a corresponding entry of the submission queue for the command, and generate a completion queue message that includes the command identifier, the submission queue identifier, and a completion request of the host tag. Additionally, at operation 1365, the completion queue interface circuit 608C generates an MSI-X (or similar MSI-related) message to alert the host system to process the entry in CQ 308.

[0182] Fig.14 14 is a flow chart of an example method 1400 for handling page request interface support in automation when caching host memory address translation data in a memory subsystem according to some embodiments. The method 1400 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 1400 is performed by Figure 6 and Fig.13 The processes of FIG. 608 and 609 are performed by the completion queue interface circuit 608C of FIG. 608C. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood to be examples only, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0183] At operation 1410 , processing logic detects an address translation request for a completion request that missed at a cache, the completion request being buffered within a particular completion request queue of the set of completion request queues and associated with an entry in a completion queue of the host system.

[0184] At operation 1420 , processing logic pauses the particular completion request queue in response to the detected miss.

[0185] At operation 1430 , processing logic triggers a page request interface (PRI) handler to send a page miss request to a translation agent of the host system, the page miss request including a virtual address of the address translation request.

[0186] At operation 1440 , when the PRI handler receives the page miss response from the translation agent, processing logic receives a restart message from the PRI handler.

[0187] At operation 1450, processing logic restarts the particular completion request queue in response to the restart message.

[0188] Fig.15 is a block diagram of a host interface circuit system 1500 including a pointer extraction interface circuit (PFIC) 1502 for retrieving pointers to support data transfer operations according to some embodiments. In some embodiments, the host interface circuit system 1500 is the host interface circuit system 603 and the PFIC 1502 is a reference Figure 6 608 discussed above. Thus, similar to that discussed with reference to host interface circuit 608, in embodiments to obtain translations of virtual addresses (VAs), PFIC 1502 is configured to interact with ATS 613 and thus with ATC 616 or 716 of ATS 613. In at least some embodiments, host interface circuit system 1500 includes pointer buffer 1510 to which PFIC 1502 is coupled. In embodiments, PFIC 1502 buffers pointers retrieved from host memory 212. In embodiments, pointer buffer 1510 is located in DRAM 222, SRAM 204 or 224, or memory device 130 (see Figure 2 ).

[0189] In various embodiments, as discussed, HDPA circuit 232 controls data transfer while activating control path 240 for extracting (or retrieving) pointer 310 ( Figure 3 ) (such as PRP / SGL, issuing completions and interrupts), and activating DMA 230 for actual data transfer between host system 220 and memory device 130 (see Figure 2 ). In an embodiment, the PFIC 1502 requests the ATC 616 to translate the virtual addresses of those pointers in order to ultimately determine the physical address destination (e.g., data transfer) of the memory commands residing in the submission queue (SQ) 306. In various embodiments, these memory commands are distinguished into logical block address (LBA) read and write commands and non-LBA read and write commands. In these embodiments, the LBA read / write commands are software-based addressing of a specific LBA address (e.g., linear address) associated with a virtual address of the host system 220, such as the host memory 212. In contrast, in some embodiments, non-LBA read and write commands do not have explicit LBA-based addresses and may be generated by legacy I / O devices coupled to the host system 220. For example, non-LBA read and write commands may be addressed using other non-linear addressing schemes, such as cylinder-head-sector (CHS), extended CHS, zone bit recording, or the like.

[0190] To facilitate retrieval of pointers in the event of a cache miss, in some embodiments, PFIC 1502 interacts with PRI handler 219 to send a translation miss message directly for an address translation miss at ATC 616 and receive a translation miss response from PRI handler 219 (see dashed line indicating an optional path). In these embodiments, PFIC 1502 triggers PRI handler 219 by sending a translation miss message to PRI handler 219. In an embodiment, the translation miss message includes a virtual address and a start write command (LBA or non-LBA) or a non-LBA read command (see Figures 16A to 17 ), or the starting point of the virtual address and chop command or the end of the LBA read command translation (see Figures 17 to 18 ).

[0191] In some embodiments, PFIC 1502 interacts with HCA circuit 1301 to support HDPA circuit 232, as HCA circuit 1301 interacts with PRI handler 219 to resolve address translation cache (ATC) 616 (see Figure 6 ) at the address translation miss. In an embodiment, as shown in reference Fig.13 As discussed, the HCA circuit 1301 checks the operation code (opcode) of the incoming memory command to determine whether to use firmware or hardware automation for completion handling. In the disclosed embodiment, the HCA circuit 1301 also arbitrates the completion of commands and coordinates sending CQ completion messages to the host system, as well as other host command automation tasks.

[0192] For example, in various embodiments in an effort to complete a memory command, the HCA circuit 1301 sends a pointer fetch request to the PFIC 1502 to cause the PFIC 1502 to retrieve a pointer for the memory command being processed. If during processing of a command and in accordance with some embodiments, the PFIC 1502 detects a translation miss at the ATC 616, the PFIC 1502 generates a translation miss message (which includes the virtual address that missed at the cache) and sends the translation miss message to the HCA circuit 1301. This translation miss message may trigger the HCA circuit 1301 to also generate a translation miss message to the PRI handler 219 and ultimately receive an address translation message back from the PRI handler 219. In response to the address translation message, in some embodiments, the HCA circuit 1301 commands the PFIC 1502 to reinsert the TU and continue with the pointer fetch for that memory command.

[0193] According to at least some embodiments, the PFIC 1502 is configured to send an address translation request 1520 for a translation unit (TU) to the ATC 616. As will be discussed in more detail, a TU contains a subset of pointers and therefore the PFIC 1502 sends an address translation request with an LBA or non-LBA corresponding to a virtual address (VA) at the beginning of the TU. Since the HDPA 232 operates at the TU level, this is sufficient in at least some embodiments.

[0194] According to some embodiments, the PFIC 1502 determines that the VA of a particular TU misses at the ATC 616. For example, the PFIC 1502 may determine that the VA of the TU misses at the ATC 616 based on a previous address translation miss for the same virtual address or due to receiving a non-acknowledge status (NAK) from the HDPA circuit 232, the latter of which will be discussed more below. In these embodiments, the PFIC 1502 sends a notification of the TU miss at the cache to the HCA circuit 1301, thereby triggering the HCA circuit 1301 to generate a translation miss message to the PRI handler 219. In other embodiments, the PFIC 1502 sends the translation miss message (dashed line) directly to the PRI handler 219. In at least some embodiments, pointers used to write data to the host memory 212 are not translated by the PFIC 1502, and the PFIC 1502 may translate only those pointers required for the PFIC 1502 to traverse the PRP / SGL that constitutes the pointer list. That is, the pointers contained within the SGL descriptor may not need to be translated by the PFIC 1502 , but may be translated by the HDPA circuit 232 .

[0195] In some embodiments, the HCA circuit 1301 provides a TU-sized command descriptor 1540 to the HDPA circuit 232, which is used to track the data transfer operation associated with the memory command for which the PFIC 1502 is retrieving a pointer. In some embodiments, the address translation request 1520 sent by the PFIC 1502 passes through or is otherwise visible to the HDPA circuit 232 so that the HDPA circuit 232 can track the state of the pointer required to perform the data transfer and obtain the pointer directly. For example, in an embodiment, the physical address (PA) returned from the ATC 616 upon a successful address translation 1530 is received in response to the address translation request 1520.

[0196] In some embodiments, upon receiving a failed address translation from the ATC 616, the HDPA 232 sends an address translation miss notification (e.g., a non-acknowledgement (NAK) of the associated command descriptor 1540) to the HCA circuit 1301. In various embodiments, upon receiving a translated address (e.g., PA) from the ATC 616 by the HDPA circuit 232, the HDPA circuit 232 also provides status in the form of an acknowledgement to the PFIC 1502. For example, a successful address translation may be reported as a zero ("0") (e.g., an acknowledgement (ACK)), and a failed address translation may be reported as a one ("1") (e.g., a non-acknowledgement (NAK)). The ACK status may trigger the PFIC 1502 to delete the pointers from the pointer buffer 1510 because those address translations have been successfully delivered to the HDPA circuit 232. The NAK status of a write command may trigger the PFIC 1502 to delete the pointers from the pointer buffer 1510 because the address translation for the write command will need to be reinserted. The NAK status of a read command may trigger storage of the TU outside of the pointer buffer 1510 (e.g., retaining an associated pointer), which must be at least partially reinserted after the PRI handler 219 resolves an address translation miss at the ATC 616, as will be discussed in more detail with reference to FIGS. 16 to 19.

[0197] Fig.16A 1 is a diagram of an example memory command 1600 (eg, a write command or a non-logical block address (LBA) read command) in response to a translation miss during a pointer fetch, according to some embodiments. Fig.16A In some embodiments, memory command 1600 is either an LBA write command, a non-LBA write command, or a non-LBA read command. As illustrated, memory command 1600 is not divided (e.g., chopped) and is therefore a series of consecutive translation units that each include a subset of pointers. In some embodiments, several such memory commands are executed sequentially, such as sequentially processing LBAs or bytes from a starting LBA to an ending LBA or covering a total number of bytes per memory command.

[0198] In some embodiments, in response to a miss in a cache (e.g., ATC 616) of a translation unit (TU) 1602 of a memory command 1600, the PFIC 1502 triggers the PRI handler 219 to send a page miss request to a translation agent (TA) of the host system 220. In an embodiment, in response to a translation miss, the PFIC 1502 sends a translation miss message to the HCA circuit 1301 or the PRI handler 219 to trigger handling of the translation miss, which in some embodiments may be queued in the HCA PRI queue. The translation miss message may include the virtual address of the TU 1602 (where the miss occurred) and the restart point at the TU 1601 (which is the beginning of the write command 1600). In some embodiments, the page miss request causes the translation agent to re-pin the physical page of the memory to the virtual address (VA). In these embodiments, the page miss response received back from the TA indicates that the physical page has been re-pinned. In some embodiments, in response to a cache miss of TU 1602, PFIC 1502 discards a pointer to the entire memory command 1600, which will be reprocessed later from the first TU 1601 (eg, the restart point at the beginning of the memory command 1600).

[0199] In at least some embodiments, to perform this discarding, each memory command that must be reprocessed, some of which have been discussed elsewhere in this disclosure, may be flushed from the command processing pipeline of the host interface circuitry 603 and / or 1500. Such flushing may be performed by continuing to execute the command processing pipeline, but without performing any actual data transfers. As a result, the pointer buffer 1510 and the queue of address translation requests 1520 between the PFIC 1502 and the HDPA 232 may be freed to handle further processing of subsequent memory commands.

[0200] In various embodiments, the PRI handler 219 collects page request addresses of translation units that are ordered after the translation unit that missed at the cache until the memory command is cleared from the host interface circuitry by, for example, a command end indicator that occurs at the end of the memory command 1600. The PRI handler 219 may further send a page request message for each respective page request address of the respective translation unit to the TA. The PRI handler 219 may further wait until the command end indicator is detected to cause the translation unit to be reintroduced into the pointer extraction interface circuitry from a restart point (e.g., the first TU 1601), for example, by tracking the PRI request according to a particular VA.

[0201] In some embodiments, a single page miss response is received back from the TA for the entire group of page requests associated with the corresponding TUs of the memory command 1600. In other embodiments, individual page miss responses are received back, one for each corresponding TU of the memory command 1600. Thus, in these embodiments, in response to receiving a page miss response from the TA for the TU 1602 that missed at the cache, the PRI handler 219 causes the memory command to be reprocessed, for example, by reinjecting the memory command into the HCA 1301, which sends a new pointer fetch request to the PFIC 1502. In some embodiments, this reintroduction reintroduces the memory command 1600 into the host queue interface circuit 608B ( Figure 6 In these embodiments, the PRI handler 219 may also clear the HCA circuit 1301 (e.g., descriptors or other tracking means associated with individual UTs) before reintroducing the memory command 1600 to the host queue interface circuit 608B, which restarts processing of the memory command 1600.

[0202] Fig. 16B is divided into chop commands and different actions of the PFIC 1502 (or other host interface circuitry) in response to a translation miss during a pointer fetch according to some embodiments. Fig.16A FIG. 16 is a diagram of an example memory command 1600 of FIG. 1600. In an embodiment, HCA circuit 1301 converts memory command 1600 ( Fig.16A ) is divided into a plurality of chop commands of a maximum data transfer size (MDTS). By way of example only, each chop command is a 2MB or 4MB (or similar MDTS) block of instruction information and each TU within the chop command is a pointer, such as a virtual address of a pointer, such as a value of 2KB or 4KB (or the like). The HCA circuit 1301 may provide an identifier for each chop command, such as "10" for the first chop command, "00" for any intermediate chop commands, and "01" for the last chop command in the memory command. The memory command 1600 includes four chop commands, but any given memory command may include more or fewer chop commands.

[0203] In an exemplary embodiment, the PFIC 1502 buffers the chop commands in the pointer buffer 1510 one TU at a time as the pointers are retrieved from system memory and provided to the HDPA 232 (which is illustrated as from left to right). In this manner, the PFIC 1502 keeps track of the virtual address of each pointer or TU so that data transfers are performed sequentially. Fig. 16BWhen the retrieval pointer is about halfway completed, the PFIC 1502 detects a translation miss at TU 1607 at about one-third of the second chop command. In an embodiment, in response to the translation miss, the PFIC 1502 sends a translation miss message to the HCA circuit 1301 or the PRI handler 219 to trigger handling of the translation miss, which in some embodiments may be queued to the HCA PRI queue. The translation miss message may include the virtual address of TU 1607 (where the miss occurred) and the restart point at TU 1601 (which is the beginning of the memory command 1600).

[0204] In addition, in at least some embodiments, PFIC 1502 discards the pointers buffered so far from pointer buffer 1510 and triggers an abort to retrieve the pointers for memory command 1600 because writes are handled sequentially, so memory command 1600 either completed concurrently or did not complete, and was therefore delayed. Specifically, a translation miss means that memory command 1600 will need to be restarted (or reprocessed) after obtaining the missed address translation. Therefore, in an embodiment, PFIC 1502 also sends an abort message to other circuit systems of host interface circuit system 1500 (including at least HCA circuit 1301 and / or PRI handler 219) for each chop command to communicate the abort to retrieve the pointers for the remainder of the chop command that has not yet been processed. For example, Fig. 16B The remainder of the chopping commands in the example embodiment of includes a third and final chopping command.

[0205] In an embodiment, in response to the translation miss message, the PRI handler 219 sends a page miss request to a translation agent (TA) of the host system, the page miss request including the virtual address. In some embodiments, the PRI handler 219 sends the page miss request to the translation agent to cause the translation agent to repin the physical page of the memory to the virtual address. In an embodiment, the page miss response from the TA indicates that the physical page has been repinned.

[0206] In at least some embodiments, in response to receiving a page miss response from the TA, the PRI handler 219 causes the memory command at the queue number in the submission queue to be reprocessed. In these embodiments, before the memory command is reprocessed, the PFIC 1502 sends an address translation request for the corresponding translation unit (TU) of the corresponding chop command for the subsequent memory command to the address translation circuit 613. In this way, the pointer buffer 1510 and other command processing queues remain active while waiting to reprocess the memory command 1600. In an embodiment, a firmware injection command to restart the processing of the memory command 1600 (after PRI processing) is injected into the host queue interface circuit 608B, which has been discussed in detail previously. However, the HCA circuit 1301 can receive an indication from the host queue interface circuit 608B to coordinate and control the completion of the memory command 1600.

[0207] In at least some embodiments, the HDPA 232 detects a translation miss for the TU 1605 in the first chop command and associated with the descriptor 1540 for which data transfer is being prepared. In these embodiments, the HDPA 232 sends a translation miss message to the HCA circuit 1301, which then notifies the PRI handler 219 to retrieve the correct physical address of the TU 1605. The translation miss message may include the virtual address of the miss at the ATC 616. The HDPA 232 may then take a further step to send a NAK message to the host interface circuitry (including the HCA circuit 1301) including the htag of this particular TU 1605. In these embodiments, the detection of a translation miss causes the HDPA 232 to also discard the associated descriptor and restart upon receiving a translation response from the PRI handler 219 or the HCA circuit 1301.

[0208] Fig.17 1 is a flow chart of an example method 1700 for handling a translation miss during a pointer fetch for a memory command according to some embodiments. The method 1700 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 1700 is performed by Fig.15 The pointer extraction interface circuit 1502 of the embodiment is executed. Although shown in a specific order or sequence, unless otherwise specified, the order of the process can be modified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0209] At operation 1710, processing logic buffers a plurality of pointers associated with a memory command in pointer buffer 1510. In some embodiments, the memory command resides in a submission queue of the host system. In some embodiments, the memory command is a write command or a non-LBA read command.

[0210] At operation 1720 , processing logic sends an address translation request for a corresponding translation unit of the memory command to the address translation circuit (ATS) 613 , each translation unit including a subset of the plurality of pointers.

[0211] At operation 1730, processing logic triggers the page request interface (PRI) handler 219 to send a page miss request to the translation agent of the host system when the address translation request of the translation unit for the memory command misses at the cache (e.g., ATC 616). In an embodiment, the page miss request includes the virtual address of the translation unit (or TU).

[0212] At operation 1740, processing logic discards the plurality of pointers from the pointer buffer 1510, for example due to having to later cause the memory command to be reinserted into the host interface circuitry for reprocessing.

[0213] Fig.18 is a diagram of an example LBA read command 1800 that has been divided into a chopping command and different actions of the PFIC (or other host interface circuitry) in response to a translation miss during a pointer fetch, according to some embodiments. Fig. 16B 1800, by way of example only, this example includes four chop commands because the HCA circuit 1301 can divide the LBA read command 1800 into fewer or more than four chop commands. In addition, in some embodiments, the pointer that retrieves the LBA read command 1800 advances from the first or starting chop command (the "10" chop command) to the end or last chop command (the "01" chop command).

[0214] exist Fig.18In the example of , the first chop command (Chop_0) starting at TU 1803 is processed without a translation miss, and PFIC 1502 detects a translation miss at TU 1805 midway through the second chop command (Chop_1). In some embodiments, in response to detecting this miss at ATC 616, PFIC 1502 causes a translation miss message to be sent to page request interface (PRI) handler 219, for example, optionally via HCA circuit 1301. In some embodiments, the translation miss message contains the virtual address of the translation unit and the restart point of the chop command. In some embodiments, the translation miss message triggers PRI handler 219 to send a page miss request to a translation agent (TA) of host system 220. In at least some embodiments, the translation miss message is triggered by PRI handler 219 to send a page miss request to a translation agent (TA) of host system 220 for a write command. Fig. 16B Unlike the embodiment of the present invention, the restart point is at the beginning of the second chop command (e.g., chop_1) or the logical block address (LBA) at the translation unit (TU) within the pointer buffer 1510. This restart point does not need to go back to the beginning of the LBA read command 1800 within the pointer buffer 1510 because the read operation is handled out of order and can be resumed from wherever the processing of the LBA read command 1800 was stopped.

[0215] Additional References Fig.18 In some embodiments, the pointer extraction interface circuit (PFIC) 1502 further causes one or more subsequent messages for subsequent chop commands following the chop command (e.g., for the third chop command (Chop_2) and the fourth chop command (Chop_3)) to be sent to the PRI handler 219. In some embodiments, the PFIC 1502 causes the subsequent messages to be sent from the HCA circuit 1301, as shown in FIG. Fig.15 In some embodiments, one or more subsequent messages each include the LBA of a corresponding subsequent chopping command within the pointer buffer 1510, such as the second (2 nd ) News and the third (3 rd ) messages. As illustrated, these messages contain a value indicating that a PRI is not required, and include the starting LBA for each corresponding subsequent chop command.

[0216] In some embodiments, the PRI handler 219 sends a page miss request to a translation agent (TA) to cause the translation agent to re-pin the physical page of the memory to the virtual address. In an embodiment, the page miss response indicates that the physical page has been re-pinned by the TA. Therefore, in an embodiment, the PRI handler 219 receives a page miss response from the translation agent. In an embodiment, the PRI handler 219 causes the pointer for which the address translation request hits at the cache to be reinserted in the pointer buffer 1510 in response to the page miss response. In some embodiments, the PRI handler 219 waits until the end of the last command (e.g., Chop_3) is received before starting to reinsert the pointer as just described. In some embodiments, when causing the pointer to be reinserted in the pointer buffer 1510, the PFIC 1502 maintains the chopping boundaries of the multiple pointers corresponding to the chopping command from the restart point.

[0217] In an embodiment, in response to the page miss response, the PRI handler 219 also causes a pointer corresponding to a chop command starting from the restart point of the pointer buffer 1510 identified by the page miss response to be inserted in the pointer buffer 1510. In some embodiments, while waiting for the page miss response and in response to the pointer buffer 1510 becoming full, the PFIC 1502 further stores the pointers of the plurality of pointers for which the address translation request hits at the cache (e.g., ATC 616) in the local memory (e.g., SRAM or DRAM). In this way, if necessary, these address translations will be readily available to be reinserted into the pointer buffer 1510 because the pointer buffer 1510 becomes too full to continue to hold that information.

[0218] Fig.19 1 is a flow chart of an example method 1900 for handling a translation miss during a pointer fetch for an LBA read command according to some embodiments. The method 1900 may be performed by processing logic, which may include hardware (e.g., a processing device, a circuit system, a dedicated logic, a programmable logic, a microcode, hardware of a device, an integrated circuit, etc.), software (e.g., instructions running or executed on a processing device), or a combination thereof. In some embodiments, the method 1900 is performed by Fig.15 The pointer extraction interface circuit 1502 of the embodiment is executed. Although shown in a specific order or sequence, unless otherwise specified, the order of the process can be modified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be performed in a different order, and some processes can be performed in parallel. In addition, one or more processes can be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0219] At operation 1910 , processing logic buffers a plurality of pointers associated with a plurality of chop commands in the LBA read commands residing in the submission queue of the host system in a pointer buffer 1510 of the host interface circuitry of the processing device.

[0220] At operation 1920, processing logic sends address translation requests for respective translation units for respective chop commands to address translation circuitry (eg, ATS 613), each translation unit including a subset of the plurality of pointers.

[0221] At operation 1930 , processing logic detects, at a cache (eg, ATC 616 ) of the address translation circuitry, an address translation request miss of a translation unit for a chop command of the plurality of chop commands.

[0222] At operation 1940, processing logic sends a translation miss message to the page request interface (PRI) handler 219. The translation miss message contains the virtual address of the translation unit and the restart point of the chop command. The translation miss message is used to trigger the PRI handler to send a page miss request to the translation agent of the host system.

[0223] Fig. 20 An example machine illustrating a computer system 2000 in which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed. In some embodiments, the computer system 2000 may correspond to a host system (e.g., Figure 1 1) a host system 120 that includes, is coupled to, or utilizes a memory subsystem (e.g., Figure 1 In some embodiments, the machine may be connected (e.g., using a network) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine may operate as a server or a client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0224] The machine may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Furthermore, while a single machine is described, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0225] The example computer system 2000 includes a processing device 2002, a main memory 2004 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) (e.g., synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), etc.), a static memory 2006 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 2018, which communicate with each other via a bus 2030.

[0226] The processing device 2002 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, or the like. More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets or a processor that implements a combination of instruction sets. The processing device 2002 may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The processing device 2002 is configured to execute instructions 2026 for performing the operations and steps discussed herein. The computer system 2000 may further include a network interface device 2008 for communicating over a network 2020.

[0227] The data storage system 2018 may include a machine-readable storage medium 2024 (also referred to as a computer-readable medium) having stored thereon one or more sets of instructions 2026 or software embodying any one or more of the methodologies or functions described herein. The instructions 2026 may also reside, completely or at least partially, within the main memory 2004 and / or within the processing device 2002 during execution by the computer system 2000, the main memory 2004 and the processing device 2002 also constituting machine-readable storage media. The machine-readable storage medium 2024, the data storage system 2018, and / or the main memory 2004 may correspond to Figure 1 Memory subsystem 110.

[0228] In one embodiment, instructions 2026 include instructions for implementing the Figure 1 The machine-readable storage medium 2024 is shown as a single medium in the example embodiment, but the term "machine-readable storage medium" should be considered to include a single medium or multiple media storing one or more sets of instructions. The term "machine-readable storage medium" should also be considered to include any medium capable of storing or encoding a set of instructions for machine execution and causing the machine to perform any one or more methodologies of the present disclosure. Therefore, the term "machine-readable storage medium" should be considered to include, but not limited to, solid-state memory, optical media, and magnetic media.

[0229] Some portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the art of data processing to most effectively convey the substance of their work to others skilled in the art. An algorithm is herein and generally considered to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulation of physical quantities. Typically, although not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. Mainly for common purposes, it sometimes proves convenient to refer to these signals as bits, values, elements, symbols, characters, items, numbers, or the like.

[0230] It should be remembered, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure may be directed to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other such information storage systems.

[0231] The present disclosure also relates to an apparatus for performing the operations described herein. This apparatus may be specially constructed for the intended purpose, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. This computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0232] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general purpose systems may be used with programs according to the teachings herein, or it may prove convenient to build more specialized equipment to perform the methods. The structures of a variety of these systems will appear as described in the following description. In addition, the present disclosure is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present disclosure as described herein.

[0233] The present disclosure may be provided as a computer program product or software that may include a machine-readable medium having instructions stored thereon, which instructions may be used to program a computer system (or other electronic device) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium, such as a read-only memory ("ROM"), a random access memory ("RAM"), a magnetic disk storage medium, an optical storage medium, a flash memory component, etc.

[0234] In the foregoing description, embodiments of the present disclosure have been described with reference to specific example embodiments thereof. It will be apparent that various modifications may be made to the present disclosure without departing from the broader spirit and scope of the embodiments of the present disclosure as set forth in the appended claims. Accordingly, the description and drawings should be regarded in an illustrative rather than a restrictive sense.

Claims

1. A memory subsystem controller, comprising: A host interface circuit system, which is used to interact with a host system and includes: a pointer extraction interface circuit coupled to the pointer buffer; and address translation circuitry coupled to the pointer fetch interface circuitry, the address translation circuitry to handle address translation requests for the host system and including a cache to store address translations; and The pointer extraction interface circuit is used to: buffering a plurality of pointers associated with memory commands in the pointer buffer, wherein the memory command is one of a write command or a non-logical block address (non-LBA) read command; sending address translation requests for corresponding translation units of the memory command to the address translation circuit, each translation unit including a subset of the plurality of pointers; When an address translation request of a translation unit for the memory command misses in the cache, triggering a page request interface PRI handler to send a page miss request to a translation agent of the host system, wherein the page miss request includes a virtual address of the translation unit; and The plurality of pointers are discarded from the pointer buffer.

2. The memory subsystem controller of claim 1 , further comprising the PRI handler coupled to the pointer extraction interface circuit, the PRI handler being configured to perform operations including: receiving a page miss response from the translation agent, wherein the page miss request is to cause the translation agent to repin a physical page of memory to the virtual address, and wherein the page miss response indicates that the physical page has been repinned; and Responsive to the page miss response, the memory command is caused to be reprocessed.

3. The memory subsystem controller of claim 2 , wherein when the memory command is a write command, causing the memory command to be reprocessed, the operation further comprising injecting the write command into a host queue interface circuit of the host interface circuit system, the host queue interface circuit being used to handle command fetch arbitration of a submission queue of the host system in which the write command resides.

4. The memory subsystem controller of claim 1 , wherein to trigger the PRI handler, the pointer extraction interface circuit is configured to send a translation miss message to the PRI handler, the translation miss message including the virtual address and a restart point at the beginning of the memory command.

5. The memory subsystem controller of claim 4, further comprising the PRI handler coupled to the pointer extraction interface circuit, the PRI handler being configured to perform operations including: In response to receiving the translation miss message, sending the page miss request to the translation agent of the host system, the page miss request including the virtual address; and In response to receiving the page miss response, the memory command is caused to be reprocessed from the restart point through the pointer fetch interface circuit.

6. The memory subsystem controller of claim 4, further comprising the PRI handler coupled to the pointer extraction interface circuit, the PRI handler being configured to perform operations including: collecting page request addresses of translation units ordered after the translation unit that missed at the cache until the memory command is cleared from the host interface circuitry by a command end indicator; sending a page request message for each corresponding page request address of the corresponding translation unit to the translation agent; and Waiting until the end-of-command indicator is detected to cause the translation unit to be reintroduced into the pointer extraction interface circuit from the restart point.

7. The memory subsystem controller of claim 1 , wherein the memory command comprises a plurality of chop commands, the translation request is directed to the corresponding translation unit of a corresponding chop command in the plurality of chop commands, and the translation unit sends the page miss request for the chop command that misses at the cache.

8. The memory subsystem controller of claim 7, wherein the pointer extraction interface circuit is further configured to: sending an abort message to other circuitry of the host interface circuitry regarding retrieving pointers to a remainder of the plurality of chop commands that have not yet been processed; and Before the memory command is reprocessed, an address translation request of a corresponding translation unit for a corresponding chopping command of a subsequent memory command is sent to the address translation circuit.

9. The memory subsystem controller of claim 1 , wherein the host interface circuit system further comprises a host command automation (HCA) circuit coupled to the pointer extraction interface circuit, the HCA circuit being configured to send a plurality of pointer extraction requests for the plurality of pointers to the pointer extraction interface circuit, wherein the pointer extraction interface circuit is configured to provide the virtual address to the HCA circuit, and wherein the HCA circuit is configured to send a translation miss message to the PRI handler that triggers the PRI handler to send the page miss request.

10. A memory subsystem controller, comprising: A host interface circuit system, which is used to interact with a host system and includes: a pointer extraction interface circuit coupled to the pointer buffer; and address translation circuitry coupled to the pointer fetch interface circuitry, the address translation circuitry to handle address translation requests for the host system and including a cache to store address translations; and The pointer extraction interface circuit is used to: buffering, in the pointer buffer, a plurality of pointers associated with a plurality of chop commands in a logical block address (LBA) read command residing in a submission queue of the host system; sending address translation requests for respective translation units for respective chopping commands to the address translation circuit, each translation unit including a subset of the plurality of pointers; detecting, at the cache, an address translation request miss of a translation unit for a chop command of the plurality of chop commands; and A translation miss message is caused to be sent to a page request interface PRI handler, wherein the translation miss message contains a virtual address of the translation unit and a restart point of the chop command, and the translation miss message is used to trigger the PRI handler to send a page miss request to a translation agent of the host system.

11. The memory subsystem controller of claim 10 , wherein the pointer extraction interface circuit is further configured to cause one or more subsequent messages for subsequent chop commands following the chop command to be sent to the PRI handler, the one or more subsequent messages each including a logical block address of a corresponding subsequent chop command within the pointer buffer.

12. The memory subsystem controller of claim 10, wherein the restart point is one of a start of the chop command or a logical block address at the translation unit within the pointer buffer, the memory subsystem controller further comprising the PRI handler coupled to the pointer extraction interface circuit, the PRI handler to perform operations comprising: receiving a page miss response from the translation proxy; and In response to the page miss response: causing some of the plurality of pointers for which address translation requests hit at the cache to be reinserted into the pointer buffer; and One of the plurality of pointers corresponding to chop commands identified by the page miss response starting from the restart point of the pointer buffer is caused to be inserted in the pointer buffer.

13. The memory subsystem controller of claim 12, wherein the operations further comprise storing in local memory the ones of the plurality of pointers for which address translation requests hit in the cache while waiting for the page miss response and in response to the pointer buffer becoming full.

14. The memory subsystem controller of claim 12, wherein in causing the ones of the plurality of pointers to be reinserted in the pointer buffer, the operations further comprise maintaining chopping boundaries of the plurality of pointers corresponding to the chopping commands starting from the restart point.

15. The memory subsystem controller of claim 12, wherein the page miss request is used to cause the translation agent to repin a physical page of memory to the virtual address, and wherein the page miss response indicates that the physical page has been repinned.

16. The memory subsystem controller of claim 10, wherein the page miss message comprises the virtual address, and the restart point comprises a logical block address at one of a start of the chop command or the translation unit within the pointer buffer.

17. The memory subsystem controller of claim 10 , wherein the host interface circuit system further comprises a host command automation (HCA) circuit coupled to the pointer extraction interface circuit, the HCA circuit being configured to send a plurality of pointer extraction requests for the plurality of pointers to the pointer extraction interface circuit, wherein the pointer extraction interface circuit is configured to provide the virtual address to the HCA circuit, and wherein the HCA circuit is configured to send the translation miss message that triggers the PRI handler to send the page miss request to the PRI handler.

18. A method comprising: buffering, in a pointer buffer of host interface circuitry of the processing device, a plurality of pointers associated with a plurality of chop commands of logical block address (LBA) read commands residing in a submission queue of the host system; sending address translation requests for respective translation units for respective chopping commands to address translation circuitry, each translation unit including a subset of the plurality of pointers; detecting, at a cache of the address translation circuit, an address translation request miss of a translation unit for a chopping command of the plurality of chopping commands; and A translation miss message is sent to a page request interface PRI handler, wherein the translation miss message contains a virtual address of the translation unit and a restart point of the chop command, and the translation miss message is used to trigger the PRI handler to send a page miss request to a translation agent of the host system.

19. The method of claim 18, further comprising sending one or more subsequent messages to the PRI handler for subsequent chop commands following the chop command, the one or more subsequent messages each comprising a logical block address of a respective subsequent chop command within the pointer buffer.

20. The method of claim 18, wherein the restart point is a logical block address at one of the beginning of the chop command or the translation unit within the pointer buffer, the method further comprising: Receiving, by the PRI handler, a page miss response from the translation agent; and In response to the page miss response: causing, by the PRI handler, some of the plurality of pointers for which address translation requests hit at the cache to be reinserted into the pointer buffer; and The PRI handler causes ones of the plurality of pointers corresponding to chop commands identified by the page miss response starting from the restart point of the pointer buffer to be inserted in the pointer buffer.

21. The method of claim 20, further comprising storing in local memory the ones of the plurality of pointers for which address translation requests hit at the cache while waiting for the page miss response and in response to the pointer buffer becoming full.

22. The method of claim 20, wherein causing the ones of the plurality of pointers to be reinserted in the pointer buffer comprises maintaining chopping boundaries of the plurality of pointers corresponding to the chopping commands starting from the restart point.

23. The method of claim 18, wherein the page miss message comprises the virtual address, and the restart point comprises a logical block address at one of a start of the chop command or the translation unit within the pointer buffer.

Images