Bandwidth allocation for storage system commands in a peer-to-peer environment

Abstract

Techniques are disclosed for allocating PCIe bus bandwidth to storage commands in a peer-to-peer environment. A non-volatile storage system has a peer-to-peer connection with a host system and a target device, such as a GPU. A memory controller in the storage system monitors a latency of a PCIe transaction performed over a PCIe bus in order to transfer data for an NVMe command. The PCIe transaction can involve direct memory access (DMA) of memory in the host system or the target device. Depending on which memory is being accessed and / or which communication link is used to access the memory, there can be significant differences in transaction latency. The memory controller allocates bandwidth on the PCIe bus to the NVMe command based on the latency of the PCIe transaction. In one aspect, the memory controller groups the PCIe addresses based on the latency of the PCIe transaction.

Description

Background Technology

[0001] This technology relates to the operation of non-volatile memory systems.

[0002] One type of non-volatile memory device is semiconductor memory. For example, non-volatile semiconductor memory is used in solid-state drives, mobile computing devices, non-mobile computing devices, and other memory systems. Typically, a memory system has a controller that controls data transfer between the memory system and a host system via a communication interface. The host system can be a computer system, a cellular phone, a server, etc. The memory system and the host system can exchange data via, for example, a peripheral computer interface Express (PCIe) bus. Non-volatile Memory Express (NVMe) is a logical device interface specification for accessing non-volatile memory devices attached via a PCIe bus. NVMe utilizes the parallelism provided by semiconductor memory, such as, but not limited to, solid-state drives.

[0003] To execute NVMe commands, the memory system can perform direct memory access (DMA) of the host system's memory. For example, for an NVMe write command, the memory controller can access data from the write data buffer in the host memory. For an NVMe read command, the memory controller can store data to read from the data buffer in the host memory. The memory controller can also access other areas in the host memory for control information. For example, the memory controller can access NVMe commands from the I / O (input / output) submission queue in the host memory.

[0004] In addition to the host system, the memory controller can also access the memory of another electronic device (such as a graphics processing unit (GPU)) to complete NVMe commands. For example, a data buffer may reside in the GPU memory. Therefore, the memory controller can directly access the GPU memory to complete NVMe commands.

[0005] The host system, GPU, and memory system can connect to the same communication bus (such as the PCIe bus). Therefore, the memory controller can issue, for example, PCIe transactions to access both host memory and GPU memory. The latency of PCIe transactions to GPU memory can differ from the latency of PCIe transactions to the host processor. Such latency differences can lead to poor utilization of the PCIe bus when PCIe bus utilization is high. Furthermore, bottlenecks can arise due to the mixing of PCIe transactions to host memory and GPU memory. For example, conflicts may occur in the data buffer of the PCIe root union. Attached Figure Description

[0006] Components with similar numbers refer to common parts in different drawings.

[0007] Figure 1A It is a block diagram of an implementation scheme for a storage system that connects to a host system and a target device.

[0008] Figure 1B This is a block diagram of one implementation scheme of the front-end processor circuit.

[0009] Figure 2A This is a block diagram of one implementation scheme for the back-end processor circuit.

[0010] Figure 2B This is a block diagram of one implementation scheme of a memory package.

[0011] Figure 3A This is a functional block diagram of one implementation scheme for a memory die.

[0012] Figure 3B This is a functional block diagram of one implementation of an integrated memory component.

[0013] Figure 4A A side view depicts one embodiment of an integrated memory assembly stacked on a substrate.

[0014] Figure 4B A side view depicts one embodiment of an integrated memory assembly stacked on a substrate.

[0015] Figure 5 This is a block diagram of a module implemented in the front end of one implementation of a memory controller.

[0016] Figure 6 This is a flowchart of an implementation of the process of allocating PCIe bus bandwidth to NVMe commands based on PCIe transaction latency.

[0017] Figure 7 This is a flowchart of an implementation scheme for the NVMe command processing procedure.

[0018] Figure 8A This is a flowchart of an implementation of a process for monitoring latency in the access I / O submission queue.

[0019] Figure 8B An implementation scheme for mapping I / O submission queues to delayed tables is described.

[0020] Figure 9A This is a flowchart of an implementation scheme for monitoring the delay in obtaining the PRP list.

[0021] Figure 9B An implementation scheme for mapping PRP list addresses to delayed tables is described.

[0022] Figure 10AThis is a flowchart of an implementation of a process for monitoring DMA latency in NVMe write commands.

[0023] Figure 10B An implementation scheme for mapping PCIe addresses to a latency table is described.

[0024] Figure 10C Another implementation of mapping PCIe addresses to a latency table is described.

[0025] Figure 11 This is a flowchart of an implementation scheme for monitoring DMA latency in NVMe read commands.

[0026] Figure 12 This is a flowchart of an implementation of the process of allocating PCIe bus bandwidth to NVMe commands based on PCIe transaction latency. Detailed Implementation

[0027] The techniques described herein relate to allocating communication bus bandwidth to storage commands used to access non-volatile memory in a non-volatile storage system. In one embodiment, PCIe bus bandwidth is allocated to NVMe commands in a peer-to-peer PCIe environment. The non-volatile storage system may have peer connections to a host system and a target device. The host system may be an NVMe host, which is a system capable of issuing NVMe commands to the storage system. The host system may have a central processing unit (CPU). The target device may be, for example, a graphics processing unit (GPU). In one embodiment, the host system is an NVMe host, and the GPU is an NVMe co-host. The NVMe co-host is capable of issuing NVMe commands to the storage system. For the sake of discussion, this document will discuss examples where the peer-to-peer PCIe environment has a CPU acting as an NVMe host and a GPU that may, but is not required to, be an NVMe co-host.

[0028] In one implementation, the memory controller in a non-volatile memory system monitors the latency of PCIe transactions executed to transfer data in response to NVMe commands. PCIe transactions may involve direct memory access (DMA) of memory in a CPU or GPU. Each PCIe transaction has an address in the PCIe bus address space. In one implementation, the memory controller groups PCIe addresses based on the latency of the PCIe transactions. The memory controller can then allocate PCIe bus bandwidth to NVMe commands based on the latency of the PCIe transactions executed to transfer data associated with the NVMe commands.

[0029] Transaction latency can vary significantly depending on the type of memory being accessed and / or the communication link used to access that memory. The memory controller allocates PCIe bus bandwidth to NVMe commands based on the latency of the PCIe transaction. For example, a PCIe transaction with higher latency may be delayed, at least temporarily, to benefit a PCIe transaction with lower latency. Therefore, the memory controller makes good use of PCIe bandwidth, especially when PCIe bus utilization is high.

[0030] Furthermore, bottlenecks can arise due to the mixing of PCIe transactions to host memory and GPU memory. For example, conflicts may occur in the data buffer of the PCIe root union. Allocating PCIe bandwidth to NVMe commands based on PCIe transaction latency reduces or eliminates conflicts in the data buffer of the PCIe root union. Therefore, bottlenecks on the PCIe bus are reduced or eliminated.

[0031] Figures 1A to 4B An example of a storage system that can be used to implement the techniques disclosed herein is described.

[0032] Figure 1A This is a block diagram of one embodiment of a storage system 100 connected to a host system 120 and a target device 140 via a communication bus. In one embodiment, the communication bus is a PCIe bus. The storage system 100 has a memory controller 102 and a memory package 104 containing non-volatile memory. In one embodiment, the target device 140 is a graphics processing unit (GPU). However, the target device 140 does not necessarily have to be a GPU. In one embodiment, the target device 140 is an NVMe co-host, meaning it is capable of issuing NVMe storage commands to the storage system 100. Therefore, in some cases, the target device 140 may be referred to as a co-host or an NVMe co-host. There may be more than one target device 140. Typically, the host system 120 and the target device 140 may each be referred to as an electronic device connected to a communication bus (e.g., a PCIe bus).

[0033] In one embodiment, storage system 100 communicates with host system 120 and target device 140 via a PCIe communication bus. In one embodiment, there is a first PCIe link between storage system 100 and host system 120, a second PCIe link between storage system 100 and target device 140, and a third PCIe link between host system 120 and target device 140. Host system 120 has a PCIe root union 126. The PCIe root union may include a root port. However, PCIe root union 126 may be external to host system 120. In one embodiment, PCIe switch 130 communicatively couples storage system 100 and target device 140 to host system 120 via PCIe root union 126 in host system 120. A PCIe switch is a device that connects two or more PCIe links. However, PCIe switch 130 is not mandatory. On the other hand, more than one PCIe switch 130 may be present.

[0034] In one embodiment, storage system 100, host system 120, and target device 140 are each PCIe devices. In one embodiment, storage system 100 is a first PCIe endpoint device, and target device 140 is a second PCIe endpoint device. Connection 132 between PCIe switch 130 and PCIe root union 126 can be a PCIe-based interface. Connection 134 between PCIe switch 130 and PCIe interface 150 in storage system 100 can also be a PCIe-based interface. Connection 136 between PCIe switch 130 and PCIe interface 146 in target device 140 can also be a PCIe-based interface. However, connections 134 and 136 may support fewer channels than connection 132. For example, connection 132 may support 32 channels, while connections 134 and 136 may each support eight channels.

[0035] In one embodiment, host system 120 issues a storage command to memory controller 102 in storage system 100 to access (e.g., read, write) non-volatile memory in memory package 104. In one embodiment, the storage command is an NVMe command. Host system 120 has the capability to make direct requests for transfers to / from memory package 104. In one embodiment, host system 120 acts as a host in the NVMe specification to send storage commands to memory controller 102. Therefore, in one embodiment, NVMe via PCIe is used.

[0036] In one embodiment, target device 140 issues a storage command (e.g., an NVMe command) to memory controller 102 to access (e.g., read, write) non-volatile memory in memory package 104. In one embodiment, storage system 100 and target device 140 have a peer-to-peer connection. The peer-to-peer connection allows target device 140 to make direct requests for transfers to / from memory package 104. In one embodiment, target device 140 acts as a co-host to send NVMe storage commands to memory controller 102; in this case, target device 140 may be referred to as a co-host or NVMe co-host. In one embodiment, target device 140 is a GPU.

[0037] The host system 120 includes a host processor 122, host memory 124, a PCIe root union 126, and a bus 128. The host memory 124 is the host's physical memory and can be DRAM, SRAM, non-volatile memory, or another type of storage device. The host system 120 is external to and separate from the storage system 100. In one embodiment, the storage system 100 is embedded within the host system 120.

[0038] Host memory 124 can be used with various data buffers and queues known in the NVMe protocol. The NVMe specification defines the command interface based on management queues (e.g., 124b), I / O commit queues (e.g., 124c), and I / O completion queues (e.g., 124d). Management queue 124b is used for tasks such as queue creation and deletion, device status queries, and feature configuration. I / O queues are used for all storage-related transfers, such as reads and writes. For example, host system 120 places an NVMe command on I / O commit queue 124c to instruct memory controller 102 to read from or write to non-volatile memory 104. Upon completion of the NVMe command, memory controller 102 places a command completion message in I / O completion queue 124d.

[0039] Data buffer 124a can be used to store data to be written to non-volatile memory 104, or to store data read from non-volatile memory 104. When writing data to non-volatile memory 104, memory controller 102 can perform DMA on the data from data buffer 124. When reading data from non-volatile memory 104, memory controller 102 can perform DMA on the data to data buffer 124.

[0040] The host memory buffer (HMB) 124e is a dedicated area of ​​host memory 124 configured for storing information in system 100. As an example, memory controller 102 can use HMB 124e to store host logical addresses into a physical address translation table (or a portion thereof). A physical address is an address in non-volatile memory 104 corresponding to a host logical address. Therefore, memory controller 102 may need to access HMB 124e to execute NVMe commands specifying a host logical address.

[0041] Target device 140 has a processor 142, target device memory 144, PCIe interface 146, and bus 148. Target device memory 144 is the physical memory of the PCIe device and can be DRAM, SRAM, non-volatile memory, or another type of storage device. Target device 140 is external to and separate from storage system 100. In one embodiment, target device 140 is a GPU.

[0042] In one implementation, target device memory 144 is used for data buffer 144a and / or I / O queues 144b, 144c. In one implementation, target device 140 uses I / O command queue 144b to issue NVMe storage commands to storage system 100. Storage system 100 can write command completion status to I / O completion queue 144c. In one implementation, host system 120 can create I / O queues 144b, 144c in target device 140. The storage device for such I / O queues can be a series of memories within the allocated base address register (BAR) space of target device 140. BAR space is a PCIe memory region allocated by the PCI enumerator at device initialization time and is accessible to all devices on a PCIe architecture. After queue creation, host system 120 can transmit memory allocation details to target device 140. Target device 140 can then autonomously execute NVMe commands from host system 120 using the NVMe protocol. According to some implementations, queue memory allocation information may be determined by target device 140 and transmitted from target device 140 to host system 120. Further details of a host system establishing an NVMe queue on a co-host are described in U.S. Patent 9,304,690, entitled "System and Method for Peer-to-Peer PCIe Storage Transfers," which is incorporated herein by reference.

[0043] The data buffer 144a in the target device memory 144 can be used to store data to be written to the non-volatile memory 104, or to store data read from the non-volatile memory 104. When writing data to the non-volatile memory 104, the memory controller 102 can perform DMA on the data from the data buffer 144a. When reading data from the non-volatile memory 104, the memory controller 102 can perform DMA on the data to the data buffer 144a.

[0044] The use of target device memory 144 is one example, but other possible scenarios exist. Optionally, target device memory 144 can be used to store HMB. It is not mandatory for target device 140 to have I / O queues 144b and 144c and data buffer 144a. In one embodiment, target device 140 has data buffer 144a but not I / O queues 144b and 144c. In this case, target device 140 does not necessarily operate as an NVMe co-host. That is, target device 140 does not need to issue NVMe commands to storage system 100.

[0045] In one embodiment, storage system 100 uses Direct Memory Access (DMA) to access host memory 124 and target device memory 144. Storage system 100 uses a common address space to communicate with other PCIe devices. In one embodiment, storage system 100 uses a PCIe address space to communicate with host system 120 and target device 140. Each PCIe device can be assigned a set of addresses in the PCIe address space. However, the internal (or local) memory addresses used by a PCIe device may differ from those in the PCIe address space. Therefore, translation between the PCIe address space and internal memory addresses is typically required.

[0046] PCIe root union 126 provides address translation services to redirect DMA transactions from one peer to another as needed, without requiring buffers within host memory 124. For example, when storage system 100 issues a transaction on a PCIe link to access memory in host system 120 or target device 140, storage system 100 includes addresses from the PCIe address space in the packets it transmits on the PCIe link. PCIe root union 126 performs PCIe address translation upon receiving a packet. That is, PCIe root union 126 translates the PCIe address from a transaction from storage system 100 into a physical (or internal) address in the target (e.g., host system 120, target device 140). However, PCIe root union 126 itself has limited buffer space. Therefore, conflicts between PCIe transactions may occur. Such conflicts can slow down data transfer rates between storage system 100 and other PCIe devices (e.g., host 120, target device 140). In one implementation, the memory controller 102 allocates PCIe bus bandwidth to NVMe commands based on the latency of PCIe transactions, which reduces or eliminates such conflicts. Therefore, the data transfer rate between the storage system 100 and other PCIe devices increases.

[0047] To execute a single NVMe command, the memory controller 102 may need to access several different areas in the host memory 124 and / or the target device memory 144. For example, the memory controller 102 may need to access the I / O commit queue (124c or 144b), HMB 124e, data buffer (124a or 144a), and I / O completion queue (124e or 144c) to execute a single read or write command. Therefore, several PCIe transactions may be required. There exists a specific order in which these PCIe transactions for a given NVMe are executed. Furthermore, one PCIe transaction may depend on another PCIe transaction. For example, the memory controller 102 may first need to access the host memory 124 to obtain a pointer to a data buffer that stores data to be written to the non-volatile memory 104 for an NVMe write command. Therefore, the high latency in obtaining the data buffer pointer will affect the DMA of the data from the data buffer.

[0048] The NVMe protocol does not separate or label PCIe addresses in any special way. Therefore, for memory controller 102, the PCIe translated address may appear to be the same as the direct physical address in host memory 124 or target device memory 144. Furthermore, NVMe transactions occur at a logical layer above PCIe. Therefore, ordering and dependency requirements in NVMe are not reflected at the PCIe layer. In some implementations, memory controller 102 monitors the latency of PCIe transactions. Based on the latency, memory controller 102 allocates PCIe bus bandwidth to NVMe commands. Therefore, by taking into account the different PCIe transactions that may be required to execute NVMe commands, memory controller 102 is able to increase throughput on the PCIe bus. Collisions in the PCIe root union 126 are also reduced or eliminated.

[0049] Storage system 100 can implement the techniques disclosed herein. Many different types of storage systems can be used with the techniques disclosed herein. An exemplary storage system is a solid-state drive (“SSD”); however, other types of storage systems may also be used. Storage system 100 includes a memory controller 102, a memory package 104 for storing data, and local memory (e.g., DRAM / ReRAM) 106. Memory controller 102 includes front-end processor circuitry (FEP) 110 and one or more back-end processor circuitry (BEP) 112. In one embodiment, FEP circuitry 110 is implemented on an ASIC. In one embodiment, each BEP circuitry 112 is implemented on a separate ASIC. The ASICs for each of the BEP circuitry 112 and FEP circuitry 110 are implemented on the same semiconductor, such that memory controller 102 is fabricated as a system-on-a-chip (“SoC”). Both FEP 110 and BEP 112 include their own processors. In one implementation, FEP 110 and BEP 112 operate in a master-slave configuration, where FEP 110 is the master and each BEP 112 is a slave. For example, FEP circuit 110 implements a flash translation layer that performs memory management (e.g., garbage collection, wear leveling, etc.), logic-to-physical address translation, communication with the host, DRAM (local volatile memory) management, and overall operation management of the SSD (or other non-volatile memory system). BEP circuit 112 manages memory operations within memory package 104 based on requests from FEP circuit 110. For example, BEP circuit 112 can implement read, erase, and program processes. Additionally, BEP circuit 112 can perform buffer management, set specific voltage levels required by FEP circuit 110, perform error correction (ECC), control the switching mode interface to the memory package, etc. In one implementation, each BEP circuit 112 is responsible for its own set of memory packages. Memory controller 102 is an example of control circuitry.

[0050] In one embodiment, there are multiple memory packages 104. Each memory package 104 may include one or more memory dies. In one embodiment, each memory die in the memory package 104 utilizes NAND flash memory (including two-dimensional NAND flash memory and / or three-dimensional NAND flash memory). In other embodiments, the memory package 104 may include other types of memory; for example, the memory package may include phase-change memory (PCM) memory.

[0051] Figure 1B This is a block diagram of one implementation scheme of FEP circuit 110. Figure 1BA PCIe interface 150 communicating with a host system 120 and a host processor 152 communicating with the PCIe interface are shown. The host processor 152 can be any type of processor known in the art suitable for implementation. The host processor 152 communicates with a network on-chip (NOC) 154. An NOC is a communication subsystem on an integrated circuit, typically between cores in a SoC. NOCs can span synchronous and asynchronous clock domains or use non-clocked asynchronous logic. NOC technology applies network theory and methods to on-chip communication and brings significant improvements compared to conventional bus and cross-switch interconnects. Compared to other designs, NOCs improve the scalability of SoCs and the power efficiency of complex SoCs. The wires and links of a NOC are shared by many signals. Because all links in a NOC can operate simultaneously on different data packets, a high degree of parallelism is achieved. Therefore, as the complexity of integrated subsystems increases, NOCs offer enhanced performance (such as throughput) and scalability compared to previous communication architectures (e.g., dedicated point-to-point signal lines, shared buses, or segmented buses with bridges). Connected to and communicating with the NOC 154 are the memory processor 156, SRAM 160, and DRAM controller 162. The DRAM controller 162 operates and communicates with the DRAM (e.g., DRAM 106). SRAM 160 is the local RAM used by the memory processor 156. The memory processor 156 runs the FEP circuitry and performs various memory operations. Two PCIe interfaces, 164 and 166, also communicate with the NOC. Figure 1B In one embodiment, the memory controller 102 includes two BEP circuits 112; therefore, there are two PCIe interfaces 164 / 166. Each PCIe interface communicates with one of the BEP circuits 112. In other embodiments, there may be more or fewer than two BEP circuits 112; therefore, there may be more than two PCIe interfaces.

[0052] Figure 2A This is a block diagram of one implementation scheme of BEP circuit 112. Figure 2A A diagram is shown for communicating with FEP circuit 110 (e.g., with...). Figure 1B PCIe interface 200 communicates with one of PCIe interfaces 164 and 166. PCIe interface 200 communicates with two NOCs 202 and 204. In one implementation, the two NOCs may be combined into one large NOC. Each NOC (202 / 204) is connected to SRAM (230 / 260), buffers (232 / 262), processor (220 / 250), and data path controller (222 / 252) via XOR engine (224 / 254) and ECC engine (226 / 256).

[0053] ECC engines 226 / 256 are used to perform error correction, as is known in the art. Hereinafter, ECC engines 226 / 256 may be referred to as the controller ECC engine. XOR engines 224 / 254 are used to perform XOR operations on data, enabling data to be combined and stored in a recoverable manner in the event of programming errors. In one implementation, XOR engines 224 / 254 can recover data that could not be decoded using ECC engines 226 / 256.

[0054] Data path controller 222 is connected to memory interface 228 for communicating with the integrated memory component via four channels. Therefore, top NOC 202 is associated with memory interface 228 for the four channels used to communicate with the integrated memory component, and bottom NOC 204 is associated with memory interface 258 for the four additional channels used to communicate with the integrated memory component. In one embodiment, each memory interface 228 / 258 includes four switching mode interfaces (TM interfaces), four buffers, and four schedulers. For each channel, there is one scheduler, one buffer, and one TM interface. The processor can be any standard processor known in the art. Data path controller 222 / 252 can be a processor, an FPGA, a microprocessor, or other type of controller. XOR engines 224 / 254 and ECC engines 226 / 256 are dedicated hardware circuitry referred to as hardware accelerators. In other embodiments, XOR engines 224 / 254 and ECC engines 226 / 256 can be implemented in software. The scheduler, buffers, and TM interfaces are hardware circuitry. In other embodiments, the memory interface (circuit for communicating with the memory die) can be... Figure 2A The different structures are depicted. Additionally, they have the same... Figure 1B and Figure 2A Controllers with different architectures can also be used with the techniques described in this article.

[0055] Figure 2B This is a block diagram of one embodiment of a memory package 104 including multiple memory dies 300 connected to a memory bus (data lines and chip enable lines) 318. The memory bus 318 is connected to a switching mode interface 228 for communication with the TM interface of the BEP circuit 112 (see, for example...). Figure 2AIn some embodiments, the memory package may include a small controller connected to the memory bus and the TM interface. In summary, the memory package 104 may have eight or 16 memory dies; however, other numbers of memory dies may also be implemented. The techniques described herein are not limited to any particular number of memory dies. In some embodiments, the memory package may also include a processor, a CPU device such as a RISC-V CPU, and a certain amount of RAM to help achieve some of the capabilities described below. The techniques described herein are not limited to any particular number of memory dies.

[0056] Figure 3A This is a block diagram illustrating an example of a memory die 300 that can implement the technology described herein. It can correspond to... Figure 2B One of the memory dies 300 includes a memory array 302, which may include any memory cells described below. The array terminal lines of the memory array 302 include various word line layers organized in rows and various bit line layers organized in columns. However, other orientations may also be implemented. The memory die 300 includes row control circuitry 320, the output of which is 308 connected to a corresponding word line of the memory array 302. The row control circuitry 320 receives a set of M row address signals and one or more various control signals from system control logic circuitry 360, and typically includes circuitry such as a row decoder 322, an array terminal driver 324, and a block select circuitry 326 for both read and write operations. The row control circuitry 320 may also include read / write circuitry. The memory die 300 also includes column control circuitry 310, which includes a sense amplifier 330, the input / output of which is 306 connected to a corresponding bit line of the memory array 302. Although only a single block is shown for array 302, the memory die may include multiple arrays that can be accessed individually. Column control circuitry 310 receives a set of N column address signals and one or more various control signals from system control logic unit 360, and typically includes circuitry such as column decoder 312, array terminal receiver or driver 314, block selection circuitry 316, and read / write circuitry and I / O multiplexers.

[0057] System control logic unit 360 receives data and commands from the host and provides output data and status to the host. In other embodiments, system control logic unit 360 receives data and commands from a separate controller circuit and provides output data to that controller circuit, wherein the controller circuit communicates with the host. In some embodiments, system control logic unit 360 may include a state machine 362 that provides die-level control for memory operations. In one embodiment, state machine 362 is programmable by software. In other embodiments, state machine 362 does not use software and is implemented entirely in hardware (e.g., circuitry). In another embodiment, state machine 362 is replaced by a microcontroller or microprocessor, wherein the microcontroller or microprocessor is on or outside the memory chip. System control logic unit 360 may also include a power control module 364 that controls the power and voltage supplied to the rows and columns of memory structure 302 during memory operations, and may include charge pump and regulator circuitry for generating regulated voltages. System control logic unit 360 includes a storage device 366 that can be used to store parameters for operating memory array 302.

[0058] Commands and data are transmitted between controller 102 and memory die 300 via memory controller interface 368 (also referred to as the "communication interface"). Memory controller interface 368 is an electrical interface for communicating with memory controller 102. Examples of memory controller interface 368 include a switching mode interface and an Open NAND Flash Interface (ONFI). Other I / O interfaces may also be used. For example, memory controller interface 368 may implement a switching mode interface that connects to the switching mode interface of memory interface 228 / 258 of memory controller 102. In one embodiment, memory controller interface 368 includes a set of input and / or output (I / O) pins connected to controller 102.

[0059] In some embodiments, all components of memory die 300 (including system control logic 360) may be formed as part of a single die. In other embodiments, some or all of the system control logic 360 may be formed on different dies.

[0060] For the purposes of this document, the phrase “one or more control circuits” may include a controller, state machine, microcontroller, microprocessor and / or other control circuits represented by system control logic unit 360, or other similar circuits for controlling non-volatile memory.

[0061] In one embodiment, memory structure 302 includes a three-dimensional memory array of non-volatile memory cells, wherein multiple memory stages are formed over a single substrate such as a wafer. The memory structure may include any type of non-volatile memory integrally formed in one or more physical stages of memory cells having active regions disposed over a silicon (or other type of) substrate. In one example, the non-volatile memory cells include vertical NAND strings with charge-trapping material.

[0062] In another embodiment, memory structure 302 includes a two-dimensional memory array of non-volatile memory cells. In one example, the non-volatile memory cells are NAND flash memory cells utilizing floating gates. Other types of memory cells (e.g., NOR flash memory) may also be used.

[0063] The exact type of memory array architecture or memory cell included in memory structure 302 is not limited to the examples described above. Many different types of memory array architectures or memory technologies can be used to form memory structure 302. Implementing the new embodiments claimed herein does not require a specific non-volatile memory technology. Other examples of suitable technologies for memory cells of memory structure 302 include ReRAM (Resistive Random Access Memory), magnetoresistive memory (e.g., MRAM, spin-torque MRAM, spin-orbit torque MRAM), FeRAM, phase-change memory (e.g., PCM), and so on. Examples of suitable technologies for memory cell architectures of memory structure 302 include two-dimensional arrays, three-dimensional arrays, cross-point arrays, stacked two-dimensional arrays, vertical bitline arrays, and so on.

[0064] One example of a ReRAM crosspoint memory includes reversible resistive switching elements arranged in a crosspoint array accessed by X-rays and Y-rays (e.g., word lines and bit lines). In another embodiment, the memory cell may include a conductive bridge memory element. A conductive bridge memory element may also be referred to as a programmable metallized cell. Based on the physical repositioning of ions within a solid electrolyte, the conductive bridge memory element can be used as a state-changing element. In some cases, the conductive bridge memory element may include two solid metal electrodes, one relatively inert (e.g., tungsten) and the other electrochemically active (e.g., silver or copper), with a thin film of solid electrolyte between the two electrodes. As temperature increases, ion mobility also increases, leading to a decrease in the programming threshold of the conductive bridge memory cell. Therefore, the conductive bridge memory element can have a wide range of programming thresholds across the entire temperature range.

[0065] Another example is magnetoresistive random access memory (MRAM), which stores data using magnetic storage elements. These elements are formed from two ferromagnetic layers separated by a thin insulating layer, each of which can remain magnetized. One of these layers is a permanent magnet set to a specific polarity; the magnetization of the other layer can be changed to match the magnetization of the memory by an external magnetic field. The memory device is constructed from a grid of such memory cells. In one implementation for programming, each memory cell is located between a pair of write lines arranged at right angles to each other, parallel to the cell, one above and one below. When current passes through them, an induced magnetic field is generated. MRAM-based memory implementations will be discussed in more detail below.

[0066] Phase-change memories (PCMs) utilize the unique properties of chalcogenide glasses. One embodiment uses a GeTe-Sb₂Te₃ superlattice to achieve a non-thermal phase transition by changing the coordination state of germanium atoms using only a laser pulse (or a light pulse from another source). Therefore, the programming dose is the laser pulse. Memory cells can be suppressed by preventing them from receiving light. In other PCM embodiments, memory cells are programmed by current pulses. It should be noted that the use of "pulse" in this document does not require a rectangular pulse, but includes (continuous or discontinuous) vibrations or pulse trains of sound, current, voltage, light, or other waves. These memory elements within the individual selectable memory cells or bits may include additional series elements as selectors, such as bidirectional threshold switches or metallic insulator substrates.

[0067] Those skilled in the art will recognize that the techniques described herein are not limited to a single particular memory structure, memory configuration, or material composition, but encompass many related memory structures within the technical essence and scope as described herein and as understood by those skilled in the art.

[0068] Can Figure 3A The components are divided into two parts: the memory structure 302 of the memory cells; and the peripheral circuitry, including all other components. A key characteristic of the memory circuitry is its capacity, which can be increased by increasing the area of ​​the memory die reserved for the memory structure 302 within the memory system 100; however, this reduces the area of ​​the memory die available for the peripheral circuitry. This can impose significant limitations on these peripheral components. For example, the need to mount sense amplifier circuitry within the available area can be a major constraint on sense amplifier design architecture. The reduced available area relative to the system control logic unit 360 may limit the available functionality that can be implemented on the chip. Therefore, a fundamental trade-off must be made between the amount of dedicated area for the memory structure 302 and the amount of dedicated area for the peripheral circuitry in the design of the memory die for the memory system 100.

[0069] Another area where memory structure 302 often conflicts with peripheral circuitry lies in the processing involved in forming these areas, as these areas typically involve different processing techniques and trade-offs when implementing different techniques on a single die. For example, when memory structure 302 is NAND flash memory, it is an NMOS structure, while the peripheral circuitry is typically CMOS-based. For instance, elements such as sense amplifier circuitry, charge pumps, logic elements in state machines, and other peripheral circuitry in system control logic unit 360 typically employ PMOS devices. The processing operations used to manufacture CMOS dies will differ in many ways from those optimized for NMOS flash NAND memory or other memory cell technologies.

[0070] To mitigate these limitations, the implementation scheme described below can... Figure 3A The components are separated onto individually formed dies, and then these dies are bonded together. More specifically, memory structure 302 can be formed on a single die, and some or all of the peripheral circuitry elements (including one or more control circuits) can be formed on separate dies. For example, a memory die can be formed solely of memory elements, such as flash NAND memory, MRAM memory, PCM memory, ReRAM memory, or other memory cell arrays of other memory types. Some or all of the peripheral circuitry (even including elements such as decoders and sense amplifiers) can then be moved to separate dies. This allows each die in the memory die to be optimized individually according to its technology. For example, a NAND memory die can be optimized for an NMOS-based memory array structure without worrying about CMOS elements now moved to a separate peripheral circuitry system die that can be optimized for CMOS processing. This provides more space for peripheral elements, and additional capabilities that might not be easily combined can now be incorporated if peripheral elements are confined to the edge of the same die that houses the memory cell array. Two dies can then be bonded together in a bonded multi-die memory circuit, with an array on one die connected to peripheral elements on another memory circuit. For example, although the following description will focus on a combined memory circuit with one memory die and one peripheral circuit die, other implementations may use more dies, such as two memory dies and one peripheral circuit die.

[0071] Figure 3B It shows Figure 3A An alternative arrangement of the arrangement can be implemented using wafer-to-wafer bonding to provide bonded die pairs. Figure 3BA functional block diagram of one embodiment of an integrated memory assembly 307 is depicted. The integrated memory assembly 307 can be used in a memory package 104 in a memory system 100. The integrated memory assembly 307 includes two types of semiconductor dies (or more simply, "dies"). A memory structure die 301 includes a memory structure 302. The memory structure 302 may contain non-volatile memory cells. A control die 311 includes control circuitry 360, 310, 320. In some embodiments, the control die 311 is configured to be connected to the memory structure 302 within the memory structure die 301. In some embodiments, the memory structure die 301 and the control die 311 are coupled together.

[0072] Figure 3B An example of peripheral circuitry is shown, including control circuitry formed in the peripheral circuitry or control die 311, which is coupled to the memory structure 302 formed in the memory structure die 301. General components are similar to... Figure 3A The system control logic unit 360, row control circuitry 320, and column control circuitry 310 are located in control die 311. In some embodiments, all or part of the column control circuitry 310 and all or part of the row control circuitry 320 are located on memory structure die 301. In some embodiments, some circuitry in the system control logic unit 360 is located on memory structure die 301.

[0073] System control logic unit 360, row control circuitry 320, and column control circuitry 310 can be formed using conventional processes (e.g., CMOS processes), making it possible to add elements and functions more commonly found on memory controller 102, such as ECC, with few or no additional process steps (i.e., the same process steps used to manufacture controller 102 can also be used to manufacture system control logic unit 360, row control circuitry 320, and column control circuitry 310). Therefore, while removing such circuitry from a die (e.g., memory structure die 301) reduces the number of steps required to manufacture such a die, adding such circuitry to a die (e.g., control die 311) may not require any additional process steps. Because some or all of the control circuitry 360, 310, and 320 are implemented using CMOS technology, control die 311 may also be referred to as a CMOS die.

[0074] Figure 3BA column control circuit 310, including a sense amplifier 350, is shown on a control die 311. This column control circuit is coupled to a memory structure 302 on a memory structure die 301 via an electrical path 306. For example, the electrical path 306 can provide electrical connections between the column decoder 312, driver circuitry 314, block selector 316, and bit lines of the memory structure 302. The electrical path can extend from the column control circuitry 310 in the control die 311 through pads on the control die 311 that bond to corresponding pads on the memory structure die 301 that connect to bit lines of the memory structure 302. Each bit line of the memory structure 302 can have a corresponding electrical path in the electrical path 306, including a pair of bonded pads connected to the column control circuitry 310. Similarly, a row control circuitry 320 (including a row decoder 322, an array driver 324, and a block selector 326) is coupled to the memory structure 302 via an electrical path 308. Each electrical path in electrical path 308 may correspond to a word line, a dummy word line, or a select gate line. Additional electrical paths may also be provided between the control die 311 and the memory structure die 301.

[0075] For the purposes of this document, the phrase "control circuitry" may include one or more of the following: memory controller 102, system control logic unit 360, column control circuitry 310, row control circuitry 320, microcontroller, state machine, and / or other control circuitry, or other similar circuitry for controlling non-volatile memory. Control circuitry may consist solely of hardware or a combination of hardware and software (including firmware). For example, a memory controller programmed by firmware to perform the functions described herein is an example of control circuitry. Control circuitry may include a processor, FGA, ASIC, integrated circuit, or other types of circuitry.

[0076] In some embodiments, the integrated memory assembly 307 contains more than one control die 311 and more than one memory structure die 301. In some embodiments, the integrated memory assembly 307 includes a stack of multiple control dies 311 and multiple memory structure dies 301. Figure 4A A side view of one embodiment of an integrated memory assembly 307 (e.g., a stack including control dies 311 and memory structure dies 301) stacked on a substrate 402 is depicted. The integrated memory assembly 307 has three control dies 311 and three memory structure dies 301. In some embodiments, there are more than three memory structure dies 301 and more than three control dies 311.

[0077] Each control die 311 is attached (e.g., bonded) to at least one memory structure die in the memory structure die 301. Some of the bonding pads 470, 474 are depicted. There may be more bonding pads. The space between the two bondsed dies 301, 311 is filled with a solid layer 448, which may be formed of epoxy resin or other resins or polymers. The solid layer 448 protects the electrical connection between the dies 301, 311 and further secures the dies together. Various materials can be used as the solid layer 448, but in this embodiment, the material may be Hysol epoxy resin from Henkel Corporation, which has offices in California, USA.

[0078] The integrated memory component 307 can be stacked, for example, in a stepped offset manner, such that the bonding pads at each stage are not covered and can be reached from above. Wire connections 406, attached to the bonding pads, connect the control die 311 to the substrate 402. Multiple such wire connections can be formed over the width of each control die 311 (i.e., formed to...). Figure 4A (on the page).

[0079] A through-silicon via (TSV) 412 for memory structure die 301 can be used to route signals through memory structure die 301. A through-silicon via (TSV) 414 for control die 311 can be used to route signals through control die 311. TSVs 412 and 414 can be formed before, during, or after the formation of integrated circuits in semiconductor dies 301 and 311. TSVs can be formed by etching holes through the wafer. These holes can then be lined with a barrier to prevent metal diffusion. The barrier layer can in turn be lined with a seed layer, and the seed layer can be plated with an electrical conductor, such as copper, although other suitable materials such as aluminum, tin, nickel, gold, doped polysilicon, and alloys or combinations thereof can be used.

[0080] Solder balls 408 may optionally be attached to contact pads 410 on the lower surface of substrate 402. Solder balls 408 may be used to electrically and mechanically couple integrated memory component 307 to host devices such as printed circuit boards. Solder balls 408 may be omitted if integrated memory component 307 will be used as an LGA package. Solder balls 408 may form part of the interface between integrated memory component 307 and memory controller 102.

[0081] Figure 4BA side view of one embodiment of an integrated memory assembly 307 stacked on a substrate 402 is depicted. The integrated memory assembly 307 has three control dies 311 and three memory structure dies 301. In some embodiments, there are more than three memory structure dies 301 and more than three control dies 311. In this example, each control die 311 is bonded to at least one memory structure die 301. Optionally, a control die 311 may be bonded to two memory structure dies 301.

[0082] Some of the bonding pads 470 and 474 are depicted. There may be more bonding pads. The space between the two joined dies 301 and 311 is filled with a solid layer 448, which may be formed of epoxy resin or other resins or polymers. Figure 4A Compared to the examples in, Figure 4B The integrated memory component 307 in the memory structure has no stepped offset. A through-silicon via (TSV) 412 for memory structure die 301 can be used to route signals through the memory structure die 301. A through-silicon via (TSV) 414 for control die 311 can be used to route signals through the control die 311.

[0083] Solder balls 408 may optionally be attached to contact pads 410 on the lower surface of substrate 402. Solder balls 408 may be used to electrically and mechanically couple integrated memory component 307 to host devices such as printed circuit boards. Solder balls 408 may be omitted if integrated memory component 307 will be used as an LGA package.

[0084] As briefly discussed above, the control die 311 and the memory structure die 301 can be bonded together. Bonding pads on each die 301, 311 can be used to bond the two dies together. In some embodiments, in a so-called Cu-Cu bonding process, the bonding pads are bonded directly to each other without solder or other additional material. In the Cu-Cu bonding process, the bonding pads are controlled to be highly flat and formed in a highly controlled environment that is essentially free of environmental particles that would otherwise deposit on the bonding pads and prevent a tight bond. Under these properly controlled conditions, the bonding pads are aligned and pressed against each other to form a bond based on surface tension. This bond can be formed at room temperature, although heat can also be applied. In embodiments using Cu-Cu bonding, the bonding pads can be approximately 5 μm square and spaced apart from each other at a pitch of 5 μm to 5 μm. Although this process is referred to herein as Cu-Cu bonding, the term can also be applied when the bonding pads are formed from materials other than copper.

[0085] When the area of ​​the bonding pads is small, it can be difficult to bond semiconductor dies together. The size and spacing of the bonding pads can be further reduced by providing a film layer on the surface of the semiconductor die, including the bonding pads. The film layer is disposed around the bonding pads. When the dies are placed together, the bonding pads can bond to each other, and the film layers on the individual dies can bond to each other. This bonding technique can be called hybrid bonding. In embodiments using hybrid bonding, the bonding pads can be approximately 5 μm square and spaced apart from each other with a pitch of 1 μm to 5 μm. Bonding techniques can be used to provide bonding pads with even smaller sizes and pitches.

[0086] Some embodiments may include a membrane on the surfaces of dies 301, 311. If such a membrane is not initially provided, the space between the dies may be underfilled with epoxy resin or other resins or polymers. The underfill material may be applied as a liquid and then allowed to harden into a solid layer. This underfilling step protects the electrical connection between dies 301, 311 and further secures the dies together. Various materials are available as underfill materials.

[0087] Figure 5 This is a block diagram of the components in the front-end 110 of one embodiment of the memory controller 102. The front-end 110 includes an address range identification module 502, an address range cache 504, a DMA engine 506, a command parser 508, and a bandwidth allocator 520. The address range identification module 502 includes a transaction latency measurement 510 and a transaction address grouper 512. The transaction latency measurement 510 measures the latency of a transaction on the communication bus between the storage system 100 and the host system 120 or the target device 140. In one embodiment, the transaction is a PCIe transaction via the PCIe bus. The transaction can be used for DMA of the host memory 124 or the target device memory 144. In one embodiment, for reading from the host memory 124 or the target device memory 144, the latency is the time between when the storage system 100 initiates a PCIe transaction and when the storage system 100 receives the data. In one embodiment, for writing to host memory 124 or target device memory 144, the latency is the time between when storage system 100 initiates a PCIe transaction to write data and when storage system 100 receives an acknowledgment of the write.

[0088] In one implementation, the transaction address grouper 512 groups the transaction addresses based on their respective delays. In another implementation, the transaction addresses are placed into buckets, where each bucket corresponds to a different delay range.

[0089] Address range identification module 502 writes to address range cache 504 to store transaction latency information. In one embodiment, address range cache 504 stores a mapping between PCIe addresses (or ranges of PCIe addresses) and PCIe transaction latency. For example, PCIe addresses may be grouped into different buckets based on their latency. In one embodiment, the PCIe address is the address of a data buffer (e.g., 124a, 144a).

[0090] In one implementation, command parser 508 parses NVMe commands. NVMe commands may include fields identifying the command type (e.g., read, write, etc.). NVMe commands may include an LBA field indicating the starting logical block address (LBA) for the read or write operation, and a length field indicating the length of the read or write operation. NVMe commands may also include pointers to data buffers, which are locations in host memory 124 (or target device memory 144) where data will be read (for writing) or data will be stored (for reading). NVMe commands may also include pointers to a list of Physical Region Pages (PRPs), which point to a list of PRP entries. A PRP entry may be a pointer to a physical memory page in host memory 124 (or target device memory 144). In one implementation, a PRP entry may be a 64-bit (e.g., 8-byte) physical memory page address. Each PRP entry points to a different data buffer. Therefore, there may be many data buffers for a read or write command. As an example, the PRP list may have 512 PRP entries. For example, the PRP list could be 4KB in size, with 512 PRP entries, each 64 bits long. In one implementation, the size of the physical memory pages in host memory 124 (or target device memory 144) is configured by host system 120. For example, host system 120 could specify that each physical memory page is 4KB, 8KB, 16KB, or some other size.

[0091] In one implementation, DMA engine 506 controls direct memory access (DMA) with respect to other PCIe devices on the PCIe bus. For example, DMA engine 506 controls DMA to / from host memory 124 and DMA to / from target device memory 144. In one implementation, DMA engine 506 uses PRP entries (i.e., data buffer pointers) to perform DMA. In some implementations, a single DMA may use multiple PRP entries, such as eight PRP entries. Therefore, the DMA can access eight data buffers (e.g., eight physical memory pages).

[0092] Bandwidth allocator 520 allocates PCIe bus bandwidth to NVMe commands. Bandwidth allocator 520 has a command cost estimate 522 and a command current limiter 524. Command cost estimate 522 estimates the cost of executing an NVMe command based on the latency of the PCIe transactions required to execute the NVMe command. Therefore, the cost is estimated based on the latency of transactions on the PCIe bus. Command current limiter 524 can limit the NVMe command based on the aforementioned cost. In one embodiment, command current limiter 524 uses an integral-based method, where integrals are allocated to the NVMe command. In one embodiment, if the cost of executing an NVMe command (in terms of data transfer across the PCIe bus) is too high for a given current PCIe bus usage, the NVMe command is current-limited. That is, the PCIe bus is then used for other NVMe commands.

[0093] Address cache 504 is a non-transitory storage device. Address cache 504 may include volatile memory or non-volatile memory. Various modules (address range identification module 502, DMA engine 506, command parser 508, and bandwidth allocator 520) are implemented in software only, hardware only, or a combination of hardware and software (including firmware). For example, the host processor in front-end 110 ( Figure 1B (152) The module's functions can be programmed by firmware. The module can be implemented using a processor, FGA, ASIC, integrated circuit, or other types of circuitry.

[0094] Figure 6 This is a flowchart of one implementation of a process 600 for allocating PCIe bus bandwidth to NVMe commands based on PCIe transaction latency. Process 600 can be executed by memory controller 102. More generally, process 600 can be applied to a first communication protocol and a second communication protocol (e.g., NVMe), where the first communication protocol can be a physical layer protocol (e.g., PCIe) and the second communication protocol can be a logical layer protocol higher than the first communication protocol.

[0095] Step 602 includes the memory controller 102 determining the latency of a PCIe transaction for direct access to memory in a PCIe device connected to the PCIe bus via the PCIe bus. The PCIe transaction is used to access the memory of the PCIe transaction in order to execute an NVMe command. The PCIe transaction may include a data payload transaction and a data control transaction. A data payload transaction involves DMA to / from memory of another PCIe device, reading data from or writing data to non-volatile memory 104. A data control transaction involves DMA to / from memory of another PCIe device, containing control information required to complete the execution of an NVMe command, but does not include DMA of the data payload. Relative to Figure 7 , Figure 8A , Figure 9A , Figure 10A and Figure 11 Additional details on determining the latency of PCIe transactions are shown and described.

[0096] Step 604 includes the memory controller 102 constructing a table storing the PCIe transaction latency of the PCIe bus addresses. The table can store latency information in various ways. In one embodiment, PCIe addresses are placed into “buckets,” where each bucket corresponds to a different latency range. In the most basic example, there are two buckets. One bucket contains PCIe addresses of data buffers with latency below a threshold, and the other bucket contains PCIe addresses of data buffers with latency above the threshold. Any number of buckets can exist. In one embodiment, the PCIe addresses include the addresses of data buffers (e.g., 124a, 144a). However, PCIe addresses may correspond to I / O queues (e.g., 124c, 124d, 144b, 144c) or other locations of control data.

[0097] Step 606 includes the memory controller 102 allocating PCIe bus bandwidth to NVMe commands based on PCIe transaction latency. The purpose of allocating bandwidth to NVMe commands is to allow underlying PCIe transactions to occur on the PCIe bus in order to transfer data associated with the NVMe commands. Therefore, the phrase "allocating PCIe bus bandwidth to NVMe commands," etc., implies allowing one or more underlying PCIe transactions necessary to transfer data associated with NVMe commands.

[0098] In one implementation, step 606 includes estimating the cost of transferring the data associated with the NVMe command across the PCIe bus based on the latency of the underlying PCIe transaction required to transfer the data associated with the NVMe command. The underlying PCIe transaction may include both data payload transactions and data control transactions. However, in one implementation, only PCIe transactions for data payload transactions are considered in terms of cost. The NVMe command is then scheduled for the PCIe bus based on cost. The phrase "schedule NVMe commands for the PCIe bus" refers to when a PCIe transaction is allowed to transfer data for an NVMe command (relative to other NVMe commands). For example, NVMe commands with costs greater than a threshold may be rate-limited, where underlying PCIe transactions for low-cost NVMe commands are allowed to execute first.

[0099] In one implementation, step 606 includes allocating PCIe bus bandwidth based on the dependency between data payload transactions and data control transactions. For example, a memory controller might need to execute a control transaction to read a PRP list, after which DMA to / from a data buffer can be performed. Therefore, such dependencies can be taken into account in the scheduling of NVMe commands for the PCIe bus. It should be noted that bandwidth allocation would not be as efficient if scheduling were based solely on PCIe transactions without considering such dependencies.

[0100] As described above, the memory controller 102 monitors PCIe transaction latency when processing NVMe commands. Figure 7 This is a flowchart of one embodiment of the NVMe command processing procedure 700. Procedure 700 will be described to provide an overview of how PCIe transaction latency is monitorable during NVMe command execution. Typically, procedure 700 is divided into data control transactions and data payload transactions. In one embodiment, the memory controller 102 distinguishes the latency of data payload transactions from that of data control transactions. In one embodiment, the latency of data payload transactions is weighted more heavily than the latency of data control transactions in PCIe bus bandwidth allocation. The steps in procedure 700 are described in a specific order for ease of interpretation. However, some steps may be performed multiple times for a single NVMe command. The steps may be performed in a different order than depicted.

[0101] In one implementation, process 700 is initiated when the host system 120 or the target device 140 rings the NVMe doorbell. That is, the host system 120 or the target device 140 may write to registers in the storage system 100 to indicate that an NVMe command has been placed on the I / O commit queue. Step 702 includes the memory controller 102 reading the command from the I / O commit queue. This could be I / O commit queue 124c in the host memory 124 or I / O commit queue 144b in the target device memory 144. Therefore, step 702 includes DMA of the memory of the host system 120 or the target device 140. DMA involves a PCIe transaction with a PCIe address. In one implementation, the memory controller 102 measures the latency of this PCIe transaction. Step 702 is an example of a data control transaction.

[0102] Step 704 includes determining whether the NVMe command includes a pointer to a PRP list. If so, the PRP list is obtained in step 706. The PRP list may be in host memory 124 or target device memory 144. Therefore, step 706 includes DMA of the PCIe device's memory. DMA involves PCIe transactions with PCIe addresses. As described above, the PRP list contains PRP entries, each of which points to a data buffer. The data buffer may be in host memory 124 or target device memory 144. The data buffer may be in a different PCIe device than the PRP list. For example, the PRP list may be in host memory 124, while the data buffer may be in target device memory 144. In one embodiment, the memory controller 102 measures the latency of this PCIe transaction. Step 706 is an example of a data control transaction.

[0103] Step 708 includes determining whether the NVMe command includes data transfer bits. For example, this determines whether it is a read command or a write command. The path where the NVMe command is a read command will be discussed first. Step 710 includes the memory controller 102 initiating execution of the read command. Note that this refers to internal execution including reading data from the non-volatile memory 104. Note that step 710 may occur before obtaining the list of all PRPs in step 706. Step 710 may include determining the physical address in the non-volatile memory 104 corresponding to the logical address in the NVMe command. To make this determination, the memory controller 102 may access the HMB 124e in the host memory 124. That is, the memory controller 102 may use the HMB 124e to store the LTP (Logical Address to Physical Address) mapping. Therefore, step 710 may include DMA of the host memory 124. DMA involves a PCIe transaction with a PCIe address. In one embodiment, the memory controller 102 measures the latency of this PCIe transaction. Step 710 is an example of a data control transaction. However, in some cases, the storage system 100 may contain the required LTP mapping in, for example, DRAM 106. Therefore, access to HMB 124e is not required in step 710. Step 710 also includes the memory controller 102 reading data from non-volatile memory 104. The memory controller 102 may perform error correction on the data and temporarily store the data in volatile memory (e.g., DRAM 106) within the storage system 100.

[0104] Step 712 includes writing a payload via PCIe to a data buffer identified by the PRP entry. The payload contains data read from non-volatile memory 104. The data buffer may be in host memory 124 or target device memory 144. Therefore, step 712 includes one or more DMAs to the memory of the PCIe device. Each DMA involves a PCIe transaction with a PCIe address. In one embodiment, memory controller 102 measures the latency of each PCIe transaction. Step 712 is an example of a data payload transaction.

[0105] Step 714 involves the memory controller 102 writing the command completion status to the I / O completion queue. This could be I / O completion queue 124d in host memory 124 or I / O completion queue 144c in target device memory 144. Typically, this would be the same PCIe device storing the I / O commit queue for this NVMe command. Therefore, step 714 includes DMA of the PCIe device's memory. DMA involves a PCIe transaction with a PCIe address. In one embodiment, the memory controller 102 measures the latency of this PCIe transaction. Step 714 is an example of a data control transaction.

[0106] Step 716 includes the memory controller 102 sending a command interrupt. The command interrupt may be sent to either the host system 120 or the target device 140, depending on the location of the I / O queue for this NVMe command. This concludes the processing of the NVMe read command.

[0107] Next, the processing of NVMe write commands will be discussed. Referring now to step 718, the memory controller 102 reads the payload from the address identified by the PRP via PCIe. Note that step 718 may begin before reading the entire PRP list in step 706. Therefore, in step 718, the data buffer identified by the PRP entry is read. The data buffer may be in host memory 124 or target device memory 144. Thus, step 718 includes one or more DMAs from the memory of the PCIe device. Each DMA involves a PCIe transaction with a PCIe address. Data may be temporarily stored in volatile memory within storage system 100. In one embodiment, the memory controller 102 measures the latency of each PCIe transaction. Step 718 is an example of a data payload transaction.

[0108] Step 720 includes the memory controller initiating NVMe command execution. Note that this refers to internal execution including writing data to non-volatile memory 104. Similar to an NVMe read command, the memory controller 102 can use the LTP mapping in HMB 124e to determine the physical address in non-volatile memory 104 where data is written. Therefore, step 720 may include DMA of host memory 124. DMA involves a PCIe transaction with a PCIe address. In one embodiment, the memory controller 102 measures the latency of this PCIe transaction. Step 720 is an example of a data control transaction. However, access to HMB 124e is not required in step 720. Step 720 also includes the memory controller 102 writing data to non-volatile memory 104. After step 720, the memory controller executes steps 714 and 716, which are similar to those discussed above for NVMe read commands. This concludes the processing of the NVMe write command.

[0109] Next, the processing of NVMe commands other than read or write will be discussed. Referring now to step 722, memory controller 102 executes the NVMe command. Typically, the execution of step 722 does not involve memory access to host memory 124 or target device memory 144. After step 722, the memory controller executes steps 714 and 716. As discussed above, step 714 includes writing the command completion status to an I / O completion queue, which may include PCIe transactions and DMA to memory of a PCIe device. In one embodiment, memory controller 102 measures the latency of this PCIe transaction. As discussed above, step 714 is an example of a data control transaction. Step 716 includes sending a completion interrupt, as discussed above.

[0110] Figure 8A This is a flowchart of one implementation of a process 800 for monitoring latency in accessing I / O commit queues. Process 800 is executed in one implementation of step 702 of process 700. Process 800 is one implementation for monitoring latency in data control transactions. Process 800 can be executed individually for each I / O commit queue in host system 120 and for each I / O commit queue in target device 140.

[0111] Step 802 includes the memory controller 102 issuing a PCIe transaction for a direct access (e.g., DMA) NVMe command from the NVMe I / O commit queue. In one embodiment, the memory controller 102 sends a packet to a PCIe switch 130, which forwards the packet to a PCIe root union 126. The packet may contain a PCIe address. The PCIe root union 126 may perform address translation. If the PCIe address is in the BAR of the target device 140, the PCIe root union 126 may forward the packet to the PCIe interface 146 in the target device 140. The result of step 802 is a DMA from memory of a PCIe device (e.g., host system 120 or target device 140) to storage system 100.

[0112] Step 804 includes the memory controller 102 recording the delay of the PCIe transaction. In this case, the delay is the time between when the memory controller 102 issues the PCIe transaction and when the memory controller 102 receives the data (i.e., the NVMe command).

[0113] The memory controller 102 may continue monitoring PCIe transactions for this I / O submission queue until the memory controller determines that a sufficient number of measurements have been performed to achieve the target accuracy (in step 806). The memory controller 102 may perform a fixed number of measurements, which may be as few as a single measurement. Therefore, steps 802 and 804 may be repeated for other NVMe commands on this I / O submission queue. In step 808, the memory controller 102 records the latency value for this I / O submission queue. The value may be, for example, an average (e.g., mean, median) latency. If the mean is used, outlier values ​​(e.g., very long, very short) may be ignored. In one embodiment, the value is recorded in an address range cache 504. After step 808, the process may return to step 802 to continue monitoring the latency so that the value is updated in step 808.

[0114] Figure 8BAn implementation of Table 850, which maps I / O commit queues to latency, is depicted. Table 850 has columns labeled as addresses, containing entries for each I / O commit queue. Each commit queue can be identified by its PCI address or in another way. For example, NVMe can be used to identify each I / O commit queue. For example, NVMe can have a unique identifier for each I / O commit queue, which can be used instead of a PCIe address. The latency column has entries for each I / O commit queue to specify the latency of that I / O commit queue. As mentioned above, the latency of each queue can be determined based on "n" measurements. Latency can be updated frequently, but this is not mandatory. The cost column has the cost of a PCIe transaction, which is based on latency. Therefore, a longer latency will correspond to a higher cost. In one implementation, cost is used when allocating bandwidth.

[0115] Figure 9A This is a flowchart of one implementation of process 900 for monitoring the delay in obtaining the PRP list. Process 900 is executed in one implementation of step 706 of process 700. Process 900 is one implementation for monitoring the delay in data control transactions.

[0116] Step 902 includes the memory controller 102 issuing a PCIe transaction for a direct access (e.g., DMA) PRP list. The PRP list resides in either host memory 124 or target device memory 144. In one embodiment, the memory controller 102 sends a packet to a PCIe switch 130, which forwards the packet to a PCIe root union 126. The packet may contain a PCIe address corresponding to the location where the PRP list resides in the memory (e.g., host memory 124 or target device memory 144). The PCIe root union 126 may perform address translation. If the PCIe address is in the BAR of the target device 140, the PCIe root union 126 may forward the packet to PCIe interface 146 in the target device 140. The result of step 902 is a DMA from memory of a PCIe device (e.g., host system 120 or target device 140) to storage system 100.

[0117] Step 904 includes the delay of this PCIe transaction in which the memory controller 102 records this PRP list. In this case, the delay is the time between when the memory controller 102 issues the PCIe transaction and when the memory controller 102 receives the data (i.e., the PRP list).

[0118] The memory controller 102 may continue monitoring PCIe transactions against the PRP list until the memory controller determines that a sufficient number of measurements have been performed to achieve the target accuracy (in step 906). The memory controller 102 may perform a fixed number of measurements, which may be as few as a single measurement. Therefore, steps 902 and 904 may be repeated for other PRP lists. In step 908, the memory controller 102 records the latency values ​​for the PRP lists. As mentioned above, there may be PCIe addresses corresponding to each PRP list. Values ​​may be recorded for each PCIe address. Another option is to bucket the PCIe addresses based on latency. The simplest example is to place PCIe addresses into two buckets. A fast bucket is used for PCIe addresses with latency below a threshold, and a slow bucket is used for PCIe addresses with latency above a threshold. More than two buckets may be used. Note that the latency of each PCIe address may be based on a single measurement for that PCIe address, or based on multiple measurements for that PCIe address. In one embodiment, the values ​​are recorded in an address range cache 504. After step 908, the process can return to step 902 to continue monitoring the delay so that the value can be updated in step 908.

[0119] Figure 9B An implementation of Table 950, which maps PRP list addresses to latency, is depicted. Table 950 has columns labeled as addresses, each containing several entries. Each entry contains a group of addresses with similar latencies. For example, PRP list addresses with the lowest latency or grouped (or “bucketed”) into a low-latency group. PRP list addresses with the highest latency or grouped (or “bucketed”) into a high-latency group. More than two groups may exist to allow for finer granularity in latency. The cost column contains the cost of the PCIe transaction for retrieving the PRP list, which is latency-based. Therefore, a longer latency will correspond to a higher cost. In one implementation, cost is used when allocating PCIe bus bandwidth.

[0120] Figure 10A This is a flowchart of one embodiment of a process 1000 for monitoring DMA latency of data for NVMe write commands. The memory controller 102 can access data from, for example, host memory 124 or target device memory 144. Process 1000 is executed in one embodiment of step 718 of process 700. Process 1000 is one embodiment for monitoring the latency of data payload transactions.

[0121] Step 1002 includes the memory controller 102 issuing a PCIe transaction to read (e.g., DMA) data directly from the data buffer of a PCIe device (e.g., host system 120, target device 140). In one embodiment, the memory controller 102 sends a packet to a PCIe switch 130, which forwards the packet to a PCIe root union 126. The packet may contain a PCIe address corresponding to the location where the data buffer in the memory (e.g., host memory 124 or target device memory 144) resides. The PCIe root union 126 may perform address translation. If the PCIe address is in the BAR of the target device 140, the PCIe root union 126 may forward the packet to the PCIe interface 146 in the target device 140. The result of step 1002 is a DMA from the memory of the PCIe device (e.g., host system 120 or target device 140) to the storage system 100.

[0122] Step 1004 includes the memory controller 102 recording the delay of this PCIe transaction for this data buffer. In this case, the delay is the time between when the memory controller 102 issues the PCIe transaction and when the memory controller 102 receives the data (i.e., the data to be written to the non-volatile memory 104).

[0123] The memory controller 102 may continue to monitor PCIe transactions for reads from the data buffers until the memory controller determines that a sufficient number of measurements have been performed to achieve the target accuracy (in step 1006). The memory controller 102 may perform a fixed number of measurements, which may be as few as a single measurement. Therefore, steps 1002 and 1004 may be repeated for other data buffers. In step 1008, the memory controller 102 records a mapping of PCIe addresses to latency. As described above, a PCIe address corresponding to each data buffer may exist. A value may be recorded for each PCIe address. Another option is to bucket the PCIe addresses based on latency, similar to what is described for the PRP list in step 908 of process 900. Therefore, PCIe addresses may be mapped to two or more groups based on latency. Note that the latency of each PCIe address may be based on a single measurement for that PCIe address or based on multiple measurements for that PCIe address. In one embodiment, the value is recorded in address range cache 504. After step 1008, the process may return to step 1002 to continue monitoring the delay so that the value can be updated in step 1008.

[0124] Figure 10BAn implementation of a table 1050 mapping PCIe addresses to latency is described. Table 1050 may be created or updated in step 1008. Table 1050 has multiple entries, each containing PCIe addresses bucketed into different latency groups. The PCIe addresses correspond to addresses of data buffers (e.g., data buffers 124a, 144a). Each entry contains a group of addresses with similar latencies. For example, PCIe addresses with latency L1 or faster are grouped (or “bucketed”) into a low-latency group. Other groups are for PCIe addresses with latencies between L1 and L2, between L2 and L3, and above L3. Typically, there are two or more latency groups. The cost column contains the cost of PCIe transactions in each corresponding bucket, which is based on latency. Therefore, a longer latency will correspond to a higher cost. In one implementation, cost is used when allocating PCIe bus bandwidth.

[0125] Figure 10C An alternative implementation of Table 1070, which stores PCIe transaction latency information, is described. Table 1070 may be created or updated in step 1008. Table 1070 is based on I / O commit queues. In this implementation, it is assumed that the data buffers used in conjunction with a given I / O commit queue will likely be in the same PCIe device. For example, the data buffers may be in one of the host system 120 or the target device 140 (but not both). Therefore, the latency of the data buffers associated with a given I / O commit queue may be similar. Thus, each entry in the "I / O Commit Queue Data Buffer Grouping" column relates to the latency of accessing the data buffer of one of the I / O commit queues. The cost column has the cost of a PCIe transaction accessing the data buffer associated with each corresponding I / O commit queue. In one implementation, the cost is used when allocating PCIe bus bandwidth.

[0126] Figure 11 This is a flowchart of one embodiment of process 1100 for monitoring DMA latency of data for NVMe read commands. Memory controller 102 may write data to, for example, host memory 124 or target device memory 144. Process 1100 is executed in one embodiment of step 712 of process 700. Process 1100 is one embodiment for monitoring the latency of data payload transactions.

[0127] Step 1102 includes the memory controller 102 issuing a PCIe transaction to write (e.g., DMA) data directly to the data buffer of a PCIe device (e.g., host system 120, target device 140). In one embodiment, the memory controller 102 sends a packet to a PCIe switch 130, which forwards the packet to a PCIe root union 126. The packet may contain a PCIe address corresponding to the location where the data buffer in the memory (e.g., host memory 124 or target device memory 144) resides. The PCIe root union 126 may perform address translation. If the PCIe address is in the BAR of the target device 140, the PCIe root union 126 may forward the packet to the PCIe interface 146 in the target device 140. The result of step 1102 is DMA from the storage system 100 to the memory of the PCIe device (e.g., host system 120 or target device 140).

[0128] Step 1104 includes the memory controller 102 recording the delay of this PCIe transaction for this data buffer. In one embodiment, the delay is the time between when the memory controller 102 issues the PCIe transaction and when the memory controller 102 receives confirmation that data has been written to the memory of another PCIe device.

[0129] The memory controller 102 may continue to monitor PCIe transactions for writes to the data buffers until the memory controller determines that a sufficient number of measurements have been performed to achieve the target accuracy (in step 1106). The memory controller 102 may perform a fixed number of measurements, which may be as few as a single measurement per PCIe address. Therefore, steps 1102 and 1104 may be repeated for other data buffers. In step 1108, the memory controller 102 records the mapping of PCIe addresses to latency. As described above, there may be a PCIe address corresponding to each data buffer. A value may be recorded for each PCIe address. Another option is to bucket the PCIe addresses based on latency, similar to what is described for the PRP list in step 908 of process 900. Therefore, PCIe addresses may be mapped to two or more groups based on latency. Therefore, NVMe reads may be used to form groups such as Figure 10B Table 1050 in the table. Another option is to build a table similar to... Figure 10CTable 1070 is a table in the table. In one embodiment, when forming tables 1050 and 1070, the memory controller 102 does not distinguish between NVMe reads and writes. That is, tables 1050 and 1070 can be formed based on the PCIe address of the data buffer DMA, regardless of whether it is a read or write NVMe command. In another embodiment, separate tables are created for NVMe read DMA and NVMe write DMA. Note that the latency for each PCIe address can be based on a single measurement for that PCIe address or based on multiple measurements for that PCIe address. In one embodiment, the value is recorded in address range cache 504. After step 1108, the process can return to step 1102 to continue monitoring the latency so that the value is updated in step 1108.

[0130] Figure 12 This is a flowchart of one implementation of a process 1200 that allocates PCIe bus bandwidth to NVMe commands based on PCIe transaction latency. Process 1200 is used in one implementation of step 606 of process 600.

[0131] Step 1202 involves estimating the cost of the corresponding NVMe command based on the latency of the PCIe transaction required by the NVMe command. Cost refers to the cost of the underlying PCIe transaction that transfers data across the PCIe bus, where the data is associated with the NVMe command. The data can be payload data and / or control data. One option is to consider both control data transactions and data payload transactions. Another option is to use only the cost of the data payload transaction. Return to Reference Figure 7 In one implementation, the data payload transaction is used for DMA to / from the data buffer (e.g., data buffers 124a, 144a) in step 712 or 718. There may be many PCIe transactions for a given NVMe read or write transaction. For example, each PCIe transaction may access a data buffer of, for example, 4KB, 8KB, 16KB, or some other size. However, NVMe commands can read or write data that is much larger in size. Optionally, however, the cost of controlling one or more of the transactions may be taken into account in the calculations in step 1202.

[0132] The following example illustrates a possible way to estimate costs. In step 702 of process 700, an NVMe command is accessed from the I / O submission queue. In one implementation, the cost of the NVMe command is estimated at this time. As an example, the NVMe command may specify the total amount of data to be transferred across the PCIe bus for reading or writing. That is, this is the total amount to be transferred to or from the data buffer (e.g., 124a, 144a). In one implementation, memory controller 102 tracks the latency of accessing the data buffer associated with that particular I / O submission queue. Figure 10C Table 1070 shows an example of the latency of accessing the data buffer based on the I / O submission queue. Therefore, the memory controller 102 can estimate the cost of accessing the data buffer for this NVMe command at this time.

[0133] As another example of cost estimation, when the PRP list is retrieved, the memory controller 102 may wait for step 706 of process 700. The PRP list has pointers to data buffers. In one embodiment, the memory controller 102 tracks the latency (based on PCIe address) of accessing the corresponding data buffer. Figure 10B Table 1050 illustrates an example of latency tracking based on PCIe addresses, where PCIe addresses are bucketed based on latency. Therefore, memory controller 102 can estimate costs based on the latency of the corresponding PCIe addresses. It should be noted that memory controller 102 can estimate costs before having a list of all PRPs for this NVMe command. One option is to assume that data buffers pointed to by other PRP lists will have similar latencies to those for the obtained PRP lists. This assumption can be made because data buffers for a given NVMe command are likely to have similar latencies, such as in the same PCIe device. For example, the data buffers will be entirely in host memory 124 or entirely in target memory 144. However, memory controller 102 will generally not be able to determine whether a data buffer is in host system memory 124 or target device memory 144 based solely on the PCIe address. This is because the NVMe protocol does not in any special way separate or label PCIe addresses. Therefore, for memory controller 102, the PCIe address may appear to be the same as the direct physical address in host memory 124 or target device memory 144.

[0134] Step 1204 involves adjusting data transfer across the PCIe bus for NVMe commands based on estimated costs. It should be noted that if the PCIe bus is not currently saturated with PCIe transactions, all NVMe commands can be allowed without any rate limiting. However, if the PCIe bus is saturated, rate limiting some NVMe commands can improve efficiency. That is, delaying some NVMe commands until the PCIe bus is no longer saturated can improve efficiency, for example, by avoiding bottlenecks and / or collisions in the buffers of the PCIe root union 126.

[0135] In one implementation, an integral-based method is used in step 1204. In this method, NVMe commands with costs below a threshold are allocated bandwidth on the PCI bus, while NVMe commands with costs above the threshold are rate-limited. Rate-limited NVMe commands can be delayed, thus temporarily not using PCIe bus bandwidth. When the PCIe bus is not busy, rate-limited NVMe commands can be allowed to execute (i.e., allocated bandwidth on the PCIe bus). When the PCIe bus remains busy, rate-limited NVMe commands can be allowed to execute after a certain amount of time.

[0136] In view of the foregoing, it can be seen that the first embodiment includes an apparatus comprising: a first communication interface configured to issue transactions via a communication bus according to a first communication protocol; a second communication interface configured to control access to non-volatile memory according to a second communication protocol; and control circuitry coupled to the first communication interface and the second communication interface. The control circuitry is configured to instruct the first communication interface to issue transactions of the first communication protocol for direct access to the memory of an electronic device connected to the communication bus via the communication bus. The transaction is associated with a store command for accessing the non-volatile memory according to the second communication protocol. The control circuitry is configured to determine the delay of the transaction, each transaction having a communication bus address. The control circuitry is configured to allocate bandwidth on the communication bus to the store command of the second communication protocol based on the delay of the transaction according to the first communication protocol.

[0137] In the second embodiment, in order to advance the first embodiment, the first communication protocol is a physical layer protocol, and the second communication protocol is a logical layer protocol that is higher than the first communication protocol.

[0138] In the third embodiment, in order to advance the first or second embodiment, the first communication protocol is Peripheral Computer Interface Express (PCIe), and the second communication protocol is Non-Volatile Memory Express (NVMe).

[0139] In a fourth embodiment, to advance any of the first to third embodiments, the control circuitry is configured to estimate the cost of transmitting data associated with the store command across the communication bus based on the transaction's latency. The control circuitry is configured to adjust the data transmission across the communication bus for the store command based on the estimated cost in order to allocate communication bus bandwidth to the store command.

[0140] In a fifth embodiment, to advance any of the first to fourth embodiments, the control circuit is further configured to determine the delay of a data control transaction, determine the delay of a data payload transaction, and allocate bandwidth on the communication bus to the storage command of the second communication protocol based on the dependency between the data payload transaction and the data control transaction for the storage command.

[0141] In a sixth embodiment, to advance any of the first to fifth embodiments, the control circuit is further configured to measure the latency of a transaction of the first communication protocol that stores data to a read data buffer via the communication bus. Each read data buffer resides in the memory of one of the electronic devices connected to the communication bus. The control circuit is further configured to allocate bandwidth on the communication bus to the store command of the second communication protocol, wherein the store command includes a read command, based on the latency of the transaction of the first communication protocol accessing the read data buffer.

[0142] In a seventh embodiment, to advance any of the first to sixth embodiments, the control circuitry is further configured to measure the latency of a transaction of the first communication protocol that reads data from a write data buffer via the communication bus. Each write data buffer resides in the memory of one of the electronic devices connected to the communication bus, wherein the store command includes a write command. The control circuitry is configured to allocate bandwidth on the communication bus to the store command of the second communication protocol based on the latency of the transaction of the first communication protocol accessing the write data buffer.

[0143] In the eighth embodiment, in order to advance any of the first to seventh embodiments, the control circuit is configured to determine the total latency of the first communication protocol transaction required for the storage command of the second communication protocol, and to schedule the storage command to access the communication bus based on the total latency.

[0144] In a ninth embodiment, to advance any of the first to eighth embodiments, the control circuitry is configured to measure the latency of a transaction of the first communication protocol accessing a command submission queue of the second communication protocol via the communication bus. Each command submission queue resides in the memory of one of the electronic devices connected to the communication bus. The control circuitry is configured to allocate the bandwidth on the communication bus to the stored command of the second communication protocol based on the latency of the transaction of the first communication protocol accessing the command submission queue of the second communication protocol.

[0145] In a tenth embodiment, to advance any of the first to ninth embodiments, the control circuit is further configured to measure the latency of a transaction of the first communication protocol accessing a list of pointers to a data buffer via the communication bus. Each list of pointers resides in the memory of one of the electronic devices connected to the communication bus. The list of pointers to the data buffer conforms to the second communication protocol. The control circuit is further configured to allocate bandwidth on the communication bus to the store command of the second communication protocol based on the latency of the transaction of the first communication protocol accessing the list of pointers to the data buffer.

[0146] In the eleventh embodiment, in order to advance any of the first to tenth embodiments, the electronic device connected to the communication bus includes a central processing unit (CPU) and a graphics processing unit (GPU).

[0147] One embodiment includes a method of operating a non-volatile memory system. The method includes a memory controller of the non-volatile memory system determining the latency of a Peripheral Computer Interconnect Express (PCIe) transaction that directly accesses memory in a corresponding PCIe device connected to the PCIe bus via a PCIe bus. Each PCIe transaction has a PCIe address. The method includes the memory controller grouping the PCIe address based on the latency of the PCIe transaction. The method includes the memory controller scheduling non-volatile memory Express (NVMe) commands to access the PCIe bus based on the latency of the PCIe transaction. The method includes the memory controller initiating PCIe transactions on the PCIe bus to execute the NVMe commands according to the scheduling, including initiating multiple PCIe transactions for each NVMe command.

[0148] One embodiment includes a non-volatile memory system comprising: non-volatile memory; a Peripheral Computer Interface Express (PCIe) interface configured to control PCIe transactions via a PCIe bus; and a Non-Volatile Memory Express (NVMe) interface configured to process NVMe commands received on the PCIe bus to access the non-volatile memory. The PCIe interface is configured to issue PCIe transactions that directly access data buffers in a plurality of processors connected to the PCIe bus. The non-volatile memory system includes means for determining the latency of the PCIe transaction that directly accesses the data buffers in the plurality of processors. The PCIe transaction that directly accesses the data buffers includes a data payload transaction. The non-volatile memory system includes means for adjusting data transfers across the PCIe bus for the NVMe command based on the latency of the PCIe transaction that directly accesses the data buffers in the plurality of processors.

[0149] In one embodiment, the apparatus for determining the latency of a PCIe transaction that directly accesses the data buffer in the plurality of processors includes one or more of a host processor 152, an address range identification module 502, a transaction latency measurement 510, a transaction address packet, an FGA, an ASIC, an integrated circuit, or other types of circuitry. In one embodiment, the apparatus for determining the latency of a PCIe transaction that directly accesses the data buffer in the plurality of processors performs one or more of processes 1000 and / or 1100.

[0150] In one embodiment, the apparatus for adjusting data transfer across the PCIe bus for the NVMe command based on the latency of the PCIe transaction that directly accesses the data buffers in the plurality of processors includes one or more of a host processor 152, a bandwidth allocator 520, an FGA, an ASIC, an integrated circuit, or other types of circuitry. In one embodiment, the apparatus for adjusting data transfer across the PCIe bus for the NVMe command based on the latency of the PCIe transaction that directly accesses the data buffers in the plurality of processors performs process 1200.

[0151] In one embodiment, the means for determining the latency of a PCIe control transaction that directly accesses a list of NVMe input / output (I / O) queues or physical region pages (PRPs) in memory of the plurality of processors connected to the PCIe bus includes one or more of a host processor 152, an address range identification module 502, a transaction latency measurement 510, a transaction address packet, an FGA, an ASIC, an integrated circuit, or other type of circuitry. In one embodiment, the means for determining the latency of a PCIe control transaction that directly accesses a list of NVMe input / output (I / O) queues or physical region pages (PRPs) in memory of the plurality of processors connected to the PCIe bus executes one or more of processes 800 and / or 900.

[0152] For the purposes of this document, the terms “implementation scheme,” “one implementation scheme,” “some implementation schemes,” or “another implementation scheme” used in the specification may be used to describe different implementation schemes or the same implementation scheme.

[0153] For the purposes of this document, a connection may be a direct connection or an indirect connection (e.g., via one or more other components). In some cases, when an element is mentioned as being connected or coupled to another element, the element may be directly connected to the other element or indirectly connected to the other element via an intermediary element. When an element is mentioned as being directly connected to another element, there is no intermediary element between the two elements. If two devices are directly or indirectly connected, the two devices are “communicating”, enabling them to communicate electronic signals between them.

[0154] For the purposes of this document, the term “based on” may be understood as “at least partially based on”.

[0155] For the purposes of this document, the use of numerical terms such as “first” object, “second” object, and “third” object without additional context may not imply an ordering of objects, but may be used for identification purposes to distinguish different objects.

[0156] For the purposes of this document, the term "group" of objects may refer to a "group" of one or more objects.

[0157] The detailed description above has been provided for purposes of illustration and description. It is not intended to be exhaustive or to limit the precise forms disclosed in the invention. Many modifications and variations are possible based on the teachings above. The described embodiments were chosen to best explain the principles of the proposed technology and its practical application, thereby enabling others skilled in the art to best utilize it in various embodiments and various modifications suitable for the specific intended use. The scope of the invention is intended to be defined by the appended claims.

Claims

1. An apparatus comprising: A first communication interface is configured to send transactions to an electronic device connected to the communication bus via a communication bus according to a first communication protocol, wherein the first communication protocol is a physical layer protocol. A second communication interface is configured to process storage commands of a second communication protocol received via the communication bus to control access to non-volatile memory according to the second communication protocol, wherein the second communication protocol is a logic layer protocol higher than the first communication protocol. as well as A control circuit, coupled to the first communication interface and the second communication interface, is configured to: The first communication interface is instructed to issue a transaction via the communication bus for the execution of direct memory access (DMA) of the electronic device connected to the communication bus using the first communication protocol. Determine the delay of the issued transactions, each transaction having a communication bus address; as well as Based on the delay of the transaction issued by the first communication protocol, bandwidth on the communication bus is allocated to the storage command of the second communication protocol.

2. The apparatus according to claim 1, wherein: The first communication protocol is the peripheral computer interface Express, i.e., PCIe; and The second communication protocol is non-volatile memory Express, or NVMe.

3. The apparatus of claim 1, wherein the control circuit is configured to: Estimate the cost of transmitting data associated with the storage command across the communication bus based on the transaction's latency; and The data transmission across the communication bus for the storage command is adjusted based on the estimated cost in order to allocate the communication bus bandwidth to the storage command.

4. The apparatus of claim 1, wherein the control circuit is further configured to: Determine the delay of data control transactions; Determine the latency of data payload transactions; and Based on the dependency between the data payload transaction and the data control transaction for the storage command, bandwidth on the communication bus is allocated to the storage command of the second communication protocol.

5. The apparatus of claim 1, wherein the control circuit is further configured to: The latency of a transaction of the first communication protocol, which stores data into read data buffers via the communication bus, is measured, and each read data buffer resides in the memory of one of the electronic devices connected to the communication bus; and Based on the latency of the transaction of the first communication protocol accessing the read data buffer, bandwidth on the communication bus is allocated to the storage command of the second communication protocol, wherein the storage command includes a read command.

6. The apparatus of claim 1, wherein the control circuit is further configured to: The latency of a transaction of the first communication protocol for reading data from write data buffers via the communication bus is measured, each write data buffer residing in the memory of one of the electronic devices connected to the communication bus, wherein the storage command includes a write command; and Based on the latency of the transaction of the first communication protocol accessing the write data buffer, bandwidth on the communication bus is allocated to the storage command of the second communication protocol.

7. The apparatus of claim 1, wherein the control circuit is configured to: Determine the total latency of the transaction of the first communication protocol required for the storage command of the second communication protocol; and The storage commands are scheduled to access the communication bus based on the total latency.

8. The apparatus of claim 1, wherein the control circuit is configured to: Measuring the latency of transactions of the first communication protocol accessing the command submission queue of the second communication protocol via the communication bus, each command submission queue residing in the memory of one of the electronic devices connected to the communication bus; and Based on the delay of the transaction of the first communication protocol in the command submission queue of the second communication protocol, the bandwidth on the communication bus is allocated to the stored command of the second communication protocol.

9. The apparatus of claim 1, wherein the control circuit is further configured to: Measure the latency of transactions of the first communication protocol for accessing a list of pointers to a data buffer via the communication bus, each list of pointers residing in the memory of one of the electronic devices connected to the communication bus, the list of pointers to the data buffer conforming to the second communication protocol; and Based on the latency of the transaction of the first communication protocol that accesses the list of pointers to the data buffer, bandwidth on the communication bus is allocated to the storage command of the second communication protocol.

10. The apparatus of claim 1, wherein the electronic device connected to the communication bus comprises: The central processing unit, or CPU, is used for processing data. as well as The graphics processing unit is called the GPU.

11. A method of operating a non-volatile memory system, the method comprising: The memory controller of the non-volatile memory system determines the latency of the peripheral computer interconnect Express transaction, i.e., the PCIe transaction, which is a PCIe transaction with a PCIe address, for direct access to the memory in the corresponding PCIe device connected to the PCIe bus via the PCIe bus. The memory controller groups the PCIe addresses based on the latency of the PCIe transaction; The memory controller schedules non-volatile memory Express commands (NVMe commands) that access the PCIe bus based on the latency of the PCIe transaction. as well as The memory controller initiates PCIe transactions on the PCIe bus to execute the NVMe commands according to the schedule, including initiating multiple PCIe transactions for each NVMe command.

12. The method according to claim 11, wherein: Determining the latency of PCIe transactions that directly access the memory of the corresponding PCIe device connected to the PCIe bus via the PCIe bus includes measuring the latency of PCIe transactions that directly access NVMe commit queues, each NVMe commit queue residing in the memory of one of the PCIe devices. as well as Grouping the PCIe addresses based on the latency of the PCIe transactions includes grouping the NVMe commit queue based on the latency of the PCIe transactions that directly access the NVMe commit queue.

13. The method according to claim 11, wherein: Determining the latency of PCIe transactions that directly access the memory of the corresponding PCIe device connected to the PCIe bus via the PCIe bus includes measuring the latency of PCIe transactions that directly access NVMe commit queues, each command commit queue residing in the memory of one of the PCIe devices, and measuring the latency of PCIe transactions that directly access NVMe data buffers, each data buffer residing in the memory of one of the PCIe devices. as well as Grouping the PCIe addresses based on the latency of the PCIe transaction includes placing the PCIe addresses into different groups based on the latency of the PCIe transaction that directly accesses the NVMe data buffer.

14. The method of claim 11, wherein scheduling the NVMe command to access the PCIe bus based on the latency of the PCIe transaction comprises: The cost of implementing the NVMe command is estimated based on the latency of the PCIe transaction that implements the NVMe command; as well as Determine whether the estimated cost exceeds the threshold.

15. A non-volatile memory system, comprising: Non-volatile memory; The peripheral computer interface Express interface is also known as the PCIe interface. The PCIe interface is configured to control PCIe transactions through the PCIe bus and to issue PCIe transactions that directly access data buffers in multiple processors connected to the PCIe bus. The non-volatile memory Express interface, also known as the NVMe interface, is configured to process NVMe commands received on the PCIe bus to access the non-volatile memory. A means for determining the latency of a PCIe transaction that directly accesses a data buffer in one of the plurality of processors, the PCIe transaction including a data payload transaction; as well as A means for adjusting data transfer across the PCIe bus for the NVMe command based on the latency of the PCIe transaction that directly accesses the data buffers in the plurality of processors.

16. The non-volatile memory system of claim 15, wherein the means for adjusting data transfer across the PCIe bus for the NVMe command based on the latency of the PCIe transaction directly accessing the data buffers in the plurality of processors is configured to: Based on the latency of the PCIe transaction that directly accesses the data buffers in the plurality of processors, estimate the cost of accessing the data buffer associated with the NVMe command; and The NVMe commands are scheduled based on the estimated cost.

17. The non-volatile memory system of claim 15, further comprising means for determining the latency of PCIe control transactions that directly access NVMe input / output queues, i.e., NVMe I / O queues, or physical region page lists, i.e., PRP lists, in memory of the plurality of processors connected to the PCIe bus; and The means for regulating data transmission across the PCIe bus for the NVMe command is also used to schedule the NVMe command based on the latency of the PCIe control transaction.

18. The non-volatile memory system of claim 15, wherein the means for determining the latency of the PCIe transaction for directly accessing the data buffers in the plurality of processors is configured to: Determine the latency of PCIe transactions that directly access the data buffer in the central processing unit, i.e., the CPU; and Determine the latency of PCIe transactions that directly access the data buffer in the graphics processing unit, i.e., the GPU.

19. The non-volatile memory system of claim 15, wherein the means for adjusting data transfer across the PCIe bus for the NVMe command based on the latency of the PCIe transaction directly accessing the data buffers in the plurality of processors is configured to: Schedule PCIe transactions that directly access the data buffer in the first processor of the plurality of processors.

Images