A multi-port, multimode memory subsystem with scalable virtualization capabilities.

Abstract

This disclosure relates to a memory subsystem that supports both PCIe and Ethernet without the need for separate switches or bridges. The memory subsystem includes a processing device that detects a first host system connected to a first interface port of a memory device having a plurality of interface ports. The processing device further detects a second host system connected to a second interface port of the plurality of interface ports. The processing device also uses Root I / O Virtualization (SR-IOV) to assign a first subset of a plurality of virtual functions (VFs) associated with the memory device to the first host system and uses SR-IOV to assign a second subset of the plurality of VFs to the second host system. Furthermore, the processing device assigns a first range of logical block addresses (LBAs) corresponding to each VF in the first VF subset and a second range of LBAs corresponding to each VF in the second VF subset.

Description

Technical Field

[0001] The present disclosure generally relates to a memory system, and more specifically to a multi-mode memory subsystem having a plurality of ports with scalable virtualization capabilities.

Background Art

[0002] A memory subsystem may include one or more memory components for storing data. These memory components can be, for example, non-volatile memory components and volatile memory components. Generally, a host system can store data in the memory components and retrieve data from the memory components using the memory subsystem.

Brief Description of the Drawings

[0003] The present disclosure will be more fully understood by reference to the following detailed description and the accompanying drawings that illustrate various embodiments of the present disclosure. [Figure 1] FIG. 1 shows an example of a computing system including a memory subsystem according to some embodiments of the present disclosure. [Figure 2] FIG. 2 shows an example of a computing system including a memory subsystem having a plurality of ports with scalable virtualization capabilities according to some embodiments of the present disclosure. [Figure 3] FIG. 3 shows virtual function memory region mapping in a memory subsystem for SR-IOV / S-IOV virtualization according to some embodiments of the present disclosure. [Figure 4] FIG. 4 shows an example of a multi-mode memory subsystem having a plurality of ports with scalable virtualization capabilities according to some embodiments of the present disclosure. [Figure 5] FIG. 5 shows an example of a multi-mode memory subsystem having a plurality of ports with scalable virtualization capabilities according to some embodiments of the present disclosure. [Figure 6]This is a flow diagram illustrating an example of a method for managing multiple SR-IOV / S-IOV compatible interface ports of a memory subsystem, according to some embodiments of the present disclosure. [Figure 7] This figure shows an example of a method for assigning virtual functions to a virtual machine in order to support multiple SR-IOV / S-IOV compatible interface ports of a memory subsystem, according to some embodiments of the present disclosure. [Figure 8] This block diagram shows an example of a computer system in which several embodiments of this disclosure can operate. [Modes for carrying out the invention]

[0004] Each aspect of this disclosure relates to supporting multiple ports having SR-IOV (Single Root Input / Output Virtualization) or S-IOV (Scalable Input / Output Virtualization) in a memory subsystem. The memory subsystem may be a storage device, a memory module, or a hybrid of a storage device and a memory module. Examples of storage devices and memory modules are described later with reference to Figure 1. Generally, a host system can utilize a memory subsystem that includes one or more memory components, such as a memory device that stores data. The host system can provide data stored in the memory subsystem and can request data to be retrieved from the memory subsystem.

[0005] In a memory subsystem, data can be sent and received between the memory subsystem and the host system using a single interface port. Multiple hosts running virtual machines may also interact with the memory subsystem. Virtual machines can emulate the physical host system or other physical resources. Therefore, the memory subsystem can be used to store and retrieve data for virtual machines running on the host system. To manage the transmission and reception of data, i.e., data transmission from the memory subsystem's memory devices to virtual machines on the host system, the memory subsystem's storage resources may be shared via a single interface port utilizing Single Root Input / Output Virtualization (SR-IOV). In some embodiments, SR-IOV can isolate the resources of interfaces such as PCI Express (PCIe: Peripheral Component Interconnect Express) used by different virtual machines to read and write data to the memory subsystem. For example, SR-IOV can provide different virtual functions (VFs), each used by individual virtual machines. PCI Express (PCIe) virtual functions (VFs) are lightweight PCIe functions on network adapters that support single-route I / O virtualization (SR-IOV). VFs are associated with PCIe physical functions (PFs) on network adapters and represent virtualized instances of the network adapter. Each VF has its own PCI configuration space and shares one or more physical resources, such as external network ports, with PFs and other VFs.

[0006] When a memory subsystem is used by multiple host systems, a single interface port of the memory subsystem can be used to share storage resources between virtual machines running on the host systems and the memory subsystem. To manage the use of a single interface port by multiple host systems, a switch can be used as an intermediary between the memory subsystem and each host system. For example, the switch may be a PCIe switch, providing each host system with access to the memory subsystem via a single interface port. Thus, the switch can sequentially (i.e., at different access times) open a single interface port utilizing single-route I / O virtualization to different host systems. For example, all virtual functions (VFs) provided by SR-IOV can be opened to all host systems. However, if the switch is provided as a separate component, it is an independent hardware component connected to the host systems, which may incur additional costs and power consumption for the memory subsystem. Furthermore, since all host systems connect to the memory subsystem via the switch, there is a risk that the switch will become a single point of failure in the memory subsystem. Therefore, if the switch fails, all host systems may lose access to the memory subsystem.

[0007] Each aspect of this disclosure addresses the aforementioned problems and other drawbacks by introducing multiple interface ports within the memory subsystem, thereby enabling the memory subsystem to be shared as storage among multiple host systems. Each of the multiple interface ports supports virtualization, including Single Root Input / Output Virtualization (SR-IOV) and Scalable Input / Output Virtualization (S-IOV). For example, the memory subsystem may have multiple SR-IOV-enabled interface ports or multiple S-IOV-enabled interface ports, thereby enabling access to multiple host systems without the use of separate switches or bridges. The interface ports may be PCIe ports, Ethernet ports, or physical ports. The multiple interface ports of the memory subsystem are simultaneously accessible to each other, allowing multiple host systems to access the memory subsystem simultaneously, or at least for a period where some of the access time overlaps. Each interface port (e.g., PCIe interface port and Ethernet port) can provide a separate set of virtual functions to each host system using SR-IOV / S-IOV. In some embodiments, the maximum number of virtual functions that the memory subsystem can provide may be defined. Therefore, if the memory subsystem has a large number of interface ports, the number of virtual functions assigned to each port can be reduced so that the total number of virtual functions assigned to all ports does not exceed the maximum number of virtual functions supported by the memory subsystem. Similarly, if the memory subsystem has a small number of interface ports, more virtual functions can be assigned to each port.

[0008] In some embodiments, the memory subsystem may have two or more SR-IOV / S-IOV compatible interface ports. For example, the memory subsystem may be used by two or more host systems, with multiple virtual machines running on each host system. Each interface port (e.g., a PCIe port or an Ethernet port) is SR-IOV / S-IOV compatible and can provide a set of virtual functions to a group of virtual machines on the host system. SR-IOV is a specification that enables the isolation of Peripheral Component Interconnect Express (PCIe) resources among multiple hardware functions for manageability and performance reasons, and further enables the sharing of a single physical PCIe device within a virtual environment. SR-IOV and S-IOV provide different virtual functions (VFs) to different virtual components (e.g., network adapters) on a physical server machine. Furthermore, SR-IOV and S-IOV enable different virtual machines within a virtual environment to share a single PCIe or Ethernet hardware interface without compromising performance.

[0009] In one example of this embodiment, the memory subsystem can generate multiple virtual instances of a physical device and assign each to a different virtual machine using hardware-assisted virtualization technologies such as an Input / Output Memory Management Unit (IOMMU) and Direct Memory Access (DMA) remapping. One advantage of using an IOMMU is that it provides memory protection and isolation for input / output processing. By mapping the virtual address of each virtual machine (VM) to a separate set of physical addresses, the IOMMU ensures that one VM cannot access memory areas used by other VMs. This provides an additional layer of security and helps prevent unauthorized access such as buffer overflow attacks and other memory-based attacks. The memory subsystem can provide the virtual functions supported by each interface port (e.g., address or other identification information) and SR-IOV / S-IOV compatible interface ports to the virtual machines of the host system. Each virtual machine in the host system is assigned one of the virtual functions of the interface port. Therefore, since the memory subsystem has multiple interface ports, each port opening up a separate set of virtual functions that can be used by different host systems, there is no need to provide a switch or bridge between the memory subsystem and the host system.

[0010] In some embodiments, each virtual function (VF) can be assigned a portion of the memory subsystem's namespace or logical block address space (LBA space). For example, each virtual machine assigned a different virtual function can access a different region of the memory subsystem's logical block address space. The logical block address space may be mapped to the memory subsystem's physical block address space. Each virtual instance has its own virtual function (VF) identifier, which the hypervisor uses to map I / O requests from virtual machines to the appropriate virtual instance. This allows multiple virtual machines to access the same physical device simultaneously without interfering with each other or degrading performance.

[0011] In some embodiments, the memory subsystem controller can divide its total bandwidth among multiple interface ports. Thus, each interface port may be connected to a separate internal memory buffer within the memory subsystem controller, which may be used to temporarily store data received from or transmitted from the controller through the interface port.

[0012] One of the advantages of this disclosure, though not limited to, is that it reduces overall costs by eliminating the need for separate switches or bridges when coordinating multiple host systems and memory subsystems. Virtualization environments with multi-host SoCs and multi-VMs (virtual machines) are becoming increasingly common in enterprise data centers, such as in-vehicle infotainment (IVI) and advanced driver assistance systems (ADAS), making cost and power consumption reduction of multi-host SoCs desirable. Furthermore, by eliminating the use of separate switches, a single point of failure (i.e., the switch itself) can be eliminated. Since access to multiple host systems is provided through individual interface ports, if a port fails, the impact is limited to the host systems connected to that port, and other ports can continue to function normally. This improves the reliability of the memory subsystem. In addition, power consumption can be reduced by not including separate switches in the memory subsystem configuration. Furthermore, by using SR-IOV / S-IOV, a hypervisor for virtualizing the storage environment becomes unnecessary. Therefore, the software overhead caused by the hypervisor can be eliminated, enabling significant cost and power reductions at the system level, and allowing for bare-metal connectivity between the memory subsystem and the host SoC.

[0013] Figure 1 is a diagram showing an example of a computing system 100 according to some embodiments of the present disclosure, which includes a memory subsystem 110. The memory subsystem 110 may include media such as one or more volatile memory devices (e.g., memory device 140), one or more non-volatile memory devices (e.g., memory device 130), or a combination thereof.

[0014] The memory subsystem 110 may be a storage device, a memory module, or a hybrid of a storage device and a memory module. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash storage (UFS) drives, and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small outline DIMMs (SO-DIMMs), and non-volatile dual in-line memory modules (NVDIMMs).

[0015] The computing system 100 may be a computing device such as a desktop computer, laptop computer, network server, mobile device, vehicle (e.g., aircraft, drone, train, automobile, or other transportation equipment), IoT (Internet of Things) enabled device, or embedded computer (e.g., one embedded in a vehicle, industrial equipment, or networked commercial equipment). Alternatively, it may be such a computing system comprising memory and processing devices.

[0016] The computing system 100 may include a host system 120 connected to one or more memory subsystems 110. In some embodiments, the host system 120 may be connected to different types of memory subsystems 110. Figure 1 shows an example of a host system 120 connected to one memory subsystem 110. As used herein, the expression “coupled to” generally refers to a connection between components, which may be either a direct communication connection without other components interposed, or an indirect communication connection via intervening elements. This connection may be wired or wireless, and may include electrical, optical, magnetic, and the like.

[0017] The host system 120 may include a processor chipset and a software stack run by that processor chipset. The processor chipset may include one or more cores, one or more caches, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). The host system 120 uses the memory subsystem 110, for example, to write data to and read data from the memory subsystem 110.

[0018] The host system 120 may be connected to the memory subsystem 110 via a physical host interface. Examples of physical host interfaces include, but are not limited to, Serial ATA (SATA) interfaces, Peripheral Component Interconnect Express (PCIe) interfaces, Universal Serial Bus (USB) interfaces, Fibre Channel, and Serial Attached SCSI (SAS). The physical host interface is used to send and receive data between the host system 120 and the memory subsystem 110. Furthermore, if the memory subsystem 110 is connected to the host system 120 via a PCIe interface, the host system 120 can access memory components (e.g., memory device 130) using an NVM Express (NVMe) interface. The physical host interface can provide an interface for transmitting control signals, address signals, data signals, and other signals between the memory subsystem 110 and the host system 120.

[0019] The memory device can include any combination of multiple types of non-volatile memory devices and / or volatile memory devices. The volatile memory device (e.g., memory device 140) may be, but is not limited to, a random access memory (RAM), and may include, for example, a dynamic random access memory (DRAM) or a synchronous dynamic random access memory (SDRAM).

[0020] Examples of non-volatile memory devices (e.g., memory device 130) include NAND flash memory and write-in-place memory (e.g., three-dimensional cross-point (3D cross-point) memory). A 3D cross-point memory device is composed of a cross-point array of non-volatile memory cells, and in combination with a stackable lattice data access array, can store bits based on changes in bulk resistance. Also, unlike many flash-based memories, cross-point type non-volatile memory can perform a "write-in-place" operation, which means it is programmable without pre-erasing non-volatile memory cells.

[0021] Although non-volatile memory components such as 3D cross-point type and NAND type flash memory have been described, memory device 130 is not limited to these and may be configured based on other types of non-volatile memory. Examples include read-only memory (ROM), phase change memory (PCM), magnetic random access memory (MRAM), NOR type flash memory, electrically erasable programmable read-only memory (EEPROM), and cross-point arrays of non-volatile memory cells, etc.

[0022] For example, a memory cell such as a single-level cell (SLC) can store one bit per cell. Other types of memory cells include multi-level cells (MLC), triple-level cells (TLC), and quad-level cells (QLC), which can store multiple bits per cell. In some embodiments, each memory device 130 may include one or more memory arrays consisting of SLC, MLC, TLC, QLC, or any combination thereof. In some embodiments, a particular memory device may include memory cells in SLC areas and MLC, TLC, or QLC areas. The memory cells of the memory device 130 may be grouped as pages, which are logical units of the memory device used to store data. In some types of memory (e.g., NAND), pages may be grouped to form blocks. Also, in some memory types such as 3D crosspoint, pages can be grouped across dies or channels to form Management Units (Mus).

[0023] The memory subsystem controller 115 (hereinafter also simply referred to as controller 115) communicates with the memory device 130 and can perform various operations on the memory device 130, such as reading, writing, and erasing data. The memory subsystem controller 115 may include hardware such as one or more integrated circuits and / or discrete components, buffer memory, or a combination thereof. This hardware may include digital circuits with dedicated (i.e., hardcoded) logic for performing the operations described herein. The memory subsystem controller 115 may be a microcontroller, an application-specific logic circuit (e.g., a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)), or another suitable processor.

[0024] Memory subsystem controller 115 can include a processor (processing device) 117 configured to execute instructions stored in local memory 119. In the illustrated example, local memory 119 of memory subsystem controller 115 includes embedded memory, and this embedded memory is configured to store instructions for executing various processes, operations, logic flows, and routines for controlling the operation of memory subsystem 110. This control also includes communication processing between memory subsystem 110 and host system 120.

[0025] In some embodiments, local memory 119 can include memory registers for storing memory pointers, fetched data, and the like. Also, local memory 119 may include a read-only memory (ROM) for storing microcode. Memory subsystem 110 in the example shown in FIG. 1 is shown as including memory subsystem controller 115, but in other embodiments of the present disclosure, memory subsystem 110 may not include memory subsystem controller 115, and in that case, it may depend on external control (e.g., control provided by an external host or a processor or controller separate from the memory subsystem).

[0026] Generally, the memory subsystem controller 115 can receive commands or operations from the host system 120, translate those commands or operations, and generate instructions or appropriate commands to achieve the desired access to the memory device 130. The memory subsystem controller 115 can also be responsible for operations such as wear leveling, garbage collection, error detection and error correction code (ECC) processing, encryption, caching, and address translation between logical addresses (e.g., logical block addresses (LBAs) or namespaces) and physical addresses (e.g., physical block addresses) associated with the memory device 130. Furthermore, the memory subsystem controller 115 may include a host interface circuit for communicating with the host system 120 via a physical host interface. This host interface circuit can translate commands received from the host system into command instructions for performing access to the memory device 130, and translate responses associated with the memory device 130 into information for the host system 120.

[0027] The memory subsystem 110 may include additional circuitry or components not shown. In some embodiments, the memory subsystem 110 includes a cache or buffer (e.g., DRAM) and addressing circuits (e.g., a row decoder and a column decoder) that receive addresses from the memory subsystem controller 115 and decode those addresses to access the memory device 130.

[0028] In some embodiments, the memory device 130 includes a local media controller 135 that operates in cooperation with a memory subsystem controller 115 and performs various operations on one or more memory cells of the memory device 130. An external controller (e.g., the memory subsystem controller 115) may externally manage the memory device 130 (e.g., it may perform media management operations on the memory device 130). In some embodiments, the memory device 130 may be a managed memory device that combines a raw memory device (unmanaged memory device) with a local controller (e.g., the local controller 135) to perform media management within the same memory device package. An example of a managed memory device is a managed NAND (MNAND) device.

[0029] The memory subsystem 110 includes a multi-port virtualization component 113, which can be used to support multiple SR-IOV-enabled ports within the memory subsystem. This allows the memory subsystem to be shared as storage among multiple host systems. In some implementations, the memory subsystem 110 can be shared by multiple host systems using its multiple SR-IOV-enabled interface ports without requiring a separate switch (e.g., a multi-host PCIe switch). The multiple interface ports of the memory subsystem 110 can operate simultaneously with each other, allowing multiple host systems to access the memory subsystem 110 simultaneously, or at least partially, by connecting each host system to one of the memory subsystem's interface ports. Each interface port (e.g., a PCIe interface port or an Ethernet port) can use SR-IOV to provide the virtual functions of the memory subsystem 110 to the host system connected to that port. In some implementations, the memory subsystem 110 may have a maximum number of virtual functions that can be provided. In this case, the number of virtual functions assigned to each interface port can be determined by dividing the total number of virtual functions of the memory subsystem by the number of interface ports of the memory subsystem 110.

[0030] In some implementations, the memory subsystem 110 may include two or more SR-IOV / S-IOV compatible interface ports. For example, the memory subsystem may be used by multiple host systems 120, with multiple virtual machines running on each host system. Each interface port (e.g., a PCIe port or an Ethernet port) is SR-IOV compatible, thereby providing a set of virtual functions to the virtual machine group on each host system. SR-IOV is a specification that enables the isolation of Peripheral Component Interconnect Express (PCIe) resources among multiple hardware functions for manageability and performance reasons, while simultaneously allowing a single physical PCIe device to be shared within a virtual environment. SR-IOV provides different virtual functions (VFs) to different virtual components (e.g., network adapters) on a physical server machine. SR-IOV also allows different virtual machines within a virtual environment to share a single PCIe hardware interface. Scalable I / O virtualization (SIOV) allows multiple virtual machines (VMs) to share a single physical device, such as a network adapter or storage controller, without compromising performance. SIOV uses hardware-assisted virtualization technologies such as input / output memory management units (IOMMUs) and direct memory access (DMA) remapping to create multiple virtual instances of a physical device and assign each to a different virtual machine.

[0031] In one implementation, the multi-port virtualization component 113 can provide virtual functions supported by each interface port (e.g., address or other identification information) and SR-IOV / S-IOV compatible interface ports to a group of virtual machines on the host system 120. Each virtual machine on the host system 120 is assigned one of the virtual functions of the interface port. Thus, since the memory subsystem 110 has multiple interface ports, each port opening up a separate set of virtual functions available on different host systems, there is no need to provide a switch between the memory subsystem 110 and the host system 120.

[0032] In some implementations, the multiport virtualization component 113 can assign a portion or namespace of the logical block address space (LBA space) of the memory subsystem 110 to each virtual function. For example, each virtual machine to which a virtual function is assigned can access a different logical block address space (LBA) region of the memory subsystem 110. The logical block address space may be mapped to the physical block address space of the memory subsystem. In some implementations, the memory subsystem controller 115 can determine that a group of virtual functions is associated with a specific LBA range and assigned to each interface port. In the same or a different embodiment, the controller 115 may change the number of virtual functions assigned to each interface port. For example, the number of virtual functions assigned to a virtual machine of a host system connected to a particular interface port can be increased or decreased based on the host system's use of storage resources in the memory subsystem 110 by the virtual machine. In some embodiments, different LBA ranges (e.g., logical block address ranges with different corresponding physical block address ranges) may be assigned to different virtual functions. For example, a virtual function on one interface port may be assigned a wider LBA range than a virtual function on another interface port. Different LBA ranges may be determined based on the usage patterns of the virtual machines or applications on the host system connected to the interface port in question.

[0033] In some implementations, each interface port of the memory subsystem controller 115 may be connected to a separate memory buffer within the memory subsystem controller 115. This allows the controller 115 to divide bandwidth among the interface ports, enabling all ports to operate in parallel. Thus, each interface port can temporarily store data received from the controller 115 in a buffer assigned to that port. Similarly, the controller 115 can store data received from each interface port in a buffer dedicated to that port and hold it until the controller 115 is ready to process the data from that port.

[0034] Figure 2 shows an example of a computing system 200 including a memory subsystem 110 having multiple ports with single-root virtualization functionality, according to some embodiments of the present disclosure. The memory subsystem 110 may be connected to a plurality of host systems 210-240. Each host system 210-240 may be identical or similar to host system 120 shown in Figure 1. In some implementations, each host system 210-240 may be an independent system such as a system-on-a-chip (SOC), or another host system that provides a group of virtual machines. For example, each host system 210-240 may be an independent processor core or processor device that provides or runs a group of virtual machines. Each host system 210-240 may be connected to the memory subsystem 110 via one of the interface ports 250A-D provided by the memory subsystem controller 115 of the memory subsystem 110. For example, a PCIe interface may be used between each host system and the interface port, thereby connecting each host system 210-240 to a different interface port 250A-D of the memory subsystem 110. As another example, two host systems can be connected to quad interface ports 250A-D (for example, using separate PCIe switches), which increases the number of host systems that can access the memory subsystem 110.

[0035] In one example, host systems 210-240 are system-on-chip (SOC) hosts, and the memory subsystem 110 may have one or more PCIe endpoint ports and one or more Ethernet ports. Each interface port has one lane, and each link can be automatically detected for connection to the root complex (RC) of each host SOC. In some implementations, the link / lane configuration combinations of the interface ports may include one lane each in a 4-port configuration, one lane each in a 3-port configuration, two lanes each in a 2-port configuration, and four lanes in a 1-port configuration. The PCIe PHY layer may be divided into up to four parts to share the bandwidth of the backend storage of the memory subsystem 110, with one allocated to each interface port. Each interface port opens up a set of virtual functions (VFs) on that port to each host SOC, and the host SOC runs multiple virtual machines on its CPU cores.

[0036] In one implementation, interface port 250A may be connected to a host system 210. Virtual functions (VFs) 251A to C may be assigned to interface port 250A. To isolate the resources of the shared memory subsystem 110, each VF 251A to C on port 250A may be used by different virtual machines (VMs) 211A to C on the host system 210. In this case, VF 251A is assigned to VM 211A on the host system 210, VF 251B to VM 211B, and VF 251C to VM 211C. In some implementations, each VF 251A to L in the memory subsystem 110 may be assigned a corresponding LBA range of memory devices 130 to 140, and each VF 251A to L may have its own dedicated namespace within the memory subsystem 110.

[0037] SR-IOV-enabled port 250B may be assigned VF251D~F. To isolate resources used by different virtual machines and host systems, each VF251D~F on port 250B may be used by one of the virtual machines (VMs) 221A~C on host system 220. In this case, VF251D is assigned to VM221A on host system 220, VF251E to VM221B, and VF251F to VM221C. Similarly, SR-IOV-enabled port 250C is connected to host system 230 and may be assigned VF251G~I. VF251G is assigned to VM231A on host system 230, VF251H to VM231B, and VF251I to VM231C. In a similar configuration, SR-IOV compatible port 250D is connected to host system 240, and VF251J~L may be assigned to it. VF251J may be assigned to VM241A on host system 240, VF251K to VM241B, and VF251L to VM241C.

[0038] In some implementations, each VM211A~C, 221A~C, 231A~C, and 241A~C can access a separate namespace in one of the memory devices 130~140 using its assigned VF. For example, each VF251A~L may be assigned a specific LBA range within memory devices 130~140 dedicated to that VF. This allows the virtual machine associated with that VF to access a separate area of ​​memory. This point will be explained in more detail below.

[0039] In one implementation, each interface port 250A-D of the memory subsystem controller 115 may be connected to a separate buffer within the controller 115's memory buffer group 258. This separate buffer allows the controller 115 to process each interface port 250A-D individually and isolate the data transmitted and received between each port, thus enabling parallel operation of the interface ports 250A-D. Therefore, each interface port 250A-D can store the data received from the controller 115 in the buffer assigned to that port and then process it. Similarly, the controller 115 can store the data received from each interface port in a buffer dedicated to that port and hold it until the controller 115 is ready to process the data from that port. In one illustrative example, the memory buffer 258 enables simultaneous reception of memory access requests on interface ports 250A-D. In this case, the memory access request is held in the associated memory buffer while the current request is being processed on the corresponding interface port. Once the interface port has finished processing the current request, the next request is retrieved from the associated buffer and processed, and then any further memory access requests are added to that buffer.

[0040] Figure 3 shows the memory area mapping of virtual functions within a memory subsystem for SR-IOV / S-IOV virtualization according to some embodiments of the present disclosure. As described above, the multiport virtualization component 113 allocates a corresponding area of ​​one of the memory devices 130 to 140 to each virtual function (VF) 320A to D in the memory subsystem 110. Multiple VFs 320A to D are provided within the memory subsystem 110 using the SR-IOV 313, as detailed above. As shown in Figure 3, VF 320A is allocated area 302 of memory device 130, VF 320B is allocated area 304 of memory device 130, VF 320C is allocated area 306 of memory device 140, and VF 320D is allocated area 308 of memory device 140. In other embodiments, each of these areas may span two or more memory devices 130 to 140. Each region 302-308 may be of a fixed size or of different sizes. For example, region 302 may be larger than region 304, region 304 may be larger than region 306, and regions 306 and 308 may be the same size. In one embodiment, each region 302-308 is represented by a unique namespace. This namespace represents one or more units of memory devices 130-140 that can be formatted into logical blocks (e.g., ranges of LBA space) when the memory devices are configured with the NVMe protocol. The NVMe protocol provides access to this namespace, thereby recognizing it as a standard block device from which file systems and applications can be deployed without modification. Each virtual function 320A-D may have one or more independent namespaces, each namespace identified by a unique namespace ID (NSID).

[0041] In one illustrative example, the memory subsystem 110 can provide identification information for each VF320A-D (e.g., a virtual PCIe interface or a virtual Ethernet interface) that can be supported via the PCIe interface for device identification information. In some embodiments, each region 302-308 may be configured as a range of LBA space for memory devices 130-140. Thus, each virtual machine of a host system to which a virtual function is assigned (e.g., assigned by connecting to a VF using the device identification information of the VF) may be assigned to a different region of the logical block address space of the memory subsystem 110. The logical block address space may be mapped to the physical block address space of the memory subsystem 110.

[0042] In some embodiments, the controller 115 of the memory subsystem 110 may specify that virtual function groups be assigned to interface ports having a specific LBA range for each virtual function. In similar or different embodiments, the controller may also change the number of virtual functions assigned to each interface port. For example, the number of virtual functions provided to a virtual machine on a host system connected to a particular interface port may be increased or decreased based on the virtual machine's use of the memory subsystem's storage resources. In some embodiments, different LBA ranges (e.g., different numbers of logical block addresses with different corresponding physical block addresses) may be assigned to different virtual functions. For example, a virtual function on one interface port may be assigned a wider LBA range than a virtual function on another interface port. The different LBA ranges may be determined based on the usage patterns of different virtual machines or applications on the corresponding host systems.

[0043] Figure 4 shows an example of a memory device 400 including a multimode memory subsystem 110 with multiple ports having scalable virtualization capabilities, according to some embodiments of the present disclosure. The memory device 400 includes a multiport (e.g., quadport) physical layer (PHY) multiplexer (MUX) 402, which identifies input signals (e.g., PCIe or Ethernet), transmits PCIe signals to a PCIe physical layer (PHY) 404, and transmits Ethernet signals to a physical medium layer 410. In some embodiments, the PHY MUX 402 can process PCIe and Ethernet signals simultaneously without compromising bandwidth or reliability.

[0044] The PHY MUX402 enables the sharing of physical layer (PHY) resources, such as physical layer transceivers, across multiple network interfaces or ports. Using the PHY MUX402 reduces the number of required physical layer resources, thereby lowering the overall cost and complexity of the memory subsystem. The PHY MUX402 selectively connects physical layer resources to the appropriate network interface or port based on data flow requirements. For example, if a particular port is transmitting data, the PHY MUX402 connects the corresponding physical layer transceiver to that port, enabling transmission. Similarly, if a port is receiving data, the PHY MUX402 connects the corresponding physical layer transceiver to that port, enabling reception.

[0045] The PHY MUX402 may be connected to the PCIe PHY404, which is the physical layer interface of the Peripheral Component Interconnect Express (PCIe). The PCIe PHY404 is responsible for implementing the physical layer of the PCIe protocol, including the electrical, timing, and signaling characteristics of the PCIe interface. The PCIe PHY404 has the capability to send and receive data at high speed between PCIe devices connected to the memory subsystem 110. The PCIe PHY404 is also responsible for encoding and decoding data and sending and receiving signals on the physical layer of the PCIe interface. In some embodiments, four-level pulse amplitude modulation (PAM4) encoding may be used, which uses four voltage levels to represent a two-bit logic combination (00, 01, 10, 11). PAM4 encoding may be used on some 56GHz channels and all 112GHz channels. The PCIe PHY404 supports multiple lanes (e.g., four lanes), which can increase the overall bandwidth of the PCIe interface. Each lane operates at a specific speed, such as 2.5Gbps, 5Gbps, 8Gbps, or 16Gbps, depending on the PCIe version and the specific implementation of the PHY.

[0046] The PHY MUX 402 may be connected to a PCIe controller 406 that manages communication between the processor 430 and the PCIe bus. The PCIe bus may be used to connect various peripheral devices, such as network cards and graphics cards, to one or more processors. The PCIe controller 406 acts as a bridge between the processor 430 and the PCIe bus, enabling the processor 430 to communicate with peripheral devices connected to the bus. When a peripheral device is connected to the PCIe bus, the PCIe controller 406 assigns a unique address to the device and manages data transfer between the device and the processor 430. The PCIe controller 406 operates by sending and receiving commands and data between the processor 430 and peripheral devices over the PCIe bus. The controller manages the configuration and state of PCIe devices using a set of registers and also provides interrupt handling, error reporting, and power management functions. In some embodiments, the PCIe controller 406 may use a set of rules to determine which request gains access to the bus first, based on factors such as the device's priority level, the type of data being transferred, or the number of other devices currently using the bus.

[0047] In some embodiments, the PCIe controller 406 may include an Address Translation Service (ATS) engine 408 for translating virtual addresses to physical addresses. The ATS engine is initialized when the memory subsystem starts up and is responsible for maintaining a translation table that maps virtual addresses to physical addresses. When a program running on the memory subsystem needs to access memory, the program sends a virtual address to the ATS engine. The ATS engine uses the translation table to look up the corresponding physical address. To improve performance, the ATS engine may store recently used translation results in cache memory. This allows the ATS engine to quickly translate frequently used virtual addresses without accessing the translation table. The ATS engine may also use a coherency protocol so that all processors and devices can consistently recognize the contents of the translation table. If a translation error occurs, such as an invalid virtual address, the ATS engine may generate an exception or interrupt to notify the processor 430 of the error.

[0048] The PHY MUX402 may be connected to the Physical Medium Layer (PML) 410, which is a component of a network interface card (NIC) that transmits and receives data over a physical communication channel. The data to be transmitted is first encoded into a digital format, such as binary, and then modulated onto a carrier wave. The modulation scheme used depends on the type of communication channel and the desired transmission speed. The modulated signal is transmitted over the physical communication channel. During this process, the signal may be amplified, shaped, or filtered to compensate for attenuation, distortion, or interference caused by the channel. The received signal is processed by the PML 410 to extract the original modulated signal. This processing includes demodulation, equalization, and synchronization. The modulated signal is decoded back into its original digital format, and error correction codes are applied to correct errors that occurred during transmission. The decoded data is then passed to the link layer of the network protocol stack, where protocol headers are added, flow control is performed, and data packet transmission management is handled.

[0049] The PML410 may be connected to a Physical Coding Sublayer (PCS) 412, which is responsible for encoding and decoding data at the bit level. The PCS 412 receives data from the Media Access Control (MAC) layer and encodes it into a format suitable for transmission over the physical communication channel. This encoding may include the addition of error detection and correction codes, data scrambling to prevent pattern-dependent errors, and data formatting to facilitate transmission and reception. The encoded data is mapped to a specific modulation scheme suitable for the physical communication channel. Depending on the channel type and the desired transmission speed and reliability, the modulation scheme may include amplitude modulation, phase modulation, or frequency modulation. The mapped signal is transmitted over the physical communication channel. This signal may be amplified, filtered, or otherwise corrected to ensure accurate transmission and proper reception at the receiving end. At the receiving end, the transmitted signal is received and decoded back into the original data. The receive process extracts the original data from the signal by performing the reverse of the encoding, mapping, and transmission processes. The decoded data may contain errors due to noise, interference, or other factors. PCS412 uses error detection and correction codes added during encoding to correct errors and ensure that data is received accurately. The error-corrected data is then sent to the MAC layer for transfer to higher layers of the network protocol stack.

[0050] The PCS412 may be connected to an Ethernet Media Access Controller (MAC) 414, which controls access to the shared communication channel. The MAC414 receives data from the upper layers of the protocol stack and formats it into an Ethernet frame, which includes the source and destination addresses, frame type, and payload. The MAC414 checks the destination address of the frame and determines whether it is destined for a local memory device. If the frame is destined for a local memory device, it is processed by the upper layers of the protocol stack. On the other hand, if the frame is destined for another device on the network, the MAC414 begins transmitting the frame to the communication channel. The MAC414 also monitors the communication channel to detect if other devices may be transmitting simultaneously and checks for collisions on the channel. If a collision is detected, the MAC414 waits for a certain random time and then attempts to retransmit the frame. Furthermore, the MAC414 performs flow control to ensure that data is transmitted at an appropriate rate. This process includes adjusting the amount of data that can be sent before waiting for an acknowledgment and controlling the transmission rate based on network traffic and available bandwidth. In addition, MACs can perform address resolution, which maps higher-level addresses such as IP addresses to MAC addresses used in Ethernet frames. This process involves communicating with other devices on the network to identify their MAC addresses and maintaining a cache of these correspondences for efficient address resolution.

[0051] The Ethernet MAC 414 may be connected to a multi-target offload engine 416 to reduce the load on the processor 430 and / or host systems 210-240 and improve performance. The multi-target offload engine 416 can receive both PCIe and Ethernet signals and perform memory copy operations to move data from one location in memory to another. By performing these operations in hardware, the multi-target offload engine 416 can accelerate memory copy operations without involving the processor 430 and / or host systems 210-240. Furthermore, the multi-target offload engine 416 can implement an algorithm that moves data directly from a source memory location to a destination memory location without involving the processor 430 and / or host systems 210-240 at all. This configuration significantly improves performance and reduces latency, especially with large datasets.

[0052] In some embodiments, storing data as data objects enables memory modules implemented within a computing system to perform data processing operations, thereby improving the system's operational efficiency by, for example, offloading processing performed by host processing circuits. Specifically, memory processing circuits implemented within a memory module can access (e.g., receive, read, or retrieve) data objects containing data blocks and metadata. The memory processing circuits can determine the context of a data block based at least on the metadata and perform data processing operations accordingly. In this way, memory processing circuits implemented within a memory module can, at least in some cases, offload processing performed by host processing circuits by post-processing the data, i.e., performing data processing (e.g., encoding or compression) on the data before storage, thereby improving the operational efficiency of the corresponding computing system. Similarly, memory processing circuits implemented within a memory module can, at least in some cases, offload processing performed by host processing circuits by pre-processing the data, i.e., performing data processing (e.g., decoding and / or decompression) on the data before output, thereby improving the operational efficiency of the corresponding computing system. Furthermore, the technology of this disclosure can further improve operational efficiency by leveraging the data communication efficiency provided by an internal bus implemented on the memory module, in addition to offloading (e.g., reducing) processing performed by the host processing circuit. By implementing and / or operating a memory module in accordance with the technology described herein, the memory module can perform data processing operations that offload (e.g., reduce) processing performed by the main (e.g., host) processing circuit of the computing system. For example, a dedicated (e.g., memory) processing circuit implemented within the memory module can preprocess data before outputting it to the main processing circuit, or postprocess data received from the main processing circuit before storing it.

[0053] The multi-target offload engine 416 may be connected to an NVMe-oF offload engine 418 to accelerate the processing of NVMe over Fabric (NVMe-oF) traffic. When a network interface card (NIC) receives an NVMe-oF packet, the offload engine 418 decapsulates the packet, separating NVMe commands and data from the network header. The offload engine 418 processes the NVMe commands within the packet and forwards them to the NVMe controller 420 for execution. The offload engine 418 also processes the data within the packet, handling tasks such as buffer allocation, data copying, and data integrity verification. In some implementations, the offload engine 418 may also be able to handle remote direct memory access (RDMA) operations. Once the execution of the NVMe commands is complete, the offload engine 418 generates a completion packet and sends it back to the host system. By offloading the processing of NVMe-oF traffic, the offload engine 418 can improve performance and reduce CPU utilization. Furthermore, by utilizing RDMA, network latency and CPU overhead can be further reduced.

[0054] Both the PCIe controller 406 and the NVMe-oF offload engine 418 are connected to the NVMe controller 420. The NVMe controller 420 is responsible for managing data transfer between the host system and memory devices 130-140, as well as performing various memory management and error correction processes. The NVMe controller 420 receives commands from the host system via the PCIe interface and processes these commands to read or write data to memory devices 130-140. Once the controller 420 has processed the commands, it controls data transfer between the host system and memory devices 130-140. This process includes managing read and write paths, encrypting and decrypting data as needed, and ensuring data integrity. The controller 420 manages the memory on memory devices 130-140 and performs operations including wear leveling and garbage collection. Wear leveling is a process that distributes write operations across memory devices 130-140 to prevent premature degradation of specific areas. Garbage collection is the process of identifying and erasing data blocks that are no longer needed, thereby freeing up storage space. Furthermore, the controller 420 is also responsible for error correction, performing tasks such as detecting and correcting data errors and handling bad blocks. In some embodiments, the NVMe controller 420 also includes power management functions, enabling a reduction in power consumption and extension of the lifespan of the memory subsystem 110. For example, these power management functions may include techniques such as power gating, which turns off the power to unused portions of memory devices 130-140.

[0055] The NVMe controller 420 may include a multi-port (e.g., quad-port) NVMe controller 422. Furthermore, the NVMe controller may include a multi-DMA (e.g., quad-DMA) engine 424 and a multi-port multifunction (MPMF) arbiter 426. The multi-direct memory access (multi-DMA) engine 424 can accelerate data transfer between different devices, such as between the processor 430 and memory devices 130-140, or between the processor 430 and input / output (I / O) devices. When the memory subsystem 110 is powered on, the multi-DMA engine 424 is initialized and configured based on system parameters. The engine 424 manages a request queue that holds pending data transfer requests from the processor 430 or I / O devices. Upon receiving a data transfer request, the multi-DMA engine 424 performs the transfer using one or more DMA channels. A DMA channel is a hardware resource that enables direct data transfer between devices without going through the processor 430. Once data transfer is complete, engine 424 generates an interrupt to notify processor 430 that the data is available. The multi-DMA engine 424 includes an error handling mechanism to detect and correct any errors that may occur during data transfer. To optimize performance, the multi-DMA engine 424 can utilize advanced techniques such as scatter / gather DMA. This allows data to be transferred from multiple non-contiguous memory address regions to a single destination without intermediate copies.

[0056] A multi-port multifunction (MPMF) arbiter 426 is responsible for controlling access to shared resources (such as memory or I / O buses) by multiple devices. The arbiter 426 manages a request queue that holds pending requests from multiple devices. The arbiter 426 can use a priority scheme to determine which device should be granted access to the shared resource next. This priority scheme is configured based on factors such as device type, request type, or round-robin scheduling. Upon receiving a request, the arbiter 426 determines which device should be granted access next based on the priority scheme and sends a grant signal to the selected device indicating permission to access. Furthermore, the arbiter 426 may incorporate timing and synchronization features to ensure devices access shared resources in a fair and predictable manner. These include features such as fixed arbitration intervals, time slicing, and backoff algorithms. Additionally, the MPMF arbiter 426 may include error handling mechanisms to detect and resolve potential conflicts and errors that may occur during arbitration. Furthermore, the MPMF arbiter 426 can support multiple functions or device types by assigning individual resources to each function or device type and independently managing access to each resource.

[0057] Device 400 may include one or more processors 430, which may be connected to a memory buffer and memory manager 432, an Advanced Cryptographic Standard (AES) engine and a RAIN / NAND management controller 434, and one or more NAND Error Correction Code (ECC) engines 436. The AES engine 434 can speed up encryption and decryption processes. Before encryption or decryption, the AES engine 434 first sets the keys. This process involves initializing a key expansion algorithm using a secret key and generating a set of round keys used in the encryption or decryption process. In AES encryption, the plaintext is divided into several blocks, and each block is processed by a series of encryption rounds. In each round, a transformation combining substitution, transposition, and mixed operations is applied to the block, and a round key is added at each stage. AES decryption is the reverse process of encryption, where each block is transformed through a series of decryption rounds. In each round, the reverse transformation of the operations performed during encryption is performed, and a round key is added at the same stages as during encryption. To optimize performance, the AES engine 434 can utilize techniques such as pipeline processing, parallel processing, and data caching. Pipelining is a method that divides the encryption or decryption process into multiple stages and processes multiple blocks simultaneously. Parallel processing uses multiple processing units to perform encryption or decryption simultaneously. Data caching is a technique that reduces access time by keeping frequently used data in a small, high-speed memory. Furthermore, the AES engine 434 also includes an error handling mechanism to detect and correct errors such as data corruption and key mismatches that may occur during encryption or decryption.

[0058] An independent NAND redundant array (RAIN) / NAND management controller 434 may be used to manage data storage and retrieval. For example, NAND flash memory is organized into blocks consisting of multiple pages. The NAND management controller 434 is responsible for managing these blocks and performing wear leveling, error correction, and bad block management. Wear leveling distributes writes evenly across all blocks to prevent any particular block from degrading prematurely. Error correction codes are used to detect and correct errors that occur during data transfer. Bad block management, on the other hand, is the process of identifying and marking blocks that have become unusable due to physical defects. The NAND management controller 434 also manages each page within a block. This includes performing read and write operations, as well as erase operations. The controller 434 must ensure that data is written and read accurately, and that erased blocks are in a reusable state. Memory devices 130-140 store data in a series of pages and blocks, and they need to be properly organized and managed to enable efficient access and retrieval. The NAND management controller 434 organizes data into logical blocks, facilitating data management and retrieval. Furthermore, the NAND management controller 434 connects to the host system via standard interfaces such as SPI (Serial Peripheral Interface) and SDIO (Secure Digital Input / Output), managing communication protocols and ensuring accurate and efficient data transfer between the host system and flash memory. To optimize performance, the NAND management controller 434 can utilize techniques such as data caching, compression, and error correction code. Data caching is a method of reducing access time by storing frequently accessed data in small-capacity, high-speed memory, while compression is used to reduce the amount of data that needs to be written to flash memory. Additionally, error correction code can reduce the impact of errors on data integrity.

[0059] One or more NAND error correction code (ECC) engines and channels 436 may be used to detect and correct errors occurring within a NAND flash memory device. The NAND ECC engine 436 operates by adding additional data bits to each memory page written to the NAND flash device. These additional bits, called parity bits or ECC bits, are calculated by complex algorithms designed to detect and correct errors in the data. When reading data from the NAND flash memory device, the NAND ECC engine 436 reads the data and its corresponding parity bits. The engine then uses the parity bits to check for errors in the data. If the NAND ECC engine 436 detects an error in the data, it reports the error to the host device, allowing the host device to perform appropriate corrective actions. Conversely, when writing data, the NAND ECC engine 436 calculates parity bits based on the data being written. The engine writes the data to the NAND flash memory device along with the calculated parity bits. If an error is detected during a subsequent read operation, the NAND ECC engine 436 uses the parity bits to correct the error in the data. In this way, the accuracy and reliability of the data stored in the NAND flash memory device are ensured.

[0060] Figure 5 shows an example of a multimode memory subsystem 110 with multiple ports having scalable virtualization capabilities, according to some embodiments of the present disclosure. In this embodiment, device 500 is similar to device 400, except that the Ethernet media access controller 414 can identify the type of Ethernet signal received and route it to the appropriate offload engine based on the type of Ethernet packet received. For example, the Ethernet media access controller 414 can identify Remote Direct Memory Access over Converged Ethernet Version 2 (ROCEv2) signals and forward them to the ROCEv2 offload engine 428, or identify TCP / IP data packets and forward them to the TCP / IP offload engine 438. ROCEv2 is a protocol that enables low-latency, high-bandwidth Ethernet communication between ports and devices connected to memory devices 130-140. The ROCEv2 offload engine 428 reduces the load on processors 430 and / or host systems 210-240 and improves performance. The ROCEv2 offload engine 428 performs memory copy operations, moving data from one location in memory to another. By performing these operations in hardware, this engine can accelerate memory copy operations without involving the processor 430 and / or the host systems 210-240. Furthermore, the ROCEv2 offload engine 428 can implement algorithms that directly transfer data from a source memory location to a destination memory location without going through the processor 430 and / or the host systems 210-240 at all. This significantly improves performance and reduces latency, especially with large datasets.

[0061] The ROCEv2 offload engine 428 may forward ROCEv2 data packets to the NVMe over Fabric (NVMe-oF) remote direct memory access (RDMA) offload engine 440, since ROCEv2 contains one or more RDMA packets. The NVMe-oF RDMA offload engine 440 speeds up the processing of the NVMe-oF protocol by offloading some of the processing performed on the host systems 210-240 to the offload engine 440. The NVMe-oF RDMA offload engine 440 is initialized by a software driver on the host systems 210-240. This driver sets the necessary parameters, such as the target IP address and the data size to be transferred for the offload engine. When the software driver requests data transfer, the NVMe-oF RDMA offload engine 440 takes over the data transfer processing and sends the data to the target system using the NVMe-oF protocol. The NVMe-oF RDMA offload engine 440 performs various functions, such as establishing RDMA connections, data partitioning and reconfiguring, and flow control processing, in order to offload processing from host systems 210-240.

[0062] Once data transfer is complete, the NVMe-oF RDMA offload engine 440 sends an acknowledgment message to the host software driver to notify it that the data has been successfully transferred. By offloading some of the processing from the host CPU to the offload engine, the NVMe-oF protocol is accelerated, latency is reduced, and bandwidth is improved.

[0063] Similarly, the TCP / IP offload engine 438 may forward the TCP / IP packets to the NVMe-oF TCP offload engine 442 because the signals contain TCP / IP packets. The NVMe-oF TCP offload engine (TOE) 442 speeds up the processing of the NVMe-oF protocol over the TCP / IP network by offloading some of the processing that would otherwise be performed on the host systems 210-240 to the offload engine.

[0064] In this embodiment, one or more NAND ECC engines and channels 436 may include a media QoS scheduler 444 that assigns a priority corresponding to the media type from among multiple priority levels based on the quality of service (QoS) requirements for the media type. The media QoS scheduler 444 manages the transmission of multimedia data such as video and audio at different priority levels and is responsible for maintaining a predetermined quality of service for each stream. First, the media QoS scheduler 444 classifies the incoming traffic into multiple priority levels based on the media type and its quality requirements. Subsequently, it applies traffic shaping to control the amount of traffic for each priority level so as not to exceed a predetermined maximum bandwidth.

[0065] In some embodiments, the multiport multifunction (MPMF) arbiter 426 may include a Quality of Service (QoS) bandwidth control component. QoS bandwidth control is used to allocate and prioritize bandwidth on a network, aiming to ensure that critical applications have the bandwidth necessary to function properly. QoS bandwidth control categorizes communications into several categories based on traffic priority. This categorization can be performed by parsing packet headers or by identifying traffic flows using application-level information. Once traffic is categorized, the QoS bandwidth controller allocates bandwidth to each traffic category based on its priority. Various techniques can be used for this, such as rate limiting, traffic shaping, or traffic policing.

[0066] Figure 6 is a flow diagram illustrating an example of a method for managing a memory subsystem with multiple interface ports having SR-IOV / S-IOV functionality, according to some embodiments of the present disclosure. Method 600 may be executed by processing logic including hardware (e.g., processing devices, circuits, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions executed on the processing devices), or a combination thereof. In some embodiments, Method 600 is executed by the multi-port virtualization component 113 shown in Figure 1. Although shown in a specific order or sequence, the order of processing may be changed unless otherwise specified. Therefore, the illustrated embodiments are merely examples, the illustrated processing may be executed in a different order, and some processing may be executed in parallel. Furthermore, one or more processing may be omitted in each embodiment. Therefore, not all processing is required in all embodiments. Other process flows are also possible.

[0067] In operation 610, the processing logic detects a first host system, which is one of several host systems that can be connected to the memory device. The first host system is connected to a first interface port among several interface ports of the memory device. In one embodiment, this first interface port may be a Peripheral Component Interconnect Express (PCIe) port, and as already described in detail, each PCIe port may be SR-IOV / S-IOV compatible.

[0068] In one example, multiple interface ports can be accessed simultaneously by multiple host systems without the need for separate switches or bridges. Therefore, the memory subsystem can provide the host systems with simultaneous access to its storage device via multiple interface ports, as described above.

[0069] In operation 620, the processing logic detects a second host system, which is one of several host systems that can be connected to the memory device. The second host system is connected to a second interface port of the memory device, which is different from the first interface port. In some implementations, the second interface port may be an Ethernet port, and as already described in detail, each Ethernet port may be SR-IOV / S-IOV compatible.

[0070] In operation 630, the processing logic assigns a first subset of virtual functions (VFs) associated with a memory device to a first host system using root input / output virtualization (SR-IOV). In one implementation, the first VF subset corresponds to a group of virtual PCIe interfaces that share the physical resources of each interface port. Furthermore, for each of multiple host systems, the processing logic can assign the corresponding VF from the corresponding VF subset assigned to that host system to the corresponding virtual machine among multiple virtual machines running on that host system. This point will be explained in more detail below.

[0071] In operation 640, the processing logic assigns a first range of logical block addresses (LBAs) of a memory device to each VF of a first subset of virtual functions (VFs) assigned to the first host system. In one implementation, the logical block address space may be mapped to the physical block address space of one or more memory devices of the memory subsystem, as described above.

[0072] In operation 650, the processing logic assigns a second subset of virtual functions (VFs) associated with a memory device to a second host system using root input / output virtualization (SR-IOV). In one implementation, the second VF subset corresponds to a group of virtual Ethernet interfaces that share the physical resources of each Ethernet port. Furthermore, for each of the multiple host systems, the processing logic can assign the corresponding VF from the corresponding VF subset assigned to that host system to the corresponding virtual machine among the multiple virtual machines running on that host system. This point will be explained in more detail below.

[0073] In operation 660, the processing logic assigns a second range of logical block addresses (LBAs) of a memory device to each VF in a second subset of virtual functions (VFs) assigned to the second host system. In some implementations, the logical block address space may be mapped to the physical block address space of one or more memory devices in the memory subsystem, as has been described in detail.

[0074] Figure 7 shows an example of how to assign virtual functions to a virtual machine to support multiple SR-IOV-enabled interface ports of a memory subsystem. This method 700 can be executed by processing logic including hardware (e.g., processing devices, circuits, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions executed on the processing device), or a combination thereof. In one embodiment, this method 700 is executed by the multi-port virtualization component 113 in Figure 1. Although shown in a specific order, the order of the steps can be changed unless otherwise specified. Therefore, the illustrated embodiment is illustrative only, the illustrated steps may be executed in a different order, and some steps may be executed in parallel. Furthermore, depending on the embodiment, one or more steps may be omitted. Therefore, not all steps are mandatory. Other processing flows are also possible.

[0075] In operation 710, the processing logic provides the host SOC with access to the memory device using an SR-IOV-enabled port. In one implementation, the memory subsystem can detect that the host SOC is connected to one of several PCIe ports, as will be described in more detail below. Furthermore, upon detecting the host SOC, the memory subsystem can assign that PCIe port to the host SOC by providing the host SOC with the device identification information of the SR-IOV-enabled PCIe port.

[0076] In operation 720, the processing logic detects multiple virtual functions assigned to the PCIe port. In one implementation, the memory subsystem can identify the virtual functions supported by the PCIe bus or interface. For example, the memory subsystem can provide identification information for each virtual function (e.g., a virtual PCIe interface) that can be supported via the PCIe interface of the identified port, as will be described in more detail below.

[0077] In operation 730, the processing logic detects a first virtual machine (VM) and a second VM running on the host SOC. In one implementation, each VM on the host SOC may be assigned a dedicated virtual function (VF) of a PCIe port to access the corresponding storage area of ​​the memory subsystem. Thus, in operation 740, the processing logic assigns the first VF of the PCIe port to the first VM on the host SOC. All memory access requests from the first VM are handled by the first VF of the PCIe port. Similarly, in operation 750, the processing logic assigns the second VF of the PCIe port to the second VM on the host SOC. All memory access requests from the second VM are handled by the second VF of the PCIe port. As will be described in more detail below, each VM may have a dedicated area in the memory device of the memory subsystem by using the VF of the PCIe port assigned to the host SOC.

[0078] Figure 8 shows an example of a computer system 800 machine, in which a set of instructions for causing the machine to perform any of the methodologies described herein is executable. In one embodiment, the computer system 800 may correspond to a host system (e.g., host system 120 in Figure 1) that includes, is connected to, or utilizes a memory subsystem (e.g., memory subsystem 110 in Figure 1), or may be used to perform controller operations (e.g., running an operating system that performs operations corresponding to the multi-port virtualization component 113 in Figure 1). In another embodiment, the machine may be connected to other machines (e.g., network connection) over a LAN, intranet, extranet, and / or the internet. The machine may operate as a server or client machine in a client-server network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment.

[0079] This machine may be any personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, server, network router, switch or bridge, or any other machine capable of executing a set of instructions (whether sequential or otherwise) that specify the actions that the machine should perform. Although a single machine is illustrated, the term “machine” should be understood to also include a collection of multiple machines that individually or collectively execute a set (or multiple sets) of instructions in order to perform one or more methodologies described herein.

[0080] An example of a computer system 800 includes a processing device 802, main memory 804 (e.g., read-only memory (ROM), flash memory, synchronous DRAM (SDRAM), or rhombus DRAM (RDRAM) or other dynamic random access memory (DRAM)), static memory 806 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 818, which communicate with each other via a bus 830.

[0081] Processing device 802 represents one or more general-purpose processing devices, such as a microprocessor or a central processing unit (CPU). More specifically, the processing device may be a complex instruction set computing (CISC) microprocessor, a simple instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or a processor implementing a combination of these instruction sets. Alternatively, processing device 802 may be one or more dedicated processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor. Processing device 802 is configured to execute instruction 826 for performing the operations and steps described herein. Furthermore, the computer system 800 may include a network interface device 808 for communication over network 820.

[0082] The data storage system 818 includes a machine-readable storage medium 824 (also called a computer-readable medium) on which one or more sets of instructions 826 or software are stored, which embody one or more methodologies or functions described herein. When the instructions 826 are executed by the computer system 800, all or at least part of them may be stored in the main memory 804 and / or processing device 802. The main memory 804 and processing device 802 also constitute a machine-readable storage medium. The machine-readable storage medium 824, the data storage system 818, and / or the main memory 804 may correspond to the memory subsystem 110 in Figure 1.

[0083] In one embodiment, instruction 826 includes instructions for implementing functions corresponding to multiple SR-IOV / S-IOV port components 113 in Figure 1. Although the machine-readable storage medium 824 is shown as a single medium in the exemplary embodiment, the term “machine-readable storage medium” should be understood to include a single or multiple mediums that store one or more sets of instructions. Furthermore, the term “machine-readable storage medium” should be understood to include any medium that can store or encode a set of instructions to be executed by a machine, thereby causing the machine to execute any methodology of the Disclosure. Accordingly, the term “machine-readable storage medium” is not limited to solid memory, optical media, and magnetic media, but includes them.

[0084] Some of the detailed explanations above describe operations on data bits in computer memory in the form of algorithms and symbolic representations. These descriptions and representations of algorithms are methods used by those skilled in the field of data processing to most effectively communicate the essence of their work to others skilled in the field. An algorithm, here and generally, is considered a self-contained set of processing steps to produce a desired result. These operations require the physical manipulation of physical quantities. These quantities, though not always necessary, often take the form of electrical or magnetic signals that can be stored, combined, compared, and otherwise manipulated. For the convenience of common terminology, these signals are often referred to as bits, values, elements, symbols, characters, terms, numbers, etc.

[0085] However, it should be noted that all these terms and similar terms should be understood in relation to the appropriate physical quantities, and are merely labels conveniently assigned to those physical quantities. This disclosure may refer to the operation and processing of computer systems or similar electronic computing devices. These devices manipulate and transform data represented as physical (electronic) quantities in the registers and memory of the computer system, and similarly transform it into other data represented as physical quantities, or data in other information storage systems.

[0086] This disclosure also relates to an apparatus for performing the processes described herein. This apparatus may be specifically configured for the intended purpose, or it may include a general-purpose computer that is selectively started or reconfigured by a computer program stored within the computer. Such computer programs may be stored on a computer-readable storage medium connected to a computer system bus, such as various disks including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic or optical cards, or any other medium suitable for storing electronic instructions.

[0087] The algorithms and representations disclosed herein are not inherently related to any particular computer or other device. Programs based on the disclosures herein can be used in a variety of general-purpose systems, or it may be convenient to configure more specialized equipment to perform such methods. These various system configurations are described below. Furthermore, this disclosure is not written with reference to any particular programming language. It will be understood that a variety of programming languages ​​can be used to implement the disclosures described herein.

[0088] This disclosure may be provided as a computer program product or software. This program product or software may include instructions stored on a machine-readable medium, which may be used to program a computer system (or other electronic device) to perform processing in accordance with this disclosure. A machine-readable medium is any mechanism that stores information in a format that can be read by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes machine (e.g., computer)-readable storage media such as read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, and flash memory components.

[0089] In the above specification, embodiments of the present disclosure have been described with reference to specific exemplary embodiments. It will be apparent that various modifications can be made without departing from the broad spirit and scope of the embodiments of the present disclosure described in the following claims. Accordingly, this specification and the drawings should be understood in an exemplary sense, not in an restrictive sense.

Claims

1. Memory devices and, Multiple interface ports operably connected to the aforementioned memory device, A system comprising a processing device operably connected to the aforementioned memory device, The processing device is A process to identify a first host system connected to a first interface port among the plurality of interface ports, wherein the first interface port includes a PCIe (Peripheral Component Interconnect Express) port, and the first host system is running a plurality of first virtual machines (VMs), A process for identifying a second host system connected to a second interface port among the plurality of interface ports, wherein the second interface port includes an Ethernet port, and the second host system is running a plurality of second VMs. A process of assigning a first subset of a plurality of virtual functions (VFs) associated with the memory device to the first host system, A process of assigning a first range of logical block addresses (LBAs) of the memory device to each VF of the subset of the first virtual function, A process of assigning a second subset of the plurality of virtual functions associated with the memory device to the second host system, A process of assigning a second range of the LBA of the memory device to each VF of the subset of the second virtual function, A process of assigning a first VF from a first subset of the plurality of VFs to a first VM among the plurality of first VMs, The operation includes the process of assigning a second VF from a second subset of the plurality of VFs to a second VM among the plurality of second VMs, A system in which a first subset of the plurality of VFs corresponds to a plurality of virtual PCIe interfaces that share the physical resources of a PCIe port, and a second subset of the plurality of VFs corresponds to a plurality of virtual Ethernet interfaces that share the physical resources of an Ethernet port.

2. The system according to claim 1, wherein the plurality of VFs are represented by at least one of Scalable I / O Virtualization (S-IOV) VFs, Single Root I / O Virtualization (SR-IOV) VFs, Multi-Physical Function (Multi-PF) Virtualization VFs, or Multi-Target Network Function Virtualization (NFV) VFs.

3. The system according to claim 2, wherein the first interface port and the second interface port are activated by at least one of S-IOV or SR-IOV.

4. The system according to claim 1, wherein the process further comprises: The process involves routing Ethernet signals from the second interface port to the physical media layer transceiver of the memory subsystem controller, The process of routing the Ethernet signal from the physical media layer transceiver to the Ethernet media access controller of the memory subsystem controller, A system that includes this.

5. The system according to claim 4, wherein the processing device further comprises A process to identify a first type of Ethernet frame based on the header of the frame decoded from the Ethernet signal, The process of routing the first type of Ethernet signal to a first off-road engine operating on the network adapter of the output interface, A system configured to perform the following actions.

6. The system according to claim 1, wherein the processing device further comprises A system configured to perform a process of assigning one of several priority levels to a data type based on quality of service (QoS) requirements relating to the type of frame decoded from the Ethernet signal.

7. A system according to claim 1, wherein the plurality of host systems are configured to access the memory device without going through individual switches or individual bridges.

8. A processing device detects that a first host system, which is one of a plurality of host systems, is connected to a first interface port among a plurality of interface ports of a memory device, wherein the first interface port includes a Peripheral Component Interconnect Express (PCIe) port. A step of detecting that a second host system, which is one of a plurality of host systems, is connected to a second interface port among a plurality of interface ports of a memory device, wherein the second interface port includes an Ethernet port. A step of assigning a first subset of a plurality of virtual functions (VFs) associated with the memory device to the first host system using virtualization, A step of assigning a first corresponding range of logical block addresses (LBAs) of the memory device to each VF of the subset of the first virtual function, A step of assigning a second subset of a plurality of virtual functions associated with the memory device to the second host system using virtualization, A step of assigning a second correspondence range of the LBA of the memory device to each VF of the subset of the second virtual function, A method that includes this.

9. A method according to claim 8, wherein the plurality of virtual functions (VFs) are represented by at least one of scalable input / output virtualization (S-IOV), single-root input / output virtualization (SR-IOV), multi-physical function (Multi-PF) virtualization, or multi-target network function virtualization (NFV).

10. The method according to claim 9, wherein the first interface port and the second interface port are S-IOV compatible or SR-IOV compatible.

11. A method according to claim 8, wherein a first VF subset corresponds to a plurality of virtual PCIe interfaces sharing the physical resources of a PCIe port, and a second VF subset corresponds to a plurality of virtual Ethernet interfaces sharing the physical resources of an Ethernet port.

12. The method according to claim 8, further, A step of assigning a first VF of a first subset of the plurality of VFs to a first VM among a plurality of first virtual machines (VMs) of a first host system, A step of assigning a second VF of a second subset of the plurality of VFs to a second VM among a plurality of second VMs of a second host system, Methods that include...

13. The method according to claim 8, further, The process involves routing Ethernet signals from the second interface port to the physical media layer transceiver of the memory subsystem controller, The process of routing the Ethernet signal from the physical media layer transceiver to the Ethernet media access controller of the memory subsystem controller, Methods that include...

14. The method according to claim 13, further, A step of identifying a first type of Ethernet signal based on the header within the Ethernet signal, The process involves routing the first type of Ethernet signal to a first off-road engine operating on the network adapter of the output interface, A step of identifying a second type of Ethernet signal based on the header in the Ethernet signal, The process involves routing the second type of Ethernet signal to a second off-road engine operating on the network adapter of the output interface, Methods that include...

15. A method according to claim 8, wherein the plurality of host systems are provided with access to the memory device without the need for individual switches or individual bridges.

16. Memory devices and, Multiple interface ports operationally coupled to the memory device, A system comprising a processing device which is operationally coupled to the memory device and performs the following processes, The processing device is A process that provides access to multiple host systems utilizing the memory device using multiple interface ports, wherein the multiple interface ports include at least a Peripheral Component Interconnect Express (PCIe) port and an Ethernet port, and each of the multiple interface ports is connected to a different host system among the multiple host systems, A process of assigning a corresponding subset of multiple virtual functions (VFs) of the memory device to each of the multiple host systems using virtualization, A system that performs actions including those mentioned above.

17. The system according to claim 16, wherein the processing device further comprises the plurality of host systems, A system that performs the process of assigning a corresponding range of logical block addresses (LBAs) of the memory device to each virtual function in a subset of the corresponding virtual functions assigned to each of the host systems.

18. The system according to claim 16, wherein the virtualization includes at least one of scalable input / output virtualization (S-IOV), single-root input / output virtualization (SR-IOV), multi-physical function (Multi-PF) virtualization, or multi-target network function virtualization (NFV).

19. The system according to claim 18, wherein the PCIe port and the Ethernet port are S-IOV compatible or SR-IOV compatible.

20. The system according to claim 16, wherein the processing device further comprises A process of assigning a first VF of a first subset of the plurality of VFs to a first VM among a plurality of first virtual machines (VMs) of a first host system, A process of assigning a second VF from a second subset of the plurality of VFs to a second VM among a plurality of second VMs of the second host system, A system that performs actions including those mentioned above.

Images