Storage system, storage device, and method for controlling storage device
By designing multi-mode and multi-speed storage operation capabilities in NVMe-oF systems, using motherboard, substrate management controller and midplane, combined with FPGA and GPIO control, multi-mode and multi-mode in NVMe-oF configurations in the prior art are solved. The difficulty of speed operation achieves efficient storage performance and flexibility.
Patent Information
- Application Number
- CN201910053414.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-06-13
- Filing Date
- 2019-01-21
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2039-01-21
AI Technical Summary
Prior art In NVMe-oF configurations, especially in Ethernet-attached SSD systems, it is difficult to effectively support multi-mode and multi-speed storage operations, resulting in performance and cost challenges, and network-attached SSD designs require handling point-to-point connectivity. and erase encoded data protection, increasing design complexity.
A system is designed, including a motherboard, substrate management controller, a middle plane and a storage device, receive inputs through multiple device ports and select operating modes and speeds, and control the storage device in NVMe and NVMe-oF modes using FPGA and GPIO pins. It operates under the operation, supports 10G to 100G Ethernet speeds, and uses U.2 connectors and PCIe bus to achieve multi-mode and multi-speed storage operations.
It enables storage devices that support NVMe and NVMe-oF protocols to operate at multiple Ethernet speeds without changing the hardware, reducing development and deployment costs, improving system flexibility and adaptability, and able to support different types of NVMe-oF device and network architecture.
Smart Images

Figure CN110221769B_ABST
Abstract
Description
[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 637,340, filed on March 1, 2018, and U.S. Patent Application No. 16 / 007,949, filed on June 13, 2018, the contents of which are incorporated herein by reference in their entirety. Technical Field
[0002] One or more aspects of embodiments according to the present invention relate to network attached devices, and more particularly, to systems and methods for supporting multi-mode and / or multi-speed Non-volatile memory (NVM) transport standard over a network (NVMe) (Non-volatile memory express over fabrics, NVMe-oF) devices. Background Art
[0003] In the case of NVMe-oF configurations (e.g., configurations using Ethernet-attached NVMe solid-state drives (SSDs)), improving Ethernet and SSD costs and performance may be challenging. For example, while SSD performance may rely on the Peripheral Component Interconnect Express (PCIe) interface and NAND technology, Ethernet speeds have increased with the advent of 50G / 100G technology. Network-attached SSDs may present additional unique design challenges for supporting erasure coding data protection because each device may provide point-to-point connectivity.
[0004] Therefore, a storage device that can support both NVMe and NVMe-oF protocols and operate at different Ethernet speeds may be desired.
[0005] The information disclosed in the above Background section is only for enhancement of understanding of the background of the present invention and therefore it may contain information that does not constitute the prior art. Summary of the invention
[0006] Aspects of embodiments of the present disclosure are directed to a system comprising: at least one mainboard; at least one baseboard management controller (BMC); a midplane; and at least one storage device, wherein the at least one storage device is configured to: operate in a first mode or a second mode based on a first input received from the at least one mainboard or the at least one BMC via a plurality of device ports through the midplane; and when operating in the second mode, the at least one storage device is configured to: operate at a first speed among a plurality of operating speeds based on a second input received from the midplane via the plurality of device ports.
[0007] In one embodiment, the at least one storage device includes a plurality of solid state drives (SSDs) communicating with at least one field programmable gate array (FPGA) via a connector and at least one bus. In one configuration, the connector is an SSD connector and the at least one bus is a peripheral component interconnect express (PCIe) bus.
[0008] In another embodiment, the first mode and the second mode of the at least one storage device are Non-Volatile Memory Express (NVMe) mode and NVMe over a Network (NVMe-oF) mode, respectively. In one configuration, the first input is controlled by using a physical pin on a chassis of the at least one motherboard or by an in-band command from the at least one BMC.
[0009] In one embodiment, the second input is controlled by using two general purpose input / output (GPIO) pins, wherein the two GPIO pins are controlled by one or more internal registers within a local central processing unit (CPU) of the at least one BMC or the at least one motherboard or a field programmable gate array (FPGA) of the at least one storage device. In another embodiment, the plurality of operating speeds of the at least one storage device are two or more speeds above 10G (e.g., between 10G and 100G, or between 10G and speeds above 100G).
[0010] In one aspect, the plurality of device ports are connected to the at least one storage device via a U.2 connector, and the storage device is configured to operate in a high availability (HA) mode or a non-HA mode. In another aspect, the at least one storage device supports two serially attached small computer system interface (SAS) ports and up to four PCIe X4 lanes of a peripheral component interconnect express (PCIe) X4 bus via a U.2 connector, wherein the two SAS ports are used as network attached ports, wherein the network attached ports include an Ethernet port, a network channel port, and an InfiniBand port.
[0011] In one configuration, when operating in the second mode, the at least one storage device supports two of the four PCIe X4 lanes for control planes of two network attached ports and the remaining two of the four PCIe X4 lanes as additional network attached ports via the U.2 connector. In another configuration, the at least one storage device includes a first storage device and a second storage device, wherein at a first time, the first storage device and / or the second storage device operates in the first mode or the second mode at a first speed or a second speed among the plurality of speeds.
[0012] According to an embodiment of the present invention, a method is provided, the method comprising: receiving a first input at a storage device, wherein the first input is received at the storage device from at least one mainboard or baseboard management controller (BMC) via a midplane via a plurality of device ports; determining, by the storage device, whether to operate in a first operating mode or a second operating mode based on the first input received at the storage device; when operating in the second mode, receiving a second input from the midplane at the storage device via the plurality of device ports; and selecting, by the storage device, an operating speed of the storage device from a plurality of operating speeds based on the second input.
[0013] In one embodiment, the first mode and the second mode of the storage device are non-volatile memory transport standard (NVMe) mode and NVMe over network (NVMe-oF) mode. In one embodiment, the first input is controlled by using a physical pin on a chassis of the at least one motherboard or by an in-band command from a BMC.
[0014] In one aspect, the second input is controlled by using two general purpose input / output (GPIO) pins, wherein the two GPIO pins are controlled by one or more internal registers within a BMC or a local central processing unit (CPU) of the at least one motherboard or a field programmable gate array (FPGA) of the storage device. In another aspect, wherein the plurality of operating speeds of the storage device are two or more speeds above 10G.
[0015] According to another embodiment of the present invention, a storage device is provided, comprising: a plurality of solid-state drives (SSDs), communicating with at least one field programmable gate array (FPGA) via a first connector and at least one bus, wherein the storage device is configured to operate in a first mode or a second mode based on a first input received via a second connector, wherein when operating in the second mode, the storage device is configured to operate at a first speed among a plurality of operating speeds based on a second input received via the second connector.
[0016] In one aspect, a first input is received from a motherboard or a baseboard management controller (BMC) of the switch via a midplane via a plurality of device ports, and a second input is received from the midplane via the plurality of device ports, wherein the first input is controlled by using a physical pin on a chassis of the motherboard or by an in-band command from the BMC, and the second input is controlled by using two general purpose input / output (GPIO) pins, wherein the two GPIO pins are controlled by one or more internal registers within a local central processing unit (CPU) of the BMC or motherboard or a field programmable gate array FPGA of the at least one storage device.
[0017] On the other hand, the first connector is an SSD connector, the at least one bus is a Peripheral Component Interconnect Express (PCIe) bus, the second connector is a U.2 connector, the first mode and the second mode of the storage device are a Non-Volatile Memory Express (NVMe) mode and an NVMe over a network (NVMe-oF) mode, respectively, and the multiple operating speeds of the storage device are two or more speeds above 10G.
[0018] In one configuration, at a first time, a first SSD and / or a second SSD of the plurality of SSDs operates in a first mode or a second mode at a first speed or a second speed among the plurality of speeds. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] These and other features of some example embodiments of the present invention will be appreciated and understood with reference to the specification, claims and drawings, in which:
[0020] Figure 1A illustrates the configuration of an NVMe-oF device according to some example embodiments of the present invention;
[0021] Figure 1B A block diagram illustrating an example SSD according to some example embodiments of the present invention;
[0022] Figure 2 A block diagram illustrating a mid-plane or complex programmable logic device (CPLD) according to some example embodiments of the present invention;
[0023] Figure 3 showing an example table illustrating example usage of a U.2 connector according to configuration of an example multi-mode NVMe-oF device according to some example embodiments of the present invention;
[0024] Figure 4A and Figure 4B A block diagram illustrating an example NVMe-oF device operating in a non-high availability (non-HA) mode according to some example embodiments of the present invention;
[0025] Figure 4C and Figure 4D A block diagram illustrating an example NVMe-oF device operating in a high availability (HA) mode according to some example embodiments of the present invention;
[0026] Figure 5A A block diagram illustrating an example field programmable gate array (FPGA) device according to some example embodiments of the present invention;
[0027] Figure 5B Another block diagram illustrating an example FPGA device according to some example embodiments of the present invention;
[0028] Figure 6 Another block diagram illustrating an example FPGA device according to some example embodiments of the present invention;
[0029] Figure 7 A block diagram illustrating an example switch according to some example embodiments of the present invention;
[0030] Figure 8 A flowchart of a method for operating a storage device according to some example embodiments of the present invention is shown. DETAILED DESCRIPTION
[0031] The specific embodiments described below in conjunction with the accompanying drawings are intended as embodiments of some example embodiments of systems and methods for supporting multi-mode and / or multi-speed NVMe-oF devices provided according to the present invention, and are not intended to represent the only form in which the present invention can be constructed or utilized. This specification describes the features of the present invention in conjunction with the illustrated embodiments. However, it will be understood that the same or equivalent functions and structures can be implemented by different embodiments that are also intended to be included within the scope of the present invention. As shown elsewhere herein, the same element numbers are intended to indicate the same elements or features.
[0032] NVMe is a standard that defines a register-level interface for host software to communicate with a non-volatile memory subsystem (e.g., SSD) over a PCIe bus. NVMe can be a replacement for the Small Computer System Interface (SCSI) standard, which is used to connect and transfer data between a host and a peripheral target storage device or system. PCIe-connected NVMe SSDs allow applications to talk directly to storage.
[0033] The physical connection of NVMe is based on the PCIe bus. A typical Ethernet SSD (eSSD) has a U.2 connector to connect to the system via the midplane through the PCIe bus. The U.2 connector (SFF-8639) is a computer interface for connecting SSDs and computers. The U.2 connector supports up to four lanes (X4) of parallel I / O in one Serial Advanced Technology Attachment (SATA) port, two Serial Attached SCSI (SAS) ports, or PCIe SSDs. The U.2 connector is standardized for NVMe and supports PCIe 3.0X4, providing five times the speed of a typical SATA SSD.
[0034] NVMe-oF is an extension of the NVMe standard that enables operations over various networks (or interconnects) other than PCIe. Here, the term "fabric" refers to a network topology in which network nodes can pass data to each other through various interconnect protocols, ports, and switches. For example, an Ethernet-attached SSD can be directly attached to the network, in which case the network is Ethernet.
[0035] NVMe-oF implements the use of alternative transmission to PCIe, wherein the alternative transmission extends the distance that the NVMe host device and the NVMe storage drive or subsystem can be connected. Therefore, NVMe-oF is a technical specification designed to use commands based on non-volatile memory transmission standard messages to transmit data between a host computer and a target solid-state storage device (e.g., eSSD or NVMe-oF device) or system through a network (such as Ethernet, Fibre Channel (FC) or InfiniBand). When configured to support the NVMe-oF standard, the system can support various networks, including not only Ethernet but also Fibre Channel, InfiniBand and other network architectures. For example, an eSSD can be directly attached to a network, in which case the network is Ethernet. An eSSD may represent an SSD that can support the NVMe-oF protocol. For ease of illustration, the following examples and embodiments may illustrate an Ethernet-attached NVMe-oF device. However, it should be noted that any other type of NVMe-oF device may be used without departing from the scope of the present disclosure.
[0036] As described above, the physical connection of NVMe is based on the PCIe bus. Recently, with the advent of PCIe 4.0, the bandwidth mismatch error can be higher than PCIe 3.0. A single 25G Ethernet does not have enough bandwidth to keep up with the back end from the SSD via X4PCIe 4.0 (up to 8GB / s). 50G or 100G Ethernet provides a better match of X4PCIe 4.0 from the SSD. Therefore, embodiments of the present invention may include a storage device (e.g., eSSD) that can support both NVMe and NVMe-oF protocols, and when in NVMe-oF mode, the storage device can operate at Ethernet speeds from 10G to 100G without any changes to its hardware.
[0037] In addition, embodiments of the present invention may enable any platform system to support different types of NVMe-oF devices from different vendors. By using common building blocks (such as switch boards, midplanes, and Ethernet SSDs), ecosystem providers are able to enter the market faster than existing systems and provide a variety of price / performance products to satisfy a variety of consumers. Some example embodiments may also enable a general system platform that can support NVMe-oF devices with standard U.2 connectors (such as PM1725a or PM1735) and emerging general purpose SSDs such as M.3 SSDs. In one configuration, the M.3 SSD may also be referred to as an NGSFF-based SSD.
[0038] Different embodiments of the present system provide a common building block and a single platform that can support both NVMe and NVMe-oF devices, and when in NVMe-oF mode, the system can operate at Ethernet speeds from 10G to 100G without changing their hardware. Common building blocks that can support both NVMe and NVMe-oF devices may include switch boards, midplanes, and eSSDs. The present system can be linearly expanded by adding more similar devices and / or chassis. The present system may also provide a common system platform that can support NVMe-oF devices (such as PM1725a or PM1735) with standard U.2 connectors and SSDs based on M.3 or NGSFF. Different configurations of the present system may also be compatible with technological advances (such as 50G and 100G Ethernet and PCIe 4.0). In particular, different configurations of the present application provide systems that can support both NVMe and NVMe-oF protocols and various types of network-attached SSDs (e.g., eSSDs). Additionally, different configurations of the present application may provide information about how the chassis and device (eg, eSSD) can adjust the device to understand where it is deployed and what performance it should employ as a result of its deployment at a particular location in a data set.
[0039] The network-attached SSD (eSSD) disclosed herein is a single universal device that can be used in multiple systems compatible with NVMe and NVMe-oF standards. Ethernet SSDs can use U.2 connectors to connect to host devices via midplanes. U.2 connectors can support 25Gbps (100G-KR4) Ethernet. The eSSD disclosed herein can be a device that can be used in multiple systems compatible with NVMe and NVMe-oF standards. As such, the network-attached SSD (eSSD) can also be referred to as a multi-mode NVMe-oF device. Multi-mode NVMe-oF devices can also support NVMe or NVMe-oF standards by detecting product information from a known location (e.g., midplane or chassis type pin E6 of the motherboard). For example, a limiting signal (e.g., a mode signal) on a U.2 connector can indicate whether the drive is operating in NVMe mode or in NVMe-oF mode. If the limiting signal appears in an NVMe chassis, the X4 lane PCIe of the U.2 connector will be driven by the PCIe engine. In this case, the device will disable the network attached port (e.g., at least one of the Ethernet port, the network channel port, and the InfiniBand port), while all NVMe protocols and functions are supported or enabled. If present in an NVMe-oF chassis, the network attached port will only use unused SAS pins.
[0040] Figure 1A The configuration of the NVMe-oF device 100 is shown. Figure 1A The NVMe-oF device 100 includes an eSSD FPGA / ASIC 102, a first M.3 format SSD 114, a second M.3 format SSD 116, two double data rate fourth generation (DDR4) memories (DDR4 memories 122 and 144) and four capacitors CAP (capacitors 126, 128, 130 and 132). The NVMe-oF device 100 is also connected to a U.2 connector 112, and, via the U.2 connector 112, the NVMe-oF device 100 can support up to four PCIe X4 lanes of two SAS ports (SAS0 and SAS1) and a PCIe X4 bus. The two SAS ports (SAS0 and SAS1) can be used as Ethernet ports by the NVMe-oF device 100. The NVMe-oF device 100 can operate at different Ethernet speeds from 10G to 100G without any changes to the hardware.
[0041] Figure 1B A block diagram of an example M.3 format SSD 134 is shown according to one embodiment. The M.3 format SSD 134 may be Figure 1AThe new form factor 1 (NF1) of the M.3 format SSD 134 can accommodate two rows of NAND flash memory packages, thereby maximizing the capacity of the modular SSD. The M.3 format SSD 134 can be connected to the FPGA via the PCIe X4 bus 136. The FPGA can be Figure 1A FPGA 102, PCIe X4 bus 136 can be Figure 1A One of the PCIe X4 buses 118 or 120. The M.3 format SSD 134 can also be used to connect to an FPGA (e.g., Figure 1A The at least one Ethernet port 138 may be configured to accommodate at least one optional Ethernet port 138 (Ethernet port A and / or Ethernet port B) by using unused or reserved (e.g., high-speed) pins of an M.2 connector of an FPGA 102 in the M.3 format to accommodate at least one optional Ethernet port 138 (Ethernet port A and / or Ethernet port B). The at least one Ethernet port 138 may enable the M.3 format SSD 134 to operate in NVMe-oF mode.
[0042] In some embodiments, the M.3 format SSD 134 may conform to a 2.5-inch hard drive form factor (or small form factor (SFF)) standard. In other embodiments, the M.3 format SSD 134 may conform to a standard PCIe card form factor (e.g., a full-height, full-length (FH-FL) card profile, or a full-height, half-length (FH-HL) card profile). The M.3 format SSD 134 may also include a controller, a buffer memory, and flash memory. The controller may execute software, NVMe commands, and / or firmware stored in, for example, a read-only memory or a buffer memory in the controller (or separate from the controller). Figure 1B In the embodiment of the present invention, the eSSD ASIC 140 is an enhanced SSD controller including a flash translation layer (FTL) and a flash controller.
[0043] Back to Figure 1A, the NVMe-oF device 100 can be configured to operate in NVMe mode or NVMe-oF mode. In NVMe-oF mode, the NVMe-oF device 100 can configure two PCIe lanes (PCIe 1 and PCIe 2) as Ethernet port 106 (Ethernet port B) and Ethernet port 110 (Ethernet port D). The NVMe-oF device 100 can also configure two SAS ports (SAS0 and SAS1) as Ethernet port 104 (Ethernet port A) and Ethernet port 108 (Ethernet port C). The first 25G Ethernet port 104 may be connected to pins S2, S3, S5, and S6 of the U.2 connector 112, the second 25G Ethernet port 106 may be connected to pins S17, S18, S20, and S21 of the U.2 connector 112, the third 25G Ethernet port 108 may be connected to pins S9, S10, S12, and S13 of the U.2 connector 112, and the fourth 25G Ethernet port 110 may be connected to pins S23, S24, S26, and S27 of the U.2 connector 112. When in NVMe-oF mode, the NVMe-oF device 100 can operate at different Ethernet speeds from 10G to 100G.
[0044] exist Figure 1AIn the embodiment, when the NVMe-oF device 100 is configured in NVMe-oF mode, the FPGA 102 can provide an interface between four 25Gbps Ethernet ports (Ethernet ports 104, 106, 108, and 110) and two M.3 format SSDs (SSDs 114 and 116). The four 25G Ethernet ports (Ethernet ports 104, 106, 108, and 110) can be connected to the motherboard through the midplane according to the mode of operation of the NVMe-oF device 100. In one configuration, the motherboard may include one or more switch elements, one or more memory elements, one or more I / O elements, and the like. The FPGA 102 is connected to the first M.3 format SSD 114 via the first PCIe X4 bus 118, and the FPGA 102 is connected to the second M.3 format SSD 116 via the second PCIe X4 bus 120. The first PCIe X4 bus 118 and the second PCIe X4 bus 120 can be connected to the first M.3 format SSD 114 and the second M.3 format SSD 116 via their respective M.3 connectors. In this case, the unused or reserved (e.g., high-speed) pins of the M.2 connectors attached to the SSD 114 and SSD 116 can be used for Ethernet connections. In this mode, the FPGA 102 can be used as an NVMe-oF target. The NVMe-oF target implemented on the eSSD ASIC / FPGA 102 provides all network and storage protocol processing, which eliminates the need for an x86-based central processing unit (CPU) on the motherboard in the target device. For NVMe-oF-based systems, an x86-based CPU on the motherboard is no longer required to move data between the initiator (e.g., host software) and the target device (i.e., NVMe-oF device) because the target device can move data by itself.
[0045] In one configuration, the NVMe-oF device 100 may be configured in NVMe mode. In NVMe mode, the NVMe-oF device 100 may use all four PCIe X4 lanes (in single-port mode) to carry PCIe signals through the PCIe X4 bus. The PCIe X4 bus is connected to the midplane, and the PCIe X4 bus is shared between data and control signals. In one aspect, at a given time, the first M.3 format SSD 114 and the second M.3 format SSD 116 of the NVMe-oF device 100 may both operate in NVMe mode or NVMe-oF mode. In another aspect, at a given time, the first M.3 format SSD 114 may operate in NVMe mode, and the second M.3 format SSD 116 may operate in NVMe-oF mode. In another aspect, at a given time, the first M.3 format SSD 114 may operate in NVMe-oF mode, and the second M.3 format SSD 116 may operate in NVMe mode.
[0046] The operating mode of the NVMe-oF device 100 can be self-configured or set externally using a physical pin (e.g., a presence pin (pin E6) on the chassis of the motherboard) or through an in-band command from the baseboard management controller (BMC) of the motherboard. Manageability information obtained through Ethernet is referred to as "in-band" information, while manageability information obtained through the PCIe bus is referred to as "out-of-band" information. When configured as an NVMe-oF device, the multi-mode NVMe-oF device 100 (chassis type pin E6 = low) can be configured in single-port NVMe-oF mode or dual-port NVMe-oF mode. In single-port NVMe-oF mode, pin E25 of the U.2 connector 112 can be high; in dual-port NVMe-oF mode, pin E25 of the U.2 connector 112 can be low.
[0047] In a configuration such as Figure 2As shown, different operating speeds of the NVMe-oF device 100 can be achieved by using two additional general purpose input / output (GPIO) (ESpeed[1:0]) pins 202 located on the midplane or CPLD 204, wherein the ESpeed[1:0] pins 202 can be controlled by internal registers within the FPGA 102 of the NVMe-oF device 100 or the local CPU or BMS of the motherboard, and can be accessed via the I2C bus connected to pins E23 and E24 of the U.2 connector 112. In the aforementioned option, the ESpeed pin 202 is multiplexed with the U.2I2C pin within the CPLD 204 and can be latched after reset Reset#207 has been asserted (logical high to logical low). In one configuration, the input to the multiplexer MUX is driven by the local CPU or BMC or CPLD 204 of the motherboard. However, in some configurations, the location of the MUX is within the CPLD 204. The number of MUXs is equal to the maximum number of NVMe-oF devices supported by the chassis. In this way, each device can be reset individually and independently. This approach can support hot-plug events (hot-add and hot-remove). If there is one MUX shared by all devices, then when a new device is hot-plugged, it can affect existing devices in the chassis due to the reset cycle, which may not be desirable.
[0048] If the CPLD 204 is controlled by the BMC of the motherboard or the local CPU, the control signal from the BMC or the local CPU may be received at the CPLD / BMC GPIO pin 206 located at the CPLD 204. Table 1 below shows the state of the ESpeed pin during different operating speeds of the NVMe-oF device 100.
[0049]
[0050]
[0051] Table 1
[0052] In addition, during power-on or reset, the Ethernet speed is selected using the system management bus SMBus 208. For example, the SMBus 208 can transmit the SMBus clock (SMBus CLK) and SMBus data (SMBusData) from the BMC / local CPU, etc. In one configuration, the multiplexer is selected by the power-on or reset signal 207, and when the reset 207 is low, the multiplexer can select Espeed[0:1] 202 for the FPGA 102, and when the reset 207 is high, the multiplexer can connect the SMBus 208 between each slot to the BMC on the switch board.
[0053] In one configuration, the multi-mode NVMe-oF device 100 may be configured in a single-port NVMe mode, a dual-port NVMe mode, a single-port NVMe-oF mode, and a dual-port NVMe-oF mode.
[0054] Figure 3 An example use of the U.2 connector 112 according to the configuration of the multi-mode NVMe-oF device 100 is shown. When configured as an NVMe device (chassis type pin E6 = low), the multi-mode NVMe-oF device 100 can be configured in single-port NVMe mode or dual-port NVMe mode. In single-port NVMe mode, the dual-port EN# pin E25 of the U.2 connector 112 can be asserted high. In single-port NVMe mode, PCIe lanes 0-3 of the U.2 connector 112 are used to carry PCIe signals. In dual-port NVMe mode, the dual-port EN# pin E25 of the U.2 connector 112 can be asserted low. In dual-port NVMe mode, PCIe lanes 0-3 are divided into 2×2 lanes; PCIe lanes 0 and 1 are used for the first port (port A), and PCIe lanes 2 and 3 are used for the second port (port B).
[0055] In one configuration, when the multi-mode NVMe-oF device 100 is in NVMe-oF mode, the NVMe-oF device 100 may have two X1 PCIe lanes (PCIe 0 and PCIe 3) for the control plane of the first Ethernet port (Port A) and the second Ethernet port (Port B). Such a configuration may enable two PCIe lanes (PCIe 1 and PCIe 2) to be used for additional 25G Ethernet ports (Ethernet ports B and D). In some configurations, when the NVMe-oF device 100 operates at 10G (single or dual port) or 25G (single or dual port) speed in NVMe-oF mode, SAS port 0 is used for Ethernet port A (first port) and SAS port 1 is not used. PCIe lanes 0 and 3 are used as the control plane of the NVMe-oF controller for the first (Port A) and second (Port B) Ethernet attachments. PCIe lane 1 is used for Ethernet port B (second port) and PCIe lane 2 is not used. In some configurations, when the NVMe-oF device 100 operates in NVMe-oF mode at 50G (single or dual port) or 100G (single port only) speed, SAS port 0 is used for Ethernet port A (first port) and SAS port 1 is used as Ethernet port C (third port). PCIe lanes 0 and 3 are used as the control plane of the NVMe-oF controller for the first (port A) and second (port B) Ethernet attachments, PCIe lane 1 is used for Ethernet port B (second port), and PCIe lane 2 is used for Ethernet port D (fourth port).
[0056] In some configurations, if the product information is stored in the chassis, two lanes (in single-port mode) or four lanes (in dual-port mode) of the PCIe bus on the U.2 connector 112 are driven by the PCIe engine. In this case, the multi-mode NVMe-oF device 100 may disable one or more Ethernet engines, and the NVMe protocol and functions are supported or enabled. If the product information is stored in the NVMe-oF chassis, the Ethernet-attached NVMe-oF controller uses only PCIe lanes 1 and 2 and / or SAS pins according to the design of the multi-mode NVMe-oF device.
[0057] In some cases, product information for self-configuration is stored in the chassis as vital product data (VPD). During startup, the multi-mode NVMe-oF device 100 can obtain the VPD from the chassis and configure itself based on the VPD. In some configurations, the multi-mode NVMe-oF device 100 can be configured in various ways without departing from the scope of the present disclosure. For example, the multi-mode NVMe-oF device 100 can be configured by a control command issued by the BMC of the motherboard connected to the multi-mode NVMe-oF device 100 via the PCIe bus. The system provides a platform that can support various types of NVMe and NVMe-oF devices in non-high availability (non-HA) mode (i.e., single-path input / output (I / O)) or HA mode with minimal hardware changes.
[0058] Figure 4A and Figure 4B A block diagram illustrating an example NVMe-oF device 400 operating in non-HA mode according to one embodiment. The NVMe-oF device 400 may be Figure 1ANVMe-oF device 100 in. In this example, NVMe-oF device 400 can support single path I / O and can operate at different Ethernet speeds from 10G to 100G without any hardware changes. NVMe-oF device 400 may include FPGA / ASCI 402, a first M.3 format SSD 414, a second M.3 format SSD 416, and two DDR4 memories (422 and 424). FPGA 402 is connected to the first M.3 format SSD 414 via a first PCIe X4 bus 418, and FPGA 402 is connected to the second M.3 format SSD 416 via a second PCIe X4 bus 420. NVMe-oF device 400 is also connected to U.2 connector 412, and via U.2 connector 412, NVMe-oF device 400 can support two SAS ports (SAS0 and SAS1) and up to 4 PCIe X4 lanes of the PCIe X4 bus. Two SAS ports (SAS0 and SAS1) can be used as Ethernet ports. The NVMe-oF device 400 can configure two PCIe lanes (PCIe 1 and PCIe 2) as Ethernet port 406 (Ethernet port B) and Ethernet port 410 (Ethernet port D). The NVMe-oF device 400 can also configure two SAS ports (SAS0 and SAS1) as Ethernet port 404 (Ethernet port A) and Ethernet port 408 (Ethernet port C). The first 25G Ethernet port 404 can be connected to pins S2, S3, S5 and S6 of the U.2 connector 412, the second 25G Ethernet port 406 can be connected to pins S17, S18, S20 and S21 of the U.2 connector 412, the third 25G Ethernet port 408 can be connected to pins S9, S10, S12 and S13 of the U.2 connector 412, and the fourth 25G Ethernet port 410 can be connected to pins S23, S24, S26 and S27 of the U.2 connector 112.
[0059] In one configuration, the Ethernet ports 404, 406, 408, and 410 can be connected to the motherboard 401 through the midplane 426 via several high-speed Molex connectors 428, wherein the high-speed Molex connectors 428 can collectively carry all of the Ethernet ports 404, 406, 408, and 410 and other non-high-speed control signals (such as SMBus, reset, and clock, etc.). Several high-speed Molex connectors 428 may include PCIe clock distribution and 12V power B to B. The midplane 426 may also include 12V power B to B. Both 12V power B to B can supply power to the components in the motherboard 401 and the midplane 426. Specifically, the 12V power B to B of the midplane 426 can supply power to the VPD electrically erasable programmable read-only memory (EEPROM) in the midplane 426. The 12V hot-swap controller in the motherboard 401 can control the 12V power B to B in the midplane 426. The power supplied by both 12V power B to B can be generated and distributed by dual redundant power supply units (PSUs), power distribution units, and 12V power inputs. The motherboard 401 may include a local CPU 434 (i.e., CPU Broadwell-DE 434) connected to a management physical layer (MGMT PHY), a BMC 432, an Ethernet switch controller 436 connected to a 2×3 quad small form factor pluggable (QSFP) 28-100G port, a PCIe switch 438, and two DDR4 memories (DDR4 memories 440 and 442). In some configurations, the motherboard 401 may push various signals to the NVMe-oF device 400 via Ethernet ports 404, 406, 408, and 410 through the midplane 426, and perform various services on the NVMe-oF device 400. For example, the motherboard 401 may receive device-specific information from the NVMe-oF device 400 through the Ethernet ports 404, 406, 408, and 410, wherein the device-specific information includes, but is not limited to, health status information, field replaceable unit (FRU) information, and sensor information of the NVMe-oF device 400. The motherboard 401 may also perform various services through the Ethernet ports 404, 406, 408, and 410, wherein the various services include, but are not limited to, a discovery service for a BMC (e.g., BMC 432) or a local host CPU (e.g., CPU 434) and a download service for performing a firmware upgrade for new eSSD firmware.
[0060] Figure 4C and Figure 4DA block diagram of an example NVMe-oF device 400 operating in HA mode according to one embodiment is shown. In this example, the NVMe-oF device 400 can support multipath I / O and can operate at different Ethernet speeds from 10G to 100G without any hardware changes. In HA mode, the NVMe-oF device 400 can be connected to two motherboards 401 and 403 via a midplane 426 to support multipath I / O in a dual-port configuration (in HA mode). The midplane 426 is a common midplane that can support both HA mode and non-HA mode. Depending on the system configuration, signal integrity may need to be tested to ensure that the common midplane 426 can support two configurations. If the signal integrity is not enough, the system can have two versions of the midplane including a first midplane for HA mode and a second midplane for non-HA mode. The E25 pin of the U.2 connector 412 can be used to implement a dual-port configuration. The NVMe-oF device 400 can self-configure its operating mode using a physical pin (e.g., a presence pin (pin E6) on the chassis of the motherboard (motherboard 401 or 403)) or through an in-band command (not shown) from the BMC (BMC432 or 444) of the motherboard (motherboard 401 or 403). Both BMCs (BMC432, 444) can be combined with a CPLD (CPLD 430) in the midplane 426 via 3×SMBus. The CPLD (CPLD 430) can activate a BMC identification (ID) based on a 3.3V auxiliary power supply (aux).
[0061] In one configuration, when operating in HA mode, the Ethernet port 404 (Ethernet port A) and the Ethernet port 406 (Ethernet port B) of the NVMe-oF device 400 can be connected to the motherboard 401 via a high-speed Molex connector 428 through a midplane 426, wherein the high-speed Molex connector 428 can carry the Ethernet ports 404 and 406 and other non-high-speed control signals (such as SMBus, reset, and clock, etc.) together. In addition, in HA mode, the Ethernet port 408 (Ethernet port C) and the Ethernet port 410 (Ethernet port D) of the NVMe-oF device 400 can be connected to a second motherboard 403 (not shown) through the midplane 426 via several high-speed Molex connectors 456, wherein the high-speed Molex connector 456 can carry the Ethernet ports 408 and 410 and other non-high-speed control signals (such as SMBus, reset, and clock, etc.) together. The second motherboard 403 may include a local CPU 446 (ie, CPU Broadwell-DE 446 ), a BMC 444 , an Ethernet switch controller 448 , a PCIe switch 450 , and two DDR4 memories ( 452 and 454 ).
[0062] Because the multi-mode NVMe-oF device 400 can operate in both NVMe mode and NVMe-oF mode, the cost for developing and deploying the device can be reduced because the same device can be used in NVMe mode and NVMe-oF mode. For the same reason, the multi-mode NVMe-oF device 400 can have a faster time to market. Multi-mode NVMe-oF devices can be used in a variety of products and chassis. The two lanes of the PCIe bus are reserved for standard features through the control plane. The CPU, BMC, and other devices can use the two lanes of the PCIe bus as a control plane for communicating with each NVMe-oF device within the chassis without additional overhead. The NVMe midplane can be used unmodified, and because of the additional GPIO pins located on the midplane (for example, Figure 2 ESpeed[1:0]202) without the need for a new connector for the NVMe-oF device 400. In addition, it is expected that any platform system supports different types of NVMe-oF devices from different vendors. By using common building blocks such as switch boards, midplanes, and Ethernet SSDs, different embodiments of the present application enable existing ecosystem providers to enter the market much faster and provide products of various prices / performance to meet different consumers. It is also expected to have a common system platform that can support NVMe-oF devices with standard U.2 (such as PM1725a or PM1735) and emerging SSDs such as those based on M.3 or NGSFF.
[0063] Figure 5A-5B is a block diagram illustrating an example FPGA device 500 according to one embodiment. FPGA 500 may be Figure 1A FPGA 500 may include clock and reset (Clk / Reset), debug tools, laser emitting diode (LED), temperature sensor inter-integrated circuit (I2C), universal asynchronous receiver / transmitter (UART), and joint test action group (JTAG). FPGA 500 may provide U.2 connector 502 and multiple flash drives (e.g., Figure 1A FPGA 500 can also be connected to multiple DDR4 memories (e.g., Figure 1AFor example, FPGA 500 may be connected to a PCIe X4 bus (e.g., Figure 1A The port 518 of the first PCIe X4 bus 118) is connected to the first SSD SSD1 (for example, Figure 1A 114), the FPGA 500 may be connected to another PCIe X4 bus (e.g., Figure 1A The port 522 of the second PCIe X4 bus 120) is connected to the second SSDSSD2 (for example, Figure 1A FPGA 500 can also be connected to a first DDR4 memory DDR4 1 (for example, Figure 1A DDR4 memory 122) and connected to a second DDR4 memory DDR4 2 (eg, Figure 1A DDR4 memory 124) is connected.
[0064] The U.2 connector 502 connected to the FPGA 500 can support two SAS ports (SAS0 and SAS1) and up to four PCIe X4 lanes of the PCIe X4 bus. The two SAS ports (SAS0 and SAS1) can be used as Ethernet ports. In NVMe-oF mode, the two PCIe lanes (PCIe 1 and PCIe 2) can be configured as Ethernet port 510 (Ethernet port 1) and Ethernet port 512 (Ethernet port 3). In NVMe-oF mode, the two SAS ports (SAS0 and SAS1) can be configured as Ethernet port 506 (Ethernet port 0) and Ethernet port 508 (Ethernet port 2). Ethernet port 506 (Ethernet port 0) can be connected to pins S2, S3, S5 and S6 of the U.2 connector 502, Ethernet port 510 (Ethernet port 1) can be connected to pins S9, S10, S12 and S13 of the U.2 connector 502, Ethernet port 508 (Ethernet port 2) can be connected to pins S23, S24, S26 and S27 of the U.2 connector 502, and Ethernet port 512 (Ethernet port 3) can be connected to pins S17, S18, S20 and S21 of the U.2 connector 502.
[0065] In one configuration, in NVMe-oF mode, the two X1 PCIe lanes (PCIe 0 and PCIe 3) 504 and 514 of the U.2 connector 502 can be configured as control planes for the first (port 0) and second (port 1) Ethernet NVMe-oF controllers. The control plane PCIe lane (PCIe 0) 504 can be connected to pins E10, E11, E13, and E14 of the U.2 connector 502, and the control plane PCIe lane 514 (PCIe 3) can be connected to pins E17, E18, E20, and E21 of the U.2 connector 502. In some configurations, such as Figure 5A As shown, in NVMe-oF mode, when FPGA 500 operates at 50G (single port or dual port) speed, SAS port 0 is used for Ethernet port 0 (Ethernet port 506), and SAS port 1 is used as Ethernet port 2 (Ethernet port 508). PCIe lane 504 (PCIe 0) and PCIe lane 514 (PCIe 3) are used as control planes, PCIe lane 1 (Ethernet port 510) is used for Ethernet port 1, and PCIe lane 2 (Ethernet port 512) is used for Ethernet port 3.
[0066] In another configuration, Figure 5B As shown, in NVMe-oF mode, when FPGA 500 operates at 100G (single port) speed, SAS port 0 is used for Ethernet port 0 (Ethernet port 506), and SAS port 1 is used as Ethernet port 2 (Ethernet port 508). PCIe lane 504 (PCIe 0) is used as a control plane, PCIe lane 1 (port 510) is used for Ethernet port 1, and PCIe lane 2 (port 512) is used for Ethernet port 3. Control plane PCIe lane 514 (PCIe 3) is not used.
[0067] In some configurations, ports 504-510 may be connected to the FPGA 500 through a midplane (eg, Figure 4A and Figure 4B midplane 426) connected to the main board (e.g., Figure 4A Motherboard 401 or Figure 4B In some configurations, FPGA 500 can be used as an NVMe-oF target and eliminates the need for an x86-based CPU on the motherboard in the target device. For an NVMe-oF-based system, in order to move data between an initiator (e.g., host software) and a target device (i.e., an NVMe-oF device), an x86-based CPU on the motherboard is no longer required because the target device is able to move data by itself. In addition, multiple flash drives (e.g., Figure 1AThe different operating speeds of the two M.3 format SSDs (SSDs 114 and 116) can be achieved by using a CPLD located in the midplane (e.g., Figure 2 Two additional GPIO (ESpeed[1:0]) pins on the CPLD204 (e.g., Figure 2 The GPIO (ESpeed[1:0]) pin can be controlled by one or more internal registers within the FPGA 500.
[0068] In one embodiment, when an Ethernet data packet is received at FPGA 500, logic blocks within FPGA 500 (such as processor subsystem 1, processor subsystem 2, write buffer (WDB), read buffer (RDB), DDR controller, DDR4 controller, Ethernet 1-physical layer (Eth 1-PHY), Ethernet 2-physical layer (Eth 2-PHY), PCIe 1-physical layer (PCIe1-PHY), PCIe 2-physical layer (PCIe 2-PHY), Ethernet media access control (Eth-MAC), extensible authentication protocol packet (EAPP), password authentication protocol packet (PAPP), express remote direct memory access network interface card (eRNIC), NVM-oF front end (NVMF FE), NVM-oF back end (NVMF FE), etc.) are sent to the FPGA 500. BE) and PCIe controller) can participate in stripping off various layers of the header file in that Ethernet packet. The processor subsystem can be an independent redundant disk array on a chip (RoC). For example, within the Ethernet packet, there can be a TCP or UDP packet, within the TCP or UDP packet, there can be a remote direct memory access (RDMA) packet, within the RDMA packet, there can be an NVMe-oF packet, and within the NVMe-oF packet, there can be an NVMe command. The logic blocks within FPGA 500 can represent various layers, wherein the Ethernet packets are stripped out in each layer to expose the next layer of protocol. A flash drive or SSD connected to FPGA 500 via ports 518 and 522 can consume NVMe commands as if it had been connected to a host or motherboard at the other end of Ethernet ports 506, 508, 510, and 512 via U.2 connector 502.
[0069] Figure 66 is a block diagram illustrating an example FPGA device 600 according to one embodiment. FPGA device 600 may include FPGA 601. FPGA 601 may provide an interface between U.2 connector 606 and multiple flash drive 2Gbit SPI NOR (e.g., flash drives 603A and 603B). For example, FPGA 601 may be connected to flash drive 603A via port 607 that may be connected to a PCIe X4 bus, and FPGA 601 may be connected to flash drive 603B via port 608 that may be connected to another PCIe X4 bus. FPGA 601 may also be connected to multiple DDR4 memories (e.g., DDR4 memories 602A-602J). FPGA 601 is also connected to clock circuit CLK Gen 614.
[0070] The U.2 connector 606 can be connected to the FPGA 601 via two PCIe X4 buses 604 and 605 through two multiplexers 610 and 612. The PCIe X4 bus 604 can be used to send signals or data packets to the motherboard through the midplane via the U.2 connector 606 through the multiplexer 610, and the PCIe X4 bus 605 can be used to receive signals or data packets from the motherboard through the midplane via the U.2 connector 606 through the multiplexer 612. In some configurations, the different operating speeds of multiple flash drives (e.g., flash drives 603A and 603B) connected to the FPGA 601 can be achieved by using a CPLD located in the midplane or CPLD (e.g., Figure 2 Two additional GPIO (ESpeed[1:0]) pins on the 204) (e.g., Figure 2 The GPIO (ESpeed[1:0]) pins may be controlled by one or more internal registers within the FPGA 601. Table 1 shows the states of the ESpeed pins during different operating speeds of the flash memory drivers 603A and 603B connected to the FPGA 601.
[0071] Figure 7 700 includes two motherboards 701A and 701B for supporting multiple I / O in a dual-port configuration (in HA mode) via a midplane 761. Motherboard 701A includes an Ethernet switch controller 704A and a PCIe switch 705A, and motherboard 701B includes an Ethernet switch controller 704B and a PCIe switch 705B. Each of motherboards 701A and 701B may include other components and modules (e.g., Figure 4AThe example motherboard 401 shown includes local CPUs (CPU 706A and 706B (ie, CPU Skylake-D 706A and CPU Skylake-D 706B)), BMCs (BMCs 707A and 707B), and optional repeaters, among others.
[0072] Some eSSDs may be inserted into device ports of switch 700. For example, each eSSD is connected to switch 700 using a U.2 connector. Each eSSD may be connected to both motherboard 701A and motherboard 701B. In this example, the eSSDs inserted into switch 700 are configured as NVMe-oF devices that need to be connected to switch 700 through midplane 761 via a PCIe bus and an Ethernet port.
[0073] Figure 8 A flow chart of a method for operating a storage device according to an embodiment of the present invention is shown. The storage device may be Figure 4A The NVMe-oF device 400 is provided.
[0074] At 801, a storage device receives a first input. The storage device may receive the first input from a motherboard or BMC via a plurality of device ports via a midplane. For example, the NVMe-oF device 400 may receive the first input via a plurality of device ports via a midplane. Figure 4A The midplane 426 receives a first input from the mainboard 401 or the BMC 432 via the device ports 404 - 406 .
[0075] In one configuration, the first input may be controlled by using a physical pin on the chassis of the motherboard or by an in-band command from the BMC. For example, the first input received at the NVMe-oF device 400 may be controlled by using Figure 4A The physical pin (E6) on the chassis of the motherboard 401 is controlled.
[0076] At 802, based on the first input, the storage device determines whether to operate in a first operating mode or a second operating mode. In one configuration, the first operating mode and the second operating mode of the storage device can be an NVMe mode and an NVMe-oF mode. For example, based on the first input, the NVMe-oF device 400 determines whether to operate in an NVMe mode or an NVMe-oF mode.
[0077] At 802, based on the first input, if the storage device determines to operate in the first operation mode, then at 803, the storage device performs a storage operation according to the first operation mode. For example, if the NVMe-oF device 400 determines to operate in the NVMe mode based on the first input, the NVMe-oF device 400 performs a storage operation according to the NVMe protocol.
[0078] However, if the storage device determines to operate in the second operation mode based on the first input at 802, the storage device performs storage operations according to the second operation mode at 804. For example, if the NVMe-oF device 400 determines to operate in the NVMe-oF mode based on the first input, the NVMe-oF device 400 performs storage operations according to the NVMe-oF protocol.
[0079] At 805, when operating in the second operating mode, the storage device receives a second input. The storage device may receive the second input from the midplane via a plurality of device ports. For example, when operating in the NVMe-oF operating mode, the NVMe-oF device may receive the second input via a plurality of device ports. Figure 4A The plurality of device ports 404 - 406 receive a second input from the midplane 426 .
[0080] In one configuration, the second input is controlled by using two GPIO pins located on the midplane, wherein the two GPIO pins are controlled by one or more internal registers within the local CPU of the BMC or motherboard or the FPGA of the storage device. For example, the second input can be controlled by using two GPIO pins located on the midplane 426 (e.g., Figure 2 The (ESpeed[1:0]) pin 202) is controlled by the two GPIO pins. Figure 4A The BMC 432 or the local CPU 434 of the mainboard 401 or one or more internal registers within the FPGA 402 of the storage device 400 are controlled.
[0081] At 806, based on the second input, the storage device selects an operating speed from a plurality of operating speeds for the storage device. For example, based on the second input, the NVMe-oF device 400 selects an operating speed from a plurality of operating speeds for the NVMe-oF device 400. For example, when operating in NVMe-oF mode, based on the second input, the NVMe-oF device 400 selects an operating speed of 100G. In one configuration, the plurality of operating speeds of the storage device may be operating speeds of any number between 10G and 100G.
[0082] At 807, when operating in the second mode, the storage device operates at the selected operating speed. For example, when operating in the NVMe-oF mode, the NVMe-oF device 400 operates at an operating speed of 100G.
[0083] It will be understood that although the terms "first", "second", "third", etc. may be used herein to describe various elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or part from another region, layer or part. Therefore, without departing from the spirit and scope of the present invention, the first element, component, region, layer or part discussed herein may be referred to as a second element, component, region, layer or part.
[0084] Spatially relative terms (such as, "below," "below," "below," "below," "above," "above," etc.) may be used herein to facilitate description of the relationship of one element or feature shown in the drawings to another element or features. It will be understood that such spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the drawings. For example, if the device in the drawings is flipped, an element described as being "below" or "below" or "below" other elements or features may be oriented "above" the other elements or features. Thus, the example terms "below" and "below" may encompass both an orientation of above and below. The device may be otherwise oriented (rotated 90 degrees or oriented in other directions), and the spatially relative descriptors used herein should be interpreted accordingly. In addition, it will be understood that when a layer is described as being "between" two layers, the layer may be the only layer between the two layers, or one or more intermediate layers may also be present.
[0085] The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the inventive concept. As used herein, the terms "substantially" and "approximately" and similar terms are used as terms of approximation rather than terms of degree, and are intended to illustrate the inherent deviation of measured or calculated values that will be understood by those skilled in the art.
[0086] As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. It will also be understood that when the terms "including" and / or "comprising" are used in this specification, the existence of the described features, wholes, steps, operations, elements and / or components is indicated, but the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their groups is not excluded. As used herein, the term "and / or" includes any and all combinations of one or more associated listed items. When a statement such as "at least one of" appears after a column of elements, it modifies the entire column of elements, rather than a single element of the modified list. In addition, the use of "may" when describing an embodiment of the present invention's conception means "one or more embodiments of the present invention". In addition, the term "exemplary" is intended to indicate an example or illustration. As used herein, the term "use" may be considered synonymous with the term "utilize".
[0087] It will be understood that when an element or layer is referred to as being “on,” “connected to,” “coupled to,” or “adjacent to” another element or layer, it can be directly on, connected to, coupled to, or adjacent to the other element or layer, or one or more intervening elements or layers may be present. In contrast, when an element is referred to as being “directly on,” “directly connected to,” “directly coupled to,” or “directly adjacent to” another element or layer, there are no intervening elements or layers.
[0088] Any numerical range recited herein is intended to include all subranges of the same numerical precision contained within the recited range. For example, the range "1.0 to 10.0" is intended to include all subranges (such as, for example, 2.4 to 7.6) between (and including) the recited minimum value 1.0 and the recited maximum value 10.0, that is, all subranges have a minimum value equal to or greater than 1.0 and a maximum value equal to or less than 10.0. Any maximum numerical limitation recited herein is intended to include any lower numerical limitation contained therein, and any minimum numerical limitation recited in this specification is intended to include any higher numerical limitation contained therein.
[0089] Although exemplary embodiments of systems and methods for supporting multi-mode and / or multi-speed NVMe-oF devices have been specifically described and illustrated herein, various modifications and variations are clear to those skilled in the art. Therefore, it will be understood that systems and methods for supporting multi-mode and / or multi-speed NVMe-oF devices constructed according to the principles of the present invention may be implemented in ways other than those specifically described herein. The present invention is also defined in the above claims and their equivalents.
Claims
1. A storage system, include: At least one motherboard; at least one baseboard management controller; midplane; at least one storage device, wherein the at least one storage device is configured to: operate in a first mode or a second mode based on a first input received from the at least one mainboard or the at least one baseboard management controller via a plurality of device ports through a midplane; When operating in the second mode, the at least one storage device is configured to: select a first speed from a plurality of operating speeds and operate at the first speed based on a second input received from the midplane via the plurality of device ports, wherein the plurality of operating speeds respectively correspond to a plurality of network speeds, Among them, the first mode and the second mode of the at least one storage device are respectively a non-volatile memory transmission standard NVMe mode and a non-volatile memory transmission standard NVMe-oF mode through a network.
2. The storage system according to claim 1, in, The at least one storage device includes a plurality of solid state drives (SSD) communicating with at least one field programmable gate array (FPGA) via a connector and at least one bus.
3. The storage system according to claim 2, in, The connector is an SSD connector, and the at least one bus is a Peripheral Component Interconnect Express (PCIe) bus.
4. The storage system according to claim 1, in, The first input is controlled by using a physical pin on a chassis of the at least one motherboard or by an in-band command from the at least one baseboard management controller.
5. The storage system according to claim 1, in, The second input is controlled by using two general purpose input / output GPIO pins, wherein the two GPIO pins are controlled by one or more internal registers within the local central processing unit CPU of the at least one baseboard management controller or the at least one mainboard or the field programmable gate array FPGA of the at least one storage device.
6. The storage system according to claim 1, in, The plurality of operating speeds of the at least one storage device are two or more speeds above 10G.
7. The storage system according to claim 1, in, The plurality of device ports are connected to the at least one storage device via a U.2 connector, the at least one storage device being configured to operate in a high availability HA mode or a non-HA mode.
8. The storage system according to claim 7, in, The at least one storage device supports two serially attached small computer system interface ports and up to four PCIe X4 lanes of a peripheral component fast interconnect PCIe X4 bus via a U.2 connector, wherein the two serially attached small computer system interface ports are used as network attached ports, wherein the network attached ports include at least one of an Ethernet port, a network channel port, and an Infiniband port.
9. The storage system according to claim 8, in, When operating in the second mode, the at least one storage device supports two of the four PCIe X4 lanes for control planes of two network attached ports and the remaining two of the four PCIe X4 lanes as additional network attached ports via the U.2 connector.
10. The storage system according to claim 1, in, The at least one storage device includes a first storage device and a second storage device, wherein at a first time, the first storage device and / or the second storage device operates in a first mode or a second mode at a first speed or a second speed among the plurality of operating speeds.
11. A method of controlling a storage device, include: receiving a first input at the storage device, wherein the first input is received at the storage device from at least one mainboard or at least one baseboard management controller via a plurality of device ports through a midplane; determining, by the storage device, whether to operate in a first operating mode or a second operating mode based on a first input received at the storage device; receiving, at the storage device, a second input from the midplane via the plurality of device ports while operating in the second operating mode; Based on the second input, selecting, by the storage device, an operating speed of the storage device from a plurality of operating speeds, wherein the plurality of operating speeds respectively correspond to a plurality of network speeds, Among them, the first operation mode and the second operation mode of the storage device are respectively the non-volatile memory transmission standard NVMe mode and the non-volatile memory transmission standard NVMe-oF mode through the network.
12. The method according to claim 11, in, The first input is controlled by using a physical pin on a chassis of the at least one motherboard or by an in-band command from the at least one baseboard management controller.
13. The method according to claim 11, in, The second input is controlled by using two general purpose input / output GPIO pins, wherein the two GPIO pins are controlled by one or more internal registers within a local central processing unit CPU of the at least one baseboard management controller or the at least one mainboard or a field programmable gate array FPGA of a storage device.
14. The method according to claim 11, in, The plurality of operating speeds of the storage device are two or more speeds above 10G.
15. A storage device, include: A plurality of solid-state drives in communication with at least one field programmable gate array (FPGA) via a first connector and at least one bus, wherein the storage device is configured to operate in a first mode or a second mode based on a first input received via a second connector, wherein When operating in the second mode, the storage device is configured to: operate at a first speed among a plurality of operating speeds based on a second input received via the second connector, wherein the plurality of operating speeds respectively correspond to a plurality of network speeds, Among them, the first mode and the second mode of the storage device are respectively a non-volatile memory transmission standard NVMe mode and a non-volatile memory transmission standard NVMe-oF mode through a network.
16. The storage device according to claim 15, in, receiving a first input from a mainboard or a baseboard management controller (BMC) of the switch via a plurality of device ports through the midplane, receiving a second input from the midplane via the plurality of device ports, The first input is controlled by using a physical pin on the chassis of the motherboard or by an in-band command from the BMC, and the second input is controlled by using two general-purpose input / output GPIO pins, wherein the two GPIO pins are controlled by one or more internal registers within the BMC or the local central processing unit CPU or FPGA of the motherboard.
17. The storage device according to claim 15, in, The first connector is a solid-state drive connector, the at least one bus is a Peripheral Component Interconnect Express (PCIe) bus, the second connector is a U.2 connector, and the multiple operating speeds of the storage device are two or more speeds above 10G.
18. The storage device according to claim 15, in, At a first time, a first solid state drive and / or a second solid state drive of the plurality of solid state drives operates in a first mode or a second mode at a first speed or a second speed among the plurality of operating speeds.
