Baseboard management controller located at the server network interface card

Abstract

The present disclosure provides a baseboard management controller located at a server network interface card. The process includes an application layer of a host of a computer platform using an intelligent network interface card (NIC) of the computer platform to provide input / output (I / O) services to the application layer. The I / O services include services associated with a cloud operator domain; the intelligent NIC is installed in a connector; and the application layer is associated with a cloud tenant domain. The process includes a baseboard management controller of the intelligent NIC managing a computer platform. Managing the computer platform includes the baseboard management controller managing the host; the baseboard management controller manages components of the intelligent NIC other than the baseboard management controller; and managing the host includes the baseboard management controller communicating with the host via the connector to control the system power state of the computer platform.

Description

Background Art

[0001] For the purpose of communicating with an external network, a computer platform may have a network interface card (NIC) (e.g., a Peripheral Component Interconnect Express (PCIe) bus card) that provides network connectivity for components of the platform. Computer technology continues to advance, and recent computer platforms may have "intelligent NICs." In addition to providing network connectivity, intelligent NICs may also offload processing operations traditionally performed by a general-purpose central processing unit (CPU) of a conventional computer platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Figure 1A is a schematic diagram of a cloud-based computer system according to an example implementation.

[0003] Figure 1B is a schematic diagram of an intelligent network interface card (NIC) according to an example embodiment, wherein the intelligent network interface card is installed in Figure 1A The invention relates to a computer platform of a computer system of the present invention and includes a baseboard management controller (BMC) for managing the computer platform.

[0004] Figure 1C is an illustration of a virtual BMC for a smart NIC according to an example implementation.

[0005] Figure 2 is a schematic diagram of a hardware BMC of a smart NIC according to an example implementation.

[0006] Figure 3 is a flow chart depicting a process for a BMC located at an intelligent NIC to manage a computer platform according to an example implementation.

[0007] Figure 4 is a flow chart illustrating management of a boot process of a host computer platform by a BMC located at an intelligent NIC according to an example implementation.

[0008] Figure 5 is a flow chart depicting management of system power states by a BMC located at a smart NIC according to an example implementation.

[0009] Figure 6 is a flow diagram depicting management of a boot path of a host of a computer platform by a BMC located at an intelligent NIC according to an example implementation.

[0010] Figure 7 is a flow chart depicting thermal management of a computer platform by a BMC located at a smart NIC according to an example implementation.

[0011] Figure 8is a flow chart depicting a process for providing a BMC located at an intelligent NIC to manage a computer platform according to an example implementation.

[0012] Fig. 9 is a schematic diagram of a system including a host, a smart NIC, and a BMC of the smart NIC that manages the system according to an example implementation.

[0013] Fig.10 is an illustration of instructions stored on a non-transitory machine-readable storage medium that, when executed by a machine, causes the machine to provide a virtual BMC on a smart NIC to manage a system including the smart NIC and a host of the system according to an example implementation. DETAILED DESCRIPTION

[0014] The information technology (IT) industry is becoming increasingly cloud-centric, and server architectures are transforming accordingly to address the challenges that arise when providing cloud services. A multi-tenant cloud operator that provides cloud services to a large number of clients or tenants may rely on networked servers (e.g., blade servers) that may be located in one or more data centers. A given server may execute cloud-native software for multiple tenants; and as such, certain resources of the server may be considered to reside in an untrusted cloud tenant domain (or "client domain") because the domain is shared by more than one tenant. The execution of tenant software may rely on input / output (I / O) services provided by the cloud operator, and these I / O services may be located in a trusted cloud operator domain that is separated from the client domain by security restrictions implemented by the server.

[0015] More specifically, a set of resources of a given server can be considered part of a "host". In general, a host is traditionally considered to be the primary control point for a server and includes the server's general processing hardware (e.g., general central processing unit (CPU), memory, etc.) and software that executes software instances (application instances, operating system instances, etc.) for tenants and provides virtualization-based tenant isolation for software instances (e.g., virtual machines and containers). As such, at least a portion of a host can be considered part of a client domain.

[0016] I / O services provided by cloud operators may include networking services such as network virtualization services (e.g., overlay network services, virtual switching services, virtual routing services, and network function virtualization services); network storage services; networking monitoring services; etc. I / O services may also include storage acceleration services (e.g., services based on non-volatile memory express (NVMe)) and security services (e.g., encryption services and network firewall services).

[0017] In response to the increasingly cloud-centric nature of server processing, a cloud-native server architecture can offload processing operations that are traditionally performed by main CPU processing operations in a traditional server architecture to one or more peripheral devices of the server. In this context, a "peripheral device" refers to a component of a server that provides one or more services or functions to the host of the server; and specifically, according to an example embodiment, a peripheral device can be a smart NIC that provides one or more I / O services to a tenant application instance that is executing on the server.

[0018] In this context, a "smart NIC" generally refers to a NIC that provides one or more I / O services affiliated with a cloud operator (and cloud operator domain) in addition to establishing network connectivity for a server. Here, "I / O services" refers to services that include the use of one or more network resources (e.g., resources accessible to a server via an external network) and provide functionality in addition to providing network connectivity for components of a server. As described above, an I / O service can be any of the services mentioned above, such as a network service, a storage service, or a security service.

[0019] Unlike traditional server architectures where the host acts as the primary control point for all server-related operations, as part of a cloud-native server architecture, a smart NIC can act as a control point for the server. The smart NIC can implement a boundary or isolation between the client domain and the cloud operator domain by providing the following two dual air-gap security management and control interfaces on the server: a first air-gap security management and control interface associated with transactions (e.g., communications, requests, responses, etc.) transmitted from the tenant domain to the cloud operator domain; and a second air-gap security management and control interface for transactions transmitted from the cloud operator domain to the tenant domain. In this context, an "air-gap interface" (such as a first air-gap security management or a second air-gap security management and control interface) refers to a communication interface that is isolated from an unsecured network by software and / or hardware. Depending on the specific implementation, the air-gap interface may or may not correspond to an actual physical isolation (e.g., the air-gap interface may or may not include the presence of an actual air gap in the communication path).

[0020] A server may include a dedicated service processor known as a "baseboard management controller" or "BMC", which is typically remotely accessible (e.g., from a remote server located in a different geographic location from the server, such as a remote server located outside a data center containing the server). In general, a BMC acts as an agent that can manage the operation and functionality of the server in response to communications from a remote management server. In general, such management is independent of the host, and the BMC is able to manage the host even if no software is installed on the host; and according to example embodiments, the BMC can perform management functions when the host is powered off due to a power supply that is independent of the host that supplies power to the BMC. In this context, "managing a host" means that the BMC performs one or more actions related to monitoring, maintaining, configuring, or controlling the host. As examples of its role in managing a host, the BMC can power on the host; power off the host; monitor sensors (e.g., temperature sensors, cooling fan speed sensors); monitor operating system status; monitor power supply status; log computer system events; control boot paths; control the use of virtual media; control security checks; update firmware; verify software; verify hardware; enable the host's boot function; and so on. In addition, the BMC can manage operations of the host before the host is powered on; manage operations before the host's operating system has booted; and perform recovery operations after an operating system or computer system failure.

[0021] Although traditional servers may include a single BMC for managing the host, servers with cloud-native architectures may have multiple BMCs (e.g., one BMC for the host and one BMC for each smart NIC of the server). In this way, the smart NIC may include a BMC for managing operations and functions on the smart NIC, such as powering on the smart NIC, configuring the smart NIC, booting the smart NIC, controlling the health of the smart NIC, updating the firmware of the smart NIC, and so on. However, using multiple BMCs on a server (such as one BMC for the host and another BMC for the smart NIC) may pose challenges. For example, the host BMC may be responsible for the overall management of the server. This means that management actions of the host and the smart NIC may require the BMC of the smart NIC to coordinate its management actions with the host BMC, and possibly vice versa. As an example, such coordination may involve communicating the smart NIC BMC with the host BMC for managing power-on and / or power-off operations of the host, smart NIC, and / or server. As other examples of coordination between BMCs, the smart NIC BMC can communicate with the host BMC for the following purposes: determining whether the host is powered on; determining whether the host has passed the fault check associated with the host powering on; determining whether the host has passed the safety check associated with the host powering on; the smart NIC BMC notifies the smart NIC BMC that the server is powered off; and so on. As another example of coordination between BMCs, the host BMC and the smart NIC BMC can communicate about the compatibility or timing of firmware upgrades. As yet another example, the host BMC and the smart NIC BMC can communicate about thermal issues regarding the smart NIC so that the host BMC can adjust the cooling fan speed. Therefore, in general, the two BMCs can coordinate with each other on many management issues related to the host, the smart NIC, and the entire server platform. Such coordination between the two BMCs may lead to overall complexity and cost of the server.

[0022] According to example embodiments described herein, a computer platform (e.g., a server, a blade server, etc.) includes a host and an intelligent NIC installed in an expansion bus connector (e.g., a PCIe bus connector) of the computer platform. According to example embodiments, instead of the host and the intelligent NIC each containing a BMC, a single BMC is located on the intelligent NIC; and the single BMC manages both the intelligent NIC and the host. In other words, according to example embodiments, the BMC located at the intelligent NIC manages the operation of the host (e.g., power on, configuration, boot, platform health control, firmware updates, etc.) and manages the operation of the intelligent NIC (e.g., power on, configuration, boot, platform health control, firmware updates, etc.). Among the possible advantages of this arrangement, the cost and complexity of the server is reduced because the overhead that would otherwise be associated with coordination and collaboration between multiple BMCs is eliminated.

[0023] refer to Figure 1A As a more specific example, according to some embodiments, a cloud-based computer system 99 affiliated with a particular cloud operator can provide multi-tenant cloud services to multiple clients or tenants. The cloud service can be any of a variety of different cloud services, such as software as a service (SaaS), infrastructure as a service (IaaS), platform as a service (PaaS), etc. In addition, according to specific embodiments, the cloud service can be affiliated with one of several different cloud infrastructures, such as a public cloud that is generally available to all potential users through a public network; a limited access private cloud provided through a private network (e.g., a cloud service provided by an on-site data center); or a hybrid cloud that combines public and private clouds.

[0024] Tenants can access the cloud computing resources 100 of the system 99 (which provides cloud services) via cloud clients 128 (e.g., laptop computers, desktop computers, smart phones, tablet computers, wearable computers, etc.). Figure 1A As depicted, cloud computing resources 100 and clients 128 may be interconnected via a network fabric 159. In general, network fabric 159 may be associated with one or more types of communications networks such as, by way of example, a Fibre Channel network, a Gen-Z fabric, a dedicated management network, a local area network (LAN), a wide area network (WAN), a global network (e.g., the Internet), a wireless network, or any combination thereof.

[0025] Generally speaking, cloud computing resources 100 are a shared resource pool that includes physical hardware resources, such as physical servers (eg, server blades), network components, management resources, physical storage devices, physical storage networks, and the like. Figure 1A A physical computer platform 100-1 is illustrated that can provide at least a portion of cloud computing resources 100. In this context, a "computer platform" refers to a unit that includes a chassis and hardware mounted to the chassis, wherein the hardware is capable of executing machine-executable instructions (or "software"). According to an example embodiment, a blade server is an example of computer platform 100-1. However, according to further embodiments, computer platform 100-1 can be a platform other than a blade server, such as a rack-mounted server, a client, a desktop computer, a smart phone, a laptop computer, a tablet computer, etc.

[0026] For example embodiments where computer platform 100-1 is a blade server, the server may have a frame or chassis; one or more motherboards may be mounted to the chassis; and each motherboard may contain one or more multi-core central processing unit (CPU) semiconductor packages (or "sockets" or "chips"). According to example embodiments, the blade server may have a form factor, mechanical latch(es), and corresponding electrical connectors such that a server blade will be installed in or removed from a corresponding server blade opening or slot in a rack-mounted blade chassis.

[0027] Regardless of its specific form, according to an example embodiment, computer platform 100-1 can have a cloud native architecture in which the hardware resources and machine executable instruction resources (i.e., "software resources") of computer platform 100-1 are divided into two security domains: a cloud tenant domain; and a cloud operator domain. In this context, the "tenant domain" refers to the portion of computer platform 100-1 associated with executing tenant software and data; and according to an example embodiment, the cloud operator (the entity providing cloud services) cannot access or at least has limited access to tenant data and software. The "cloud operator domain" refers to the portion of computer platform 100-1 associated with providing I / O services for the tenant software being executed.

[0028] More specifically, according to some embodiments, a computer platform 100-1 includes a host 101 associated with a client domain. The host 101 includes an application layer 160, which contains application instances that can be associated with one or more tenants at any particular time. Virtualization technology can be used on the computer platform 100-1 to provide security and fault isolation between tenants. For example, an application instance of a particular tenant can be executed within one or more virtual machines (VMs) 170, and these VMs 170 can then reside inside a given container 172 associated with the tenant. In this way, according to some embodiments, a given tenant can be associated with one or more VMs 170 and one or more containers 172.

[0029] In this context, a "virtual machine" or "VM" (also referred to as a "client virtual machine," "virtual machine instance," or "client virtual machine instance") refers to a virtual environment that functions as a virtual server or virtual computer system, with its own physical resources (e.g., CPU, system memory, (multiple) network interfaces, and storage). In addition, a VM may have its own abstraction of an operating system (such as operating system 176); and in general, a VM is a virtual abstraction of the hardware and software resources of computer platform 100-1. The lifecycle (e.g., deployment and termination) of a VM may be managed by a virtual machine monitor (VMM) or hypervisor (such as hypervisor 174).

[0030] As used herein, a "container" (also referred to as an "instantiated container," "container instance," or "software container") generally refers to a virtual runtime environment for one or more applications and / or application modules, and the virtual runtime environment is constructed to interface with an operating system kernel. For example, a container for a given application may include the application's executable code and its dependencies, such as the application's system tools, libraries, configuration files, executable files, and binaries. According to an example embodiment, the container includes an operating system kernel mount interface, but does not include the operating system kernel. Thus, for example, a given computer platform may include multiple containers that share an operating system kernel through corresponding operating system kernel mount interfaces. Docker containers and rkt containers are examples of software containers.

[0031] According to an example embodiment, host 101 includes a bus infrastructure that includes one or more expansion bus connectors 150 (e.g., a peripheral component interconnect express (PCIe) bus connector). A given expansion bus connector 150 can accommodate a smart NIC 140. In addition, according to an example embodiment, smart NIC 140 provides a dual air gap security management and control interface to create a boundary between the tenant domain and the cloud operator domain. Figure 1A As depicted, according to an example embodiment, particular Intelligent NIC 140 includes BMC 129 ; and according to an example embodiment, this single BMC 129 manages the operation of the entire computer platform 100 - 1 , ie, BMC 129 manages host 101 , and BMC 129 also manages Intelligent NIC 140 .

[0032] As used herein, a "BMC" or "baseboard management controller" is a dedicated service processor that uses sensors to monitor the physical state of a server or other hardware and communicates with a management system through a management network. The baseboard management controller can also communicate with applications executed at the operating system level through the following: an input / output controller (IOCTL) interface driver, a representative state transfer (REST) ​​application program interface (API), or a certain system software agent that facilitates communication between the baseboard management controller and the application. The baseboard management controller can perform hardware-level access to hardware devices located in a server chassis including system memory. The baseboard management controller can be able to directly modify hardware devices. The baseboard management controller can operate independently of the operating system of the system in which the baseboard management controller is provided. The baseboard management controller can be located on a motherboard or main circuit board of a server or other device to be monitored. The fact that the baseboard management controller is installed on the motherboard of the managed server / hardware or otherwise connected or attached to the managed server / hardware does not prevent the baseboard management controller from being considered "separated" from the server / hardware. As used herein, the baseboard management controller has the management capability of the subsystems of the computing device and is separated from the processing resources of the operating system that executes the computing device. The baseboard management controller is separate from the processor (eg, central processing unit) that executes a high-level operating system or hypervisor on the system.

[0033] The BMC 129 can be remotely accessed by a remote management server 197 that is coupled to the network structure 159. In this manner, the remote server 197 can transmit requests (e.g., Intelligent Platform Management Interface (IPMI) messages containing IPMI commands) to the BMC 129 so that the BMC 129 manages and controls the functions of the host 101 and the intelligent NIC 140; and the remote server 197 can receive messages (e.g., IPMI messages) from the BMC 129 representing status information, health information, configuration information, configuration options, event notifications, etc. from the BMC 129.

[0034] According to example embodiments, the BMC 129 located at the smart NIC manages the host 101 via one or more out-of-band communication channels 109 using communications through a signaling interface 152 of a connector 150. In this context, the signaling interface 152 of the connector 150 refers to a physical communication terminal (e.g., a pin, a socket, or a terminal) of the connector 150. In this context, an "out-of-band communication channel" with the smart NIC 140 refers to using a secure communication channel with the smart NIC 140, rather than a primary communication channel of the smart NIC. For example, according to some embodiments, the smart NIC 140 may be a PCIe card having a primary PCIe bus communication channel; and for these embodiments, the out-of-band communication channel 109 may be, for example, an internal integrated bus (I 2 C) bus, improved internal integration (I 3 The signaling interface 152 may include a bus, a serial peripheral interface (SPI) bus, an enhanced SPI (eSPI) bus, or a bus associated with another standard. According to an example embodiment, the signaling interface 152 may include a communication channel associated with the communication of control and / or telemetry signals to / from the BMC 129. According to a specific embodiment, the signaling interface 152 may correspond to all terminals of the connector 150, or may correspond to a smaller subset of all terminals of the connector 150.

[0035] According to example embodiments, smart NIC 140 may be disposed on a circuit card substrate having a card edge connector configured to be inserted into connector 150 to mechanically secure smart NIC 140 to connector 150 and to form an electrical connection between a host and smart NIC 140. For example, according to some embodiments, connector 150 is a slot connector; and the circuit card substrate of smart NIC 140 may have conductive traces disposed on a card edge having a form factor configured to be received inside connector 150, such that when the card edge is received in the connector, the traces contact and are electrically connected to terminals (e.g., spring terminals) of connector 150.

[0036] In accordance with an example embodiment, connector 150 may have a card edge pinout that corresponds to the Data Center-Ready Security Control Interface (DC-SCI) of the Data Center Security Control Module (DC-SCM) specification published by the Open Compute Project Version 1.0 (March 11, 2021).

[0037] As further described herein, according to some embodiments, BMC 129 may include dedicated hardware that performs BMC-related management operations alone and does not share non-management-related BMC operations with other components of intelligent NIC 140. For example, according to some embodiments, BMC 129 may include a main semiconductor package (or "chip") containing one or more semiconductor dies. More specifically, according to some embodiments, BMC 129 may include a main semiconductor package containing one or more main hardware processing cores (e.g., CPU cores, Advanced Reduced Instruction Set Computer (RISC) Machine (ARM) processing cores, etc.) that execute machine-executable instructions (or "software", such as firmware) for managing the operation of host 101 and intelligent NIC 140. Figure 2 (described further herein) is a schematic diagram of a BMC based on such specialized hardware, according to an example implementation.

[0038] Combination Figure 1A For reference Figure 1C According to further example embodiments, BMC 129 may be a virtual BMC, i.e., an abstraction of the actual hardware and software of smart NIC 140. For example, according to some embodiments, smart NIC 140 may provide one or more client VMs 149 that provide BMC 129. Client VMs 149 may, for example, execute inside a virtualized environment (e.g., a container). According to some embodiments, as further described herein, smart NIC 140 may include a hardware processor (e.g., a CPU, a CPU processing core, an ARM processing core, etc.) that executes machine-executable instructions for providing one or more I / O services for smart NIC 140. According to example embodiments, one or more of these (multiple) hardware processors of smart NIC 140 may further execute instructions to provide VM machines 149 and BMC 129.

[0039] As a more specific example, according to some embodiments, the hardware processor of the smart NIC 140 may execute machine executable instructions to provide a container including a virtual machine 149, the virtual machine 149 being associated with the management plane of the BMC 129 and including one or more application instances corresponding to the management stack of the BMC for managing the computer platform 100-1; and the container may include another virtual machine 149 associated with the security plane of the BMC 129 and including one or more application instances that provide security services for the computer platform 100-1. Figure 2 The security plane 230 and the management plane 210 of the hardware BMC 129 are discussed.

[0040] According to further example embodiments, BMC 129 may be a hybrid combination of a virtual BMC and a hardware BMC. For example, hybrid BMC 129 may include dedicated hardware components for providing certain management and / or security plane functions of hybrid BMC 129; a hardware processor in smart NIC 140 that executes machine executable instructions to provide I / O services for smart NIC 140 may further execute machine executable instructions to provide management and / or security plane functions of hybrid BMC 129.

[0041] Back to reference Figure 1A ,for Figure 1A In the example embodiment depicted in FIG. 1 , host 101 may include one or more general-purpose hardware processors 102 (e.g., one or more CPU packages, one or more CPU processing cores, one or more GPU cores, one or more FPGAs, etc.); system memory 104; and bus infrastructure. According to an example embodiment, general-purpose hardware processor(s) 102 execute machine-executable instructions (i.e., "software") of host 101. For example, hardware processor(s) 102 may execute instructions associated with: instances of VM 170, instances of container 172, hypervisor 174, operating system 176, application instances associated with application layer 160, boot service firmware 175, etc. According to an example embodiment, system memory 104 and other memories discussed herein are non-transitory storage media that may be generally formed of storage devices such as semiconductor storage devices, memristor-based storage devices, magnetic storage devices, phase-change storage devices, combinations of devices of one or more of these storage technologies, and the like. System memory 104 may represent a collection of both volatile memory devices and non-volatile memory devices. Boot services firmware 175 represents firmware (eg, Basic Input / Output Operating System (BIOS) firmware and / or Unified Extensible Firmware Interface (UEFI) firmware) executed by computer platform 100-1 during booting of computer platform 100-1 after computer platform 100-1 is powered on or reset.

[0042] According to an example embodiment, the bus infrastructure of the host 101 may include one or more bridges 106, which may be coupled to the system memory 104 and other components of the host 101, such as a trusted platform module (TPM) 177; one or more USB devices 126; and the like. The bridge(s) 106 may include one or more PCIe ports connected to one or more PCIe bus expansion cards 150, such as the depicted connector 150 housing the smart NIC 140, via one or more corresponding PCIe links or buses 108. In general, the bridge(s) 106 may include various buses of the host 101, such as a PCIe bus, an SPI bus, an enhanced SPI (eSPI) bus, a low pin count (LPC) bus, an I2C bus, a SPI ... 2 C bus, I 3 C bus, and possible buses associated with other bus standards).

[0043] According to some embodiments, the bridge 106 may include a north bridge 106 and a separate south bridge 106. In this way, according to some embodiments, the general hardware processor 102 may include one or more semiconductor packages (or "chips"), and the processor 102 may include a north bridge 106, which includes a memory controller and a PCIe root port. The south bridge 106 may provide I / O ports, such as serial advanced technology attachment (SATA) ports, universal serial bus (USB) ports, LPC ports, SPI ports, eSPI ports, etc. According to some embodiments, the north bridge 106 may not be part of the hardware processor 102. According to further embodiments, the north bridge and the south bridge may be combined into a single bridge 106; and according to some embodiments, the single bridge 106 may be part of the processor 102.

[0044] According to an example embodiment, among other hardware components of the host 101, the host 101 may include a power controller 180 that may be controlled by the operating system 176 for setting a particular system power state for the computer platform 100-1. In this manner, according to an example embodiment, the operating system 176 may communicate with the power controller 180 (e.g., causing assertion of one or more signals to the power controller 180) for changing the system power state. In this context, a "system power state" refers to the power state of all components of the computer platform 100-1, except for components of the computer platform 100-1 that are involved in platform management, such as the BMC 129. For a given system power state, some components of the computer platform 100-1 may be powered on at different levels than other components (e.g., for a given power consumption state, some components of the computer platform 100-1 may be powered off for power conservation purposes, while other components may be powered on to a relatively higher power consumption state). For example, operating system 176 may communicate with power controller 180 to transition computer platform 100-1 to a power-on reset; transition computer platform 100-1 from a higher power consumption state to a lower power consumption state; transition computer platform 100-1 from a lower power consumption state to a higher power consumption state; power off computer platform 100-1; and the like.

[0045] According to example embodiments, the power controller 180 may be controlled by an entity other than the operating system 176. For example, according to some embodiments, the boot services firmware 175 may communicate with the power controller 180 for controlling the system power state. Additionally, as further described herein, according to some embodiments, the BMC 129 may communicate with an appropriate entity (e.g., the power controller 180, the boot services firmware 175, or the operating system 176) for changing the system power state.

[0046] Also like Figure 1A As depicted, according to some embodiments, computer platform 100-1 may include one or more sensors 105. According to an example embodiment, sensor 105 provides signals or indications representing various sensed conditions related to the environment and / or health of computer platform 100-1. In this manner, according to an example embodiment, BMC 129 may monitor the signals provided by sensor 105 for the following purposes: monitoring the health of computer platform 100-1; monitoring the temperature of computer platform 100-1 for performing thermal management; monitoring for tamper detection; and the like. As examples, sensor 105 may be a temperature sensor; a tamper indication sensor; an overvoltage sensor; an undervoltage sensor; a fan speed sensor; and the like. The signals provided by sensor 105 are routed to BMC 129 via signaling interface 152.

[0047] refer to Figure 1B According to example embodiments, the smart NIC 140 may include one or more hardware components mounted to one or more circuit substrates 141. In addition, a given circuit substrate 141 may have a form factor and corresponding features (electrical traces, etc.) that allow the smart NIC 140 to be mounted in a connector 150. According to some embodiments, the circuit substrate 141 may have a card edge with conductive traces arranged in a pinout according to a DC-SCI, and according to further embodiments, the circuit substrate 141 may have a card edge with conductive traces arranged in a pinout other than or in addition to a DC-SCI. According to example embodiments, the smart NIC 140 includes hardware components, such as one or more hardware processors 142 (e.g., a CPU processing core, such as an ARM processing core, an embedded processing core, an ARM processing core, etc.); a memory 144; one or more sensors 154; a network interface (not shown); and a power controller 156.

[0048] Processor(s) 142 execute machine executable instructions 146 stored in memory 144. According to some embodiments, processor(s) 142 execute instructions 146 for performing one or more cloud operator domain based I / O services for host 101. In this manner, according to example embodiments, processor(s) 142 may execute instructions 146 for executing one or more application instances of application layer 148, as well as instances associated with virtual machines, containers, etc.

[0049] Sensors 154 sense conditions of smart NIC 140 (eg, environmental events, fault events, and / or tampering events), and indications or signals provided by sensors 154 may be monitored by BMC 129 .

[0050] As described herein, according to some embodiments, BMC 129 may be formed of discrete hardware components separate from other components of intelligent NIC 140, such as a separate semiconductor package and possibly associated memory, an application specific integrated circuit (ASIC), etc. In addition, according to further example embodiments, BMC 129 may be a virtual BMC formed by one or more processors 142 of intelligent NIC 140 executing machine-executable instructions; or according to further embodiments, BMC 129 may be a hybrid BMC including virtual and actual hardware components.

[0051] Figure 2 FIG. 1 depicts a hardware BMC 129 formed of actual hardware components according to an example embodiment. Figure 1A , Figure 1B and Figure 1C For reference Figure 2 , BMC 129 includes a management plane 210 and a security plane 230 isolated from the management plane 210. According to an example embodiment, the security plane 230 includes a secure enclave 240, which refers to a portion of the BMC 129 that provides security services for the computer platform 100-1. In this context, a "secure enclave" refers to a subsystem for which access to and from the subsystem is strictly controlled. According to an example embodiment, the management plane 210 is a portion of the BMC 129 that executes a management firmware stack 257 (referred to herein as "management stack 257") for providing management services (including management host 101 and management intelligent NIC 140) for the computer platform 100-1. As an example, the management stack 257 can be an open BMC firmware stack version 2.9.0 released on January 12, 2021. According to a further embodiment, the management stack 257 can be a management stack different from the open BMC stack and different from the open source stack.

[0052] like Figure 2 As depicted, according to example embodiments, BMC 129 may be coupled to one or more external memories (e.g., external to a semiconductor package containing BMC 129), such as volatile memory 264 and non-volatile memory 268. As an example, non-volatile memory 268 may store firmware 270 (e.g., a firmware image) and may be connected to BMC 129 via a bus (e.g., an SPI bus). As further described herein, according to some embodiments, BMC 129 may validate firmware 270 at boot time of computer platform 100-1.

[0053] According to an example embodiment, the secure enclave 240 includes, among other features: a secure processor 242 (e.g., an embedded CPU processing core); a volatile memory 251 (e.g., a memory for storing firmware that is loaded into the volatile memory and executed by the secure processor); a secure bridge for controlling access into the secure enclave and controlling communications out of the secure enclave; peripherals (e.g., cryptographic accelerators, random number generators, tamper detection circuits, etc.); and a hardware root of trust (RoT) engine 243 (referred to herein as a silicon RoT engine or "SRoT engine 243"). According to an example embodiment, the SRoT engine 243 verifies the firmware to be executed by the secure processor 242 before the SRoT engine 243 loads the firmware into the volatile memory 251 of the secure enclave and allows the secure processor 242 to execute the firmware.

[0054] As used herein, a "trust root device" (such as SRoT engine 243) can be a device that operates in an expected manner. In other words, SRoT device 243 can be essentially trusted software, hardware, or some combination thereof. SRoT device 243 may include a computing engine. The computing engine can be software that operates using hardware in SRoT device 243, hardware of SRoT device 243, or some combination thereof. SRoT device 243 may include a verification trust root (RTV). RTV performs integrity measurement or digital signature on program code (e.g., code loaded into a secure enclave) and verifies the code against a predetermined expected value or policy. SRoT device 243 may include a storage trust root (RTS). RTS may be a computing engine that can maintain an accurate summary of tamper-evident values. For example, SRoT device 243 may include registers that store reference hashes or metric hashes. Further, SRoT device 243 may include multiple such registers. In another example, SRoT device 243 may include a reporting trust root (RTR). RTR may be a computing engine that can send requested information to a requesting device. The information may include the contents of a register of the SRoT device 243 (or the contents of the RTS) and information specified by the requester. The SRoT device 243 may include other computing engines not described herein, such as a computing engine for measuring a specified value or a computing engine for performing authentication.

[0055] like Figure 2 As depicted, according to some embodiments, BMC 129 includes one or more in-band communication interfaces 205 (e.g., PCIe bus interfaces) and one or more out-of-band communication interfaces 206 (e.g., DC-SCI). In addition, BMC 129 includes one or more embedded hardware processing cores 254 (e.g., embedded CPU processing cores, such as ARM processing cores) that execute machine executable instructions 256 stored in non-volatile memory 255 of BMC 129 to provide a management stack 257 of the BMC. Figure 2 As depicted, according to some embodiments, the signaling interface 152 may include signals for communicating with the out-of-band communication interface(s) 206. According to further embodiments, the signaling interface 152 may also include signals associated with the in-band communication interface(s) 204.

[0056] Figure 3 An example management plane process 300 of the BMC 129 is depicted in accordance with some embodiments. Figure 1A and Figure 2 For reference Figure 3According to some embodiments, process 300 includes BMC 129 communicating with remote server 197 (block 304); BMC 129 communicating with components of host 101 via signal interface 151; and BMC 129 communicating with components of intelligent NIC 140. According to block 308, in response to these communications, BMC 129 manages platform 100-1, including BMC 129 managing host 101 via signal interface 151 and BMC 129 managing intelligent NIC 140.

[0057] As an example of BMC managing host 101, according to some embodiments, BMC 129 can manage the booting of host 101, such as Figure 4 The example process 400 is depicted in conjunction with Figure 1A and Figure 2 For reference Figure 4 According to some embodiments, after power-on or reset, the BMC 129 holds its general purpose processing core(s) 254 in a reset state until certain actions are performed, such as Figure 4 In this way, after the initial trusted root security check and other checks (eg, hardware fault check) of the computer platform 100-1 pass (eg, no security check or hardware fault check fails), the BMC 129 releases the reset of the general processing core 254.

[0058] According to an example embodiment, the secure enclave 240 stores an immutable fingerprint used by the SRoT engine 243 to authenticate machine executable instructions. More specifically, according to an example embodiment, in response to the BMC 129 powering on or resetting, the SRoT engine 243 authenticates an initial portion of the firmware 270 and loads the authenticated portion of the firmware 270 into the non-volatile memory 251 of the BMC 129, pursuant to block 404, such that the firmware portion is now trusted. Pursuant to block 408, the security processor 242 is then allowed to boot and execute the loaded firmware instructions. By executing the firmware instructions, pursuant to block 408, the security processor 242 may then authenticate another portion of the firmware 270 corresponding to a portion of the management firmware stack 257; and after authenticating this portion of the management firmware stack 257, the security processor 242 may then load the authenticated portion into the memory 255 of the BMC 129. This portion of the management firmware stack 257 may then be executed by the general processing core(s) 254, causing the processing core(s) 254 to load additional portions of the firmware 270 and place the loaded portions into the memory 264. Access to the memory 264 may involve additional training and initialization steps (e.g., those set forth by the DDR4 specification). These instructions may be executed from the authenticated portion of the management firmware stack 257 in the memory 255. According to an example embodiment, the secure enclave 240 may lock the memory 255 to prevent modification or tampering of the authenticated portion(s) stored in the memory 255.

[0059] According to decision block 412, BMC 129 determines whether the security and hardware fault checks performed by BMC 129 have passed. If not, then according to an example embodiment, BMC 129 may terminate or end the booting of computer platform 100-1. Otherwise, BMC 129 will release (block 416) reset of general processing core 254 to allow (multiple) processing cores 254 to execute management stack 257 to manage computer platform 100-1, including management of intelligent NIC 140 and host 101.

[0060] Figure 5 A process 500 of the BMC 129 is depicted for controlling the power system state of the computer platform 100-1. Figure 1A and Figure 2 For reference Figure 5According to an example embodiment, BMC 129 receives (block 504) a communication (e.g., a message containing a command) from remote server 197 requesting a change in the system power state of computer platform 100-1. For example, the request may be a request to: power off computer platform 100-1; a request to reset computer platform 100-1; a request to put computer platform 100-1 into a sleep mode of operation; and the like. In accordance with process 500, BMC 129 determines (decision block 508) whether the change in the system power state involves communication with OS 176. For example, computer platform 100-1 may be fully booted; and, as such, control of power management may be accomplished using OS 176. Conversely, in a pre-operating system environment, BMC 129 may communicate with boot services firmware 175 (e.g., BIOS or UEFI firmware). Figure 5 As depicted, pursuant to decision block 508, if BMC 129 determines that OS 176 is involved, pursuant to block 512, BMC 129 communicates with OS 176 via signaling interface 152 to change the system power state as requested. Otherwise, pursuant to block 516, BMC 129 communicates with power controller 180 or BIOS or UEFI via signaling interface 152 to change the system power state.

[0061] Figure 6 Another management process 600 of the BMC 129 according to an example implementation is depicted. The example process 600 involves the BMC 129 controlling the boot path of the host 101. In this manner, the "boot path" refers to the path taken to load the operating system boot loader, obtain the OS image, obtain OS related files, etc. Figure 1A and Figure 2 For reference Figure 6 According to block 604, the BMC 129 selects a boot path based on the specific boot path configuration. In this regard, the remote server 197 may, for example, communicate with the BMC 129 to select a virtual media device as a boot device; select a pre-boot execution environment (PXE) network boot, etc. According to block 608, the BMC 129 communicates with the boot services firmware 75 via the signaling interface 152 to enable loading and booting of the operating system 176.

[0062] Figure 7 Another example management process 700 of the BMC 129 is depicted. Here, the process 700 illustrates a thermal management process. Figure 1A , Figure 1B and Figure 2 For reference Figure 7According to process 700, BMC 129 monitors (block 704) signals from cooling fan sensor 105 and temperature sensor 105 of host 101 (via signaling interface 152), as well as signals from temperature sensor 154 of smart NIC 140, for evaluating the thermal status of host 101 and / or smart NIC 140. For example, the evaluation may involve BMC 129 determining that one or more temperatures of host 101 or smart NIC 140 are outside of an acceptable operating range. BMC 129 may then determine that corrective action is to be taken, according to decision block 708. For example, the corrective action may include: turning on one or more cooling fans; increasing the speed of one or more cooling fans; powering off or disconnecting certain components of host 101 or smart NIC 140, changing the system power state, etc. If thermal management corrective action is to be taken, then according to block 712, BMC 129 takes the appropriate corrective action.

[0063] According to an example implementation, the BMC 129 can perform management functions for the smart NIC 140, which are similar to the management functions described above for the host 101. For example, via remote messaging, the BMC 129 can be remotely controlled to change the power state of the smart NIC 140 (e.g., shut down the smart NIC 140, reset the smart NIC 140, power on the smart NIC 140, etc.); select a boot path for booting and loading an operating system 143 of the smart NIC 140; select and configure virtual media used by the smart NIC 140; update the firmware of the smart NIC 140; etc. In addition, the BMC 129 can monitor one or more sensors 154 of the smart NIC; report environmental conditions and events resulting from the monitoring to the remote server 197; perform corrective actions based on the sensor monitoring by controlling components of the smart NIC 140; and the like. According to example implementations, BMC 129 may validate firmware executed by processor(s) 142 of smart NIC 140; detect hardware failures on smart NIC 140; perform recovery actions in response to operating system failures on smart NIC 140; and so on.

[0064] Because a single BMC 129 manages both the intelligent NIC 140 and the host 101, the single BMC 129 can coordinate and cooperate management actions of the host 101 and management actions of the intelligent NIC 140. Therefore, as described herein, according to example embodiments, a single BMC 129 manages the entire computer platform 100-1, including managing the host 101 and managing the intelligent NIC 140. As is apparent from the foregoing description, some management actions performed by the BMC 129 are only for actions performed on the host 101, some management actions are only for actions performed on the intelligent NIC 140, and some management actions may involve both the host 101 and the intelligent NIC 140.

[0065] refer to Figure 8 According to an example implementation, process 800 includes an application layer of a host of a computer platform using (block 804) a smart network interface card (NIC) of the computer platform to provide input / output (I / O) services to the application layer. The I / O services include services associated with a cloud operator domain; the smart NIC is installed in a connector; and the application layer is associated with a cloud tenant domain. Process 800 includes a baseboard management controller of the smart NIC managing (block 808) the computer platform. Managing the computer platform includes the baseboard management controller managing the host; the baseboard management controller managing components of the smart NIC other than the baseboard management controller; and managing the host includes the baseboard management controller communicating with the host via the connector to control a system power state of the computer platform.

[0066] refer to Fig. 9 According to an example embodiment, system 900 includes a host 904; a connector 916; and an intelligent network interface card (NIC) 920 to be installed in connector 916. Host 904 includes a central processing unit (CPU) 908 and memory 912. Intelligent NIC 920 includes memory 924; at least one hardware processor 932; and baseboard management controller 936. (Multiple) hardware processors 932 execute instructions 928 stored in memory 924 to provide input / output (I / O) services for host 904 other than network communications. I / O services are associated with a cloud operator domain. Baseboard management controller 936 manages components of intelligent NIC 920 other than baseboard management controller 936; and manages host 904, including communicating with host 904 via connector 916 to control system power state.

[0067] refer to Fig.10According to an example embodiment, a non-transitory storage medium 1000 stores machine-readable instructions 1010, which, when executed by a machine, cause the machine to provide input / output (I / O) services to a host of a computer platform via an intelligent network interface card (NIC). The I / O services include services associated with a cloud operator domain; the intelligent NIC is housed in a connector of a motherboard of the host; and the host includes an application layer associated with a cloud tenant domain. The instructions 1010, when executed by the machine, further cause the machine to provide a virtual baseboard management controller on the intelligent NIC to manage components of the intelligent NIC other than the baseboard management controller; and manage the host. Managing the host includes communicating with the host via the connector to control a system power state.

[0068] According to an example embodiment, the management platform includes an intelligent NIC executing instructions to provide a virtual baseboard management controller. A particular advantage is that no dedicated hardware is used for the baseboard management controller.

[0069] According to an example embodiment, providing I / O services using a smart NIC includes at least one central processing unit (CPU) of the smart NIC executing instructions to provide I / O services; and a baseboard management controller managing the computer platform further includes an embedded hardware subsystem of the smart NIC other than the CPU(s) managing the computer platform. A particular advantage is that a single baseboard management controller can manage both the smart NIC and the host.

[0070] According to an example embodiment, the intelligent NIC is installed in a connector of a computer platform; and the baseboard management controller manages the computer platform further including the baseboard management controller communicating telemetry signals with components of the host via the connector. The management computer platform further includes the baseboard management controller communicating with at least one general-purpose central processing unit (CPU) of the host via the connector; and the (multiple) CPUs are associated with the application layer. A particular advantage is that a single baseboard management controller can be used to manage both the intelligent NIC and the host.

[0071] According to an example implementation, a host may include an application layer associated with a cloud tenant domain; the cloud tenant domain may include an untrusted domain; the cloud operator domain may include a trusted domain; and a baseboard management controller may be associated with the cloud operator domain. Of particular advantage, in cloud-centric applications, a single baseboard management controller may be used to manage both the host and the smart NIC.

[0072] While the present disclosure has been described with respect to a limited number of embodiments, numerous modifications and variations of the present disclosure will become apparent to those skilled in the art having the benefit of this disclosure. The appended claims are intended to cover all such modifications and variations.

Claims

1. A method comprising: an application layer of a host of a computer platform using an intelligent network interface card NIC of the computer platform to provide input / output I / O services to the application layer, wherein the I / O services include services associated with a cloud operator domain, the intelligent network interface card NIC is installed in a connector, and the application layer is associated with a cloud tenant domain; and The baseboard management controller of the intelligent network interface card NIC manages the computer platform, wherein managing the computer platform includes: The baseboard management controller manages the host; The baseboard management controller manages components of the intelligent network interface card NIC other than the baseboard management controller, wherein managing components of the intelligent network interface card NIC other than the baseboard management controller includes the baseboard management controller changing a power state of the intelligent network interface card NIC in response to remote messaging; and Managing the host includes the baseboard management controller communicating with the host via the connector to control a system power state of the computer platform.

2. The method of claim 1, wherein: The baseboard management controller manages the computer platform including the intelligent network interface card NIC executing instructions to provide a virtual baseboard management controller.

3. The method of claim 1, wherein: Providing the I / O service using the intelligent network interface card NIC includes at least one central processing unit (CPU) of the intelligent network interface card NIC executing instructions to provide the I / O service; and The baseboard management controller manages the computer platform further including an embedded hardware subsystem of the intelligent network interface card NIC except the at least one CPU to manage the computer platform.

4. The method of claim 1, wherein: The intelligent network interface card NIC is installed in the connector of the computer platform; The baseboard management controller managing the computer platform further includes the baseboard management controller communicating telemetry signals with components of the host via the connector; The baseboard management controller managing the computer platform further comprises the baseboard management controller communicating with at least one general purpose central processing unit CPU of the host via the connector; and The at least one general purpose CPU is associated with the application layer.

5. The method of claim 1, wherein: The components other than the baseboard management controller that manage the intelligent network interface card NIC also include the baseboard management controller selecting a boot path for loading an operating system of the intelligent network interface card NIC.

6. The method of claim 1, wherein: Managing the components other than the baseboard management controller of the intelligent network interface card NIC also includes the baseboard management controller updating the firmware of the intelligent network interface card NIC.

7. The method of claim 1, wherein: The components other than the baseboard management controller that manage the intelligent network interface card NIC also include the baseboard management controller performing management actions only for the intelligent network interface card NIC but not for the host.

8. A system comprising: A host computer, the host computer comprising a central processing unit (CPU) and a memory; Connectors; as well as An intelligent network interface card NIC installed in the connector, wherein the intelligent network interface card NIC comprises: A memory, the memory being used to store instructions; at least one hardware processor configured to execute the instructions to provide input / output (I / O) services for the host other than network communications, wherein the I / O services are associated with a cloud operator domain; and A baseboard management controller, the baseboard management controller is used to perform the following operations: managing components of the intelligent network interface card NIC other than the baseboard management controller, wherein managing components of the intelligent network interface card NIC other than the baseboard management controller includes the baseboard management controller changing a power state of the intelligent network interface card NIC in response to remote messaging; and Managing the host, wherein managing the host includes communicating with the host via the connector to control a system power state.

9. The system of claim 8, wherein: The baseboard management controller includes a virtual baseboard management controller, and the at least one hardware processor is to further execute the instructions to provide the virtual baseboard management controller.

10. The system of claim 8, wherein: The baseboard management controller includes an embedded hardware subsystem separate from the at least one hardware processor, and the embedded hardware subsystem includes an embedded hardware processor for executing instructions to manage the system.

11. The system of claim 8, wherein: The host further includes an application layer associated with a cloud tenant domain; The cloud tenant domain includes an untrusted domain; The cloud operator domain includes a trusted domain; and The baseboard management controller is associated with the cloud operator domain.

12. The system of claim 11, wherein: The CPU includes at least one hardware processor configured to further execute the instructions to perform the I / O service in response to a request from the application layer.

13. The system of claim 8, further comprising: A circuit substrate separate from the intelligent network interface card NIC, wherein the CPU and the connector are mounted to the circuit substrate.

14. The system of claim 8, wherein: The connector includes a Data Center Ready Security Control Interface DC-SCI.

15. The system of claim 8, wherein: The at least one hardware processor is configured to further execute the instructions to communicate telemetry signals of the baseboard management controller with components of the host via the connector, and to communicate signals associated with a host interface of the baseboard management controller via the connector.

16. The system of claim 8, wherein: The baseboard management controller includes a hardware root of trust.

17. The system of claim 8, wherein: The at least one hardware processor is configured to execute the instructions to provide virtualization services for the host.

18. The system of claim 8, wherein: The at least one hardware processor is configured to execute the instructions to provide a virtualized network for the host.

19. The system of claim 8, wherein: The at least one hardware processor is configured to execute the instructions to provide network function virtualization for the host.

20. The system of claim 8, wherein: The at least one hardware processor is configured to execute the instructions to provide at least one of the following: a storage service for the host, a network security service for the host, or a network control plane service for the host.

21. A non-transitory storage medium for storing machine-readable instructions, the machine-readable instructions, when executed by a machine, causing the machine to perform the following operations: Providing input / output I / O services to the host of the computer platform via the intelligent network interface card NIC, wherein: The I / O services include services associated with a cloud operator domain, the intelligent network interface card NIC is accommodated in a connector of a motherboard of the host, and the host includes an application layer associated with a cloud tenant domain; as well as A virtual baseboard management controller is provided on the intelligent network interface card NIC to manage components of the intelligent network interface card NIC other than the virtual baseboard management controller and to manage the host, including communicating with the host via the connector to control a system power state, wherein managing components of the intelligent network interface card NIC other than the virtual baseboard management controller includes the virtual baseboard management controller changing the power state of the intelligent network interface card NIC in response to remote messaging.

22. The storage medium of claim 21, wherein: When executed by the machine, the instructions further cause the machine to provide at least one of the following: a network security service for the application layer, a network storage service for the application layer, a virtualization function for the application layer, or a virtual network for the application layer.

23. The storage medium of claim 21, wherein: The instructions, when executed by the machine, further cause the machine to perform the following operations: communicating telemetry signals with components of the motherboard using the connector; and The connector is used to communicate bus interface requests and responses with the host central processing unit CPU of the motherboard.

Images