Systems and methods for high speed and secure provisioning and management of bare metal resources

Abstract

Bare metal resources can be securely provisioned to computer systems using a switch that toggles connections between (1) a storage target such as a disk and a remote source of data and (2) the storage target and the bare metal resource. In this fashion, the toggled connections provide isolations of the bare metal resource that make it less susceptible to cyberattack. While the remote source-storage target connection is enabled, data such as an operating system and / or boot information for the bare metal resource can be transferred from the remote source to the storage target. While the bare metal resource-storage target connection is enabled, the remote source-storage target connection is disabled and the operating system and / or boot information can be transferred from the storage target to the bare metal resource. The bare metal resource can then boot based on this operating system and / or boot information.

Description

[0001] PCT Patent Application

[0002] Attorney Docket Number: 61732-249611

[0003] Systems and Methods for High Speed and Secure Provisioning and Management of Bare Metal Resources

[0004] Cross-Reference and Priority Claim to Related Patent Application:

[0005] This patent application claims priority to U.S. provisional patent application serial no. 63 / 688,822, filed August 29, 2024, and entitled “Systems and Methods for High-Speed and Secure Provisioning and Management of Bare Metal Resources”, the entire disclosure of which is incorporated herein by reference.

[0006] Introduction:

[0007] Disclosed herein are a variety of examples of systems and methods for provisioning and managing bare metal resources within a computing system, such as an IT system, in a secure and high speed manner.

[0008] There are technical challenges in the art with respect to how bare metal resources can be provisioned and managed within computing systems, particularly with respect to reducing the risk that the bare metal resources will be vulnerable to cyberattacks.

[0009] A bare metal resource is a physical computer which has physical electronic components and is capable of being physically connected / disconnected with other physical hardware. A physical server is an example of a bare metal resource. More generally, a motherboard with a processor (e.g., CPU or other chip), physical memory, and other electronic components is another example of a bare metal resource. By contrast, a virtual machine or a virtual container as such is not a bare metal resource, although it should be understood that virtual machines and virtual containers are software arrangements that will run on one or more underlying bare metal resources (where the one or more bare metal resources that host a virtual machine / container are abstracted away from a user). Thus, setting up or configuring a virtual machine or virtual container is fundamentally different than provisioning a bare metal resource because the primary point of using a virtual machine or virtual container is to avoid having to address the underlying bare metal resource configuration.

[0010] According to embodiments, a secure or safe bare metal provisioning device and method are provided that enable booting of one or more bare metal resources from a storage target such as a disk while removing potential attack surfaces. The number of attack surfaces in a computer system is the number of all possible points (or attack vectors) where an authorized user can access a system (or components of that system) for malicious activity such as unauthorized data extraction. The use of bare metal resources themselves may be ideal for multi -tenant environments; however many multi-tenant environments employ bare metal resources that have preloaded operating systems, which may expose the orchestration layer and control plane and thus may provide a vulnerable attack surface. According to embodiments, a provisioner enhances safety and security by removing an attack surface that may make the bare metal resource vulnerable to an attack when provisioning and / or using the bare metal resource.

[0011] According to embodiments, the provisioning device provides enhanced security when a bare metal resource is booting and / or operating by way of a storage target such as a disk. The bare metal provisioning device provides a connection to a management device that can be used for managing deployment of a bare metal resource. For example, an administrator interface or a controller of an IT system may remotely connect to the disk via the management device and may transfer data to the disk. The management device may make remote access to a disk possible through techniques such as, but not limited to, iSCSI, iSER, NVMEof, Excelero, NFS, NFS over RDMA, VAST, DDN and / or WEKA. In some embodiments, the operating system or boot information for the bare metal resource may be loaded onto the disk by way of the management device.

[0012] Boot information as used herein may include: (1) boot files (or portions thereof), (2) operating system files (or portions thereof), and / or (3) files or data (i) used in a boot process, (ii) used as the main hard drive, or (iii) used in a partition of a hard drive that is used for booting and / or operating a computer (including but not limited to kernels, initrds, rootfs, operating systems, initramfs, files that are on a boot disk, data blocks, copies of data blocks, data, root filesystems, or other information that can be used to boot the system, optionally with a BIOS or any device that powers on the system).

[0013] The management device may be configured to power on the bare metal resource (for example, in response to a command received from a controller or an administrator interface), and the bare metal resource can boot from the disk. The provisioning device can be triggered when the bare metal resource is turned on. In response to triggering, the provisioning device can operate to segment or isolate the bare metal resource and the disk from the management device (and in doing so, also segment or isolate the disk from a network, user, and / or controller). The provisioning device can further provide a connection that couples the disk to the bare metal resource in response to the bare metal resource being turned on. The disk may contain an operating system or other booting information for the bare metal resource that is accessible by the bare metal resource through the connection to the disk while the bare metal resource is turned on.

[0014] The management device may be coupled to the bare metal resource by way of the provisioning device. The management device may comprise additional management connections to the bare metal resource. The provisioning device is configured to disconnect the disk connection to the management device when the bare metal resource is on and connected to the disk. Accordingly the bare metal resource may securely boot an image while an attack surface that could potentially be presented by a running management device is removed due to the disconnection between the disk and the management device. A management device is circuitry such as a computer that has read and / or write access to a storage target such as a disk and may share this read / write access with other system components. As such, the management device can also be referred to as management circuitry. A management device may connect to a network and may be referred to as providing out of band management for the storage target. An example of a management device can be a baseboard management controller (BMC). A BMC can be configured to provide remote management, monitoring, and / or control of hardware on a bare metal resource. Moreover, the BMC can perform these operations even if the bare metal resource is otherwise not powered or non- responsive. Another example of a management device can be a remote access device (e.g., a Redfish device) for a storage target. Another example of a management device can be circuitry such as an administrator interface that can connect to a server using keyboard, video, and / or mouse. Yet another example of a management device can be a computer that manages a larger computer and / or manages settings on a bare metal computer including BIOS settings. Still another example of a management device can be a controller of an IT system, where the controller automatically manages the IT system (examples of such a controller are disclosed in published PCT patent application WO 2020 / 252088, the entire disclosure of which is incorporated herein by reference.

[0015] In use, a remote source of data such as a controller or an administrator interface may remotely log into or access the disk by way of a management device, and the management device may provide data to the disk (where such provided data may include an operating system or other booting information). This information may be provided to the management device by, for example: a controller that automatically manages an IT system (including the bare metal resource); connecting to a disk of a bare metal resource and subsequently powering on the bare metal resource that can boot from the disk (or storage). An administrator interface or controller may initiate turning on of the bare metal resource. Upon the bare metal resource turning on, the provisioning device automatically disables the connection between the management device and the disk and enables the connection between the disk and the bare metal resource, whereby the bare metal resource is then able to access the operating system or booting information by loading the operating system or booting information from the disk to the bare metal resource.

[0016] According to some embodiments, a safe provisioner is configured to provide OS images, boot data, boot disk image, credentials from a management device to bare metal resource. In addition, the safe provisioner includes security features that allow it to toggle its disk connection between the management device and bare metal resource or to otherwise turn on and off its connections between the management device and the bare metal resource. The safe provisioner may also provide or restore access to the disk for the management device where the bare metal resource is powered down.

[0017] A safe provisioner may, as an example, be used with or included with a BMC that is used to manage the bare metal resource. The BMC can serve as the management device in this example. BMC’s are management devices that operate out of band to remotely access a server though a network and manage some aspects of a bare metal resource for the server. For example, BMCs have been used to for remote viewing of screen, keyboard or mouse inputs to a remote server. BMCs have also been used to upload virtual CDROM images to a server and to turn on or off a connection to a hard drive and move a physical hard drive to a system. However, in conventional systems, a BMC’s usage is designed to be always on for management or remote administration. Accordingly, BMCs in conventional systems present an vulnerable attack surface. When BMCs have been conventionally used for multi-tenant environments they have typically been networked together, which unfortunately increases the attack surface(s).

[0018] A controller used to automatically manage an IT system (including the bare metal resource) may be used with or through the BMC to manage the deployment of the bare metal resource to the IT system. As noted above, an example of such a controller for use in automatically managing an IT system is described with reference to published PCT patent application WO 2020 / 252088 incorporated herein by reference. For example, WO 2020 / 252088 describes an IT system where a controller can use system rules, a system state, and templates to automatically manage the IT system’s physical infrastructure. This automated management can include adding resources to the IT system such as bare metal compute resources, bare metal storage resources, and bare metal networking resources while coupling these resources together to implement a desired system configuration. As explained in greater detail below, the safe provisioner described herein can be used in such an IT system to securely provision bare metal resources to the IT system as the IT system is being built out.

[0019] A provisioner may comprise a connection that provides a communication path between a storage target and a bare metal resource, a connection that provides a communication path between the storage target and a remote data source, and one or more switches that control which of these connections are enabled and disabled. These connections may include connectors with pins and / or receptacles that allow for the storage target and management device to be detachably connected with the provisioner. The provisioner may also include an input that triggers which the switch state the switch will enter. However, it should also be understood that some practitioners may choose to include additional components as part of the provisioner, and the provisioner may be incorporated into other system components as discussed herein.

[0020] The enablement of a connection between two components as described herein (which can also be referred to as “connecting”) refers to controlling the connection so that there is an operative communication path for data between the two components (e.g., for an enabled connection between the management device and the storage target, there would be an operative communication path where data can be written by the management device to the storage target via the enabled connection; similarly, for an enabled connection between the storage target and the bare metal resource, there would be an operative communication path where data can be read by the bare metal resource from the storage target). The enablement of a connection can be achieved by electronic switching that physically permits signal flow between the two components (e.g., as described below with reference to Figure 3 where transistors can be switched on) or by virtualized controls that use software techniques to permit traffic between the two components. Furthermore, it should be understood that an enabled connection encompasses a communication path where traffic flow between components in a desired direction is permitted while traffic might be blocked in the other direction (e.g., permitting a communication path to be operative that allows data to be written by a management device to a storage target while not permitting a communication path to be operative that would allow data to be read by the management device from the storage target).

[0021] The disablement of a connection between two components as described herein (which can also be referred to as “disconnecting”) refers to controlling the connection so that there is not an operative communication path for data between the two components. The disablement of a connection can be achieved by electronic switching that physically prevents signal flow between the two components (e.g., as described below with reference to Figure 3 where transistors can be switched off) or by virtualized controls that use software techniques to block traffic between the two components. As an example of virtualized control over enabling / disabling connections, network partitioning can be used to enable / disable connections. Furthermore, it should be understood that a disabled connection encompasses a communication path where traffic flow between components in a desired direction is blocked while traffic might be permitted in the other direction (e.g., rendering a communication path non-operative to prevent data being written by a management device to a storage target while permitting a communication path to be operative that would allow data to be read by the management device from the storage target).

[0022] The management device may be a BMC that can be automatically electronically disabled when a bare metal resource is provisioned, thereby removing all access for the bare metal resource to any control planes for the IT system (and vice versa). Bare metal resources may be managed by a controller, and the controller may be disconnected from the bare metal resources when the system is powered up or booted. The safe provisioner, upon power on of the bare metal resource, is configured to physically segment network connections including but not limited to out of band management connect! on(s), SAN connect! on(s), and / or parts of out of band management connection(s) as well as a network itself.

[0023] According to some embodiments, a safe provisioner tool can be used to remotely boot a server where the boot image is recognized by the server as a hard drive requiring no special BMC tooling.

[0024] A user or controller herein may also provision credentials to the disk using a provisioner 600.

[0025] A secure provisioner herein may allow one way data flow or may limit data flow to one direction. For example, the one way data flow can be a flow from a remote data source to the storage target via the connection controlled by the provisioner.

[0026] In some embodiments, user data may be stored in a cold storage and may be encrypted at rest. This data may be not currently in use by a user, but the data can remain encrypted in an IT system. At a later point a storage resource and / or a SAN (optionally coupled to compute and storage resources) may be provisioned (optionally using a safe provisioner and / or a controller). A main controller or a cloud provider may provision resources in an environment using an asymmetric credential; or users may provide encryption keys directly to a resource to decrypt the encrypted at rest data. The main controller may use this arrangement to deploy a plurality of cloud nodes or bare metal resources from encrypted long term storage for a user. In this fashion, data can be stored at rest in the IT system in an encrypted format, and this data can be decrypted by the bare metal resource when the storage target is isolated through action of the provisioner. Brief Description of the Drawings:

[0027] Figure 1 A shows an example of a provisioner in accordance with an example embodiment.

[0028] Figure IB shows an example of a bare metal resource configured with a provisioner in accordance with an example embodiment.

[0029] Figure 2 shows another example of a provisioner in accordance with an example embodiment.

[0030] Figure 3 shows an example architecture for toggling of connections between a disk and a bare metal source or a management device in accordance with an example embodiment.

[0031] Figure 4 shows another example system for secure provisioning of a bare metal resource.

[0032] Figure 5 shows an example process flow for securing provisioning a bare metal resource.

[0033] Figure 6 shows an example bare metal resource with an embedded provisioner.

[0034] Figure 7 shows an example of secure provisioning in the context of a bare metal resource pool.

[0035] Figure 8 shows an example of an IT system with different secure provisioners for different bare metal resources.

[0036] Figure 9 shows an example of secure provisioning for an IT system with a sub-controller.

[0037] Figure 10 shows an example process flow where a securely provisioned bare metal resource is coupled to a sub-controller.

[0038] Figure 11 shows an example IT system where a PCIe switch is positioned between a controller and a safe provisioner.

[0039] Figure 12A shows an example architecture in which the provisioner may be employed according to an embodiment.

[0040] Figure 12B shows another example architecture in which the provisioner may be employed according to an embodiment.

[0041] Detailed Description of Example Embodiments:

[0042] According to various examples disclosed herein, an IT system may be configured to improve security in bare metal cloud nodes or other bare metal resources by selectively controlling how the storage target used for configuring the bare metal resource can be accessed. FIG. 1A illustrates an example embodiment of a system that employs a safe provisioner 600. The provisioner 600 may comprise a management device connection 610 (for connecting the provisioner 600 to a management device 500) and a bare metal resource connection 620 (for connecting the provisioner 600 to a bare metal resource 300). The safe provisioner 600 in use can be further coupled to a storage target 400 such as a disk. During use, the safe provisioner 600 provides a connection between the storage target 400 and either the bare metal resource 300 (via connection 620) or the management device 500 (via connection 610) depending on a status of the bare metal resource 300. The provisioner 600 can be implemented as a provisioning circuit that manages connections to the storage target 400 so that access to the storage target is limited at times when the storage target is connected to the bare metal resource 300. Examples of such a provisioning circuit are discussed in greater detail below.

[0043] The storage target 400 can be configured to provide block-based storage and / or a file system storage. As an example, the storage target 400 can be a disk such as any disk that has electrical connections through which data can be communicated. For ease of reference, examples described herein will often refer to the storage target as a disk. However, it should be understood that the storage target for embodiments described herein can be non-disk storage. Examples of disk 400 can include but are not limited to an NVMe, SATA, SCSI, etc. Furthermore, a disk 400 may be a volatile or non-persistent disk that is erased when it loses power; or the disk 400 may a non-volatile or persistent disk that retains its data when it is powered down.

[0044] FIG. IB shows an example where the provisioner 600 is incorporated into a bare metal resource 300. For example, as shown by Figure IB, a motherboard 302 of the bare metal resource 300 can include a bus 208 such as a PCIe bus. The storage target 400 can connect to the bus 208 via connection 620. Also on the motherboard 302 and connected to the bus 208 are one or more processors 202, a memory 204, and other component(s) 206 (such as a network interface, etc.). Management device 500 can be deployed on the motherboard and connected to the storage target 400 via connection 610. Management device 500 can also include a connection 210 with one or more data sources such as a controller of the IT system that is provisioning the bare metal resource 300, an administer computer or interface for the IT system, etc. The connection 210 can include a network interface that interfaces the management device 500 to the data source(s) via a network. As discussed in greater detail below, the provisioner 600 can receive an input 212 that indicates whether the bare metal resource has been powered on, and the provisioner 600 can control which of the connections 610 and 620 is enabled based on the input 212. .

[0045] While Figure IB shows an example where the storage target 400, management device 500, and provisioner 600 are co-located on the bare metal resource 300, it should be understood that a practitioner may choose to distribute these components in a different fashion (e.g., see Figures 12A and 12B discussed below). For example, the management device 500 can be located off the motherboard 302 but still connected to the storage target 400 on the motherboard 302. As another example, both the management device 500 and the storage target 400 could be located off the motherboard 302 but still connected to the relevant components on the motherboard 302 for operations as described herein. As an example of such a configuration, the management device 500 and storage target 400 can be co-located on a different board or card and connected to the motherboard 302. Further still, in any of these configurations, the provisioner 600 could similarly be located on or off the motherboard 302, and optionally co-located with the storage target 400 and / or management device 500.

[0046] FIG 2 illustrates where the disk 400 has pins or other connectors 410 that may be used to connect the disk 400 to a safe provisioner 600 by way of a receptacle 615. The safe provisioner 600 comprises a circuit 650 that connects to the pins / connectors 410 of the disk 400. The circuit 650 is configured to toggle the pin receptacle connection between the management device connection 610 (when in use, the circuit 650 is coupled to the management device 500) and the bare metal connector 620 (when in use the circuit 650 is coupled to the bare metal device 300).

[0047] The circuit 650 toggles to form a connection between the management device connection 610 and the disk 400 (via pins / connectors 410) when the bare metal resource 300 is off (during which time the disk 400 may communicate with management device 500). For example a management device 500 may share the disk 400 or share access to the disk 400 so that the management device 500, an administrator who interfaces with the management device 500 or a controller that interfaces with the management device 500 (in instances where the controller is not serving as the management device itself) may put data into this disk 400. The data loaded onto the disk via the connection 610 may include but is not limited to a root filesystem, boot information (including kernels, a bootloader, BIOS information for use when the system powers up, etc.), or any other data. This communication between the management device 500 and the disk 400 may also be used so that settings for the system (where such settings may include configuration data, firmware updates, BIOS configurations, firmware configuration instructions, etc.) can be uploaded either by a controller or by any configuration tool that is sending data to the system. Upon boot, the bare metal resource 300 may process the data, use the disk 400 for booting, or use a partition of the disk 400 for booting.

[0048] When the bare metal resource 300 is turned or powered on, a power indicator (which may be a power signal from the bare metal resource 300 or another a power indicator that indicates power to the bare metal resource 300 is turned on) can be read by the provisioner 600, and the power indicator can cause the provisioner 600 to toggle the disk connection to enable the connection 620 between the disk 400 and the bare metal resource 300 while disabling the connection 610 between the disk 400 and the management device 500. When the connection 620 is enabled, for example, the bare metal resource 300 may access boot files or an operating system or other boot information from the disk 400 for deployment on the bare metal resource 300.

[0049] FIG 3 illustrates an example of a circuit 650 that may be used to toggle the connection of the disk 400 between the management device 500 and the bare metal resource 300. The circuit 650 comprises a toggle switch that toggles between (1) a connection between the disk 400 and the bare metal resource 300 (via 620) and (2) a connection between the disk 400 and the management device 500 (via 610) based on a status of the bare metal resource 300. In the example of Figure 3, the status of the bare metal resource 300 that is used to control the toggle switch is the power status of the bare metal resource 300. For example, the toggle switch can receive as an input a power indicator 305 that indicates whether the bare metal resource 300 is powered on. If the input from power indicator 305 indicates that the bare metal resource 300 is powered on, then the toggle switch is controlled to connect the disk 400 with the bare metal resource 300 (via 620) while the management device 500 is disconnected from the disk 400. If the input from power indicator 305 indicates that the bare metal resource 300 is not powered on, then the toggle switch is controlled to connect the disk 400 with the management device 500 (via 610) while the bare metal resource 300 is disconnected from the disk 400. The power indicator 305 can be a voltage from the bare metal resource’s motherboard that indicates the bare metal resource has been powered on (such as an advanced technology extended (ATX) connector). However, it should be understood that any signal that serves to indicate that the bare metal resource 300 is powered can be used as the power indicator 305. Moreover, it should be understood that any of a number of techniques can be used to power on the bare metal resource 300. For example, a “power on” command can be provided to the bare metal resource 300 by the management device 500 (e.g., a BMC) in response to a signal from a source such as a controller for an IT system, an administrator interface or administrator computer for the IT system, or other system component. Further still, the bare metal resource 300 could also be powered in response to a physical action by a person who flips a power switch, presses a power button, etc.

[0050] In the example of Figure 3, the toggle switch comprises a plurality of transistors 630a, 630b (e.g opposing pairs of transistors) connected to the pin / connector 410 of the disk 400 via a receptacle 615. The pin / connector 410 is connected to the transistors 630a, 630b in such a way that the current or signal can flow when the transistor is saturated. The (opposing) transistor inputs 631a and 63 lb are respectively connected to the bare metal resource power indicator 305. A logical “not” gate 617 is positioned between the bare metal resource power indicator 305 and the transistor input 63 lb. Accordingly, when the bare metal resource power indicator 305 indicates that the bare metal resource 300 is “on”, then transistor 630a switches on and enables the connection 620 between the bare metal resource 300 and the disk 400 while the not gate 617 operates to cause transistor 630b to switch off and disable the connection 610 between the management device 500 and the disk 400. When the bare metal resource power indicator 305 indicates that the bare metal resource 300 is “off’, then transistor 630b switches on and enables the connection 610 between the disk 400 and the management device 500 while transistor 630a switches off and disables the connection 620 between the bare metal resource 300 and the disk 400.

[0051] While Figure 3 shows an example where the toggle switch is formed from opposing transistor pairs 630a, 630b and a not gate 617, it should be understood that there are other examples of types of circuits that may be used for the toggle switch control. Examples include but are not limited to: connectors with opposing transistor polarities, circuits where the disk 400 is connected to a switched electrical connection, relays or electrical switches that selectively connect the disk 400 to the management device 500 or the bare metal resource, etc. Still further, the switch can be configured to disconnect the storage target from a remote data source by simply switching off power to the management device 500 when the bare metal resource 300 is powered on.

[0052] FIG 4 illustrates a variation of a securely managed bare metal resource 300 as part of an IT system 100 that securely manages the bare metal resource 300. For example, the bare metal resource 300 can be a bare metal compute resource that is to be provisioned by the IT system 100. In this example, the safe provisioner 600 as described with reference to Figures 2 and 3 further comprises the management device 500, where the management device 500 is configured to share a storage target that serves as disk 400 (for example a block or filesystem target) or data therefrom to an initiator 20. As an example, the initiator 20 can be a user interface that receives input from a user such as an administrator. As another example, the initiator 20 can be an automated management controller 200 such as described in the above-referenced and incorporated published PCT patent application WO 2020 / 252088. A disk in the form of a storage target 400 may be, for example, a shared block storage (such as iSCSI, iSER, NVMEof, fibre channel, or Excelero as a transport) or a filesystem storage. The storage target 400 may be exported such that the initiator 20 may connect to it and send data to the storage target 400. When the storage target 400 is connected or turned on, the safe provisioner 600 makes storage target 400 available to the initiator 20 using a network interface 280. This data sent to the storage target 400 may include but is not limited to a boot filesystem , BIOS settings or other boot information. The management device 500 may read what is on the storage target 400 and set the bios settings. The bare metal resource 300 may use data in the storage target 400 to set BIOS settings. When the bare metal resource 300 is powered on, the filesystem will be on the storage target 400 and the bare metal resource 300 can be booted from the storage target 400.

[0053] FIG 5 illustrates an example process flow using a safe provisioner 600 with respect to the example of Figure 4. At step 101, a controller (e.g., a controller as described in the abovereferenced and incorporated published PCT patent application WO 2020 / 252088) or a user (via a user interface) connects to a data storage target (e.g. network device storage target that serves as disk 400) directly, or, for example, through a management device such as a BMC. The controller or other user then copies or loads data into the data storage target (see 110). This data may include an operating system, boot files or other data required to boot a bare metal resource. Then the bare metal resource 300 is powered on, and the safe provisioner 600 toggles the connection of the storage target 400 to disable its connection with the management device 500 and enable its connection with the bare metal resource 300 (see 120). Optionally (if needed), the BIOS of the bare metal resource 300 loads settings from the data sent to the BMC 500 or storage target 400 (see 130). Finally, the bare metal resource 300 boots using the data on the storage target 400 (see 140).

[0054] FIG. 6 illustrates an example of a secure provisioner 600 that is incorporated into a bare metal resource 300. The management device 500 may also be incorporated into the bare metal resource 300 and may or may not be incorporated with the safe provisioner 600. The disk 400 and the pins / connectors 410 as described with reference to Figure 2 and 3 may be incorporated into the bare metal resource 300 and may or may not be incorporated with the safe provisioner 600. The combined device is provided with a network interface 280 such as a SAN, out of band network or other network connector through which the combined device may connect to a user interface, a controller, network, or other external devices. In this example embodiment, similar to other embodiments, the safe provisioner 600 prevents the disk 400 from being connected to both the management device 500 and the bare metal resource 300 at the same time.

[0055] FIG 7 illustrates an example embodiment where controller 200 connects to safe provisioner 600, and where the controller 200 then connects to a bare metal resource pool 700 using a software- defined networking switch 800. For example, according to some embodiments, the controller 200 may use the SDN switch 800 to move a new network instance, virtual network, or network partition (including but not limited to network instances, slices, VLANs, VXLANs, and / or infiniband partitions) from the resource pool network into the assigned virtual network, partition, etc. However, with other examples, the bare metal resource pool may have its own switch connecting the bare metal resources rather than the SDN switch 800.

[0056] In the example of Figure 7, the main controller 200 provisions hardware 300 in the bare metal resource pool 700 using the management device 500 and / or safe-provisioner 600. The bare metal resources 300 in the resource pool 700 are shared with users through assignment of those bare metal resources from the resource pool network to a new dedicated network. User can thus be given access to their own environments that are set up by the IT system. A sub-controller can provision the bare metal resource(s) for the environment(s) or the controller 200 may set up a mechanism for the resources to connect. An example can be making a jump host by provisioning a bare metal resource 300, and the jump host can be provisioned with the user’s asymmetric credential (e.g., an SSH key) so the user can remote in. The controller 200 (or subcontroller) can set up a VPN so the user can get in or can create a mechanism for the user to set up access to just that user’s environment. According to some embodiments, resource pools may be shared in a high performance computing (HPC) system or a cloud system (or other multitenant system), e.g. where resources are rented or allocated. However, the main controller 200 and / or management device 500 will immediately lose access to the bare metal resource(s) when one or more of the bare metal resources 700 are powered on due to the switching described herein for the secure provisioner 600. The controller 200 may instruct the SDN switch 800 to put bare metal resources on customer specific VLANs (or one or a plurality of network partitions) allowing the provisioning of hardware to a network partition in such a way that the controller 200 or management device 500 (which may otherwise present an attack surface to a provisioned bare metal resource 300) are isolated from the provisioned bare metal resource 300.

[0057] Network partitions as used herein may include but are not limited to VLANs, VXLANs, network instances, InfiniBand partitions, Ethernet virtual private networks (EVPNs), other methods of separating network access, and / or combinations thereof. Network partitions can be subpartitioned. Network partitions may use smart switches and may have the ability to enable and / or disable connections or access rules. Routers and switches may route network instances together, and this may be accomplished using SDN or other intelligent switching. For example, the SDN or other intelligent switching can decide the data path from a source to a destination across one or more networks and can take data from multiple sources and route that data to a single destination. As another example, in an L3 network, data is routed instead of using address resolution protocol (ARP) and sending packets to hardware addresses. A practitioner may find it desirable some traffic to flow between network partitions while other traffic would not flow between network partitions. Network partitions may have access control rules between other network partitions. Smart switches may act as routers and can couple network partitions or route network partitions to other network partitions or networks. When reference herein is made to a VLAN, VXLAN, InfiniBand partition, network instance, and / or network slice it is understood to mean a network partition. FIG 8 illustrates an example embodiment wherein controller 200 manages multiple bare metal resources by way of secure provisioners 600 including a first provisioner 600 which provisions bare metal resources from within a bare metal resource pool 700 (optionally using a software- defined networking (SDN) switch 800) and a second provisioner 600 which provisions a bare metal resource 701. The secure provisioners 600 may include both secure provisioner circuit 650 and a management device 500 (e.g., a smaller management device such as a BMC) as described herein with reference to Figures 1 A to 7. For example, from the resource pool 700, the provisioner 600 can provision bare metal networking resources and bare metal storage resources inside an environment. After a bare metal storage resource has been deployed, encrypted data from an encrypted data store can be decrypted into a SAN, and the bare metal resource(s) can boot from the SAN.

[0058] FIG 9 illustrates an example embodiment wherein a controller 200 connects to safe provisioner 600. The controller 200 may act as a management device or it may use a management device 500 which may be included with or coupled to a safe provisioner 600 as described herein. The safe provisioner 600 and controller 200 (or management device 500) further connect to a subcontroller 201 in a resource pool 700 using software-defined networking switch 800. While Figure 8 shows the SDN switch 800 as being part of the resource pool 700, it should be understood that in some examples it may be desirable for the sub-controller 201 to use its own SDN switch. As an example, the SDN switch 800 can be managed by the sub-controller 201, and the sub-controller 201 can be further managed by the controller 200 via provisioner 600.

[0059] In this example embodiment the controller 200 may deploy a sub-controller 201 using a safe provisioner 600, enabled with the management device 500 and the circuits and software disclosed herein. The sub-controller 201 image may be sent to the disk 400 from the controller 200 to its destination on the disk 400 in the resource pool 700 using the safe provisioner 600 and optionally, the management device 500„ where the disk 400 is coupled to the bare metal resource 300 inside the resource pool 700 if the bare metal resource 300 is powered on. The controller 200 may, using SDN switch 800, put the sub-controller 201 in a network partition, optionally using SDN switch 800 such as a smart switch or other network partitioning technology. The sub-controller 201 may be used by a user to automatically manage infrastructure inside a network partition, environment, or other partition of an IT system and / or cloud and / or multitenant system.

[0060] FIG 10 illustrates an example process flow for secure provisioning with respect to the system of Figure 9. At step 1000, a controller 200 connects to a storage target 400 that is connected to a secure provisioner 600. The controller 200 may connect to the storage target 400 through a management device 500 such as a BMC that is also coupled to or included with the safe provisioner 600. At 1100, the controller 200 then copies or loads data into the storage target 400. The bare metal resource 300 is then powered on, which causes the safe provisioner 600 to toggle the connection of the storage target 400 to disable the storage target’s connection with the management device 500 and enable the storage target’s connection with the bare metal resource (see 1200). In response to the secure provisioner 600 toggling off the connection to the management device 500 and / or controller 200, the flow of information between the controller 200 and the storage target 400 may stop either through electronics or other ways including but not limited to software (see 1300). For example, rather than using physical disconnections via the toggle switch as discussed above (e.g., opening or closing connections or providing / removing power to physical components), some practitioners might choose to employ software to provide a soft disconnect of the storage target 400 (e.g., by un-sharing the storage target 400 at a software level on a network, using software to effectively place the storage target behind a firewall, etc.). At 1410, the bare metal resource 300 is booted using the data in the storage target 400, and the controller 200 no longer has access to the bare metal resource 300 due to the toggled connection. The bare metal resource 300 may query or be coupled to the sub-controller 201 (see 1420). For example, when a bare metal resource 300 has been moved to an environment and then managed by sub-controller 201 (or some other new controller), the bare metal resource 300 may announce to the sub-controller 201 (or the other new controller) that it is available. It is worth noting that the sub-controller 201 may itself be a bare metal resource that has been similarly booted and set up and coupled to the bare metal resource 300. At 1430, the bare metal resource 300 may be coupled to other components of the IT system managed by the sub-controller 201. According to some embodiments, the sub-controller 201 may be a controller for a user or a controller for an isolated environment cloud or isolated plurality of environments. In this fashion, it should be understood that the secure provisioner can be capable of provisioning multiple bare metal resources in a manner that securely builds out isolated IT system environments.

[0061] Moreover, the main controller 200 may send boot information to the storage target 400 that is encrypted (or partially encrypted). The bare metal resource 300 may partially boot waiting for a user to enter a credential or key to decrypt the encrypted data. The bare metal resource 300 may, for example, be booted into an initramfs or initrd, or have a hanging init script that waits for a credential or key. The bare metal resource 300 may also mount storage and wait for a user to enter information needed to decrypt the encrypted data.

[0062] FIG 11 illustrates an example embodiment where the management connection between the provisioner 600 and the controller 200 (where controller 200 serves as the management device 500) includes a PCIe switch 285. The safe provisioner 600 and circuit 650 are coupled to the controller 200 through the PCIe switch 285. When the bare metal resource 300 is powered down, the PCIe switch 285 allows the controller 200 to directly access a storage target (which may be located on the provisioner 600 or may be external to the provisioner 600 and connected to the PCIe switch 285) to write boot information or other data.

[0063] FIG 12A illustrates an example embodiment of a safe provisioner 600 connected to a motherboard 302 of a bare metal resource. In this example, a disk 400 may be connected to a receptacle 410 or may be embedded. The disk’s connections (in Fig. 12A, connection 610 or connection 620) to the bare metal resource or the management device 500 can be controllably enabled / disabled, for example by using a circuit 650 as disclosed herein. In this embodiment the management device 500 may be selectively connected to the disk 400 through a connector such as PCI express, SATA 286 or 287, or other connector. The receptacle 410 may also be on the management device 500, and the receptacle 410 and / or circuit 650 may be on the same board as the management device 500 or in the motherboard of the bare metal resource 300. The disk’s connections may be enabled or disabled by the circuit 650 selectively connecting the disk 400 to management device 500 (e.g., a BMC). The disk’s connection 620 may also be coupled to the motherboard 302 through PCI express, SATA, or another connector 287 to the bare metal resource where the disk’s connection 620 may be selectively enabled or disabled.

[0064] In other example embodiments, the disk 400 can be connected directly to the network interface 280 instead of a management device 500, where the network interface 280 can be an external PCI Express interface or similar type of interconnect. The network interface 280 may be connected in this embodiment to a PCI Express switch.

[0065] FIG 12B illustrates an example embodiment of a card, board, or addon card which may be plugged into a slot on motherboard 302, such as a PCI express slot. It may also be coupled to the motherboard connected to a SATA connector or another type of connector 287. The disk 400 may act as a storage target through management device 500. The disk 400 or disk receptacle 410 may also be connected to an external PCI express interface where the network interface 280 is located as described in Figure 11. The external PCI express interface may be coupled with a PCI express switch. The connectors 287 may be PCI express pins and may be put into a PCI express slot. The card, board or addon card may be configured to slide into a SATA slot or another disk connection slot. The management device 500 may be included as part of the card, board, or addon card to share the storage target when the disk 400 is connected to the management device 500. The management device 500 may be subsequently be coupled to a network 280 to provide a management interface.

[0066] The configuration described in Figure 12B may be re-configured by a practitioner into a plurality of connection formats. Accordingly, the management device 500 may receive data from the storage target 400 and transmit it through connector 410 to the network interface 280. Further, the storage target 400 may transmit data to the motherboard 302 through the PCIe / SATA / other connector 285. The management device 500 may also be separate from the card on the motherboard itself or separate from the card, board or addon card.

[0067] As noted above, examples of computer system environments and controller environments in which the features described herein may be used are described in published PCT patent application WO 2020 / 252088, the entire disclosure of which is incorporated herein by reference.

[0068] While the invention has been described above in relation to its example embodiments, various modifications may be made thereto that still fall within the invention’s scope. Such modifications to the invention will be recognizable upon review of the teachings herein.

Claims

WHAT IS CLAIMED IS:

1. A computer hardware apparatus comprising: a first controllable connection between a storage target and a remote source of data; a second controllable connection between the storage target and a bare metal resource; and a switch, wherein the switch is configured to (1) receive a signal indicative of a power status for the bare metal resource and (2) control the first and second controllable connections based on the power status so that (i) if the power status indicates the bare metal resource is powered off, the switch enables the first controllable connection and disables the second controllable connection so that data from the remote source can be transferred from the remote source to the storage target and (ii) if the power status indicates the bare metal resource is powered on, the switch enables the second controllable connection and disables the first controllable connection so the bare metal resource can access the transferred data from the storage target.

2. The apparatus of claim 1 wherein the switch, if the power status indicates the bare metal resource is powered on, isolates the bare metal resource from the remote source by disabling the first controllable connection.

3. The apparatus of claim 2 wherein the switch, if the power status indicates the bare metal resource is powered on, isolates the bare metal resource and the storage target from the remote source by disabling the first controllable connection.

4. The apparatus of any of claims 1-3 wherein the first controllable connection connects the storage target to the remote source via a management device.

5. The apparatus of claim 4 wherein the management device comprises a baseboard management controller (BMC).

6. The apparatus of claim 4 wherein the management device comprises a controller for an information technology (IT) system.

7. The apparatus of claim 4 wherein the management device comprises a remote management tool.

8. The apparatus of any of claims 4-7 further comprising the management device.

9. The apparatus of claim 8 further comprising the bare metal resource, and wherein the switch, the management device, and the bare metal resource are co-located with each other.

10. The apparatus of any of claims 1-9 wherein the remote source comprises a controller for an information technology (IT) system.

11. The apparatus of any of claims 1-10 wherein the storage target comprises a disk.

12. The apparatus of claim 11 wherein the first controllable connection includes a connection between the switch and one or more connectors for the disk.

13. The apparatus of claim 12 wherein the one or more connectors for the disk comprises one or more disk pins.

14. The apparatus of any of claims 1-13 further comprising the bare metal resource.

15. The apparatus of claim 14 wherein the bare metal resource includes a motherboard, and wherein the second controllable connection includes a connection between the switch and the motherboard.

16. The apparatus of any of claims 14-15 wherein the bare metal resource comprises a pool of bare metal resources.

17. The apparatus of claims 1-16 wherein the accessed and transferred data comprises an operating system for the bare metal resource.

18. The apparatus of claims 1-17 wherein the accessed and transferred data comprises boot information for the bare metal resource.

19. The apparatus of claim 18 further comprising the bare metal resource, and wherein the bare metal resource is configured to boot based on the boot information that is accessed from the storage target.

20. The apparatus of any of claims 1-19 wherein the accessed and transferred data comprises encrypted data, and wherein the bare metal resource is configured to decrypt encrypted data that it accessed from the storage target based on a credential or a key.

21. The apparatus of claim 20 wherein the bare metal resource is configured to receive the credential or a key from a user interface or another computer.

22. The apparatus of any of claims 1-21 wherein the first controllable connection provides the remote source with read or write access to the storage target if the first controllable connection is enabled.

23. The apparatus of any of claims 1-22 wherein the second controllable connection provides the bare metal resource with read or write access to the storage target if the second controllable connection is enabled.

24. The apparatus of any of claims 1-23 wherein the switch comprises a toggle switch.

25. The apparatus of claim 24 wherein the toggle switch comprises a first transistor and a second transistor, wherein the first transistor is connected to a first input and controls the first controllable connection based on the first input, and wherein the second transistor is connected to a second input and controls the second controllable connection based on the second input.

26. The apparatus of claim 25 wherein the toggle switch further comprises a not gate between the first and second inputs, wherein the second input comprises the power status signal, and wherein the first input comprises an inversion of the power status signal based on the not gate.

27. The apparatus of any of claims 1-23 wherein the switch comprises a relay switch.

28. The apparatus of any of claims 1-27 wherein the first controllable connection comprises an out of band management connection.

29. The apparatus of any of claims 1-28 wherein the first controllable connection includes a connection to the remote source via a network interface.

30. The apparatus of any of claims 1-29 further comprising a controller that initiates a provisioning of the bare metal resource via the remote source, the storage target, and the switch.

31. The apparatus of any of claims 1-30 wherein the switch is configured to enable and disable the first and second controllable connections unidirectionally.

32. The apparatus of any of claims 1-30 wherein the switch is configured to enable and disable the first and second controllable connections bidirectionally.

33. A computer system comprising: a storage target; a bare metal resource; and a provisioning circuit that provisions the bare metal resource to the computer system by: if the bare metal resource is powered down, enabling a first connection between the storage target and a remote source of data so that data can be transferred from the remote source to the storage target via the first connection, wherein the transferred data comprises an operating system and / or boot information for the bare metal resource; and if the bare metal resource is powered on, (1) disabling the first connection and (2) enabling a second connection between the storage target and the bare metal resource so that the bare metal resource can access the operating system and / or boot information from the storage target via the second connection.

34. The system of claim 33 wherein the provisioning circuit comprises a switch configured to toggle between the first connection and the second connection based on a power status for the bare metal resource.

35. The system of any of claims 33-34 further comprising a management device, wherein the first connection connects the storage target to the remote source via the management device.

36. The system of any of claims 33-34 further comprising a controller, wherein the controller is configured to initiate the bare metal resource being provisioned to the computer system via the provisioning circuit.

37. The system of any of claims 33-36 wherein the provisioning circuit is configured to enable and disable the first and second connections unidirectionally.

38. The system of any of claims 33-36 wherein the provisioning circuit is configured to enable and disable the first and second connections bidirectionally.

39. The system of any of claims 33-38 further comprising any feature or combination of features recited by any of claims 1-30.

40. A method for provisioning a bare metal resource to a computer system with enhanced security, the method comprising: receiving a signal indicative of whether the bare metal resource is powered on; in response to the received signal indicating that the bare metal resource is powered off, (1) connecting a storage target to a remote source of data, wherein the bare metal resource is not connected to the remote source if the storage target is connected to the remote source, and (2) transferring an operating system and / or boot information for the bare metal resource from the remote source to the connected storage target; and in response to the received signal indicating that the bare metal resource is powered on, (1) disconnecting the storage target from the remote source, (2) connecting the storage target to the bare metal resource, and (3) transferring the operating system and / or boot information from the connected storage target to the bare metal resource.

41. The method of claim 40 further comprising booting the bare metal resource based on the operating system and / or boot information.

42. The method of any of claims 40-41 further comprising initiating the method based on a command from a controller for the computer system.

43. The method of claim 42 further comprising isolating the bare metal resource from the controller after the booting.

44. The method of any of claims 40-43 wherein the connecting and disconnecting steps comprise performing at least one of the connecting and disconnecting steps unidirectionally.

45. The method of any of claims 40-43 wherein the connecting and disconnecting steps comprise performing at least one of the connecting and disconnecting steps bidirectionally46. The method of any of claims 40-45 further comprising any feature or combination of features recited by any of claims 1-39.

Images