Static network fabric in prefabricated factories

Abstract

A technology for a networking fabric in a data center of a prefabricated factory is disclosed. The networking fabric may include multiple networking cables routed through the data center and characterized by a static network fabric topology, one set of which is configured to terminate at a location within the data center. Multiple computing devices are located at this location and can be configured to form a regional network when connected to the set of networking cables according to a connectivity plan. The connectivity plan can be generated by a network service using a physical construction request. The network service can determine the configuration of the multiple computing devices and the static network fabric topology. The network service can generate a connectivity plan using the above configuration and static network fabric topology.

Description

Technical Field

[0001] Cross - reference to Related Applications This application is an international application of U.S. Non - Provisional Patent Application No. 18 / 122,676, entitled "STATIC NETWORK FABRIC AT A PREFAB FACTORY", filed on March 16, 2023 (Attorney Docket No. 088325 - 1328941(344010US)), and claims the priority and benefits thereof. This application also claims the priority and benefits of the following related applications.

[0002] (1) U.S. Non - Provisional Patent Application No. 18 / 122,674, entitled "TECHNIQUES FOR BUILDING CLOUD REGIONS AT A PREFAB FACTORY", filed on March 16, 2023 (Attorney Docket No. 088325 - 1307191(344000US)) (2) U.S. Non - Provisional Patent Application No. 18 / 122,677, entitled "MOBILE PREFAB FACTORY FOR BUILDING CLOUD REGIONS", filed on March 16, 2023 (Attorney Docket No. 088325 - 1328942(344020US)) (3) U.S. Non - Provisional Patent Application No. 18 / 122,678, entitled "TECHNIQUES FOR A CABLE TERMINATION PROTECTION APPARATUS IN A PREFAB FACTORY", filed on March 16, 2023 (Attorney Docket No. 088325 - 1328943(344030US)) (4) U.S. Non - Provisional Patent Application No. 18 / 122,675, entitled "TECHNIQUES FOR VALIDATING CLOUD REGIONS BUILT AT A PREFAB FACTORY", filed on March 16, 2023 (Attorney Docket No. 088325 - 1373430(344040US)) All of the contents of the above - mentioned applications are incorporated herein by reference for all purposes.

Background Art

[0003] field This disclosure relates to cloud computing data centers. More specifically, this disclosure describes a technology for implementing a static network fabric used in the construction of a regional data center within a prefabricated factory.

[0004] background Cloud infrastructure providers may operate one or more data centers in geographical areas around the world. A "region" is a logical abstraction of a collection of computing, storage, and networking resources in a given geographical area of ​​data centers used to deliver cloud computing infrastructure. Building a new region may involve provisioning computing resources, configuring infrastructure, and deploying code to these resources, but is typically done via network connectivity to data centers. However, building a region with physical resources located at the final target data center site requires extensive preparatory work at the data center, and scheduling logistics and completing the region's construction can be complex. [Overview of the Initiative]

[0005] Brief Overview Embodiments of this disclosure relate to the automated construction of a region using a prefabricated factory. The prefabricated factory may be a facility dedicated to configuring computing devices, networking devices, and other physical resources to be delivered to a target site (e.g., a target region (one or more data centers, customer facilities, etc. in a geographic area)). The work to construct the region may include bootstrapping (e.g., provisioning and / or deployment) resources (e.g., infrastructure components, artifacts, etc.) for any number of services that may be available in the region when delivered to the destination. Once configured in the prefabricated factory, the physical resources may be delivered to the target site, deployed in the target data center, and have their final configuration and other software resources deployed. Resources used for bootstrapping (e.g., software artifacts, software images, etc.) may be provided in an existing region bootstrap environment (e.g., one or more data centers in a host region). The construction of a region may be comprised of one or more cloud-based services that manage the inventory of physical computing devices used to construct the region in a prefabricated factory, generate and specify the configuration of the region to be constructed in the prefabricated factory, manage the bootstrapping of the region, configure the region for delivery to the target site, and test and verify the physical resources after they have been deployed at the target site. The prefabricated region may be constructed to meet the configuration preferences of a particular customer (made-to-order) or to meet common specifications that can be further customized at the time of deployment at a particular customer's site (stock built).

[0006] One embodiment relates to a system including a data center used as a prefabricated factory having a networking fabric that provides network connectivity between internal devices. The networking fabric may include networking cables and other physical components used to establish network connectivity between computing devices. For example, in a data center in a prefabricated factory, multiple networking cables can be introduced and configured to terminate at locations within the data center where devices (e.g., server racks having networking devices, server devices, etc.) can be located. In the data center, multiple networking cables can be connected to one or more networking devices (e.g., switches, routers, etc.) to form a static network fabric topology between networked devices. Computing devices can form a regional network when they are connected to the networking cables according to a connectivity plan. The connectivity plan may be generated based on the configuration of the computing devices (e.g., the configuration of the computing devices and their connections to networking devices on each server rack) and the static network fabric topology (e.g., the configuration of the networking infrastructure within the data center).

[0007] Another embodiment relates to a method for generating a connectivity plan that can be used to connect a set of networking cables in a data center's static network fabric to networking devices of multiple computing devices. The multiple computing devices may include computing devices in a region being built in a data center (e.g., a prefabricated factory). For example, during region construction, multiple server racks may be located in the prefabricated factory and connected to the factory's static network fabric. The selection and placement of the server racks can be done in response to a physical build request specifying the computing devices to be included in the region build operation. A networking service for a prefabricated service running in a cloud computing environment can use the physical build request to determine the configuration of the computing devices and the data center's static network fabric topology, and then generate a connectivity plan. The connectivity plan may include instructions that can be used (e.g., by a data center employee) to connect networking cables (e.g., each networking cable in a set of networking cables terminating at the server rack location) to the corresponding networking ports on networking devices (e.g., connecting the networking cables in the prefabricated factory to a top-of-rack switch on the server rack).

[0008] Another embodiment relates to a non-temporary computer-readable medium that stores computer-executable instructions that, when executed by one or more processors of a computing system, cause the computing system to perform the methods described above. Alternatively, the embodiment may be realized by using a computer program product that, when executed by a processor, contains a computer program / instruction that causes the processor to perform any of the methods of the Disclosure.

[0009] To facilitate the identification of discussions of any particular element or behavior, the leading one or more digits of a reference number represent the figure number in which that element was first introduced. [Brief explanation of the drawing]

[0010] [Figure 1] A block diagram showing a prefabricated factory for constructing a region and preparing region computing devices to be transported to a target data center, according to at least one embodiment. [Figure 2] A block diagram showing a prefabricated factory connected to a service provided by a CSP for building a region, according to at least one embodiment. [Figure 3] Block diagram showing a CSP system including multiple host regions that can support ViBE for deploying software resources to a prefabricated region constructed in a prefabricated factory, according to at least one embodiment. [Figure 4] A block diagram showing the arrangement of physical computing resources in a prefabricated factory managed by management services and inventory services, according to at least one embodiment. [Figure 5] This figure shows the management of network configurations for computing resources in a region built in a prefabricated factory by using management services and network services, according to at least one embodiment. [Figure 6] This figure shows the testing and evaluation of a region after delivery to the target site using management services and test services, according to at least one embodiment. [Figure 7] This figure shows an exemplary method for deploying software resources to physical resources in a region constructed in a prefabricated factory, according to at least one embodiment, in order to prepare the physical resources for transport to a target data center. [Figure 8]This figure shows an exemplary method for booting up physical resources constructed in a prefabricated factory after delivery to a target data center, according to at least one embodiment, and verifying the network configuration of the physical resources. [Figure 9] A block diagram showing an exemplary static network fabric in a prefabricated factory according to at least one embodiment. [Figure 10A] This figure shows an exemplary arrangement of physical computing resources connected to a static network fabric in a prefabricated factory, according to several embodiments. [Figure 10B] This figure shows an exemplary arrangement of physical computing resources connected to a static network fabric in a prefabricated factory, according to several embodiments. [Figure 11] This figure shows an exemplary method for generating a connectivity plan for connecting multiple computing devices to a set of networking cables in a static network fabric in a prefabricated factory, according to at least one embodiment. [Figure 12] This is a block diagram showing a pattern for implementing a service-type cloud infrastructure system according to at least one embodiment. [Figure 13] This block diagram shows another pattern for implementing a service-type cloud infrastructure system according to at least one embodiment. [Figure 14] This block diagram shows another pattern for implementing a service-type cloud infrastructure system according to at least one embodiment. [Figure 15] This block diagram shows another pattern for implementing a service-type cloud infrastructure system according to at least one embodiment. [Figure 16] A block diagram showing an exemplary computer system according to at least one embodiment. [Modes for carrying out the invention]

[0011] Detailed explanation Example of automated data center construction (region construction) infrastructure In recent years, the adoption of cloud services has been rapidly increasing. Currently, various types of cloud services are offered by different cloud service providers (CSPs). The term "cloud service" is generally used to describe services or functions that are made available on demand (for example, through a subscription model) by a CSP through the use of the CSP's systems and infrastructure (cloud infrastructure). Typically, the servers and systems that make up the CSP's infrastructure and are used to deliver cloud services to customers are separate from the customer's own on-premises servers and systems. Therefore, customers can use cloud services provided by a CSP without having to purchase separate hardware and software resources for the service. Cloud services are designed to provide registered customers with easy, scalable, and on-demand access to applications and computing resources without requiring the customer to invest in procuring the infrastructure used to deliver the service or function. Cloud services can be offered in various different types or models, such as Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). Customers can subscribe to one or more cloud services offered by a CSP. Customers can be any entity, including individuals, organizations, corporations, and government agencies.

[0012] As described above, the CSP is responsible for providing the infrastructure and resources used to provide cloud services to registered customers. The resources provided by the CSP may include both hardware and software resources. These resources may include, for example, compute resources (such as virtual machines, containers, applications, processors, bare-metal computers), memory resources (such as databases, data stores), networking resources (such as routers, host machines, load balancers), IDs, and other resources. In one embodiment, the resources provided by the CSP to provide a set of cloud services are organized as a data center. The data center may be configured to provide a specific set of cloud services. The CSP is responsible for installing the infrastructure and resources used to provide the specific set of cloud services into the data center. The CSP may build one or more data centers.

[0013] The data centers provided by the CSP may be hosted in different regions. A region is a local geographical area and can be identified by a region name. Regions are generally independent of each other and can be separated by long distances, for example, across countries or continents. Regions are grouped as realms. Examples of CSP regions include US West, US East, Eastern Australia, South-Eastern Australia, etc.

[0014] A region may include one or more data centers, and these data centers are located within a certain geographical area corresponding to the region. As an example, the data centers in a region may be located in cities within the region. For instance, in the case of a specific CSP, the data centers in the western region of the US may be located in San Jose, California, the data centers in the eastern region of the US may be located in Ashburn, Virginia, the data centers in the eastern region of Australia may be located in Sydney, Australia, and the data centers in the south - eastern region of Australia may be located in Melbourne, Australia.

[0015] As described above, a CSP provides cloud services to its customers by constructing or deploying data centers. As the customer base of the CSP expands, the CSP typically constructs new data centers in new regions or increases the capacity of existing data centers to better serve the increasing needs of customers. It is preferable that a data center be constructed in geographical proximity to the location of the customers it serves. The geographical proximity between a data center and the customers it serves helps in more efficient use of resources and provides faster and more reliable services to customers by reducing latency. Therefore, a CSP typically constructs new data centers in new regions in a geographical area that is geographically close to the customers the data centers serve. For example, if the customer base is expanding in Germany, the CSP may construct one or more data centers in a new region in Germany.

[0016] The process of building (one or more) data centers in a region and configuring them to provide cloud services is sometimes referred to as region build. The term "region build" is used to describe the construction of one or more data centers in a region. Region build involves provisioning or creating a set of new resources required or used to provide a set of services that the data centers are configured to offer. The final result of the region build process is the creation of a region that, along with the hardware and software resources it contains, is capable of providing the set of services that the region is intended for, and contains a set of resources used to provide that set of services.

[0017] Building a new region is an extremely complex operation requiring extensive coordination between various bootstrap operations. At a high level, it involves performing and coordinating various tasks such as identifying a set of services to be provided by the data center, identifying the various resources required to provide that set of services, generating, provisioning, and deploying the identified resources, and properly cabling the underlying hardware to enable the intended use. Each of these tasks has further subtasks that require coordination, further increasing the complexity. Due to this complexity, building a region currently involves multiple manually initiated or manually controlled tasks that require careful manual coordination. As a result, the task of building a new region (i.e., building one or more data centers in the region and configuring the hardware and software of each data center to provide the required cloud services) is very time-consuming. Building a region can take months, for example. Furthermore, the process is highly error-prone and may require multiple iterations before the desired configuration of the region is achieved, further increasing the time required for building a region (e.g., deploying hardware and software resources). These constraints and problems severely limit the CSP's ability to scale computing resources in a timely manner in response to growing customer needs.

[0018] Recent innovations have enabled CSPs to reduce build time, minimize wasted computing resources, and mitigate the risks associated with building regions. CSPs may also bootstrap services to new regions by adopting organization services. Organization services may be cloud-based services hosted in a separate region from the target region (for example, the organization region). To bootstrap services to the target region, an organization service can generate a bootstrap environment hosting instances of one or more cloud services. The organization service can then support the deployment of services to the target region by using the services within the bootstrap environment.

[0019] More recent innovations allow CSPs to concentrate region construction work in one or more facilities that can function as "factories" that generate partially or fully configured physical infrastructure and then deliver it to the target site. Instead of waiting for the target region's data center configuration and the deployment of physical components (e.g., servers, network switches, power supplies, etc.) within the data center before bootstrapping services to the target region, the CSP can construct the region in a prefabricated factory, send the configured physical components (e.g., racks) to the target data center, and then complete and verify the region's components after the racks arrive at the target site. Multiple regions can be constructed simultaneously in a prefabricated factory. Each region constructed in a prefabricated factory may have a distinct configuration, network topology, and services. By constructing regions in a prefabricated factory, the complexity of scheduling and logistics associated with preparing the target facility, delivering physical components to the target facility, and managing bootstrap resources within the cloud service can be significantly reduced because regions can be prefabricated and held until the target site is ready.

[0020] Prefabricated factories can also be used, for example, to build computing components that are integrated into customers' on-premises solutions when customers control and manage their own data center environments.

[0021] Centralized prefabricated factories support additional innovations for the efficient construction of regions. A prefabricated factory may include a static network fabric consisting of networking infrastructure (e.g., network switches, routers, cabling, etc.) designed to support any potential configuration of region components built within the factory. Therefore, the static network fabric enables the rapid placement of region physical resources within the factory and their connection to the existing network fabric. Even regions with different network topologies can be quickly connected to the same network fabric according to a connectivity plan that fits the static network fabric containing the region's physical components. The static network fabric reduces the complexity of network connectivity within regions of the factory, increasing the speed of deployment of region components within the factory and the removal of region components from the factory for delivery preparation. Complementarily, since the static network fabric provides a set of dedicated network connections to devices in various locations within the prefabricated factory, these connections can be protected by cable terminal protection devices (CTPAs) designed to accommodate each conceivable network connection (e.g., Ethernet, fiber optic, etc.) available for connecting to the region's factory network.

[0022] This disclosure pertains to a prefab factory where automated region construction is performed using one or more prefab services. A prefab management service can organize the entire region construction in the prefab factory. The management service, in conjunction with one or more additional prefab services, may function to manage an inventory of physical components used to configure the region in the prefab factory, configure the network (e.g., endpoints, network topology, addresses, and / or other identifiers for components within the region), bootstrap services onto the region infrastructure, prepare components for delivery of the region (including encryption of data volumes for security in transit), verify the region after delivery to the destination site and deployment at the destination site, and complete the region configuration (including performing any remaining bootstrap or update operations for services previously deployed to the region infrastructure in the prefab factory). This disclosure also describes features of the prefab factory itself that improve the automated region construction work, including a static network fabric of the prefab factory configured to support any potential region network topology without requiring special modifications, as well as a dedicated CTPA that improves the performance of the static network fabric. Finally, this disclosure also describes a mobile prefabricated factory that can perform some, some, or all of the automated region construction and related operations in the prefabricated factory during the transport of the region components to the target site.

[0023] A certain definition A "region" is a logical abstraction corresponding to a set of computing, storage, and networking resources associated with a geographical location. A region may contain any number of one or more execution targets. A region may be associated with one or more data centers. A "prefab region" represents a region built in a prefabricated factory environment prior to delivery to the corresponding geographical location. In some embodiments, the execution target may also correspond to a target data center rather than a prefabricated factory data center.

[0024] An "execution target" represents the smallest unit of change required to execute a release. A "release" represents a declaration of intent to organize specific changes to a service (e.g., a version 8 deployment, "adding an internal DNS record," etc.). For most services, an execution target represents an "instance" of the service or an instance of changes applied to the service. A single service may be bootstrapped for each of one or more execution targets. An execution target may be associated with a set of devices (e.g., a data center).

[0025] A "bootstrap" for a single service is intended to represent the collective task associated with the provisioning and deployment of any suitable number of resources (e.g., infrastructure components, artifacts, etc.) corresponding to that single service. A regional bootstrap is intended to represent the collective task associated with each individual bootstrap for each service intended within that region.

[0026] A "service" typically refers to a set of resources that provide functionality, usually in the form of an API, which a customer can invoke to achieve some useful result. This set of resources includes any suitable combination of infrastructure, platforms, or software (e.g., applications) hosted by a cloud provider that can be configured to provide the functionality of the service. The service may be available to users over the internet.

[0027] An "artifact" refers to code deployed to an infrastructure component or Kubernetes engine cluster, which can include software (e.g., applications), configuration information (e.g., configuration files), credentials, etc.

[0028] IaaS provisioning (or "provisioning") refers to acquiring computers or virtual hosts for use, and further, installing the necessary libraries or services for them. The expression "provisioning a device" refers to deploying a device to a state where it can be used by an end user for a specific purpose. A device that has gone through the provisioning process is sometimes called a "provisioned device." Creating a provisioned device (installing libraries and daemons) is considered part of provisioning, but this is different from deploying a new application or a new version of an application to the created device. In most cases, deployment does not include provisioning, and provisioning may need to be performed first. The created device is sometimes called an "infrastructure component."

[0029] An IaaS deployment (or "deployment") refers to the process of providing and / or installing a new application or a new version of an application to provisioned infrastructure components. Once infrastructure components are provisioned (e.g., acquired, allocated, created, etc.), additional software deployment (e.g., provided to and installed on the infrastructure components) may occur. After provisioning and deployment are complete, infrastructure components may be referred to as "resources" or "software resources." Examples of resources include, but are not limited to, virtual machines, databases, object storage, block storage, and load balancers.

[0030] A "virtual bootstrap environment" (ViBE) represents a virtual cloud network provisioned over an existing region (e.g., a "host region"). The provisioned ViBE connects to the new region using a communication channel (e.g., IPSec Tunnel VPN). Within the ViBE, essential core services (or "seed" services) such as a deployment orchestrator, public key infrastructure (PKI) services, dynamic host configuration protocol services (DHCP), and domain name services (DNS) can be provisioned. These services can provide the functionality necessary for bringing hardware online, establishing a trust chain to the new region, and deploying other services in the new region. By utilizing a virtual bootstrap environment, circular dependencies between bootstrap resources can be prevented by leveraging the resources of the host region. These services can be staged and tested within the ViBE before the prefabricated region (e.g., the target region) becomes available.

[0031] A "Manager Service" may represent a service configured to manage the provisioning and deployment of any number of services as part of the construction of a prefabricated region. The Manager Service, in conjunction with one or more additional prefabricated services, can manage the organization of region construction at the prefabricated factory, as well as the deployment and configuration of the prefabricated region at the target data center after construction and transport. The Manager Service and other prefabricated services may be hosted in an existing region of the CSP.

[0032] The "host region" refers to the region hosting the virtual bootstrap environment (ViBE). The host region can be used to bootstrap the ViBE.

[0033] The "target region" represents the region being constructed in the prefabricated factory. During the construction of the prefabricated region, the target region is associated with the physical space, power, and cooling provided by the prefabricated factory. After bootstrapping, the prefabricated region is sent to the target data center where it will be deployed and is then associated with that target data center.

[0034] Prefabricated Region Construction In some examples, this specification describes techniques for building regions in a prefabricated factory. Such techniques may include one or more prefabricated services (e.g., management services, network services, inventory services, test services, deployment organization systems) hosted by a CSP capable of managing the bootstrapping (e.g., provisioning and software deployment) of infrastructure components for one or more regions within the prefabricated factory, as outlined above. The prefabricated factory may be configured to support the simultaneous construction of multiple regions. For example, the physical resources for a first prefabricated region (e.g., server racks, network switches, etc.) may be deployed at one location in the prefabricated factory, while the physical resources for a second prefabricated region may be deployed at a second location in the prefabricated factory. Each prefabricated region may be connected to a dedicated network fabric of the prefabricated factory that independently provides networking connectivity to each prefabricated region, and thus be able to communicate with prefabricated services and / or other cloud services that support the region construction. Based on the build request (region specifications (e.g., the number of server racks in the region, the number of computing devices, the number and types of services hosted by the region, the network topology of the region, etc.)), the prefab service can generate instructions for the corresponding physical infrastructure to be deployed in the prefab factory (e.g., by factory personnel), which may include the integrated networking of physical devices on racks, the placement of racks at the location of the prefab factory, and the connection of devices to the static network fabric of the prefab factory. The management service can then organize the provisioning of the regional infrastructure and the deployment of software resources to the prefab regional infrastructure, configure the prefab region for delivery, manage the delivery of the prefab region (e.g., scheduling and monitoring), and perform testing and verification of the prefab region upon arrival at the target site.

[0035] Prefabricated factories can centralize the region construction process and streamline the use of computing and networking resources that support region construction. For example, a prefabricated factory may be located "near" the host region containing prefabricated services and / or ViBE (e.g., with low-latency and high-data-rate networking connectivity). Furthermore, building multiple regions with improved network connectivity to the host region can avoid the potential performance degradation that occurs when performing region construction on newly configured data center sites in a normal region construction. Prefabricated factories also improve the physical computing security for devices during region construction, because the CSP can control the prefabricated factory and its network connectivity.

[0036] Furthermore, prefabricated factories improve the management of physical component inventory. Management services can determine the computing devices required for a specific region's construction, which can be stored in or near the prefabricated factory. Along with the construction and transportation of the region, the infrastructure for the new region can be quickly brought into and deployed at the prefabricated factory, improving efficiency.

[0037] Referring here to the drawings, Figure 1 is a block diagram of a prefabricated system 100, which includes a prefabricated factory 102 for constructing regions (e.g., prefabricated region 106A, prefabricated region 106B, prefabricated region 106C) and preparing regional computing devices to be delivered to target data centers (e.g., data center 108, data center 110), according to at least one embodiment. Each region constructed in the prefabricated factory 102 may include one or more devices that constitute the computing environment of the data center. The use of the prefabricated factory 102 allows for the construction of multiple regions simultaneously. For example, the prefabricated factory 102 can simultaneously construct all of prefabricated regions 106A, 106B, and 106C. In some examples, the devices of the regions may be deployed and staged in the prefabricated factory 102 prior to the commencement of infrastructure provisioning and software deployment operations.

[0038] The prefabricated factory 102 can be a data center-like facility that includes sufficient power, cooling, and networking infrastructure to support the construction of one or more regions. The prefabricated factory 102 may be located in close proximity to the existing computing infrastructure of the CSP (e.g., CSP 104). For example, CSP 104 can operate its existing data center for one or more regions. By being located near or next to the existing data center of the host region, the prefabricated factory 102 can provide high data-rate network connectivity between the CSP's cloud services and the computing devices of the region being built in the prefabricated factory 102. As an addition or alternative, the prefabricated factory 102 may be positioned to improve logistical operations, including transportation to the target data center of the region.

[0039] A prefabricated region constructed in prefabricated factory 102 may include any suitable number of physical resources, including computing devices (e.g., servers, racks of multiple servers, etc.), storage (e.g., block storage devices, object storage devices, etc.), and networking devices (e.g., switches, routers, gateways, etc.). Each region may have different physical resources depending on the specific requirements of the target region and data center. For example, prefabricated region 106A may comprise 100 racks, each having 40 computing devices. On the other hand, prefabricated region 106B may comprise 20 racks, each having 30 computing devices. Each rack of computing devices may include one or more networking devices configured to communicate with the server devices on that rack and connect to the networking infrastructure of prefabricated factory 102 to form a network with other computing devices in the prefabricated region. Each rack may also include power supplies and cooling systems to support the operation of the computing devices on the rack.

[0040] The prefabricated factory 102 may comprise any suitable number of networking devices to support the deployment and connection of one or more computing devices in the prefabricated region to be constructed. For example, the prefabricated factory 102 may comprise any suitable number of leaf and spine switches to support the connection of computing devices on multiple racks and constitute the network of the prefabricated region. Similarly, the prefabricated factory 102 may comprise network cabling that is deployed in the facility and can provide network connectivity to the networking infrastructure of the prefabricated factory 102. The network cabling may be arranged to terminate at locations within the prefabricated factory 102 where racks of computing devices for the prefabricated region can be deployed during the region construction work. Further details regarding the configuration of the networking infrastructure and the prefabricated factory are given below with respect to Figures 9 to 11.

[0041] The prefabricated factory 102 may be connected to services provided by CSP104 via one or more networks. During region construction, CSP104 can provision infrastructure components on the physical resources of the prefabricated region and deploy software resources, configurations, and / or other artifacts to the provisioned infrastructure components. For example, CSP104 can provision computing devices in prefabricated region 106A to host one or more virtual machines, provide hostnames, network addresses, and other network settings for the provisioned physical and virtual devices, and then deploy one or more services to run on the provisioned infrastructure. The prefabricated region may be in a state close to the final production state of the devices when they are deployed in the target facility.

[0042] Once a prefabricated region is constructed, physical resources may be configured for delivery / transportation to the target facility. In the context of use herein, the term “transmission” may be used synonymously with the term “transportation” when referring to the movement of physical resources associated with a prefabricated region from the prefabricated factory to the target site. Configuring a prefabricated region for delivery may include taking a “snapshot” of the current network configuration of computing devices in the prefabricated region, storing the snapshot, providing each computing device with a portion of the snapshot containing identifiers for each device in the network and its neighbors, encrypting the data volumes of the computing devices, and configuring the devices to boot in a test state upon power-on after delivery. In addition to network snapshots, the CSP104 prefab service may also capture device snapshots, which are disk images obtained for individual switches, compute devices, and smart NICs that are fully configured in the various racks being sent to the target site. Device snapshots allow for the rapid replacement of any device in a transported rack if it becomes non-functional after arrival and requires replacement. Transportation to the target facility can be carried out by one or more methods, including transportation by truck 112 or by aircraft 114. For example, prefabricated region 106B may be configured to be delivered to data center 108 by truck 112. On the other hand, prefabricated region 106C may be configured to be delivered to data center 110 by aircraft 114.

[0043] The computing devices in the prefabricated region may be deployed at the destination facility upon arrival, according to the facility's configuration. The destination facility could be a data center built to host the prefabricated region devices, with infrastructure such as networking, power, and cooling provided according to the prefabricated region's configuration. The data center may have network connectivity to CSP104. Deployment of the prefabricated region may include manual operations and other related tasks to connect the racks and their respective computing devices to the data center's network infrastructure. Once the physical connectivity is configured, powering on the devices in the prefabricated region allows the devices to initiate one or more test operations based on the configuration performed in the prefabricated factory 102 prior to delivery. The prefabricated region can communicate with prefabricated services by also connecting to CSP104 via one or more network connections to the data center. For example, prefabricated region 106B may connect to CSP104 via connection 118, while prefabricated region 106C may connect to CSP104 via connection 116. The prefab service can deploy the final configuration of the deployed devices in the target data center, deploy software resource updates on the deployed devices, and perform additional testing and verification operations on the prefab region.

[0044] Figure 2 is a block diagram of a prefabrication system 200, which includes a prefabrication factory 202 connected to a prefabrication service 210 provided by CSP204 for the construction of a region, according to at least one embodiment. The prefabrication factory 202 may be an example of the prefabrication factory 102 in Figure 1, and CSP204 may be an example of CSP104 in Figure 1. The prefabrication factory 202 may interact with CSP204 via a network 208, which may be a public network such as the Internet, a private network, or another network. The prefabrication service 210 may include a management service 212, an inventory service 214, a test service 216, an organization service 218, and a network service 220. The prefab service 210 is capable of performing actions corresponding to the construction of prefab region 206 in prefab factory 202, including managing the bootstrap environment (e.g., ViBE 222), provisioning infrastructure components in prefab region 206, deploying software resources to prefab region 206, configuring the network of prefab region 206, testing the prefab region at various stages of the construction process, and managing the physical inventory (e.g., physical inventory 224) of computing devices used to construct prefab region 206 and other prefab regions being built in prefab factory 202.

[0045] Management service 212 can perform tasks to coordinate the operation of prefab services 210, including scheduling prefab region construction work by other prefab services 210, generating physical construction requests and corresponding instructions, initiating the transport of prefab region 206 to the target site, and managing the provisioning and deployment of resources in prefab region 206 at both the prefab factory 202 and the target site. A physical construction request may specify the number and type of physical resources to be used in prefab region 206. A physical construction request may also include a set of instructions that personnel can use to deploy the corresponding physical resources in prefab factory 202. For example, management service 212 may generate a physical construction request that specifies the number of racks and server devices in prefab region 206, the number of networking devices available to connect the server devices and constitute the network of prefab region 206, and a connectivity plan that determines the networking connections between the specified server devices, networking devices, and the existing networking infrastructure of prefab factory 20. Furthermore, a physical build request may include instructions for an agent to retrieve physical devices from a relevant location (e.g., physical inventory 224) and instructions to deploy the devices to a designated location in the prefabricated factory 202. In some embodiments, the operation of a physical build request may be performed by an automated system under the control of the management service 212. For example, the retrieval of server device racks from the physical inventory 224 and the deployment of racks in the prefabricated factory 202 may be performed by a robotic system configured to move physical racks between sites.

[0046] The inventory service 214 may be configured to track and monitor physical devices corresponding to one or more regions (e.g., one or more data centers in a region). The inventory service 214 can also track physical devices for one or more prefabricated regions in the prefabricated factory 202 (e.g., prefabricated region 206). Tracking and monitoring physical devices may include maintaining an inventory of devices according to their identifiers (e.g., serial numbers, device names, etc.) and their association with data centers. The inventory service 214 can provide inventory information to other prefabricated services 210, including the management service 212, for use in the prefabricated region construction process. For example, the inventory service 214 can determine whether a physical device is located in the prefabricated factory 202 or the target site. The inventory service 214 can determine the location and / or association with a region, prefabricated region, or data center by querying devices over a network (e.g., network 208). Furthermore, the inventory service 214 can maintain a physical inventory (e.g., physical inventory 224) of devices stored for use in the prefabricated region construction work. For example, the inventory service 214 can track physical devices that, after being accepted in the physical inventory 224, are taken out of the physical inventory 224 and used as part of a prefabricated region in the prefabricated factory 202. In some examples, the inventory service 214 can provide the management service 212 with inventory information that can be used to generate physical construction requests for a prefabricated region 206, including instructions to retrieve physical resources from the physical inventory 224 and deploy them in the prefabricated factory 202.

[0047] The physical inventory 224 may be a warehouse or storage facility for storing physical resources (e.g., computing devices) used in the prefabricated region construction work. The physical inventory 224 may be located near the prefabricated factory 202 to facilitate the retrieval of physical resources in response to physical construction requests. For example, the physical inventory 224 may be in a building adjacent to the building used for the prefabricated factory 202. In some examples, the physical inventory 224 may be located within the prefabricated factory 202. The physical resources may be placed into and retrieved from the physical inventory 224 by personnel associated with the CSP and the prefabricated factory 202. In some cases, during the prefabricated region construction work, robots, automated guided vehicles, or other similar autonomous or semi-autonomous systems may be used to retrieve and deploy physical resources from the physical inventory 224 using instructions provided by the physical construction request.

[0048] The organization service 218 may be configured to perform bootstrap operations to provision infrastructure components in the prefabricated region 206 and deploy software resources to the prefabricated region 206. The organization service 218 may also configure a bootstrap environment (e.g., ViBE222) used when bootwrapping resources to the prefabricated region 206. The organization service 218 may be an example of the deployment orchestrator described above. In some examples, the organization service 218 may be configured to bootstrap (e.g., provision and deploy) services to a prefabricated region (e.g., prefabricated region 206) based on a predetermined configuration file that identifies resources (e.g., infrastructure components and software to be deployed) for implementing a given change in the prefabricated region. The organization service 218 can identify dependencies between resources by parsing and analyzing the configuration file. The analysis may generate specific data structures, which may be used to drive operations and manage the order in which services are bootstrapped to the region. The organization service 218 may use these data structures to identify when a service can be bootstrapped, when a bootstrap is blocked, and / or when a bootstrap operation associated with a previously blocked service can be resumed.

[0049] In some embodiments, the organization service 218 may include components configured to perform bootstrap tasks associated with a single service in a prefabricated region. The organization service 218 may maintain current state data indicating an optional preferred mode of the current state of the resources associated with the service. In some embodiments, the desired state data may include settings that declare a desired state for the resources associated with the service (e.g., by declarative statements). In some embodiments, the organization service 218 may identify one or more resources that require modification by comparing the desired state data with the current state data. For example, the organization service 218 may determine that it is necessary to provision one or more infrastructure components, deploy one or more artifacts, or make an optional preferred modification to the resources of the service to bring the state of those resources to a desired state. Specific details relating to a particular embodiment of the organization service 218 are given in U.S. Patent Application No. 17 / 016,754, entitled "Techniques for Deploying Infrastructure Resources with a Declarative Provisioning Tool," which is incorporated herein by reference for all purposes.

[0050] ViBE222 may be an example of a bootstrap environment that can be used to deploy resources to a prefabricated region in a prefabricated factory 202. ViBE may include a virtual cloud network (e.g., a network of cloud resources) implemented within a preferred region of a CSP (e.g., CSP204). ViBE may have one or more nodes (e.g., compute nodes, storage nodes, load balancers, etc.) that support hosting services deployed by the organization service 218. On the other hand, the use of ViBE services can support the deployment of services to the prefabricated region 206. For example, the organization service 218 may deploy an instance of one or more of its constituent services (e.g., an instance of the organization service 218) to the bootstrap environment, and the use of this instance may be used to deploy resources from ViBE222 to the prefabricated region 206. Since ViBE is implemented as a virtual cloud network in an existing region, support for services deployed in ViBE may involve provisioning an arbitrary and preferred amount of regional infrastructure (compared to the fixed hardware resources of the seed server). The organization service 218 may be configured to deploy software resources to ViBE222, as well as provision infrastructure resources for ViBE222 (e.g., virtual machines, compute instances, storage, etc.). ViBE222 can simultaneously support the bootstrap operation of two or more prefabricated regions in the prefabricated factory 202.

[0051] If a prefab region 206 is available to support bootstrap operations, connecting ViBE222 to prefab region 206 allows services within ViBE222 to interact with services and / or infrastructure components in prefab region 206. This enables the deployment of generation-level services instead of self-contained seed services as in traditional systems, and requires connectivity to the target region over the internet. Traditionally, seed services were deployed as part of a container collection and used to bootstrap the dependencies necessary for building the region. By using existing region infrastructure / tools, hardware provisioning and service deployment may be performed by bootstrapping resources to ViBE222 and connecting to prefab region 206 until prefab region 206 becomes self-sufficient (e.g., self-sufficient with respect to services hosted within prefab region 206). Using ViBE222 enables the establishment of dependencies and services necessary for infrastructure provisioning / creation and software deployment, while resolving circular dependencies of core services by utilizing resources in the host region.

[0052] Test service 216 may be configured to perform one or more test or verification operations on the prefab region 206 following the provisioning and / or deployment of resources. The test operations may be part of user approval tests that can be used to determine whether the behavior of the constructed region conforms to the construction specifications. For example, test service 216 may be configured to interact with instances of services deployed in the prefab region 206 to perform tests that verify the expected behavior of the queried services. As another example, test service 216 may be configured to perform networking tests that retrieve the hostnames, networking addresses, and / or other identifiers of the components of the prefab region 206 and compare them with the expected identifiers of the components, such as those specified in the specifications, such as the construction request for the prefab region 206. Test service 216 may be configured to perform test operations both during the prefab region construction process in the prefab factory 202 and after the prefab region 206 has been delivered to the target site. Test operations performed in the prefab factory 202 may be the same as or different from test operations performed after the prefab region 206 has been delivered to the target site.

[0053] Network service 220 may be configured to determine the network configuration of devices in prefabricated region 206. Using configuration information from build requests, network service 220 can determine the network topology of devices (e.g., servers, networking devices, racks of servers and networking devices). In the use herein, the network topology may represent a graphical representation of all networking connections between each computing device in the prefabricated region. Using configuration information, network service 220 can determine the physical networking connections (e.g., network cabling connections) between devices in the prefabricated region. Network service 220 may also provide networking connection information to management service 212, which is used to generate instructions for physically deploying devices in the prefabricated region at prefabricated factory 202. Additionally, network service 220 may obtain device information from inventory service 214 as part of determining the network topology of devices in the prefabricated region. Further details regarding network service 220 are given below with respect to Figures 5 and 6.

[0054] Figure 3 is a block diagram of a CSP system 300 that includes multiple host regions (e.g., host regions 304A to 304C) capable of supporting ViBEs (e.g., ViBE308A to 308C) for deploying software resources to a prefabricated region 306 constructed in a prefabricated factory 302, according to at least one embodiment. The prefabricated factory 302 may be an example of the prefabricated factory 202 described above with respect to Figure 2. Similarly, each of the ViBE308A to 308C may be an example of the ViBE222 in Figure 2, and the management service 312 may be an example of the management service 212 in Figure 2. The host regions 304A to 304C may correspond to regions of the CSP and may be associated with one or more data centers having computing resources for hosting the ViBEs. The host regions 304A to 304C may correspond to different geographical locations.

[0055] As shown in Figure 3, the management service 312 may be an instance within a host region (for example, host region 304B). In some embodiments, the management service 312 may correspond to CSP tenancy and therefore may have instances in multiple regions capable of providing prefab services (for example, host regions 304A, 304C). Similarly, other prefab services (for example, prefab service 210) may also be instances of services within a host region.

[0056] To support the construction of the prefabricated region at prefabricated factory 302, ViBE may be hosted in a host region. Since ViBE may be configured with organization services (e.g., organization service 218) as needed for bootstrapping the prefabricated region, it can be built in any suitable host region. Suitability as a host region may depend on network connectivity to prefabricated factory 302 (e.g., high bandwidth, high data rate, low latency network connectivity between the host region's data center and prefabricated factory 302), sufficient infrastructure resources to support ViBE for one or more prefabricated region construction operations (e.g., availability of the host region's computing resources over the time of provisioning and deployment of the prefabricated region), and / or jurisdictional considerations (e.g., the host region being in the same country as the prefabricated factory in accordance with data security regulations). For example, host region 304A may have a data center very close to prefabricated factory 302, which would result in low latency network connectivity between ViBE 308A and prefabricated region 306. In a series of prefabricated region construction operations, the ViBE used to support the construction of the prefabricated region may be configured in different host regions. For example, for one prefabricated region, ViBE308A may be used as part of the construction of the prefabricated region in prefabricated factory 302, but for subsequent region construction operations, ViBE308B in host region 304B or ViBE308C in host region 304C may be configured and used.

[0057] Furthermore, the prefabricated factory 302 may be constructed in a location that provides suitable connectivity to one or more host regions. For example, the prefabricated factory 302 may be configured at a site adjacent to a data center in host region 304A in order to provide suitable network connectivity between host region 304A and the prefabricated factory 302.

[0058] Figure 4 is a block diagram of a CSP system 400 having the arrangement of physical computing resources of a prefabricated factory 402 to different prefabricated regions 430, 440 according to at least one embodiment. The prefabricated factory 402 may be an example of the prefabricated factory 202 in Figure 2. The prefabricated services 410 may be provided by the CSP and may also be an example of the prefabricated services 210 described above with respect to Figure 2, and include management services 412 as an example of management services 212 in Figure 2, and inventory services 414 as an example of inventory services 214 in Figure 2. Similarly, prefabricated regions 430 and 440 may be examples of other prefabricated regions described herein (including prefabricated region 206 in Figure 2).

[0059] As described above, the prefabricated factory 402 may be configured to support the construction of multiple prefabricated regions simultaneously. As shown in Figure 4, the prefabricated factory 402 comprises prefabricated region 430 and prefabricated region 440. Prefabricated region 430 may include one or more server racks 432A to 432C. Each server rack may contain one or more devices (including server devices and networking devices). For example, server rack 432A may contain a switch 434 and a server device 436. Switch 434 may be a top-of-rack switch that provides networking connections to other server racks in prefabricated region 430 (for example, via the top-of-rack switch of those other server racks) or to other physical resources. Networking connections between physical resources in prefabricated region 430 may include part of the networking infrastructure 438. The networking infrastructure 438 may include a portion of the networking infrastructure of the prefabricated factory 402 (including network cabling, network switches, routers, etc., that constitute the network fabric of the prefabricated factory 402). Similarly, the prefabricated region 440 may include one or more server racks 442A and 442B. The computing devices they comprise may be more or less than those in the server racks 432A-432C of the prefabricated region 430. For example, server rack 442A may comprise a switch 444 and a server device 446.

[0060] Each prefab region may be at a different stage of the prefab region construction process at any given time. For example, infrastructure provisioning and resource deployment may be taking place in prefab region 430, while physical resources are being deployed in prefab region 440. Also, each prefab region in prefab factory 402 may have a different arrangement of physical resources. For example, prefab region 430 may contain more server racks (e.g., racks 432A-432C) than prefab region 440, and each server rack may support more computing devices than the server racks in prefab region 440 (e.g., server racks 442A, 442B). Because the number and arrangement of physical resources in each prefab region may differ, the network topology corresponding to the connections between physical resources may also differ from prefab region to prefab region.

[0061] The inventory service 414 can track the physical resources used to configure the prefabricated region in the prefabricated factory 402. The physical resources tracked by the inventory service 414 may include server devices and networking devices, as well as racks containing server devices and networking devices. The inventory service 414 can also track physical resources in the data center of the deployed region (including prefabricated region devices after delivery to the target site and deployment). In some embodiments, the inventory service 414 can connect to the prefabricated region (e.g., via a network) and query the device identifiers of devices in the prefabricated region. The inventory service 414 may also provide the management service 412 with information corresponding to the physical resources in the prefabricated region as part of the prefabricated region construction work. For example, the management service 412 may use inventory information from the inventory service 414 to determine whether the physical resources of the prefabricated region have been deployed in accordance with the physical construction request. In some embodiments, the inventory service 414 can also maintain information corresponding to the physical inventory 424 (e.g., repositories, warehouses, or other storage for physical resources such as computing devices used to configure the prefabricated region). Maintaining the physical inventory 424 may include tracking the number and types of physical resources available for use in the prefabricated region, maintaining a database or other data store of inventory information, updating the inventory information when new physical resources are added to the physical inventory 424 (e.g., delivery of new devices, configuration of server racks, etc.), and updating the inventory information when devices leave the physical inventory for use in the prefabricated factory 402 (as indicated by the arrows in Figure 4). In some examples, CSP personnel may interact with the inventory service 414 to manually update the inventory information.

[0062] The management service 412 can obtain inventory information from the inventory service 414 and use it when generating physical construction requests. For example, the management service 412 may use the inventory information to determine the physical resources to be deployed in the prefabricated factory 402 for the prefabricated region corresponding to the physical construction request.

[0063] Figure 5 shows a CSP system 500 for managing the network configuration of computing resources in a prefabricated region 530 constructed in a prefabricated factory 502, using a management service 512 and a network service 520, according to at least one embodiment. The prefabricated factory 502 and prefabricated region 530 may be examples of other prefabricated factories and prefabricated regions described herein (including the prefabricated factory 202 and prefabricated region 206 in Figure 2). The prefabricated service 510 may be provided by the CSP and may also be an example of the prefabricated service 210 described above with respect to Figure 2, and includes a management service 512 as an example of the management service 212 in Figure 2, and a network service 520 as an example of the network service 220 in Figure 2.

[0064] As described above with respect to Figure 2, the management service 512 can perform tasks to coordinate the operation of the prefab service 510, including scheduling prefab region construction work by other prefab services 510, generating physical construction requests and corresponding instructions, and configuring the prefab region 206 for transport to the target site. The physical construction request may specify the number and type of physical resources to be used in the prefab region 206. The network service 520 can use the configuration information from the construction request to determine the network topology of the devices (e.g., servers, networking devices, racks of servers and networking devices). The network service 520 can also determine the network configuration of the devices in the prefab region 530 after provisioning the infrastructure components in the prefab region 530.

[0065] In some examples, the network service 520 may store a snapshot of the network configuration of a prefabricated region (for example, prefabricated region 530). The snapshot may contain information about the network topology of the prefabricated region at a specific point in time, including network identifiers of devices in the prefabricated region (e.g., network address, hostname, etc.), current network connections between devices, physical networking interfaces between devices in the prefabricated factory 502 and the networking infrastructure 538, and network configurations of devices (e.g., port settings, gateway settings, etc.). For example, server device 536 may be a computing device in server rack 532A of prefabricated region 530. Server device 536 may have a networking connection 540 to switch 534 in server rack 532. The network configuration of prefabricated region 530 may include information relating server device 536 to switch 534, including the type of networking connection 540, the port on switch 534 to which server device 536 is connected, and information specifying the settings of both server device 536 and switch 534 corresponding to the networking connection 540 between them. The network configuration may also include information relating server device 536 to “adjacent” devices in prefabricated region 530 via networking connections 542 and 544. Networking connections 542 and 544 may be via switch 534, and server device 536 may be connected to other devices in server rack 532A via networking connections 542 and 544. In some examples, “adjacent” devices of a given device in prefabricated region 530 may include each computing device on the same server rack. Additionally, switch 534 may have network connections to one or more other switches within the prefabricated region 530 (for example, networking connection 546 to a switch in server rack 532B).

[0066] Network snapshots may be used to verify the physical deployment (e.g., physical networking connectivity) of the prefabricated region 530 after the devices have been deployed at the target site. For example, network service 520 may provide a network snapshot (or part of a snapshot) to each device in the prefabricated region 530 as part of the configuration of the prefabricated region 530 for transport to the target site. For example, network service 520 may provide a network snapshot 526 to server device 536 for storage in server device 536. Network snapshot 526 may be part of a network snapshot corresponding to the network configuration of the entire prefabricated region 530. Network snapshot 526 may include an identifier for server device 536 (e.g., network address, hostname, etc.) and information that associates server device 536 with one or more other devices in the prefabricated region 530. Information that associates server device 536 with neighboring devices may include an identifier for the neighboring device and information about the network connection between them. For example, server device 536 can use network snapshot 526 to identify neighboring devices and communicate with them over the network connection.

[0067] Furthermore, the network service 520 may maintain the network configuration of the network fabric of the prefabricated factory 502. For example, the prefabricated factory 502 may have a networking infrastructure that supports multiple separate prefabricated regions being built simultaneously. The prefabricated factory 502 may have multiple dedicated locations for locating server racks for the prefabricated regions being built. Each location may have a set of networking cables of the networking infrastructure that terminates at a location connectable to a server rack. Based on the devices located at that location, certain cables from the set of networking cables are connected to a device (e.g., a top-of-rack switch), so that these devices can be connected to other devices in the prefabricated region using part of the network fabric of the prefabricated factory 502. For example, server rack 532A may be located at a location within the prefabricated factory 502 and connected to the networking infrastructure 538 using switch 534. Server rack 532B, on the other hand, may be located at a second location and connected to the networking infrastructure 538.

[0068] In addition to maintaining the network configuration of the prefabricated region 530, configuring the prefabricated region 530 for transport to the destination site may include the management service 512 configuring each device to enter a test state upon subsequent power-on of the device, encrypting the device's data volume with an encryption key, storing the encryption key in a device that can function as a key server for the prefabricated region 530 during initialization at the destination site, and configuring one of the devices to function as a Dynamic Host Configuration Protocol (DHCP) server during initialization of the prefabricated region 530 at the destination site. The management service 512 may also generate instructions that can be used by personnel or robotic systems associated with the prefabricated factory 502 for packaging the devices for delivery. The management service 512 may also generate instructions that can be used by personnel associated with the destination facility for deploying and connecting the devices at the destination facility.

[0069] Furthermore, in some embodiments, configuring devices in the prefabricated region 530 may include taking a device snapshot of each device. The device snapshot may include a software image of memory such as one or more disk drives of the computing device, which can be used to replicate the device's software configuration onto a replacement device. The management service 512 may generate device snapshots in conjunction with one or more of the prefabricated services 510. The device snapshots may be stored in a database or datastore in conjunction with network snapshots (e.g., snapshot 524). In a particular example, the management service 512 may generate a device snapshot 552 of the server device 550 in the prefabricated region 530 at the prefabricated factory 502. The device snapshot 552 may be used for an image of another physical device having the same or similar physical configuration as the server device 550 in order to generate a replica of the server device in the event of a failure of the server device 550 (e.g., damage or loss during transport to the target site).

[0070] Figure 6 shows a CSP system 600 for testing and evaluating a prefabricated region 530 after delivery to a target site 602 using management services 612 and test services 616, according to at least one embodiment. The target site 602 may be a data center facility in a location corresponding to a new region deployed to the CSP by using the computing resources of the prefabricated region 630. The prefabricated services 610 may be provided by the CSP and may be similar to the prefabricated services 210 in Figure 2, and include management services 612 as an example of management services 212 in Figure 2, test services 616 as an example of test services 216, and organization services 618 as an example of organization services 218.

[0071] Sending the prefabricated region 530 to the destination site 602 may include powering off each device, disconnecting the devices from the networking infrastructure of the prefabricated plant, and packaging the devices as necessary for transport. Server racks (e.g., server racks 532A, 532B) may be sent as is without disconnecting the individual devices within the server racks. The server racks delivered to the destination site 602 may be placed at the destination site 602 in the resulting data center physical layout and connected to the destination site's networking infrastructure 638. For example, a networking connection may be established between the networking infrastructure 638 and the switches of the server racks 532A, 532B by connecting one or more networking cables to a switch (e.g., switch 534).

[0072] As described above, devices in prefabricated region 530 may be configured to boot in test mode upon initial power-on at target site 602. In some embodiments, these devices may have a dedicated boot volume to support test mode during initialization at target site 602. In other embodiments, the boot volume may be configured on an external device connected to each device in prefabricated region 530. For example, each server device (e.g., server device 536) may be connected to a smart network interface card (smart NIC) that provides a low-overhead boot volume available for booting the server device in test mode. Since the boot volume is only used to support test mode, encryption of data on the boot volume is not considered necessary, unlike that of data volumes on server devices.

[0073] The test mode may be configured to have each computing device verify its connectivity to other devices in the prefabricated region 530. This verification can determine whether the device's physical network connectivity to the networking infrastructure 638 at the target site 602 is correct. To verify the connectivity, the devices in test mode may be determined by a network service (e.g., network service 520 in Figure 5) and use the network settings or a portion of the network settings stored on each device. For example, server device 536 may use a network snapshot 526 to determine neighboring computing devices that are communicating with server device 536 via networking connection 542. To verify networking connection 542, server device 536 may send a verification request to the neighboring computing device. If networking connection 542 is complete, server device 536 may receive a verification indicator from the neighboring computing device indicating that the verification request was successfully received by the neighboring computing device. Server device 536 may verify all connections specified in network snapshot 526. Similarly, in prefabricated region 530, devices on one server rack (e.g., server rack 532A) may be configured to verify connections to each of the other server racks (e.g., server rack 532B).

[0074] In some embodiments, a device in the prefabricated region 530 may be configured to function as a DHCP server (e.g., DHCP server 646). The DHCP server 646 may provide identifiers, such as network addresses, to the devices in the prefabricated region 530 during initialization. For example, in test mode, each device may, after verifying its connection to the DHCP server 646, receive an address, identifier, or other network configuration information from the DHCP server 646. The device may compare the received identifier to an identifier included in the network configuration generated by the network service during the prefabricated region construction work at the prefabricated factory. For example, after receiving an identifier from the DHCP server 646, the server device 536 may compare the received identifier to an identifier in the network snapshot 526. Since the components of the prefabricated region 530 should remain unchanged during transport, the network configuration of the prefabricated region 530 at the destination site 602 (including the configuration information from the DHCP server 646) should also remain unchanged. That is, after the devices are deployed at the destination site 602, the server devices in the prefabricated region should receive the same network address from the DHCP server 646. If the network settings change, the server device may indicate that the network settings for prefab region 530 are incorrect.

[0075] In some embodiments, if any device is damaged and becomes non-functional during transport, the operator at the destination site can successfully complete on-site post-deployment verification even in the event of hardware failure during transport by replacing the broken device with a new replacement device and configuring the new device with a device snapshot taken prior to transport. For example, server device 550 may be damaged during transport to destination site 602. The discovery of the non-functional state of server device 550 may occur during a test run verifying the network configuration of prefabricated region 530. For recovery, management service 612 can generate instructions to replace server device 550 with an identical physical device at the same location on server rack 532B. Once the replacement device is deployed, management service 612 can deploy a device snapshot 552 generated during the prefabricated region construction work at prefabricated factory 502. Deploying the device snapshot 552 may involve imaging one or more disk drives or other memory of the replacement server device so that the replacement server device has the same software configuration as the server device 550 in prefabricated region 530 prior to transport to destination site 602. Other devices, including networking devices such as switch 534, may be similarly replaced and restored using captured device snapshots.

[0076] The DHCP server 646 can perform a test mode verification operation similar to that of other devices in the prefabricated region 530. If the DHCP server 646 is able to successfully verify the network connectivity with neighboring devices, it can exit test mode and begin operating as a DHCP server for other devices in the prefabricated region 530. In some embodiments, the DHCP server 646 may complete its test mode verification operation before other devices in the prefabricated region 530 have completed their respective test mode verification operations. For example, server device 536 may boot in test mode and attempt to verify its network connectivity to the DHCP server 646 prior to verifying the networking connectivity 542 or networking connectivity 544 with neighboring computing devices. The DHCP server 646 does not need to send verification indicators to server device 536 until it has completed its own test mode verification operation. Server device 536 can then wait for a predetermined time and retry the verification request to the DHCP server 646. Similarly, other computing devices performing a test mode verification operation may wait until the DHCP server 646 is operational before retrying their verification request.

[0077] As described above, the data volumes of devices in prefabricated region 530 may be encrypted prior to their transport to the target site 602. The encryption key used to encrypt the data volume of each device may be associated with that particular device. The encryption key 644 may be stored in one of the computing devices in prefabricated region 530 that is configured to function as the key server for prefabricated region 530 during initialization (for example, it may be stored in key server 642). The encryption key 644 itself may also be encrypted by a parent key. In some embodiments, the encryption key 644 may be protected by a hardware security module (for example, a trusted platform module (TPM)). The hardware security module may be part of key server 642 or part of another device connected to key server 642 (for example, a smart NIC, an external security device, etc.). In some embodiments, the parent key or external security device may be delivered to the target site 602 separately from prefabricated region 530 (for example, by a worker) and provided to or deployed to key server 642 as part of the deployment work for prefabricated region 530. The key server 642 may perform a test mode verification operation similar to that of other computing devices in the prefab region 530. If the test mode verification operation is successful, the key server 642 may provide the cryptographic key 644 to other computing devices in the prefab region to begin decrypting the data volume. For example, the key server 642 may receive a key request from server device 536. In response, the key server 642 can decrypt the data volume containing the cryptographic key 644 (for example, by a parent key, hardware security module), read the cryptographic key corresponding to server device 536, and send the cryptographic key to server device 536.

[0078] Once the prefabricated region 530 is deployed and initialized at the target site 602 (for example, when the device boots in normal operating mode, the data volumes are decrypted, and the services deployed during the prefabricated region construction work at the prefabricated factory are running), the test service 616 may perform one or more approval tests. The approval tests may include verifying that all services are functioning as expected. For example, the test service 616 may interact with the services running in the prefabricated region 530 to verify that the services are operating according to the requirements defining the approval tests. The test service 616 may provide the management service 612 with the results of the approval tests indicating that the construction of the prefabricated region is complete.

[0079] During the transport of prefabricated region 530 to the destination site 602, changes such as updates may be specified for one or more infrastructure components and / or software resources that were provisioned and / or deployed to prefabricated region 530 at the prefabricated factory. For example, services may be updated to a newer version during transport. Before the prefabricated region construction work is completed, the organization service 618 can deploy the updated software resources to prefabricated region 530 at the destination site 602. The deployment of updated software resources can occur in the same way as the deployment of software resources to prefabricated region 530 at the prefabricated factory.

[0080] Figure 7 shows an exemplary method, according to at least one embodiment, for deploying software resources to physical resources in a region constructed in a prefabricated factory, to prepare the physical resources for delivery to a target data center. Method 700 may be performed by one or more components of a computer system (including one or more components of a computer system of a CSP (e.g., CSP204 in Figure 2) that runs a management service (e.g., management service 212 in Figure 2)). The operations of Method 700 may be performed in any preferred order, and the operations included in Method 700 may be more or less than those shown in Figure 7.

[0081] Method 700 (or any other process and / or method described herein, or any variation thereof, and / or combination thereof) may be executed under the control of one or more computer systems comprising executable instructions, or may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is collectively executed on one or more processors by hardware or a combination thereof. The code may be stored in a computer-readable storage medium in the form of a computer program containing, for example, multiple instructions executable by one or more processors. The computer-readable storage medium may be non-temporary.

[0082] Method 700 may begin in block 702, causing a management service to receive a build request. The management service may be an example of any management service described herein (including management service 212 in Figure 2). The management service may be configured to run on one or more computing devices of the CSP computer system. The management service may be one of several services of the CSP configured to perform the operation of building a prefabricated region (e.g., prefabricated region 206 in Figure 2) in a prefabricated factory (e.g., prefabricated factory 202 in Figure 2). A build request may be a specification or configuration containing information characterizing the prefabricated region. For example, a build request may include information specifying the size of the prefabricated region (e.g., the number of computing devices, server racks, etc.), the number and types of services, applications, and other software to run on the computing devices in the prefabricated region, requirements regarding the computing power in the prefabricated region (e.g., the number of processors in each computing device, the processing speed of the processors, etc.), requirements regarding the type of storage to be provided in the prefabricated region, and other similar definitions. In some embodiments, the build request may be provided by the worker or the system designer who designs the prefabricated region.

[0083] In block 704, the management service may generate a physical build request for constructing physical resources within a first data center. The first data center may be a prefabricated factory (for example, prefabricated factory 202 in Figure 2). The management service can use the information in the build request to generate a physical build request. The physical resources may include server devices, networking devices, and other computing devices that can be used to construct a prefabricated region in the first data center. The physical build request may include information that identifies a specific physical resource to be constructed in the first data center. For example, the management service can interact with an inventory service (for example, inventory service 214 in Figure 2) to obtain an inventory of server devices and server racks available in the physical inventory of devices (for example, physical inventory 224 in Figure 2). The management service can then determine a specific server rack to be used to construct the prefabricated region corresponding to the build request and include that information in the physical build request. The physical build request may also include instructions that can be used, for example, when a worker retrieves, moves, and deploys the physical resources in the first data center. For example, these instructions may identify a specific location within the first data center where the server racks will be placed, and instructions for completing specific network connections to the server racks to configure the network of the prefabricated region.

[0084] In block 706, the management service may implement ViBE (for example, ViBE222 in Figure 2) in a second data center. The second data center may be communicated to the first data center. For example, the management service may implement ViBE in a host region of a CSP connected to a prefabricated factory via a network (for example, network 208 in Figure 2). The management service may implement ViBE in conjunction with an organization service (for example, organization service 218 in Figure 2). For example, the management service may build ViBE by providing the organization service with an indicator specifying the second data center in which ViBE will be configured. The management service may implement ViBE in response to an indicator that the physical resources corresponding to a physical build request have been built (for example, deployment, power-on, and normal operation in the first data center). This indicator may be provided by an operator after the deployment of the physical resources. In some embodiments, this indicator may be provided by one or more of the physical resources after the completion of verification such as a self-check of the deployment.

[0085] In block 708, the management service can use ViBE to deploy software resources to physical resources. The software resources may be associated with cloud services running on the physical resources. For example, the software resources may be components of a generation service (e.g., a database service) that runs in a prefabricated region after it has been delivered to the target site. The management service, in conjunction with the organization service, can deploy the software resources.

[0086] In block 710, the management service can generate an inventory of physical resources. The management service may generate the inventory in conjunction with an inventory service. In block 712, the management service can use the inventory to generate a network configuration corresponding to the network topology of the physical resources in the first data center. The management service may generate the network configuration in conjunction with a network service (for example, network service 220 in Figure 2). The network configuration may be a network snapshot of a prefabricated region. The network configuration may include identifiers of the physical resources in the inventory (for example, the network addresses of server devices) and information relating the physical resources to neighboring physical resources according to the network topology. For example, the network configuration may identify each computing device and the network connections between that computing device and one or more neighboring computing devices. The operations in blocks 710 and 712 may be operations for configuring physical resources for delivery to a target site. The target site may be a third data center. In some embodiments, the management service may send a portion of the network configuration to the physical resources. This portion of the network configuration may include identifiers corresponding to the physical resources and information relating the physical resources to neighboring physical resources in the network topology. In some embodiments, configuring physical resources for delivery to a target site may include encrypting at least a portion of the software resources deployed on each physical resource using an encryption key associated with each physical resource, and storing each encryption key in one of the physical resources designated to host the key service in a second data center (for example, the key server 642 in Figure 6).

[0087] In some embodiments, the management service may receive an indicator that the physical resource has been delivered to the target site and constructed (e.g., deployed). Based on this indicator, the management service may verify the topology of the physical resource at the target site. For example, the management service, in conjunction with a network service, may obtain the network configuration of the physical resource at the target site and compare it with information contained in a stored network snapshot taken before the physical resource was sent to the target site. Once the network topology of the physical resource at the target site has been verified, the management service may deploy one or more update software resources to the physical resource. For example, the management service, working with an organization service, may deploy update software components to services that were deployed in a prefabricated region at a prefabricated factory and then migrated to a newer version upon delivery of the physical resource to the target site.

[0088] In some embodiments, the management service can perform actions to support the initialization of physical resources at the target site. The management service can determine the dependencies of a first cloud service (e.g., a deployed application) on a second cloud service (e.g., a database service). The first cloud service may include software resources hosted on the first physical resource, while the second cloud service may include software resources hosted on the second physical resource. Due to the dependencies, the first cloud service may not function correctly until the second cloud service is operational. Because the physical resources can independently perform test mode verification during initialization, some parts of the deployed region may become available before others. In this case, the management service can determine whether the verification of the portion of the network topology associated with the second physical resource was successful, and then send an indicator that the first cloud service is available. This indicator may be sent to a system, such as an operations console, configured to report the availability of services and applications in a prefabricated region at the target site when they become available. For example, the use of this metric may be used to initiate one or more user approval tests for a newly available first cloud service.

[0089] In some embodiments, during the construction of a prefabricated region in a prefabricated factory, changes may be made to the configuration of the prefabricated region. For example, after being delivered to the target site, the prefabricated region may need additional computing resources to support the addition or expansion of applications and / or services. The techniques described herein can address modifications to a prefabricated region under construction in a prefabricated factory. The management service can generate an update physical build request that can be used to modify physical resources. For example, an update physical build request may specify the deployment of additional server racks to the prefabricated region in the prefabricated factory. As another example, one or more server devices may be replaced with different types of server devices (e.g., devices with high-speed processors, additional processors, additional memory, etc.). Similar to a physical build request, an update physical build request may include instructions that can be used, for example, by workers in the prefabricated factory to acquire, deploy, and / or modify physical resources. After the modifications are made, the management service can deploy the updated software resources to the modified physical resources. For example, the management service can use the organization service and ViBE to deploy the software components of the new services to the new server racks in the prefabricated region. The management service may deploy updated software resources upon receiving an indicator that the physical resource repair was successful.

[0090] In some embodiments, configuring physical resources for transport to a second data center may include generating device snapshots of one or more of the physical resources. For example, a management service may generate a software image of each server device in a prefabricated region and store this software image in a datastore or similar repository. When verifying the prefabricated region after deployment at the target site, the management service may determine that one of the physical resources has failed. For example, a server device may have been damaged or lost during transport to the target site. In response, the management service may generate instructions that can be used to replace the non-functional physical resource with a functional replacement (e.g., replacing a non-functional server device with a functional replacement server device with the same physical configuration). Once a functional replacement device is deployed, the management service may configure the replacement device using a device snapshot of the failed device. For example, the management service may deploy an image of the replacement device to generate a device that is configured and functions the same as the replaced device.

[0091] Figure 8 shows an exemplary method 800 for booting up physical resources constructed in a prefabricated factory after delivery to a target data center and verifying the network configuration of the physical resources, according to at least one embodiment. Method 800 may be performed by one or more components of a computer system (including one or more components of a computer system in a prefabricated region (e.g., prefabricated region 206 in Figure 2) that is connected to a CSP hosting a management service (e.g., management service 212 in Figure 2)). For example, Method 800 may be performed by a computing device in a prefabricated region (including server device 536 in Figure 5, DHCP server 646 or key server 642 in Figure 6). The operations of Method 800 may be performed in any preferred order, and the operations included in Method 800 may be more or less than those shown in Figure 8.

[0092] Method 800 begins in block 802, in which the computing device receives network configuration (e.g., network snapshot 526 in Figure 5) from a management service. The network configuration may include information specifying the network topology of physical resources in the first data center (e.g., prefabricated factory 202 in Figure 2). The network configuration may include a first identifier associated with the computing device (e.g., network address, hostname, etc.), a second identifier associated with the neighboring computing device (e.g., network address, hostname, etc.), and information relating the computing device to the neighboring computing device (e.g., information specifying the networking connection 544 in Figure 5). The computing device can be configured to communicate with the neighboring computing device over the network connection by using the network configuration.

[0093] In block 804, the computing device may be configured for delivery to a second data center (for example, the target site 602 in Figure 6). Configuring the computing device for delivery may involve operations similar to those described above with respect to blocks 710 and 712 in Figure 7. Configuring the computing device for delivery to a second data center may also involve configuring the computing device to boot in test mode during the subsequent power-on sequence. In block 806, the computing device may be configured to boot in test mode at the second data center. For example, once the computing device and other physical resources of the prefabricated region are delivered to and deployed at the second data center, the computing device may be powered on and enter test mode. In some embodiments, booting the computing device in test mode may involve booting from a boot volume stored on a smart NIC connected to the computing device.

[0094] In block 808, the computing device may receive a new identifier. This new identifier may be received from a server device in the second data center. For example, the server device could be a device configured to function as a DHCP server in the second data center. The identifier may also be the network address of the computing device. As mentioned above, this identifier may be the same as the first identifier associated with the computing device in the prefabricated region of the prefabricated factory, because no changes should have occurred in the network configuration during the transport and deployment of the physical resources to the second data center.

[0095] In block 810, the computing device can verify a new identifier by comparing it with a first identifier. Prior to delivery, the computing device can obtain the first identifier from the network configuration stored in the computing device.

[0096] In block 812, a computing device can send a verification request to an adjacent computing device. The verification request is sent according to a second identifier associated with the adjacent computing device. For example, a computing device can verify the network connectivity of an adjacent computing device at the network address associated with the adjacent computing device. Accordingly, in block 814, a computing device can verify its network connectivity to an adjacent computing device. Network connectivity may be characterized by network configuration. In some embodiments, verification of network connectivity may include receiving a response to a verification request, which may be a verification indicator from the adjacent computing device. In some embodiments, the response to a verification request may be an indicator (e.g., a request timeout indicator) that the verification request was not received by the adjacent computing device. The verification indicator may indicate that the physical networking between the computing device and the adjacent computing device was properly deployed in the second data center. In some embodiments, once the computing device has successfully verified its connectivity to each adjacent computing device, it may send an indicator to a management service at the target site indicating that the verification of the network connectivity associated with the computing device was successful.

[0097] In some embodiments, the computing device may be configured to act as a key server for a prefabricated region in a second data center. Configuring the computing device for delivery to the second data center may include encrypting the data volume of the neighboring computing device using an encryption key associated with the neighboring computing device. The encryption key may be stored in the data volume of the computing device, which may be encrypted with a different encryption key (e.g., a parent key). The parent key may be stored in a secure storage volume (e.g., a hardware security module, a trusted platform module (TPM), a smart NIC) that is connected to the computing device and can be used to decrypt storage volumes on the computing device, thereby reading the encryption key and providing it to neighboring computing devices and other computing devices in the prefabricated region that have come online in the second data center. In some embodiments, once the computing device has verified network connectivity to the neighboring computing device, it may retrieve the parent key from the secure storage volume, decrypt the data volume storing the encryption key, and provide the encryption key in response to key requests from the neighboring computing device and other computing devices.

[0098] In some embodiments, a computing system may determine that one or more computing devices in a second data center have failed or are not functioning correctly. For example, a server device in a server rack may have been damaged during transport. To complete the deployment of a prefabricated region in the second data center, the failed or non-functioning computing device may be replaced with another device, and the failed device may be configured with a software image prior to its transport to the second data center. As an example, a computing system may configure an adjacent computing device for delivery to the second data center by generating a device snapshot of the adjacent computing device. The device snapshot may include a software image of the adjacent computing device. The device snapshot may be generated by a management service and / or other prefab services that perform prefabricated region construction work in the prefab factory.

[0099] Once a computing device is deployed to the second data center, the computing system can determine if an adjacent computing device is not functioning. For example, the computing device may receive a response to a verification request indicating that the adjacent computing device is damaged or not functioning correctly. In response to this determination, the management service can generate instructions to replace the adjacent computing device with a replacement computing device. These instructions may be available for personnel in the second data center to perform the replacement (e.g., a likeness replacement of a device on a server rack). The management service can then deploy a device snapshot of the adjacent computing device to a functional adjacent computing device, and the resulting device may be identical to the failed device. The computing device can then resend a verification request to determine if the network connection between the computing device and the adjacent computing device is functioning correctly.

[0100] Static Network Fabric As described above, a prefabricated factory (for example, prefabricated factory 102 in Figure 1) may include a static network fabric consisting of networking infrastructure (e.g., switches, routers, cabling, etc.) designed to support various network topologies of the regional components constructed in the factory. Prefabricated regions with different network topologies can also be quickly connected to the static network fabric according to a connection plan that fits the static network fabric having the physical components of the region. The static network fabric may include network cabling that terminates at configuration locations within the prefabricated factory configured for server rack deployment. The network cabling may include terminals such as multiple cable connectors of multiple types that support connections to various computing devices in the prefabricated region deployed at these locations. For example, server racks in a first prefabricated region may be deployed at a location in the prefabricated factory and connected to the static network fabric at these locations according to the network specified for the first prefabricated region. Subsequently (for example, after the prefabricated region construction work), these server racks may be removed, and server racks in a second prefabricated region with different networking interfaces may be deployed at the same location and connected to the static network fabric according to the network specified for the second prefabricated region. Thus, in a prefabricated factory, various prefabricated regions can be introduced without modifying the static network fabric, which reduces the complexity of network connectivity between prefabricated regions within the factory and increases the speed of introducing and removing prefabricated region components to support the construction of prefabricated regions.

[0101] Figure 9 is a block diagram showing an exemplary static network fabric 900 in a prefabricated factory 902 according to at least one embodiment. Prefabricated factory 902 may be an example of any of the prefabricated factories described herein (including prefabricated factory 102 in Figure 1). The static network fabric 900 may include network cables 908 routed throughout the prefabricated factory. For example, the network cables 908 may be introduced into overhead cable trays aligned with the locations of server rack rows. In other examples, the network cables 908 may be introduced to extend through trays or conduits under elevated flooring, also aligned with the locations of server rack rows. The network cables 908 can be configured to terminate at locations in the prefabricated factory 902 where server racks may be placed when the prefabricated regions are installed. For example, a pair of network cables 908 may terminate at the location of server rack 904A in prefabricated region 904.

[0102] A prefabricated factory can support multiple prefabricated regions simultaneously with respect to the construction of prefabricated regions. As shown in Figure 9, prefabricated factory 902 may include prefabricated regions 904 and 906, each of which may be an example of other prefabricated regions described herein (including prefabricated region 206 in Figure 2). Prefabricated region 904 may include server racks 904A to 904D. Similarly, prefabricated region 906 may include server racks 906A to 906D. Computing devices in prefabricated regions may communicate with each other via network cables, including one or more of the network cables 908 of the static network fabric 900, and the arrangement of associated networking devices. The arrangement of connected computing devices in a prefabricated region may be referred to as a region network. The region network of prefabricated region 904 may differ from the region network of prefabricated region 906, although in some cases, some networking devices of the static network fabric may handle traffic from both region networks. The network cable 908 may include one or more types of network cabling, and / or various combinations of types (including fiber optic cabling, copper-based cabling (e.g., Ethernet, coaxial copper wire, etc.)). In some embodiments, the network connectivity enabled by the network cable 908 may be implemented by optical links and / or wireless links such as ultra-wideband technologies. While the description herein refers to physical network cabling, embodiments of the disclosure may also include optical or other wireless network connectivity.

[0103] A set of network cables 910 terminating at the above location may include several types of cables (e.g., optical fiber, twisted-pair Ethernet wiring, coaxial cable, etc.) each terminated with a suitable cable termination connector. The cable termination connector may be connected to the termination of one of the network cables in the set of network cables 910. Examples of cable termination connectors include, but are not limited to, multi-fiber push-on (MPO), multi-fiber pull-off, SFP (Small Form-factor Pluggable), SFP+, SFP28, QSFP (Quad Small Form-factor Pluggable), QSFP+, QSFP28, and RJ45. If server racks are located at a prefabricated factory location, one or more of the set of network cables terminating at that location may connect one or more computing devices in the server rack to a regional network. For example, server rack 904A may include a network switch (e.g., a top-of-rack switch) located on top of the rack.

[0104] Furthermore, the static network fabric 900 may include one or more networking devices configured to simultaneously support network traffic from multiple regional networks. By arranging the networking devices in various architectures, they can support different levels of network traffic from different prefabricated regions in the prefabricated factory 902. For example, the static network fabric 900 may be arranged in a three-tier architecture, with aggregate switches (e.g., switches 912, 914) supporting top-of-rack switches in each server rack (e.g., server racks 904A-904D and server racks 906A-906D), and a core switch (e.g., switch 916) supporting the aggregate switches. Another example is the static network fabric 900 being arranged in a spine-leaf architecture, with leaf switches (e.g., top-of-rack switches in each server rack) supporting traffic from server devices in each server rack, and spine switches (e.g., switches 912, 914, 916) supporting traffic from each of the leaf switches in the lower layers.

[0105] The static network fabric 900 can be configured as a Clos network. A Clos network topology is a non-block architecture in which each switch in one layer of the network fabric (e.g., each leaf switch) is connected to each switch in the next layer (e.g., each spine switch), providing network paths between each device and each device in the other, and allowing traffic to be directed most efficiently along the available paths. For example, switches 912-916 may be spine switches of the static network fabric 900. Since each of switches 912-916 may be connected to a network cable 908 terminating at each location, a set of network cables (e.g., a set of network cables 910) includes network connections to each of switches 912-916. If the network connections at these locations are connected to leaf switches (e.g., top-of-rack switches in each server rack), the resulting interconnections can constitute a Clos network. Other topologies can also be supported by a suitable number of switches and other networking devices.

[0106] Figures 10A and 10B illustrate an exemplary arrangement of physical computing resources connected to a static network fabric in a prefabricated factory according to several embodiments. Figure 10A shows a CSP system 1000 including a prefabricated factory 1002 on which a prefabricated region 1004 can be constructed. Figure 10B shows a prefabricated region 1032 in the prefabricated factory 1002. The CSP systems 1000 and 1030 may include prefabricated services 1020, which may be examples of other prefabricated services described herein (including prefabricated service 510 in Figure 5). Prefabricated services 1020 may include management services 1022 and network services 1024, which may be examples of management services 512 and network services 520 in Figure 5, respectively. The prefabricated factory 1002 may be an example of any other prefabricated factory described herein (including prefabricated factory 902 in Figure 9). The prefabricated factory 1002 may include a static network fabric 1012, which may be an example of the static network fabric 900 in Figure 9. The static network fabric 1012 may include a network cable 1008 and a networking infrastructure 1010 having one or more networking devices (e.g., switches, routers, etc.) for handling traffic between computing devices in one or more regional networks.

[0107] A prefabricated region 1004 may include multiple server racks 1004A, 1004B-1004N. Each server rack may have many computing devices (including server devices and networking devices). Each server rack 1004A-1004N may have the same or different number of computing devices and / or different types of computing devices (e.g., server devices with different computing power). For example, server rack 1004N may have fewer server devices than server rack 1004A. As part of the prefabricated region construction work for prefabricated region 1004, server racks 1004A-1004N may be positioned at a location within the prefabricated factory 1002. A set of network cables 1008 (e.g., a set of network cables 1006A-1006N) configured to terminate at each location may be connected to each server rack 1004A-1004N.

[0108] As a specific example, a server rack 1004A in prefabricated region 1004 may be connected to the static network fabric 1012 of prefabricated factory 1002 by connecting a pair of network cables 1006A from network cable 1008. As described above with respect to Figure 9, a pair of network cables 1006A may include multiple network cables having cable termination connectors. Server devices on server rack 1004A may be connected to network switches. For example, server rack 1004A may comprise 40 server devices, each having a network connection to a top-of-rack switch on the server rack 1004A. The connection between the server devices and networking devices of the server rack may be made prior to the installation of the server rack in the prefabricated factory. For example, the server devices and networking devices of server rack 1004A may be connected while server rack 1004A is stored in the physical inventory (e.g., physical inventory 224 in Figure 2). When server rack 1004A is installed in a corresponding location in a prefabricated factory 1002, the server devices can be connected to the regional network by using the static network fabric 1012, by connecting one or more of a set of network cables 1006A to one or more ports of a network switch. The regional network may include a network composed of computing devices in the prefabricated region and portions of the static network fabric 1012 that enable network connectivity between various server racks.

[0109] Depending on the configuration of server rack 1004A, some of the cables in a set of network cables 1006A may not be connected to the network switch of server rack 1004A. For example, the network switch may be configured to connect to another network switch in the static network fabric via a QSFP+ fiber optic connection and may not have networking ports that support twisted-pair or coaxial cabling. Therefore, some of the cables in a set of network cables 1006A that are twisted-pair or coaxial cabling and have corresponding cable termination connectors will not be connected to the network switch of server rack 1004A. Similarly, networking devices in server rack 1004B may be connected to one or more cables in a set of network cables 1006B, and networking devices in server rack 1004N may be connected to one or more cables in a set of network cables 1006N.

[0110] To configure connectivity between the static network fabric 1012 of prefabricated factory 1002 and computing devices in prefabricated region 1004 and / or prefabricated region 1032, management service 1022 and network service 1024 can perform the operation of generating connectivity plans. The connectivity plans may include instructions for (for example, by workers in prefabricated factory 1002) to identify appropriate network cables from a set of network cables at each location that lead to server racks (e.g., server racks 1004A-1004N, server racks 1034A-1034N), and to identify the corresponding ports (e.g., top-of-rack switches) on computing devices to which the identified cables may be connected. Server racks in prefabricated region 1004 may be connected to the static network fabric 1012 via various connections. For example, server rack 1004A may be connected via one or more QSFP+ connections from a set of network cables 1006A. On the other hand, the server rack 1034A may be connected via one or more SFP connections from a set of network cables 1036A.

[0111] To generate a connectivity plan, the network service 1024 can determine the configuration of computing devices in a prefabricated region and the static network topology of the static network fabric 1012. The configuration of computing devices may include information specifying the physical networking connections between server devices on each server rack and networking devices. For example, each server device on server rack 1004A may be connected to a specific identified port on a top-of-rack switch on server rack 1004A. The configuration of server rack 1004A may include information identifying the connection between each server device and the specific port on the top-of-rack switch to which it is connected. The configuration of computing devices in prefabricated region 1004 may be predetermined, for example, as part of the initial configuration of each server rack in a physical inventory (for example, physical inventory 224 in Figure 2). This configuration may be stored as configuration parameters in a data store accessible to the network service 1024.

[0112] Similarly, the static network topology of the static network fabric 1012 can specify the IDs and types of cables terminating at each location, as well as the physical connections of network cables 1008 to switch ports in the networking infrastructure 1010, and as part of a set of network cables at each location in the prefabricated factory 1002 (for example, a set of network cables 1006A-1006N and a set of network cables 1036A-1036N). The information representing the static network topology may be stored in a data store accessible by the network service 1024.

[0113] As shown in Figures 10A and 10B, the prefabricated factory 1002 may be configured to simultaneously support prefabricated region construction work for both prefabricated region 1004 and prefabricated region 1032. In some embodiments, the server racks 1004A-1004N of prefabricated region 1004 may be installed in a different location than the server racks 1034A-1034N of prefabricated region 1032. In some embodiments, the server racks 1034A-1034N of prefabricated region 1032 may be installed in the same location where the server racks 1004A-1004N are used after they have been configured for delivery to the target site and removed from the prefabricated factory 1002. In this case, one set of network cables 1036A to 1036N may be the same as the set of network cables 1006A to 1006N used to connect server racks 1004A to 1004N to the static network fabric 1012, but the connection to server racks 1034A to 1034N will be made according to the connection plan corresponding to the prefabricated region 1032.

[0114] As described above with respect to Figure 7, the prefabricated factory 1002 can support modifications such as updating physical resources during the construction of the prefabricated region. The static network fabric 1012 can support the introduction of additional server racks to the prefabricated region 1004 and / or prefabricated region 1032 in accordance with the update construction request. When additional computing devices are added to the prefabricated region, the network service 1024 can generate an update connectivity plan that has instructions to connect the additional computing devices to the static network fabric 1012.

[0115] Figure 11 shows an exemplary method 1100 for generating a connectivity plan for connecting multiple computing devices (e.g., server rack 1004A in Figure 10) to a set of networking cables (e.g., a set of network cables 1006A in Figure 10) in a static network fabric (e.g., static network fabric 1012 in Figure 10) in a prefabricated factory (e.g., prefabricated factory 1002 in Figure 10), according to at least one embodiment. Method 1100 may be performed by one or more computing devices of a CSP hosting a prefabricated service (e.g., prefabricated service 1020 in Figure 10) which includes a network service (e.g., network service 1024).

[0116] Method 1100 begins in block 1102, enabling the network service to receive a physical build request. A physical build request can specify multiple computing devices connected to the static network fabric of a data center (e.g., prefabricated factory 1002). The physical build request may also be an example of a physical build request generated by the management service at the beginning of a prefabricated region build operation, as described above with respect to Method 700 in Figure 7. The network service may be configured to receive physical build requests from the management service.

[0117] In block 1104, the network service can determine the configuration of multiple computing devices. This configuration can specify network connections between multiple computing devices. For example, the multiple computing devices may be server devices on a server rack, each connected to a port on a top-of-rack switch. Thus, this configuration can identify the server devices, the corresponding ports on the top-of-rack switch to which the server devices are connected, and the network configuration associated with the connections. In some embodiments, determining the configuration of computing devices may include determining the placement of network connections for the computing devices to the top-of-rack switch or other networking devices. In some embodiments, determining the configuration of computing devices may include retrieving configuration parameters from a datastore. The configuration parameters may include information identifying the connection between each server device and a specific port on the networking device to which it is connected.

[0118] In block 1106, the network service can determine the static network fabric topology of a data center's static network fabric. The static network topology may define the network connections between one or more networking devices (e.g., leaf switches, spine switches, aggregate switches, core switches, etc.) of the network infrastructure of the static network fabric of a prefabricated factory. For example, the static network topology may identify the ports and devices to which each networking device is connected in the static network fabric. The static network topology may also specify one or more cable termination connectors at a location in the prefabricated factory. The network service can determine the above configuration and static network topology in response to receiving a physical construction request. In some embodiments, determining the static network fabric topology may include obtaining a predetermined topology of the data center's static network fabric from a data store. In some embodiments, the static network topology may correspond to a Clos network.

[0119] In block 1108, the network service can use the above configuration and the topology of the static network fabric to generate a connectivity plan for connecting a set of networking cables of the static network fabric to computing devices. The set of networking cables may be determined from networking cables of the static network fabric (e.g., network cable 1008) that are configured to terminate at a location in the data center. Terminating at a location may include having cable termination connectors at the ends of the network cables that can be connected to computing devices. The location may correspond to a location in a prefabricated factory where computing devices may be placed for deployment to support prefabricated region construction work. The connectivity plan may include instructions that can be used (e.g., by an operator) to configure the regional network by connecting each networking cable of the set of networking cables to the corresponding networking port of the networking device on the computing device.

[0120] In some embodiments, a network service can determine an additional configuration for an additional computing device connected to a second networking device. The additional computing device and the second networking device may be new server racks installed in a prefabricated factory to support modifications to the prefabricated region under construction. The additional configuration may be similar to the above configuration, or it may specify network connectivity between the additional computing device and the second networking device. Subsequently, using the computing device configuration, the additional configuration for the additional computing device, and the static network topology, the network service can generate an update connectivity plan that has instructions for connecting the additional computing device to a static network fabric to constitute an update region network including the installed computing devices in the prefabricated region.

[0121] Exemplary service-based infrastructure architecture As mentioned above, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the internet). In the IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, the IaaS provider may also supply a variety of services associated with these infrastructure components (exemplary services include billing software, monitoring software, logging software, load balancing software, and clustering software, etc.). Therefore, since these services can be policy-driven, IaaS users can maintain application availability and performance by implementing policies that promote load balancing.

[0122] In some cases, IaaS customers can access resources and services over a wide area network (WAN), such as the internet, and use the cloud provider's services to install other elements of their application stack. For example, a user can log into the IaaS platform and create virtual machines (VMs), install operating systems (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and even install enterprise software on those VMs. The customer can then use the provider's services to perform various functions, such as distributing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0123] In most cases, cloud computing models may require the participation of a cloud provider. This cloud provider may, but does not have to be, a third-party service specializing in providing IaaS (e.g., granting, leasing, or selling). Alternatively, an entity could deploy a private cloud and become its own infrastructure service provider.

[0124] In some cases, IaaS deployment is the process of placing a new application or a new version of an application on a prepared application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is often managed by the cloud provider under the hypervisor layer (e.g., servers, storage, network hardware, and virtualization). Therefore, the customer may be responsible for handling (OS), middleware, and / or application deployment (e.g., on self-service virtual machines that can be spun up on demand).

[0125] In some cases, IaaS provisioning represents acquiring the computers or virtual hosts to be used, and may even represent installing the necessary libraries or services for them. In most cases, a deployment does not include provisioning, and provisioning may need to be performed first.

[0126] In some cases, IaaS provisioning presents two distinct challenges. Firstly, there is the initial challenge of provisioning the initial set of infrastructure before anything is operational. Secondly, there is the challenge of evolving the existing infrastructure after all provisioning is complete (e.g., adding new services, modifying services, removing services, etc.). In some cases, these two challenges can be addressed by enabling the declarative definition of infrastructure configuration. In other words, the infrastructure (e.g., the required components and the way those components interact) can be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., resource dependencies and how each resource works together) can be described declaratively. In some cases, once the topology is defined, a workflow can be generated to form and / or manage the various components described in the configuration files.

[0127] In some examples, infrastructure can consist of many interconnected elements. For instance, there may be one or more virtual private clouds (VPCs), also known as core networks (e.g., potential on-demand pools of configurable and / or shared computing resources). In some examples, there may also be one or more inbound / outbound traffic group rules provisioning that define how inbound and / or outbound network traffic is configured and one or more virtual machines (VMs). Other infrastructure elements such as load balancers and databases may also be provisioned. As there is a demand for and / or addition of more infrastructure elements, the infrastructure can gradually evolve.

[0128] In some cases, the adoption of sequential deployment techniques can enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the techniques described can enable infrastructure management within these environments. In some examples, a service team may write code that is desirable to be deployed to one or more (but often many) different generation environments (e.g., geographically diverse locations, sometimes even worldwide). However, in some examples, the infrastructure to which the code is deployed may need to be configured first. In some cases, manual provisioning, the use of provisioning tools for provisioning resources, and / or the use of deployment tools for deploying code after infrastructure provisioning are also possible.

[0129] Figure 12 is a block diagram 1200 showing an exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1202 may be communicated with a secure host tenancy 1204 which may include a virtual cloud network (VCN) 1206 and a secure host subnet 1208. In some examples, the service operator 1202 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhone®, mobile phones, iPad®, computing tablets, personal digital assistive devices (PDAs)) or wearable devices (e.g., Google Glass® head-mounted displays) that run software such as Microsoft Windows Mobile® and / or a variety of mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, and are capable of using the Internet, email, short message service (SMS), BlackBerry®, or other communication protocols. Alternatively, a general-purpose personal computer (PC) can serve as the client computing device, including, for example, PCs and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. A workstation computer running any of the various commercially available UNIX® or UNIX-like operating systems can also serve as the client computing device, including, but is not limited to, various GNU / Linux operating systems such as Google Chrome OS.Alternatively or additionally, the client computing device may be any other electronic device, such as a thin client computer that can communicate via a network that can access the VCN1206 and / or the Internet, an Internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without a Kinect® gesture input device), and / or a personal messaging device.

[0130] VCN1206 may comprise an LPG1210 that can communicate with SSH VCN1212 via a local peering gateway (LPG) 1210 included in Secure Shell (SSH) VCN1212. SSH VCN1212 may comprise an SSH subnet 1214, and SSH VCN1212 may communicate with control plane VCN1216 via an LPG1210 included in control plane VCN1216. Furthermore, SSH VCN1212 may communicate with data plane VCN1218 via LPG1210. Control plane VCN1216 and data plane VCN1218 may be included in a service tenancy 1219 owned and / or operated by an IaaS provider.

[0131] The control plane VCN 1216 may comprise a control plane buffer zone (DMZ) layer 1220 that functions as a perimeter network (for example, part of the corporate network between the corporate intranet and the external network). DMZ-based servers can be limited in responsibility and help to mitigate breaches. The DMZ layer 1220 may also comprise a control plane application layer 1224 that may include one or more load balancer (LB) subnets 1222, an application subnet 1226, and a control plane data layer 1228 that may include database (DB) subnets 1230 (for example, a front-end DB subnet and / or a back-end DB subnet). The LB subnet 1222 included in the control plane DMZ layer 1220 is connected to the application subnet 1226 included in the control plane application layer 1224 and the Internet gateway 1234 which may be included in the control plane VCN 1216. The application subnet 1226 may be connected to the DB subnet 1230, the service gateway 1236, and the network address translation (NAT) gateway 1238 included in the control plane data layer 1228. The control plane VCN 1216 may include the service gateway 1236 and the NAT gateway 1238.

[0132] The control plane VCN 1216 may comprise a data plane mirror application layer 1240 that may include an application subnet 1226. The application subnet 1226 included in the data plane mirror application layer 1240 may comprise a virtual network interface controller (VNIC) 1242 that can run a compute instance 1244. The compute instance 1244 may communicate with the application subnet 1226 of the data plane mirror application layer 1240 to an application subnet 1226 that may be included in the data plane application layer 1246.

[0133] The data plane VCN1218 may comprise a data plane application layer 1246, a data plane DMZ layer 1248, and a data plane data layer 1250. The data plane DMZ layer 1248 may comprise an LB subnet 1222 that can be connected to the application subnet 1226 of the data plane application layer 1246 and the internet gateway 1234 of the data plane VCN1218. The application subnet 1226 may be connected to the service gateway 1236 and the NAT gateway 1238 of the data plane VCN1218. The data plane data layer 1250 may comprise a DB subnet 1230 that can be connected to the application subnet 1226 of the data plane application layer 1246.

[0134] The Internet gateway 1234 of the control plane VCN1216 and data plane VCN1218 can be connected to a metadata management service 1252, which can be connected to the public internet 1254. The public internet 1254 can be connected to the NAT gateway 1238 of the control plane VCN1216 and data plane VCN1218. The service gateway 1236 of the control plane VCN1216 and data plane VCN1218 can be connected to a cloud service 1256.

[0135] In some examples, a service gateway 1236 of the control plane VCN1216 or data plane VCN1218 can make application programming interface (API) calls to a cloud service 1256 without going through the public internet 1254. API calls from the service gateway 1236 to the cloud service 1256 can be unidirectional. The service gateway 1236 can make API calls to the cloud service 1256, and the cloud service 1256 can send the requested data to the service gateway 1236. However, the cloud service 1256 does not have to initiate an API call to the service gateway 1236.

[0136] In some examples, the secure host tenancy 1204 may be directly connected to the service tenancy 1219, or otherwise isolated. The secure host subnet 1208 may communicate with the SSH subnet 1214 via the LPG 1210, which may enable two-way communication through a system that is otherwise isolated. By connecting the secure host subnet 1208 to the SSH subnet 1214, the secure host subnet 1208 becomes able to access other entities within the service tenancy 1219.

[0137] The control plane VCN1216 may allow users of service tenancy 1219 to configure or provision desired resources. Desired resources provisioned in the control plane VCN1216 may be deployed or used in the data plane VCN1218. In some examples, the control plane VCN1216 can be isolated from the data plane VCN1218, and the data plane mirror application layer 1240 of the control plane VCN1216 can communicate with the data plane application layer 1246 of the data plane VCN1218 via a VNIC 1242 which may be included in the data plane mirror application layer 1240 and the data plane application layer 1246.

[0138] In some examples, a system user, or customer, can make a request (for example, create, read, update, or erase an operation (CRUD)) through the public internet 1254, and the public internet 1254 can send the request to the metadata management service 1252. The metadata management service 1252 can send the request to the control plane VCN 1216 through the internet gateway 1234. The request may be received by the LB subnet 1222, which is included in the control plane DMZ layer 1220. The LB subnet 1222 may determine that the request is valid, and in response to this determination, the LB subnet 1222 may send the request to the application subnet 1226, which is included in the control plane application layer 1224. If the request is validated and a call to the public internet 1254 is required, the call to the public internet 1254 may be sent to a NAT gateway 1238, which can make a call to the public internet 1254. Metadata that is deemed desirable to be stored by the request may be stored in the DB subnet 1230.

[0139] In some examples, the data plane mirror application layer 1240 may facilitate direct communication between the control plane VCN 1216 and the data plane VCN 1218. For example, it may be desirable that configuration changes, updates, or other preferred modifications be applied to resources contained in the data plane VCN 1218. The control plane VCN 1216 can perform configuration changes, updates, or other preferred modifications to resources by communicating directly with the resources contained in the data plane VCN 1218 via VNIC 1242.

[0140] In some embodiments, the control plane VCN1216 and the data plane VCN1218 may be included in the service tenancy 1219. In this case, the system user, i.e., the customer, does not have to own either the control plane VCN1216 or the data plane VCN1218, or does not have either of them running. Alternatively, the IaaS provider may own both the control plane VCN1216 and the data plane VCN1218, or have both running, and both may be included in the service tenancy 1219. This embodiment may enable network isolation that can prevent interaction between the user, i.e., the customer, and other users, i.e., other customers' resources. Furthermore, this embodiment may enable private storage of databases by the system user, i.e., the customer, without having to rely on the public internet 1254, which may not have the desired level of threat prevention for storage.

[0141] In another embodiment, the LB subnet 1222 included in the control plane VCN 1216 can be configured to receive signals from the service gateway 1236. In this embodiment, the control plane VCN 1216 and the data plane VCN 1218 may be configured to be invoked by the IaaS provider's customer without calling the public internet 1254. The IaaS provider's customer is likely to prefer this embodiment because the database used by the customer can be stored in a service tenancy 1219 that is controlled by the IaaS provider and can be isolated from the public internet 1254.

[0142] Figure 13 is a block diagram 1300 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1302 (for example, service operator 1202 in Figure 12) may be connected to a secure host tenancy 1304 (for example, secure host tenancy 1204 in Figure 12), which may include a virtual cloud network (VCN) 1306 (for example, VCN1206 in Figure 12) and a secure host subnet 1308 (for example, secure host subnet 1208 in Figure 12). VCN 1306 may comprise an LPG 1310 (for example, LPG1210 in Figure 12), which may be connected to an SSH VCN 1312 via a local peering gateway (LPG) 1210 included in the Secure Shell (SSH) VCN 1312 (for example, SSH VCN1212 in Figure 12). SSH VCN1312 may comprise SSH subnet 1314 (for example, SSH subnet 1214 in Figure 12), and SSH VCN1312 may be communicated with control plane VCN1316 via LPG1310 included in control plane VCN1316 (for example, control plane VCN1216 in Figure 12). Control plane VCN1316 may be included in service tenancy 1319 (for example, service tenancy 1219 in Figure 12), and data plane VCN1318 (for example, data plane VCN1218 in Figure 12) may be included in customer tenancy 1321, which may be owned or operated by the system's users, i.e., customers.

[0143] The control plane VCN1316 may comprise a control plane DMZ layer 1320 (for example, the control plane DMZ layer 1220 in Figure 12) which may include an LB subnet 1322 (for example, the LB subnet 1222 in Figure 12), a control plane application layer 1324 (for example, the control plane application layer 1224 in Figure 12) which may include an application subnet 1326 (for example, the application subnet 1226 in Figure 12), and a control plane data layer 1328 (for example, the control plane data layer 1228 in Figure 12) which may include a DB subnet 1330 (similar to the database (DB) subnet 1230 in Figure 12). The LB subnet 1322 included in the control plane DMZ layer 1320 is connected to the application subnet 1326 included in the control plane application layer 1324 and the Internet gateway 1334 (for example, Internet gateway 1234 in Figure 12) which may be included in the control plane VCN 1316. The application subnet 1326 may be connected to the DB subnet 1330 included in the control plane data layer 1328, the service gateway 1336 (for example, the service gateway in Figure 12), and the Network Address Translation (NAT) gateway 1338 (for example, NAT gateway 1238 in Figure 12). The control plane VCN 1316 may comprise the service gateway 1336 and the NAT gateway 1338.

[0144] The control plane VCN 1316 may comprise a data plane mirror application layer 1340 (for example, the data plane mirror application layer 1240 in Figure 12) which may include an application subnet 1326. The application subnet 1326 included in the data plane mirror application layer 1340 may comprise a virtual network interface controller (VNIC) 1342 (for example, the VNIC in Figure 12) which can run a compute instance 1344 (for example, similar to the compute instance 1244 in Figure 12). The compute instance 1344 may facilitate communication between the application subnet 1326 of the data plane mirror application layer 1340 and the application subnet 1326 included in the data plane mirror application layer 1346 (for example, the data plane application layer 1246 in Figure 12) via the VNIC 1342 included in the data plane mirror application layer 1340 and the VNIC 1342 included in the data plane application layer 1346 (for example, the data plane application layer 1246 in Figure 12).

[0145] The Internet gateway 1334 included in the control plane VCN 1316 can be connected to a metadata management service 1352 (for example, the metadata management service 1252 in Figure 12), which can be connected to the public internet 1354 (for example, the public internet 1254 in Figure 12). The public internet 1354 can be connected to a NAT gateway 1338 included in the control plane VCN 1316. The service gateway 1336 included in the control plane VCN 1316 can be connected to a cloud service 1356 (for example, the cloud service 1256 in Figure 12).

[0146] In some examples, the data plane VCN1318 may be included in customer tenancy 1321. In this case, the IaaS provider may provide a control plane VCN1316 for each customer, or the IaaS provider may configure a unique compute instance 1344 included in service tenancy 1319 for each customer. Each compute instance 1344 may enable communication between the control plane VCN1316 included in service tenancy 1319 and the data plane VCN1318 included in customer tenancy 1321. Compute instance 1344 may enable the deployment or use of resources provisioned in the control plane VCN1316 included in service tenancy 1319 in the data plane VCN1318 included in customer tenancy 1321.

[0147] In another example, the IaaS provider's customer may have a database located in customer tenancy 1321. In this example, the control plane VCN 1316 may comprise a data plane mirror application layer 1340, which may include application subnet 1326. The data plane mirror application layer 1340 may reside in data plane VCN 1318, but may not reside in data plane VCN 1318. That is, the data plane mirror application layer 1340 may be accessible to customer tenancy 1321, but may not reside in data plane VCN 1318, nor may it be owned or operated by the IaaS provider's customer. The data plane mirror application layer 1340 may be configured to make calls to data plane VCN 1318, but may not be configured to make calls to any entities included in control plane VCN 1316. The customer is expected to want to deploy or use resources in the data plane VCN1318 that are provisioned in the control plane VCN1316, and the data plane mirror application layer 1340 can facilitate the deployment or other use of the resources desired by the customer.

[0148] In some embodiments, a customer of the IaaS provider can apply filters to the data plane VCN1318. In this embodiment, the customer can determine what the data plane VCN1318 can access, and the customer may also restrict access from the data plane VCN1318 to the public internet 1354. The IaaS provider may not be able to apply filters or control the data plane VCN1318's access to any external network or database. The application of filters and controls by the customer to the data plane VCN1318 included in the customer tenancy 1321 may help isolate the data plane VCN1318 from other customers and the public internet 1354.

[0149] In some embodiments, a cloud service 1356 can access services that could not exist on the public internet 1354, the control plane VCN 1316, or the data plane VCN 1318 via a call from the service gateway 1336. The connection between the cloud service 1356 and the control plane VCN 1316 or the data plane VCN 1318 does not have to be live or continuous. The cloud service 1356 may reside on different networks owned or operated by the IaaS provider. The cloud service 1356 may be configured to receive calls from the service gateway 1336 and may be configured not to receive calls from the public internet 1354. Some cloud services 1356 may be isolated from other cloud services 1356, and the control plane VCN 1316 may be isolated from cloud services 1356 that could not be in the same region as the control plane VCN 1316. For example, the control plane VCN1316 may be located in "Region 1," and the cloud service "Deployment 12" may be located in both Region 1 and "Region 2." If a call to Deployment 12 is made by a service gateway 1336 included in the control plane VCN1316 located in Region 1, the call may be sent to Deployment 12 in Region 1. In this example, the control plane VCN1316 or Deployment 12 in Region 1 does not need to be communication-coupled to or in communication with Deployment 12 in Region 2.

[0150] Figure 14 is a block diagram 1400 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1402 (for example, service operator 1202 in Figure 12) may be connected to a secure host tenancy 1404 (for example, secure host tenancy 1204 in Figure 12), which may include a virtual cloud network (VCN) 1406 (for example, VCN1206 in Figure 12) and a secure host subnet 1408 (for example, secure host subnet 1208 in Figure 12). VCN 1406 may comprise an LPG 1410 (for example, LPG1210 in Figure 12), which may be connected to SSH VCN 1412 via an LPG 1410 included in SSH VCN 1412 (for example, SSH VCN1212 in Figure 12). SSH VCN1412 may comprise SSH subnet 1414 (for example, SSH subnet 1214 in Figure 12), and SSH VCN1412 may be connected to control plane VCN1416 via LPG1410 included in control plane VCN1416 (for example, control plane VCN1216 in Figure 12), and may be connected to data plane VCN1418 via LPG1410 included in data plane VCN1418 (for example, data plane VCN1218 in Figure 12). Control plane VCN1416 and data plane VCN1418 may be included in service tenancy 1419 (for example, service tenancy 1219 in Figure 12).

[0151] The control plane VCN1416 may comprise a control plane DMZ layer 1420 (for example, the control plane DMZ layer 1220 in Figure 12) which may include a load balancer (LB) subnet 1422 (for example, the LB subnet 1222 in Figure 12), a control plane application layer 1424 (for example, the control plane application layer 1224 in Figure 12) which may include an application subnet 1426 (for example, similar to the application subnet 1226 in Figure 12), and a control plane data layer 1428 (for example, the control plane data layer 1228 in Figure 12) which may include a DB subnet 1430. The LB subnet 1422 included in the control plane DMZ layer 1420 is connected to the application subnet 1426 included in the control plane application layer 1424 and the Internet gateway 1434 (for example, Internet gateway 1234 in Figure 12) which may be included in the control plane VCN 1416. The application subnet 1426 may be connected to the DB subnet 1430 included in the control plane data layer 1428, the service gateway 1436 (for example, the service gateway in Figure 12), and the Network Address Translation (NAT) gateway 1438 (for example, NAT gateway 1238 in Figure 12). The control plane VCN 1416 may comprise the service gateway 1436 and the NAT gateway 1438.

[0152] The data plane VCN1418 may comprise a data plane application layer 1446 (for example, the data plane application layer 1246 in Figure 12), a data plane DMZ layer 1448 (for example, the data plane DMZ layer 1248 in Figure 12), and a data plane data layer 1450 (for example, the data plane data layer 1250 in Figure 12). The data plane DMZ layer 1448 may comprise an LB subnet 1422 that can be connected to the trusted application subnet 1460 and the untrusted application subnet 1462 of the data plane application layer 1446, as well as the internet gateway 1434 included in the data plane VCN1418. The trusted application subnet 1460 may be connected to the service gateway 1436 included in the data plane VCN1418, the NAT gateway 1438 included in the data plane VCN1418, and the DB subnet 1430 included in the data plane data layer 1450. The non-trusted application subnet 1462 can be communicated to the service gateway 1436 included in the data plane VCN 1418 and the DB subnet 1430 included in the data plane data layer 1450. The data plane data layer 1450 may comprise the DB subnet 1430, which can be communicated to the service gateway 1436 included in the data plane VCN 1418.

[0153] The untrusted application subnet 1462 may comprise one or more primary VNICs 1464(1) to 1464(N) that can be connected to tenant virtual machines (VMs) 1466(1) to 1466(N). Each tenant VM 1466(1) to 1466(N) may be connected to each application subnet 1467(1) to 1467(N) that may be included in each container output VCN 1468(1) to 1468(N) that may be included in each customer tenancy 1470(1) to 1470(N). Each secondary VNIC 1472(1) to 1472(N) may facilitate communication between the untrusted application subnet 1462 included in the data plane VCN 1418 and the application subnets included in the container output VCNs 1468(1) to 1468(N). Each container output VCN 1468(1) to 1468(N) may be equipped with a NAT gateway 1438 that can communicate with the public internet 1454 (for example, the public internet 1254 in Figure 12).

[0154] The Internet gateway 1434 included in the control plane VCN1416 and data plane VCN1418 can be connected to a metadata management service 1452 (for example, the metadata management system 1252 in Figure 12), which can be connected to the public internet 1454. The public internet 1454 can be connected to a NAT gateway 1438 included in the control plane VCN1416 and data plane VCN1418. The service gateway 1436 included in the control plane VCN1416 and data plane VCN1418 can be connected to a cloud service 1456.

[0155] In some embodiments, the data plane VCN 1418 may be integrated with a customer tenancy 1470. This integration may be useful or desirable to the IaaS provider's customers, for example, if they may want support for executing code. The customer may provide code to be executed, which may be destructive, communicate with other customer resources, or have undesirable effects. In response, the IaaS provider may determine whether to execute the code provided to the IaaS provider by the customer.

[0156] In some examples, an IaaS provider's customer may grant temporary network access to the IaaS provider and request functionality to be granted to the data plane application layer 1446. The code that performs this functionality may run in VMs 1466(1) to 1466(N), and may not be configured to run elsewhere on the data plane VCN 1418. Each VM 1466(1) to 1466(N) may be connected to a single customer tenancy 1470. Each container 1471(1) to 1471(N) contained within VMs 1466(1) to 1466(N) may be configured to run code. In this case, a double isolation may exist (for example, containers 1471(1)-1471(N) executing the code may be contained within VMs 1466(1)-1466(N) that are at least in the non-trusted app subnet 1462), which may help prevent damage to the IaaS provider's network or a different customer's network by erroneous or undesirable code. Containers 1471(1)-1471(N) may be communication-coupled to customer tenancy 1470 and may be configured to send or receive data to or from customer tenancy 1470. Containers 1471(1)-1471(N) may also be configured not to send or receive data to or from any other entity in the data plane VCN 1418. Upon completion of code execution, the IaaS provider may disable or discard containers 1471(1)-1471(N).

[0157] In some embodiments, the trusted application subnet 1460 may be configured to execute code owned or operated by the IaaS provider. In this embodiment, the trusted application subnet 1460 may be connected to the DB subnet 1430 and may be configured to perform CRUD operations in the DB subnet 1430. The non-trusted application subnet 1462 may be connected to the DB subnet 1430, but in this embodiment, may be configured to perform read operations in the DB subnet 1430. Containers 1471(1) to 1471(N) contained in each customer's VMs 1466(1) to 1466(N) and capable of executing code from the customer may not be connected to the DB subnet 1430.

[0158] In other embodiments, the control plane VCN1416 and the data plane VCN1418 do not have to be directly connected. In this embodiment, direct communication between the control plane VCN1416 and the data plane VCN1418 is not required. However, communication can be performed indirectly by at least one method. An LPG1410 that can facilitate communication between the control plane VCN1416 and the data plane VCN1418 may be established by the IaaS provider. In another example, the control plane VCN1416 or the data plane VCN1418 can make a call to the cloud service 1456 via the service gateway 1436. For example, a call from the control plane VCN1416 to the cloud service 1456 may include a request for a service that can communicate with the data plane VCN1418.

[0159] Figure 15 is a block diagram 1500 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1502 (for example, service operator 1202 in Figure 12) may be connected to a secure host tenancy 1504 (for example, secure host tenancy 1204 in Figure 12), which may include a virtual cloud network (VCN) 1506 (for example, VCN1206 in Figure 12) and a secure host subnet 1508 (for example, secure host subnet 1208 in Figure 12). VCN 1506 may comprise an LPG 1510 (for example, LPG1210 in Figure 12), which may be connected to SSH VCN 1512 (for example, SSH VCN1212 in Figure 12) via an LPG 1510 included in SSH VCN 1512. SSH VCN1512 may comprise an SSH subnet 1514 (for example, SSH subnet 1214 in Figure 12), and SSH VCN1512 may be connected to control plane VCN1516 via an LPG 1510 included in control plane VCN1516 (for example, control plane VCN1216 in Figure 12), and may be connected to data plane VCN1518 via an LPG 1510 included in data plane VCN1518 (for example, data plane VCN1218 in Figure 12). Control plane VCN1516 and data plane VCN1518 may be included in service tenancy 1519 (for example, service tenancy 1219 in Figure 12).

[0160] The control plane VCN1516 may comprise a control plane DMZ layer 1520 (for example, the control plane DMZ layer 1220 in Figure 12) which may include an LB subnet 1522 (for example, the LB subnet 1222 in Figure 12), a control plane application layer 1524 (for example, the control plane application layer 1224 in Figure 12) which may include an application subnet 1526 (for example, the application subnet 1226 in Figure 12), and a control plane data layer 1528 (for example, the control plane data layer 1228 in Figure 12) which may include a DB subnet 1530 (for example, the DB subnet 1430 in Figure 14). The LB subnet 1522 included in the control plane DMZ layer 1520 is connected to the application subnet 1526 included in the control plane application layer 1524 and the Internet gateway 1534 (for example, Internet gateway 1234 in Figure 12) which may be included in the control plane VCN 1516. The application subnet 1526 may be connected to the DB subnet 1530 included in the control plane data layer 1528, the service gateway 1536 (for example, the service gateway in Figure 12), and the Network Address Translation (NAT) gateway 1538 (for example, NAT gateway 1238 in Figure 12). The control plane VCN 1516 may comprise the service gateway 1536 and the NAT gateway 1538.

[0161] The data plane VCN 1518 may comprise a data plane application layer 1546 (for example, the data plane application layer 1246 in Figure 12), a data plane DMZ layer 1548 (for example, the data plane DMZ layer 1248 in Figure 12), and a data plane data layer 1550 (for example, the data plane data layer 1250 in Figure 12). The data plane DMZ layer 1548 may comprise a trusted application subnet 1560 (for example, the trusted application subnet 1460 in Figure 14) and an untrusted application subnet 1562 (for example, the untrusted application subnet 1462 in Figure 14) of the data plane application layer 1546, as well as an LB subnet 1522 that can be connected to an internet gateway 1534 included in the data plane VCN 1518. Trusted application subnet 1560 can be connected to service gateway 1536 included in data plane VCN 1518, NAT gateway 1538 included in data plane VCN 1518, and DB subnet 1530 included in data plane data layer 1550. Non-trusted application subnet 1562 can be connected to service gateway 1536 included in data plane VCN 1518 and DB subnet 1530 included in data plane data layer 1550. Data plane data layer 1550 may include DB subnet 1530 that can be connected to service gateway 1536 included in data plane VCN 1518.

[0162] The untrusted application subnet 1562 may comprise primary VNICs 1564(1) to 1564(N) that can be connected to tenant virtual machines (VMs) 1566(1) to 1566(N) residing within the untrusted application subnet 1562. Each tenant VM 1566(1) to 1566(N) can execute code in each of the containers 1567(1) to 1567(N) and can be connected to an application subnet 1526 that may be included in the data plane application layer 1546 that may be included in the container output VCN 1568. The secondary VNICs 1572(1) to 1572(N) can facilitate communication between the untrusted application subnet 1562 included in the data plane VCN 1518 and the application subnet included in the container output VCN 1568. The container output VCN may include a NAT gateway 1538 that can be connected to the public internet 1554 (for example, the public internet 1254 in Figure 12).

[0163] The Internet gateway 1534 included in the control plane VCN1516 and data plane VCN1518 can be connected to a metadata management service 1552 (for example, the metadata management system 1252 in Figure 12), which can be connected to the public internet 1554. The public internet 1554 can be connected to a NAT gateway 1538 included in the control plane VCN1516 and data plane VCN1518. The service gateway 1536 included in the control plane VCN1516 and data plane VCN1518 can be connected to a cloud service 1556.

[0164] In some examples, the architecture pattern shown in block diagram 1500 of Figure 15 is considered an exception to the architecture pattern shown in block diagram 1400 of Figure 14, and is considered desirable for the IaaS provider's customers when the IaaS provider cannot communicate directly with the customer (for example, in an unconnected region). Each customer has containers 1567(1) to 1567(N) contained within VMs 1566(1) to 1566(N), each of which is accessible to the customer in real time. Containers 1567(1) to 1567(N) may be configured to make calls to each of the secondary VNICs 1572(1) to 1572(N) contained within the application subnet 1526 of the data plane application layer 1546, which may be contained within the container output VCN 1568. Secondary VNICs 1572(1) to 1572(N) can send calls to the NAT gateway 1538, which may then send calls to the public internet 1554. In this example, the customer-accessible containers 1567(1) to 1567(N) can be isolated from the control plane VCN 1516, as well as from other entities included in the data plane VCN 1518. Furthermore, the containers 1567(1) to 1567(N) can also be isolated from other customer resources.

[0165] In another example, a customer can use containers 1567(1) to 1567(N) to invoke cloud service 1556. In this example, a customer can execute code in containers 1567(1) to 1567(N) to request services from cloud service 1556. Containers 1567(1) to 1567(N) can send this request to secondary VNICs 1572(1) to 1572(N), which can send the request to the NAT gateway, which can send the request to the public internet 1554. The public internet 1554 can send the request to LB subnet 1522, which is included in control plane VCN 1516, via internet gateway 1534. In response to the determination that the request is valid, the LB subnet can send the request to the application subnet 1526, and the application subnet 1526 can send the request to the cloud service 1556 via the service gateway 1536.

[0166] Naturally, the IaaS architectures 1200, 1300, 1400, and 1500 shown in the drawings may have components other than those shown. Furthermore, the embodiments shown in the drawings are merely examples of cloud infrastructure systems that may encompass one embodiment of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown, may combine two or more components, or may have different configurations or arrangements of components.

[0167] In one embodiment, the IaaS system described herein may include a set of applications, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. Oracle Cloud Infrastructure (OCI), offered by the assignee, is an example of such an IaaS system.

[0168] Figure 16 shows an exemplary computer system 1600 in which various embodiments can be realized. System 1600 may be used to realize any of the computer systems described above. As shown in the drawing, computer system 1600 comprises a processing unit 1604 that communicates with many peripheral subsystems via a bus subsystem 1602. Peripheral subsystems may include a processing acceleration unit 1606, an I / O subsystem 1608, a storage subsystem 1618, and a communication subsystem 1624. The storage subsystem 1618 comprises a tangible computer-readable storage medium 1622 and a system memory 1610.

[0169] The bus subsystem 1602 provides a mechanism for various components and subsystems of the computer system 1600 to communicate with each other as intended. While the bus subsystem 1602 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 1602 may be one of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus, using any of a variety of bus architectures. For example, such architectures may include industry standard architecture (ISA) buses, microchannel architecture (MCA) buses, extended ISA (EISA) buses, video electronics standards (VESA) local buses, and peripheral interconnect (PCI) buses, which can be implemented as mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0170] A processing unit 1604, which can be implemented as one or more integrated circuits (for example, conventional microprocessors or microcontrollers), controls the operation of the computer system 1600. The processing unit 1604 may include one or more processors. These processors may include single-core or multi-core processors. In one embodiment, the processing unit 1604 may be implemented as one or more independent processing units 1632 and / or 1634, each containing a single-core or multi-core processor. In another embodiment, the processing unit 1604 may be implemented as a quad-core processing unit formed by integrating two dual-core processors onto a single chip.

[0171] In various embodiments, the processing unit 1604 can execute a variety of programs in response to program code and maintain multiple concurrently running programs or processes. At any given time, some or all of the program code to be executed may reside in the processor 1604 and / or the storage subsystem 1618. With suitable programming, the processor 1604 can provide the various functions described above. The computer system 1600 may also include a processing acceleration unit 1606 which may include a digital signal processor (DSP), a dedicated processor, and / or similar.

[0172] The I / O subsystem 1608 may include user interface input devices and user interface output devices. User interface input devices may include pointing devices such as keyboards, mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include motion detection and / or gesture recognition devices such as Microsoft Kinect® motion sensors that enable user control and interaction with input devices such as Microsoft Xbox® 360 game controllers through a natural user interface using gestures and voice commands. User interface input devices may also include eye gesture recognition devices such as Google Glass® blink detectors that detect the user's eye activity (e.g., blinking during shooting and / or menu selection) and convert eye gestures into input to an input device (e.g., Google Glass®). Furthermore, the user interface input device may include a voice recognition detection device that enables user interaction with a voice recognition system (e.g., Siri® Navigator) via voice commands.

[0173] Furthermore, user interface input devices may include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphic tablets, as well as auditory / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. User interface input devices may also include medical imaging input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, and medical ultrasound devices. Furthermore, user interface input devices may also include audio input devices such as MIDI keyboards and digital musical instruments.

[0174] User interface output devices may include non-visual displays such as display subsystems, indicator lights, or audio output devices. Display subsystems may also include flat-panel devices such as cathode ray tubes (CRTs), liquid crystal displays (LCDs), or plasma displays, projection devices, touchscreens, etc. Generally, the use of the term “output device” is intended to include all conceivable types of devices and mechanisms for outputting information from the computer system 1600 to a user or another computer. For example, user interface output devices may include, but are not limited to, a variety of display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0175] The computer system 1600 may include a storage subsystem 1618 that provides a tangible temporary computer-readable storage medium for storing software and data constructs that provide the functionality of the embodiments described in this disclosure. The software may include programs, code, instructions, scripts, etc., that provide the above-described functionality when executed by one or more cores or processors of the processing unit 1604. The storage subsystem 1618 may also provide a repository for storing data used in accordance with this disclosure.

[0176] As shown in the example in Figure 16, the storage subsystem 1618 may comprise various components, including system memory 1610, a computer-readable storage medium 1622, and a computer-readable storage medium reader 1620. The system memory 1610 may store program instructions that can be loaded and executed by the processing unit 1604. The system memory 1610 may also store data used when instructions are executed and / or data generated when program instructions are executed. Various different types of programs may be loaded into the system memory 1610, including, but not limited to, client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), virtual machines, containers, and the like.

[0177] Furthermore, system memory 1610 may be configured to store the operating system 1616. Examples of operating systems 1616 include various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems, a variety of commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS. In one embodiment in which computer system 1600 runs one or more virtual machines, the virtual machines, along with their respective guest operating systems (GOS), may be loaded into system memory 1610 and executed by one or more processors or cores of processing unit 1604.

[0178] The system memory 1610 can have various configurations depending on the type of computer system 1600. For example, the system memory 1610 may be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM) or flash memory). It may also be provided with different types of RAM configurations, including static random access memory (SRAM), dynamic random access memory (DRAM), and others. In some embodiments, the system memory 1610 may include a basic input / output system (BIOS) that includes basic routines useful for communicating information between elements within the computer system 1600, such as during startup.

[0179] The computer-readable storage medium 1622 may represent a remote, local, fixed, and / or removable storage device, as well as a storage medium for temporarily and / or permanently containing and storing computer-readable information used by the computer system 1600 (including instructions executable by the processing unit 1604 of the computer system 1600).

[0180] The computer-readable storage medium 1622 may include, but is not limited to, any suitable medium known or used in the art (including storage and communication media), volatile and non-volatile, removable and non-removable media implemented in any method or technique for storing and / or transmitting information. This may include memory technologies such as RAM, ROM, electronically erasable programmable ROM (EEPROM), and flash memory; optical storage such as CD-ROM and digital multipurpose discs (DVDs); magnetic storage devices such as magnetic cassettes, magnetic tapes, and magnetic disk storage; or other tangible computer-readable storage media.

[0181] As an example, the computer-readable storage medium 1622 may include a hard disk drive that reads and writes to a non-removable non-volatile magnetic medium, a magnetic disk drive that reads and writes to a removable non-volatile magnetic disk, and an optical disk drive that reads and writes to a removable non-volatile optical disk such as a CD-ROM, DVD, Blu-ray® disc, or other optical medium. The computer-readable storage medium 1622 may include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, digital videotapes, etc. Furthermore, the computer-readable storage medium 1622 may include SSDs based on volatile memory such as flash memory-based SSDs, enterprise flash drives, semiconductor drives (SSDs) based on non-volatile memory such as semiconductor ROM, semiconductor RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. The disk drives and their respective associated computer-readable media may provide non-volatile storage for computer-readable instructions, data structures, program services, and other data to the computer system 1600.

[0182] Machine-readable instructions executable by one or more processors or cores of the processing unit 1604 may be stored in a non-temporary computer-readable storage medium. The non-temporary computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory devices and / or non-volatile storage devices. Examples of non-temporary computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard drives, floppy drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0183] The communication subsystem 1624 provides an interface to other computer systems and networks. The communication subsystem 1624 functions as an interface for sending and receiving data between the computer system 1600 and other systems. For example, the communication subsystem 1624 may enable the computer system 1600 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1624 may comprise radio voice and / or data networks (using, for example, cellular technology, 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), advanced data network technologies such as WiFi (IEEE 802.11 family standards), or other mobile communication technologies, or any combination thereof), a Global Positioning System (GPS) receiver component, and / or radio frequency (RF) transceiver components for accessing other components. In some embodiments, the communication subsystem 1624 may provide wired network connectivity (e.g., Ethernet) as an addition to or alternative to the radio interface.

[0184] In addition, in some embodiments, the communication subsystem 1624 is capable of receiving input communications in the form of structured and / or unstructured data feeds 1626, event streams 1628, event updates 1630, etc., on behalf of one or more users who may be using the computer system 1600.

[0185] For example, the communication subsystem 1624 may be configured to receive data feeds 1626 in real time from users of other communication services, such as social networks and / or web feeds like Twitter® feeds, Facebook® updates, RSS (Rich Site Summary) feeds, and / or real-time updates from one or more third-party sources.

[0186] Furthermore, the communication subsystem 1624 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1628 of real-time events and / or event updates 1630, which may be continuous or virtually infinite with no explicit end. Examples of applications that generate continuous data include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0187] Furthermore, the communication subsystem 1624 may be configured to output structured and / or unstructured data feeds 1626, event streams 1628, event updates 1630, etc., to one or more databases that can communicate with one or more streaming data source computers connected to the computer system 1600.

[0188] The computer system 1600 can be one of a variety of types, including portable handheld devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs, etc.), wearable devices (e.g., Google Glass® head-mounted displays, etc.), PCs, workstations, mainframes, kiosks, server racks, or any other data processing systems.

[0189] Due to the constantly changing nature of computers and networks, the description of the illustrated computer system 1600 is intended only as a specific example. Many other configurations are possible, whether with more or fewer components than the illustrated system. For example, the use of customized hardware and / or the implementation of specific elements in hardware, firmware, software (including applets), or combinations thereof are also possible. Furthermore, connections to other computing devices such as network input / output devices may be employed. Based on the disclosures and teachings contained herein, those skilled in the art will recognize other methods and / or ways of realizing various embodiments.

[0190] While specific embodiments have been described above, various improvements, modifications, alternative configurations, and equivalents are also included in the scope of this disclosure. The embodiments are not limited to operation within a particular data processing environment, but can freely operate within multiple data processing environments. Furthermore, while the embodiments have been described using a specific set of transactions and steps, it will be clear to those skilled in the art that the scope of this disclosure is not limited to the set of transactions and steps described. The various features and aspects of the embodiments described above may be used individually or in combination.

[0191] Furthermore, while embodiments have been described using specific combinations of hardware and software, it will be recognized that other combinations of hardware and software are also included in the scope of this disclosure. Embodiments may be implemented in hardware only, in software only, or by using a combination of these. The various processes described herein may be implemented on the same processor or on any combination of different processors. Thus, where a component or service is described as being configured to perform a certain operation, such configuration may be achieved, for example, by designing electronic circuitry to perform this operation, by programming programmable electronic circuitry (such as a microprocessor) to perform this operation, or by any combination thereof. Processes may be communicative using a variety of techniques, including but not limited to conventional techniques for inter-process communication. Also, different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0192] Therefore, this specification and the drawings are intended to be illustrative and not limiting in any way. However, it is clear that additions, differences, deletions, and other improvements and modifications can be made without departing from the broader idea and scope set forth in the claims. For this reason, specific embodiments of this disclosure have been described, but these are not intended to be limiting in any way. Various improvements and equivalents are included in the following claims.

[0193] In the context describing embodiments of the disclosure (in particular, in the context of the following claims), the terms “a,” “an,” and “the,” and similar reference subjects, shall be interpreted to encompass both singular and plural forms unless otherwise indicated herein or there is a clear contextual inconsistency. The terms “comprising,” “having,” “including,” and “containing” shall be interpreted as open-ended terms (i.e., “including, but not limited to,”) unless otherwise indicated herein. The term “connected” shall be interpreted as including, attaching, or integrally combining, in whole or in part, even if something is intervening. Unless otherwise indicated herein, descriptions of ranges of values ​​are intended merely as a concise way of referring individually to each distinct value contained within that range, and each distinct value is incorporated herein as if it were individually described herein. Unless otherwise indicated herein or there is a clear contextual inconsistency, all methods described herein may be performed in any preferred order. The use of any examples or illustrative expressions (e.g., "such as") described herein is intended solely to facilitate the understanding of the embodiments and, unless otherwise claimed, does not limit the scope of this disclosure. Nothing described herein shall be construed as indicating that any non-claimed element is essential to the implementation of this disclosure.

[0194] Unless otherwise specified, disjunctive expressions such as the phrase "at least one of X, Y, or Z" are intended to be understood in contexts in which they are commonly used to indicate that an item, term, etc., can be any one of X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Therefore, such disjunctive expressions are not intended, nor should they, to imply, that any embodiment requires the presence of at least one of X, at least one of Y, or at least one of Z.

[0195] This specification describes preferred embodiments of the Disclosure, including the best known modes for the execution of the Disclosure. Those skilled in the art will be able to see, by reading the above description, variations of these preferred embodiments. Those skilled in the art may adopt such variations as appropriate, and the Disclosure may be executed in a manner different from the specific description herein. Therefore, to the extent permitted by applicable law, the Disclosure includes all improvements and equivalents to the subject matter described in the claims appended herein. Furthermore, unless otherwise indicated herein, any combination of all conceivable variations of the elements described herein is included in the Disclosure.

[0196] All references cited herein, including publications, patent applications, and patents, are incorporated herein by reference to the same extent as their entire contents are incorporated herein by reference, with each reference being individually and specifically indicated.

[0197] While the above specification describes aspects of the disclosure with reference to specific embodiments, those skilled in the art will recognize that the disclosure is not limited thereto. The various features and aspects of the disclosure described above may be used individually or in combination. Furthermore, various modifications and equivalents include appropriate combinations of the features of the disclosure relevant to the embodiments. Moreover, the embodiments may be used in any number of environments and applications beyond those described herein, without departing from the broader idea and scope of this specification. Accordingly, this specification and the drawings should be considered illustrative and not limiting.

Claims

1. It is a system, The data center comprises a network fabric for the data center, including multiple network cables routed through the data center, The plurality of networking cables are characterized by a static network fabric topology, and one set of the plurality of networking cables is configured to terminate at a location within the data center. The aforementioned system, The data center further comprises a plurality of computing devices that can be placed in the aforementioned location, A system in which the plurality of computing devices are configured to form a regional network when they are connected to the set of networking cables in accordance with a connectivity plan generated at least in part based on the static network fabric topology and the configuration of the plurality of computing devices.

2. The static network fabric topology corresponds to a Close network, as described in claim 1.

3. The system according to claim 1 or 2, wherein the set of networking cables comprises cable termination connectors corresponding to at least one of multifiber push-on (MPO), multifiber pull-off, SFP (Small Form-factor Pluggable), SFP+, SFP28, QSFP (Quad Small Form-factor Pluggable), QSFP+, QSFP28, or RJ45.

4. The set of networking cables is a first set of networking cables, and the plurality of networking cables includes a second set of networking cables configured to terminate at a second location in the data center. The aforementioned system, The system according to any one of claims 1 to 3, further comprising a plurality of additional computing devices that can be located in the second location of the data center and are configured to form an update regional network when connected to the second set of networking cables.

5. The system according to claim 4, wherein the updated regional network is characterized by a second connectivity plan generated at least in part based on the second configuration of the static network fabric topology, the regional network, and the additional plurality of computing devices.

6. In network services, this involves receiving physical construction requests to connect multiple computing devices to a static network fabric in a data center, In response to the receipt of the aforementioned physical construction request, The network service determines the configuration of the plurality of computing devices, The network service determines the static network fabric topology of the static network fabric of the data center, The network service generates a connectivity plan for connecting a set of networking cables in the static network fabric to the plurality of computing devices, using the configuration and the static network fabric topology. Includes, The connection plan is a method that includes instructions which can be used to connect each networking cable of the set of networking cables to the corresponding networking ports of the networking devices of the plurality of computing devices to form a regional network.

7. The method according to claim 6, wherein the plurality of computing devices are communicated to a networking device, and determining the configuration of the plurality of computing devices includes determining the arrangement of network connections of the plurality of computing devices to the networking device.

8. The method according to claim 6, wherein determining the configuration of the plurality of computing devices includes obtaining configuration parameters from a data store that correspond to the arrangement of network connections between the plurality of computing devices and one or more networking devices on a rack.

9. The method according to any one of claims 6 to 8, wherein determining the static network fabric topology includes obtaining a predetermined topology of the static network fabric of the data center from a data store.

10. The method according to claim 9, wherein the predetermined topology of the static network fabric corresponds to a Close network.

11. The method according to any one of claims 6 to 10, wherein generating the connection plan includes determining the set of networking cables from a plurality of networking cables of the static network fabric which are configured to terminate at a location in the data center.

12. The aforementioned network service determines the additional configuration of a plurality of additional computing devices that are connected to the second networking device. The network service generates a second connectivity plan for connecting a second set of networking cables of the static network fabric to the additional computing devices, using the configuration, the additional configuration, and the static network fabric topology. It further includes, The method according to any one of claims 6 to 11, wherein the second connection plan includes additional instructions that can be used to connect each networking cable of the second set of networking cables to the corresponding networking port of the second networking device to form an updated regional network.

13. Generating the second connection plan described above means Determining a second set of networking cables configured to terminate at a second location in the data center, In the updated regional network, in order to connect the additional computing devices to the plurality of computing devices, the arrangement of the connections of the second set of networking cables to the second networking devices is determined. The method according to claim 12, including the method described in claim 12.

14. When executed by one or more processors, at least Receiving physical build requests to connect multiple computing devices to a static network fabric in a data center, In response to the receipt of the aforementioned physical construction request, Determining the configuration of the aforementioned multiple computing devices, Determining the static network fabric topology of the static network fabric of the data center, Using the above configuration and the static network fabric topology, generate a connectivity plan for connecting a set of networking cables in the static network fabric to the multiple computing devices, It stores computer executable instructions that cause the computing system to perform the following actions: The connection plan is a non-temporary computer-readable medium that includes instructions that can be used to connect each networking cable of the set of networking cables to the corresponding networking ports of the networking devices of the plurality of computing devices to form a regional network.

15. The non-temporary computer-readable medium according to claim 14, wherein the plurality of computing devices are communicated to a networking device, and determining the configuration of the plurality of computing devices includes determining the arrangement of network connections of the plurality of computing devices to the networking device.

16. The non-temporary computer-readable medium according to claim 14, wherein determining the configuration of the plurality of computing devices includes obtaining configuration parameters from a data store corresponding to the arrangement of network connections between the plurality of computing devices and one or more networking devices on a rack.

17. Determining the static network fabric topology includes obtaining a predetermined topology of the static network fabric of the data center from a data store, according to any one of claims 14 to 16, for a non-temporary computer-readable medium.

18. The predetermined topology of the static network fabric corresponds to a Clos network, and is a non-temporary computer-readable medium according to claim 17.

19. The non-temporary computer-readable medium according to any one of claims 14 to 18, wherein generating the connection plan includes determining the set of networking cables from a plurality of networking cables of the static network fabric which are configured to terminate at a location in the data center.

20. The computing system, when executed by the one or more processors, To determine the additional configuration of multiple additional computing devices that are connected to the second networking device, Using the above configuration, the additional configuration, and the static network fabric topology, a second connectivity plan is generated for connecting a second set of networking cables of the static network fabric to the additional computing devices. It is made to store another instruction that causes the computing system to perform the same action, The non-temporary computer-readable medium according to any one of claims 14 to 19, wherein the second connection plan includes additional instructions that can be used to connect each networking cable of the second set of networking cables to the corresponding networking port of the second networking device to form an updated regional network.

Images