Burst capacity of data centers for hyperscale workloads

Abstract

This invention provides a method and computing system for efficiently utilizing redundant power capacity. [Solution] The method includes monitoring the primary load and the backup load of a data center. The primary load and backup load can be monitored by a computing device. The primary load of the data center is powered by one or more primary generator blocks having primary capacity, and the backup load of the data center is powered by one or more backup generator blocks having backup capacity. The method also includes detecting when the primary load of the data center exceeds primary capacity, and using a switch on the computing device to connect the backup generator block to at least one of the primary generator block and primary load.

Description

Technical Field

[0004] , , ,

[0003] ,

[0001] Cross - Reference to Related Applications This application claims priority to U.S. Application No. 17 / 903,924, filed on September 6, 2022, and U.S. Application No. 18 / 074,332, filed on December 2, 2022, both entitled "BURST DATACENTER CAPACITY FOR HYPERSCALE WORKLOADS", the disclosures of which are hereby incorporated by reference in their entireties for all purposes.

Background Art

[0002] Background Power generation can be a limiting factor in the construction of a data center or the increase in the capacity of a data center. Data centers are built using redundant power systems to ensure high availability of servers. However, this redundant power capacity is hardly used. Therefore, there is unused power capacity in the data center, and improvement in data center design is desirable.

Summary of the Invention

[0003] Brief Summary A system consisting of one or more computers can be configured to perform a particular operation or action, which is due to software, firmware, hardware, or a combination thereof that causes the system to perform the action during operation being installed in the system. One or more computer programs can be configured to perform a particular operation or action, which is due to including instructions that cause the data processing device to perform the action when executed by the device.

[0004] In one general embodiment, a method performed by a computer may include monitoring the primary and secondary loads of a data center. Monitoring can be performed by a computing device. The primary load of the data center may be configured to be powered by one or more primary generator blocks and one or more secondary generator blocks. The secondary load of the data center may be configured to be powered by secondary generator blocks. The primary generator blocks may have primary capacity, and one or more secondary generator blocks may have secondary capacity. The method may include detecting, using a computing device, that the primary load of the data center exceeds primary capacity. The method may include connecting the secondary generator blocks to at least one of the primary generator blocks and the primary load using a switch. Other embodiments of this embodiment include corresponding computer systems, devices, and computer programs recorded in one or more computer storage devices, each configured to perform the actions of the method.

[0005] The implementation may include one or more of the following features: The method by which the backup generator block is connected may further include disconnecting the backup load from the backup generator block. The backup load can be disconnected by a circuit breaker controlled by a computing device. The method by which the backup load is disconnected if the combined load exceeds the combined capacity. The combined load may include the main load and the backup load. The combined capacity may include the main capacity and the backup capacity. The method by which power is supplied to the backup load by one or more backup generator blocks and a commercial power connection, wherein the commercial power connection supplies power to at least half of the backup load. The method by which it is detected whether the main load exceeds the main capacity is This may include determining that a malfunction has occurred in the generator block. A method for achieving 99.999% availability of the main load. A method for achieving 99.9% availability of the backup load. Implementations of the described technologies may include hardware, methods or processes, or computer media.

[0006] In one general embodiment, the system may include a non-temporary computer-readable medium storing computer executable program instructions. The system may include a processing device communicatively coupled to the non-temporary computer-readable medium for executing the computer executable program instructions, and the processing device may be configured to perform an action that includes monitoring the main and backup loads of a data center. The main loads of the data center may be powered by one or more main generator blocks having main capacity. The backup loads of the data center may be configured to be powered by one or more backup generator blocks having backup capacity. The instructions may include detecting when the main loads of the data center exceed the main capacity. The instructions may include connecting the backup generator blocks to at least one of the main generator blocks and the main loads using a switch. Other embodiments of this embodiment include a computer program recorded in a corresponding computer system, apparatus, and one or more computer storage devices, each configured to perform the actions of the method.

[0007] In a general embodiment, a non-temporary computer-readable storage medium storing computer executable program instructions may include monitoring the main and backup loads of a data center. The main loads of the data center are powered by one or more main generator blocks having main capacity, and the backup loads of the data center are powered by one or more backup generator blocks having backup capacity. Instructions may include a computing device detecting when the main loads of the data center exceed the main capacity. Instructions may include connecting the backup generator blocks to at least one of the main generator blocks and the main loads using switches controlled by the computing device. Other embodiments of this aspect include corresponding computer systems, devices, and computer programs recorded in one or more computer storage devices, each configured to perform the actions of the method. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram of a block redundant power architecture during normal operation according to one embodiment. [Figure 2] This is a diagram of a block redundant power architecture in a failure scenario according to one embodiment. [Figure 3] This is a diagram of a capacity acquisition power architecture during normal operation according to one embodiment. [Figure 4] This is a diagram of a capacity acquisition power architecture under a malfunction scenario according to a certain embodiment. [Figure 5] This is a diagram illustrating a method for disconnecting a backup load from a backup block according to one embodiment. [Figure 6] This is a block diagram showing one pattern for implementing a cloud infrastructure as a service system according to at least one embodiment. [Figure 7] This block diagram shows another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 8] This block diagram shows another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 9] This block diagram shows another pattern for implementing a cloud infrastructure as a service system, according to at least one embodiment. [Figure 10] A block diagram illustrating an exemplary computer system according to at least one embodiment. [Modes for carrying out the invention]

[0009] Detailed explanation The following description covers various embodiments. For explanatory purposes, specific configurations and details are provided to ensure a complete understanding of the embodiments. However, it will also be apparent to those skilled in the art that these embodiments can be carried out without specific details. Furthermore, well-known features may be omitted or simplified to avoid complicating the described embodiments.

[0010] Data centers are designed to minimize disruptions to server availability. In many cases, systems within a data center are designed redundantly so that service is not interrupted even if a single component fails. For example, backup generator blocks can supplement the power supplied to the data center's servers from a nearby power grid connection. In addition, some generator blocks are "reserve" or backup generator blocks that may not be connected to the load under normal operating conditions. Therefore, even if the power grid and the "primary" (i.e., non-backup) generator blocks fail, the reserve generator blocks can still supply power to the servers.

[0011] Redundant data center designs can be inefficient, as redundant systems often have unused capacity that remains idle. For example, a backup generator block may only be activated in rare circumstances where the other two power sources are unavailable. Such a backup generator block may typically only need to power the main load for less than 9 hours per year. The accepted industry standard is to keep the load available 99.999% of the time and minimize downtime to less than 6 minutes per year.

[0012] For some high-priority loads where time is of the essence, this industry standard known as Five Nines Availability may be required, but in some situations, this standard can be excessive. For example, a server processing corporate payroll that may need to be completed in a few days may not require Five Nines Availability, as a delay of a few minutes in availability might not be noticeable to the customer.

[0013] Furthermore, the five nines (e.g., 99.999%) availability standard is based on server downtime assumptions made using now-outdated technology. For example, the latest IEEE standard on five nines (e.g., IEEE Std. 3006.7, IEEE Recommended Practice for Determining the Reliability of 7x24 Continuous Power Systems in Industrial and Commercial Facilities) was published in 2013. The calculations in this 3006.7 standard are based on the reliability of power system equipment such as generators and uninterruptible power supplies (UPS) as of 2013. Technological advancements since the update of this standard could mean that the time required for backup generator blocks may be even shorter than the approximately 9 hours per year suggested by multiple standards.

[0014] Thus, backup generator blocks in data centers are costly resources that are rarely used. Backup blocks can be used to provide backup power for a backup load at low availability standards, such as 99.9% availability. This backup load can be a dual-coded opportunistic load connected to both commercial power and the backup generator block. If the main load exceeds the capacity of the main generator block, the backup load can be deactivated, and the backup generator block can supply power to the main load. For example, if a component in the main generator block fails, the main load can... The load capacity may be exceeded. The backup load can be disconnected from the backup generator and will continue to receive power as long as there is no problem with the commercial power connection.

[0015] In one exemplary example, a main load requiring five-nine availability under a service level agreement is connected to five main generator blocks. If a main generator block fails, the main load of the failed block can be switched to a backup generator block. Each generator block includes utility connections, a generator, and a battery called an uninterruptible power supply (UPS) that provides power to the load while the generator is operating. Backup loads are connected to the backup power supply blocks by circuit breakers, and the backup loads are also connected to the nearest power grid via commercial power connections.

[0016] In this example, the main load experiences a power drop because the commercial power supply to the main generator block is interrupted. The main generator block attempts to provide backup power to the main load, but the block's UPS does not activate. When the power supplied by the malfunctioning generator block begins to drop, a switch called a static changeover switch (STS) detects the power drop from that block. In response, the STS disconnects the main generator block, connects a backup generator to the main load, and the backup block's UPS provides power until the backup generator starts up.

[0017] The backup generator block attempts to supply power to both the backup load and the main load, but the combined load exceeds the capacity of the backup generator block. To ensure that the main load continues to receive power, the circuit breaker opens, disconnecting the backup load from the backup generator block so that power from the backup generator block can be supplied to the main load.

[0018] FIG. 1 is a diagram of a block redundant power architecture 100 during normal operation according to an embodiment. This architecture can be part of a block redundant (BR) architecture. In a normal operating state, the main generator block 105 provides power to the load 110 using the connection 115 to commercial power. The switches in FIGS. 1 - 4, such as switch 120, are shown in the open position, which is mainly for explaining the existence of the switch at a specific position and should not be interpreted as indicating the opening or closing of the connection. Instead, switch 120 and any other switch shown in this specification can be in an open or closed state and can be controlled to change from open to closed or from closed to open. The normal operating state can include an interruption of commercial power. When commercial power is interrupted, the main generator block can use the generator 125 in addition to the uninterruptible power supply (UPS) 130 to provide power to the load 110. The load, such as load 110, can be one or more electronic devices or computing devices. For example, load 110 can include a server computer, a personal computer, a storage device, a network device, a cooling device, a fan, an environmental monitor, etc. In a normal operating state, load 110 is assumed not to exceed the capacity of the generator block 105.

[0019] The UPS 130 is an electronic device that can provide emergency power when it detects an interruption of commercial power. The main generator block can have one or more UPSs 130 and one or more generators 125. The emergency power can be power that can be supplied by the UPS 130 in a short time (e.g., 25 milliseconds or less). One or more batteries in the UPS 130 can supply emergency power to the load 110 while the generator 125 is operating. The time to readiness, i.e., the time to operate the generator 125, can be between 10 seconds and 15 seconds. In addition to providing emergency power, the UPS 130 is a transmission path for power from the power company. Also, the UPS 130 can handle problems related to commercial power. For example, the UPS 130 can correct voltage spikes, voltage sags, noise, etc.

[0020] The BR power architecture may include one or more reserve (R) blocks 135. The R block 135 may not be activated unless there is a problem with the main generator block 105, and the R block 135 may be connected to the static transfer switch (STS) 140 without providing power to the load 110. The STS 140 may be an electronic device capable of transmitting power from the main power source to the alternative power source in a short time (e.g., 4 milliseconds). The STS 140 may switch to an alternative power source when the power from the main power source (e.g., the power source providing power through the STS) falls below a threshold value. The load 110 may be connected to the STS 140 via a switchboard (SWB) 145 capable of distributing power to the load 110. One or more main generator blocks 105 such as A block 150, B block 155, or C block 160 can be connected to one or more loads, and the SWB 145 can distribute power to those one or more loads. For example, the SWB 145 can be used to disconnect the power from a server (e.g., turn off the server's power) for server maintenance. Although three main generator blocks 105 and one R block 135 are shown in the architecture 100, other configurations are contemplated, such as configurations with a ratio of main generator blocks to reserve blocks of 3:1, 5:1, 7:1, 12:1.

[0021] Figure 2 is a diagram of the block redundant power architecture in a failure scenario according to one embodiment. As described above, the main generator block 205 provides power to the load 210 under normal operating conditions. A failure in the commercial power supply may be a normal operating condition, and a failure condition may be when one of the main generator blocks 205 is unable to provide power to the load 210. A failure condition may be when the commercial power supplied via the commercial power connection 215 is interrupted, and neither the generator 225 nor the UPS 230 can provide emergency power from one or more of the main generator blocks 205. A failure condition may occur when one of the main blocks 205 fails in any of the following: 1) the commercial power connection 215 and the generator 225, 2) the commercial power connection 215 and the UPS 230, or 3) the UPS 230. If the commercial power connection 215 fails but the UPS 230 and the generator 225 are operating, the UPS 230 can obtain its power from the generator 225. If the generator 225 malfunctions but the UPS 230 and the commercial power connection 214 are operating, the UPS 230 can continue to supply power even without transmission from the commercial power connection 215.

[0022] If a malfunction occurs in the main generator block 205 (for example, when a malfunction condition occurs), the R block 235 may operate to ensure that power to the load 210 is maintained. The R block 235 may supply emergency power to the load 210 through the commercial power connection 215, or it may operate by supplying emergency power to the load 210 from the generator 225. If there is a delay in the power supplied by the generator 225, the UPS 230 may supply emergency power to the load 210. For example, the generator 225 may not be able to supply power during the starting process.

[0023] The STS240 can detect when a main generator block, such as Block A 250, Block B 255, or Block C 260, is no longer supplying power to the load 210. For example, Figure 200 shows a black "X" indicating a malfunction in Block A 250. The STS240 detects the decrease in power supplied from Block A 250 and switches from its connection to Block A 250 to its connection to Block R 235 (e.g., the dotted line connection). Block R can supply power to the load 210 until the malfunctioning block, in this case Block A 250, returns to service and resumes supplying power to the load 210. Block R 235 can supply power to the load 210 during a malfunction condition that may include damage or malfunction of a component rendering Block A 250 inoperable or routine maintenance of Block A 250.

[0024] Figure 3 shows a capacity-acquired power architecture 300 during normal operation according to one embodiment. The architecture 300 may include a main generator block 305 that supplies power to a load 310 and a spare (R) block 315 that can supply power to a spare (R) load 320. Although the architecture 300 shows three main blocks 305 and one R block 315, other configurations are envisioned, such as main block to spare block ratios of 3:1, 5:1, 7:1, and 12:1.

[0025] The main generator block 305 and the backup block 315 can supply commercial power to the load 310 or R load 320 via the UPS 325. If commercial power to the main generator block is interrupted, the UPS 325 can supply emergency power to the load 310 or R load 320 during the delay between the start of the generator 330 and the generator being able to supply enough power to support the load. For example, the turbine in the generator 330 may take several seconds to reach a speed sufficient for power generation.

[0026] During normal operation, the main generator block 305, including blocks A 335, B 340, and C 345, can supply power to load 310, and block R 315 can supply power to load R 320. Load 310, in conjunction with block R 315, can be a highly available load providing 99.999% uptime over a given period (e.g., "five nines" availability). The main generator block 305 can provide 99.999% availability because block R 315 can provide power if one of the main generator blocks 305 fails. Downtime for a system with 99.999% uptime is 5.26 minutes or less per year. The standard for a commercial power system that can provide "five nines" availability is, for example, IEEE Std 493-2007 (a revision of IEEE Std 493-1997), "IEEE Recommended Practice for the Design of Reliable Power Systems." This is defined in "Industrial and Commercial Power Systems," volume and issue not specified, pp. 1-383, June 25, 2007, doi:10.1109 / IEEESTD.2007.380668).

[0027] Load R 320 may have lower availability than load 310. For example, load R 320 may have 99.99%, 99.9%, or 99% availability. Under normal operation, power is supplied to load R via commercial power connection 350 and commercial power connection through R block 315 (e.g., via UPS 325). R block 315 can supply 50% of load R 320, and the remaining 50% can be supplied by commercial power connection 350. R block 315 may supply a larger or smaller percentage of the power delivered to load R 320 (e.g., 5%, 25%, 45%, 55%, 75%, 95% of the power to load R). Under normal operation, if R block 315 fails, commercial power connection 350 can supply all power to load R 320. If commercial power connection 350 fails, R block 315 can supply all power to load R 320.

[0028] Figure 4 shows a power generation architecture 400 during a failure scenario according to one embodiment. A failure scenario may occur when one of the main generator blocks 405 fails. A failure in a main generator block such as block A 410, block B 415, or block C 420 may occur when a failure occurs in the UPS 425 or generator 430 and Tx 417 (although not specified, Tx may be referred to as 412, 417, or 422 here, in which case it is a failure in Tx 417). In architecture 400, block B 415 is shown with a black "X" to indicate that a block has failed, but a failure scenario can occur when any main block fails. A failure scenario may occur when there is a decrease in the power supplied by the main generator block 405 to the load 435, for example, when one or more generators and Tx or UPS within the main generator block fail. A problematic scenario may occur.

[0029] If one of the main generator blocks 405 malfunctions and one or more STS 455 switches from main power to backup (R) power, the R block 440 can be disconnected from one or more R loads 445. The R loads may also be disconnected when one of the main blocks 405 malfunctions. One of the main generator blocks 405 can supply main power, and the backup block (440) can supply backup power. The R loads 445 can be disconnected from the R block 440 by a circuit breaker (e.g., breaker 450). If the sum of the loads 435 of the main generator block 405 and the R loads 445 exceeds the capacity of the R block 440, the R block 440 can be disconnected from the R loads 445 by the breaker 450. If one of the main generator blocks 504 malfunctions, the breaker 450 may disconnect one or more of the R loads 445 if the combined loads 435 and 445 exceed the capacity of the available blocks (e.g., main generator block 405, R block 440). If the combined loads 435 and R loads 445 of the main generator block 405 (e.g., A block 410, B block 415, or C block 420) exceed the capacity of R block 445, the breaker 450 may disconnect R block 440 from R loads 445. The breaker 450 may disconnect R block 440 from R loads 445 if a threshold number of STS 455 switch from main power to R power (e.g., two or more STS switch to backup power).

[0030] The computing device 460 can control the breaker 450 to disconnect or connect the R load 445 to the R block 440. In some embodiments, the computing device 460 may be an industrial control system implemented in hardware rather than software. For example, the industrial control system may be a hardware-implemented control system that disconnects the R block whenever one of the main generator blocks 405 fails. The breaker 450 is controllable based on signals from the STS 455, which may indicate, for example, whether the STS 455 is connected to the main power or the backup (R) power. The computing device 460 can open the breaker 450 if one or more of the STS 455 or a threshold amount of STS have switched to the R power (for example, to receive power from the R block 440). Other techniques for determining whether the breaker 450 should be opened are also contemplated. For example, the voltage, current, resistance, or inductance between the main generator block 405 and the load 435 can be measured to determine whether the breaker 450 should open (for example, whether the main generator block 405 is providing sufficient power to the load 435).

[0031] The computing device 460 may close the circuit breaker 450 if it determines that block R 440 does not need to supply power to load 435. For example, the computing device 460 may have opened the circuit breaker 450 because of a malfunction in the UPS 425 of block B 415, and then closed the circuit breaker 450 after block B 415 was returned to service. The computing device 460 may be a programmable logic device, personal computer, system-on-a-chip, single-board computer (SBC), field-programmable gate assembly (FPGA), integrated circuit, programmable logic circuit (PLC), etc.

[0032] Figure 5 illustrates a method for detaching a spare load from a spare block according to one embodiment. This method is shown as a logical flow diagram, and each operation can be implemented as hardware, a computer instruction, or a combination thereof. In the context of computer instructions, An operation may represent a computer executable instruction stored in one or more computer-readable storage media that, when executed by one or more processors, performs the enumerated operation. Generally, computer executable instructions include routines, programs, objects, components, data structures, etc., that perform a particular function or implement a particular data type. The order in which operations are listed is not intended to be interpreted as limiting, and any number of listed operations can be combined in any order and / or in parallel to perform a process or method.

[0033] A more detailed examination of Method 500 reveals that in block 510, the primary and secondary loads of the data center can be monitored. The primary loads may be loads 110, 210, 310, and 435, and the secondary loads may be R loads 320 and 445. The primary and secondary loads can be monitored by a computing device 460. For example, the computing device 460 can receive outputs from STSs such as STS 140, 240, and 455 indicating whether the STS is connected to power from primary blocks (e.g., 105, 205, 305, and 405) or secondary blocks such as secondary blocks 135, 235, 315, and 440. The STS can provide the computing device 460 with information about the current flowing through the STS, and this information can be used to monitor the primary loads. Generators 125, 225, 330, 430, UPS 130, 230, 325, 425, or circuit breaker 450 can provide a computing device with information about the power supplied to the main or backup load, and the computing device 460 can use the provided information to monitor the main or backup load.

[0034] In block 520, it can be determined whether the data center's main load exceeds the main capacity. The main capacity can be the generating capacity of one or more of the main generator blocks 105, 205, 305, and 405. The main capacity of a main block can be the sum of the capacities of one of the generators 125, 225, 330, and 430 in one of the main generator blocks 102, 205, 305, and 405. The reserve capacity of a reserve block can be the sum of the capacities of the generators 125, 225, 330, and 430 in R blocks 135, 235, 315, and 440. STS 140, 240, and 455 can determine that the main load exceeds the main capacity when the power passing through the STS falls below a threshold. The power passing through the STS may be the power provided by one or more of the main generator blocks.

[0035] In block 530, the backup generator block may be connected to at least one of the main load or the main generator block. The main load may receive power from both the main generator block and the backup generator block. The backup block may be connected to the main load if one or more main generator blocks connected to the main load cannot supply enough power to the main load to keep it operational. The main load or backup load may include server computers, personal computers, storage devices, network devices, cooling devices, fans, environmental monitors, etc.

[0036] STS140, 240, and 455 can connect a backup generator block to the main load by switching from a connection to the main generator block to a connection to the backup generator block. Generator blocks such as main generator blocks 105, 205, 305, 405 or R blocks 135, 235, 315, 440 can be connected to the load by one or more STS. For example, a generator block may contain multiple generators, and each generator in the block can be connected to the load by an STS. By switching some or all of the STS, a portion of the load can be moved from one generator block to another.

[0037] Connecting a backup generator block may involve disconnecting a backup load from the backup generator block. The backup load is controllable by computing device 460. Alternatively, it can be disconnected by a circuit breaker, for example, breaker 450, which may be part of the computing device 460. Commercial power connections 115, 215, and 350 can supply power to the load after the backup load has been disconnected from the backup block. The computing device 460 can close breaker 450 and restore the connection between the backup load and the backup block if the combined load of the main load connected to the backup block and the backup load does not exceed the combined generating capacity of the main generator block connected to the main load and the backup block (for example, if the backup block has sufficient spare capacity to support the backup load). The backup block can be connected to one or more backup loads by one or more circuit breakers, and the backup block can be reconnected to some or all of the backup loads. The backup block may be reconnected if the main block, which was inoperable due to a component failure or maintenance, can resume generating power. The backup load may have various tiers, with lower tiers being disconnected before higher tiers. The backup load can be reconnected to power, with higher tiers being reconnected before lower tiers. The hierarchy can be set based on priority or service level agreement (SLA), with lower tiers having lower priority and higher tiers having higher priority.

[0038] Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to deliver virtualized computing resources over a public network (e.g., the internet). In the IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer)). In some cases, the IaaS provider may also supply various services associated with these infrastructure components (e.g., billing, monitoring, logging, security, load balancing, clustering, etc.). In this case, these services may be policy-driven, so IaaS users may be able to implement policies that promote load balancing to maintain application availability and performance.

[0039] In some cases, IaaS customers may access resources and services over a wide area network (WAN), such as the internet, and use the cloud provider's services to install the rest of their application stack. For example, a user can log into an IaaS platform, create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and then install enterprise software on those VMs. The customer can then use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting application issues, monitoring performance, managing disaster recovery, and more.

[0040] In most cases, the cloud computing model requires the participation of a cloud provider. While not mandatory, a cloud provider may be a third-party service specializing in providing IaaS (e.g., offering, renting, or selling). Alternatively, an entity may choose to deploy a private cloud and become the provider of its own infrastructure services.

[0041] In some examples, IaaS deployment is the process of placing a new application, or a new version of an application, onto a prepared application server. This may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is often done at the hypervisor layer (for example) In this case, the customer is managed by the cloud provider, which is below the server, storage, network hardware, and virtualization. In this case, the customer may be responsible for handling the deployment of the (OS), middleware, and / or applications (for example, on self-service virtual machines that can be spun up on demand).

[0042] In some examples, IaaS provisioning might refer to acquiring the computers or virtual hosts to be used and then installing any necessary libraries or services on them. In most cases, provisioning is not included in the deployment, and provisioning may need to be performed first.

[0043] In some cases, IaaS provisioning presents two distinct challenges. First, there is the initial challenge of provisioning an initial set of infrastructure before anything is executed. Second, once everything is provisioned, there is the challenge of evolving the existing infrastructure (adding new services, modifying services, removing services, etc.). In some cases, these two challenges can be addressed by allowing the infrastructure configuration to be defined declaratively. In other words, the infrastructure (e.g., what components are needed and how they interact) can be defined by one or more configuration files. In this case, the overall topology of the infrastructure (e.g., which resources depend on which and how they work together) can be described declaratively. In some cases, once the topology is defined, it is possible to generate workflows to create and / or manage the various components described in the configuration files.

[0044] In some examples, the infrastructure may have many interconnected elements. For example, there may be one or more virtual private clouds (VPCs), also known as core networks (e.g., pools of configurable and / or shared computing resources, which may be on-demand). In some examples, there may be one or more security group rules provisioned to define how the network's security will be configured, and one or more virtual machines (VMs). Other infrastructure elements, such as load balancers and databases, may also be provisioned. As more infrastructure elements are desired and / or added, the infrastructure may evolve over time.

[0045] In some cases, continuous deployment techniques may be employed to enable the deployment of infrastructure code across various virtual computing environments. In addition, the techniques described enable infrastructure management within these environments. In some examples, a service team may write code that is intended to be deployed to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes even globally). However, in some examples, the infrastructure to which the code will be deployed must first be set up. In some cases, provisioning can be done manually, using provisioning tools to provision resources and / or using deployment tools to deploy the code once the infrastructure is provisioned.

[0046] Figure 6 is a block diagram 600 illustrating an exemplary pattern of an IaaS architecture according to at least one embodiment. A service provider 602 may be communicably coupled to a secure host tenancy 604 which may include a virtual cloud network (VCN) 606 and a secure host subnet 608. In some examples, the service provider 602 is one Alternatively, multiple client computing devices may be used, which may be portable handheld devices (e.g., iPhone®, mobile phones, iPad®, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass® head-mounted displays), which may run software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS®, Windows Phone, Android, BlackBerry 8, Palm OS, and may support the Internet, email, short message service (SMS), Blackberry®, or other communication protocols. Alternatively, the client computing device may be a general-purpose personal computer, including, for example, personal computers and / or laptop computers running various versions of the Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. The client computing device may be a workstation computer running one of various commercially available UNIX® or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems such as Google Chrome OS. Alternatively, or in addition, the client computing device may be any other electronic device that can communicate via a network accessible through the VCN606 and / or the Internet, such as a thin client computer, an Internet-enabled game system (e.g., a Microsoft Xbox game console with or without a Kinect® gesture input device), and / or a personal messaging device.

[0047] VCN606 may include a local peering gateway (LPG) 610, which can be communicatively coupled to Secure Shell (SSH) VCN612 via LPG610 contained within SSH VCN612. SSH VCN612 may include an SSH subnet 614, which can be communicatively coupled to control plane VCN616 via LPG610 contained within control plane VCN616. SSH VCN612 can also be communicatively coupled to data plane VCN618 via LPG610. Control plane VCN616 and data plane VCN618 may be included in a service tenancy 619, which may be owned and / or operated by an IaaS provider.

[0048] The control plane VCN616 may include a control plane demilitarized zone (DMZ) layer 620, which functions as a perimeter network (e.g., part of the corporate network between the corporate intranet and the external network). DMZ-based servers may have limited responsibilities and help mitigate security breaches. In addition, the DMZ layer 620 may include one or more load balancer (LB) subnets 622, a control plane application layer 624 which may include application subnets 626, and a control plane data layer 628 which may include database (DB) subnets 630 (e.g., a front-end DB subnet and / or a back-end DB subnet). The LB subnet 622 included in the control plane DMZ layer 620 can be communicatively coupled to the application subnet 626 included in the control plane application layer 624 and to the internet gateway 634 which may be included in the control plane VCN 616. The application subnet 626 can be communicatively coupled to the DB subnet 630 included in the control plane data layer 628, to the service gateway 636 and to the network address translation (NAT) gateway 638. The control plane VCN 616 may include the service gateway 636 and the NAT gateway 638.

[0049] The control plane VCN616 may include the application subnet 626, and the data plane mirror - May include an application layer 640. An application subnet 626 included in the dataplane mirror application layer 640 may include a virtual network interface controller (VNIC) 642 capable of running compute instance 644. Compute instance 644 can communicatively connect the application subnet 626 of the dataplane mirror application layer 640 to an application subnet 626 that may be included in the dataplane application layer 646.

[0050] The data plane VCN618 may include a data plane application layer 646, a data plane DMZ layer 648, and a data plane data layer 650. The data plane DMZ layer 648 may include an LB subnet 622 that can be communicatively coupled to the application subnet 626 of the data plane application layer 646 and the internet gateway 634 of the data plane VCN618. The application subnet 626 may be communicatively coupled to the service gateway 636 of the data plane VCN618 and the NAT gateway 638 of the data plane VCN618. The data plane data layer 650 may also include a DB subnet 630 that can be communicatively coupled to the application subnet 626 of the data plane application layer 646.

[0051] The Internet gateway 634 of the control plane VCN616 and the data plane VCN618 may be communicatively coupled to a metadata management service 652 which may be communicatively coupled to the public internet 654. The public internet 654 may be communicatively coupled to the NAT gateway 638 of the control plane VCN616 and the data plane VCN618. The service gateway 636 of the control plane VCN616 and the data plane VCN618 may be communicatively coupled to a cloud service 656.

[0052] In some cases, a service gateway 636 on the control plane VCN616 or data plane VCN618 can make application programming interface (API) calls to a cloud service 656 without going through the public internet 654. API calls from the service gateway 636 to the cloud service 656 can be one-way; that is, the service gateway 636 can make an API call to the cloud service 656, and the cloud service 656 can send the requested data to the service gateway 636. However, the cloud service 656 cannot initiate an API call to the service gateway 636.

[0053] In some cases, a secure host tenancy 604 can be directly connected to a service tenancy 619, but otherwise, the service tenancy 619 may be isolated. A secure host subnet 608 can communicate with an SSH subnet 614 through an LPG 610, which can enable two-way communication on systems that are normally isolated. By connecting a secure host subnet 608 to an SSH subnet 614, the secure host subnet 608 may gain access to other entities within the service tenancy 619.

[0054] The control plane VCN616 may allow users of service tenancy 619 to set up or otherwise provision desired resources. The desired resources provisioned in the control plane VCN616 can then be deployed or otherwise used in the data plane VCN618. In some examples, the control plane VCN616 can be isolated from the data plane VCN618, and the data plane mirror application layer 640 of the control plane VCN616 may communicate with the data plane application layer 646 of the data plane VCN618 via a VNIC 642 which may be included in the data plane mirror application layer 640 and the data plane application layer 646.

[0055] In some examples, a system user or customer can make requests via the public internet 654, which can propagate requests to the metadata management service 652, for example, perform create, read, update, or delete (CRUD) operations. The metadata management service 652 can propagate requests to the control plane VCN 616 through the internet gateway 634. Requests may be received by the LB subnet 622, which is included in the control plane DMZ layer 620. The LB subnet 622 can determine that the request is valid, and in response to this determination, the LB subnet 622 can send the request to the application subnet 626, which is included in the control plane application layer 624. If the request is verified to be valid and requires a call to the public internet 654, the call to the public internet 654 can be sent to the NAT gateway 638, which is capable of making calls to the public internet 654. Memory that may be desired to be saved by the request can be stored in the DB subnet 630.

[0056] In some cases, the data plane mirror application layer 640 may facilitate direct communication between the control plane VCN616 and the data plane VCN618. For example, it may be desirable to apply configuration changes, updates, or other appropriate modifications to resources contained in the data plane VCN618. Through VNIC642, the control plane VCN616 can communicate directly with the resources contained in the data plane VCN618, thereby enabling it to perform configuration changes, updates, or other appropriate modifications to those resources.

[0057] In some embodiments, the control plane VCN616 and data plane VCN618 may be included in the service tenancy 619. In this case, the system user or customer may not own or operate either the control plane VCN616 or the data plane VCN618. Instead, the IaaS provider may own or operate the control plane VCN616 and the data plane VCN618, both of which may be included in the service tenancy 619. This embodiment may enable network isolation that could prevent the user or customer from interacting with other users' resources or other customers' resources. This embodiment may also enable the system user or customer to store databases privately without having to rely on the public internet 654, which may not have the desired level of security.

[0058] In another embodiment, the LB subnet 622 included in the control plane VCN616 may be configured to receive signals from the service gateway 636. In this embodiment, the control plane VCN616 and the data plane VCN618 may be configured to be invoked by the IaaS provider's customer without calling the public internet 654. The IaaS provider's customer may prefer this embodiment because the database used by the customer can be controlled by the IaaS provider and stored in a service tenancy 619 that can be isolated from the public internet 654.

[0059] Figure 7 is a block diagram 700 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service provider 702 (e.g., service provider 602 in Figure 6) may be communicably coupled to a secure host tenancy 704 (e.g., secure host tenancy 604 in Figure 6), which may include a virtual cloud network (VCN) 706 (e.g., VCN606 in Figure 6) and a secure host subnet 708 (e.g., secure host subnet 608 in Figure 6). VCN706 may connect to a secure shell (SSH) VCN712 (e.g., via LPG610 included in SSH VCN712) For example, it may include a local peering gateway (LPG) 710 (e.g., LPG610 in Figure 6) which can be communicatively coupled to the SSH VCN612 in Figure 6. The SSH VCN712 may include an SSH subnet 714 (e.g., SSH subnet 614 in Figure 6), and the SSH VCN712 may be communicatively coupled to the control plane VCN716 (e.g., control plane VCN616 in Figure 6) via the LPG710 which is included in the control plane VCN716. The control plane VCN716 may be included in a service tenancy 719 (e.g., service tenancy 619 in Figure 6), and the data plane VCN718 (e.g., data plane VCN618 in Figure 6) may be included in a customer tenancy 721 which may be owned or operated by a user or customer of the system.

[0060] The control plane VCN716 may include a control plane DMZ layer 720 (e.g., control plane DMZ layer 620 in Figure 6) which may include an LB subnet 722 (e.g., LB subnet 622 in Figure 6), a control plane application layer 724 (e.g., control plane application layer 624 in Figure 6) which may include an application subnet 726 (e.g., application subnet 626 in Figure 6), and a control plane data layer 728 (e.g., control plane data layer 628 in Figure 6) which may include a database (DB) subnet 730 (e.g., similar to DB subnet 630 in Figure 6). The LB subnet 722 included in the control plane DMZ layer 720 can be communicatively coupled to the application subnet 726 included in the control plane application layer 724 and to an internet gateway 734 (e.g., internet gateway 634 in Figure 6) which may be included in the control plane VCN 716. The application subnet 726 can be communicatively coupled to the DB subnet 730 included in the control plane data layer 728, to a service gateway 736 (e.g., service gateway in Figure 6) and to a network address translation (NAT) gateway 738 (e.g., NAT gateway 638 in Figure 6). The control plane VCN 716 may include the service gateway 736 and the NAT gateway 738.

[0061] The control plane VCN 716 may include a data plane mirror application layer 740 (e.g., data plane mirror application layer 640 in Figure 6) which may include an application subnet 726. The application subnet 726 included in the data plane mirror application layer 740 may include a virtual network interface controller (VNIC) 742 (e.g., VNIC 642) capable of running a compute instance 744 (e.g., similar to compute instance 644 in Figure 6). The compute instance 744 may facilitate communication between the application subnet 726 of the data plane mirror application layer 740 and the application subnet 726 included in the data plane application layer 746 (e.g., data plane application layer 646 in Figure 6) via the VNIC 742 included in the data plane mirror application layer 740 and the VNIC 742 included in the data plane application layer 746.

[0062] The Internet gateway 734 included in the control plane VCN 716 can be communicatively coupled to a metadata management service 752 (e.g., metadata management service 652 in Figure 6), which can be communicatively coupled to the public internet 754 (e.g., public internet 654 in Figure 6). The public internet 754 can be communicatively coupled to a NAT gateway 738 included in the control plane VCN 716. The service gateway 736 included in the control plane VCN 716 can be communicatively coupled to a cloud service 756 (e.g., cloud service 656 in Figure 6).

[0063] In some examples, the data plane VCN718 can be included in the customer tenancy 721. In this case, the IaaS provider may provide a control plane VCN716 per customer, and the IaaS provider may configure a unique compute instance 744 included in the service tenancy 719 for each customer. Each compute instance 744 is connected to the control plane VCN716 included in the service tenancy 719 and the customer tenancy This may enable communication with the data plane VCN718 included in instance 721. Compute instance 744 may enable resources provisioned in the control plane VCN716 included in service tenancy 719 to be deployed or otherwise used in the data plane VCN718 included in customer tenancy 721.

[0064] In another example, an IaaS provider's customer may have a live database in customer tenancy 721. In this example, the control plane VCN 716 may include a data plane mirror application layer 740, which may include an application subnet 726. The data plane mirror application layer 740 may reside in data plane VCN 718, but does not have to reside in data plane VCN 718. That is, the data plane mirror application layer 740 may have access to customer tenancy 721, but may not reside in data plane VCN 718, or may not be owned or operated by the IaaS provider's customer. The data plane mirror application layer 740 may be configured to make calls to data plane VCN 718, but does not have to be configured to make calls to any entity included in control plane VCN 716. Customers may wish to deploy or otherwise use resources in the data plane VCN718 that are provisioned in the control plane VCN716, and the data plane mirror application layer 740 can facilitate the customer's desired deployment or other use of resources.

[0065] In some embodiments, a customer of the IaaS provider can apply filters to the data plane VCN718. In this embodiment, the customer can determine what the data plane VCN718 can access, and can also restrict the data plane VCN718's access to the public internet 754. The IaaS provider may not be able to filter or otherwise control the data plane VCN718's access to any external network or database. Applying customer filters and controls to the data plane VCN718 included in customer tenancy 721 can help isolate the data plane VCN718 from other customers and the public internet 754.

[0066] In some embodiments, a service gateway 736 can invoke a cloud service 756 to access a service that may not exist on the public internet 754, the control plane VCN 716, or the data plane VCN 718. The connection between the cloud service 756 and the control plane VCN 716 or data plane VCN 718 may not be constant or continuous. The cloud service 756 may reside on a different network owned or operated by the IaaS provider. The cloud service 756 may be configured to receive calls from the service gateway 736 and not to receive calls from the public internet 754. Some cloud services 756 may be isolated from other cloud services 756, and the control plane VCN 716 may be isolated from cloud services 756 that are not in the same region as the control plane VCN 716. For example, the control plane VCN 716 may be located in "Region 1", and the cloud service "Deployment 6" may be located in "Region 1" and "Region 2". If a call to deployment 6 is made by a service gateway 736 included in control plane VCN716 located in region 1, the call may be sent to deployment 6 in region 1. In this example, control plane VCN716, or deployment 6 in region 1, may not be communicatively coupled to deployment 6 in region 2, or may be communicating with it.

[0067] Figure 8 is a block diagram 800 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service provider 802 (e.g., service provider 602 in Figure 6) may be communicatively coupled to a secure host tenancy 804 (e.g., secure host tenancy 604 in Figure 6), which may include a virtual cloud network (VCN) 806 (e.g., VCN606 in Figure 6) and a secure host subnet 808 (e.g., secure host subnet 608 in Figure 6). VCN806 may include an LPG810 (e.g., LPG610 in Figure 6), which may be communicatively coupled to an SSH VCN812 (e.g., SSH VCN612 in Figure 6) via an LPG810 included in the SSH VCN812. SSH VCN812 may include SSH subnet 814 (e.g., SSH subnet 614 in Figure 6), and SSH VCN812 may be communicably coupled to control plane VCN816 (e.g., control plane VCN616 in Figure 6) via LPG810 included in control plane VCN816, and to data plane VCN818 (e.g., data plane 618 in Figure 6) via LPG810 included in data plane VCN818. Control plane VCN816 and data plane VCN818 may be included in service tenancy 819 (e.g., service tenancy 619 in Figure 6).

[0068] The control plane VCN816 may include a control plane DMZ layer 820 (e.g., control plane DMZ layer 620 in Figure 6) which may include a load balancer (LB) subnet 822 (e.g., LB subnet 622 in Figure 6), a control plane application layer 824 (e.g., control plane application layer 624 in Figure 6) which may include an application subnet 826 (e.g., similar to application subnet 626 in Figure 6), and a control plane data layer 828 (e.g., control plane data layer 628 in Figure 6) which may include a DB subnet 830. The LB subnet 822 included in the control plane DMZ layer 820 can be communicatively coupled to the application subnet 826 included in the control plane application layer 824, and to an internet gateway 834 (e.g., internet gateway 634 in Figure 6) which may be included in the control plane VCN 816. The application subnet 826 can be communicatively coupled to the DB subnet 830 included in the control plane data layer 828, and to a service gateway 836 (e.g., service gateway in Figure 6) and a network address translation (NAT) gateway 838 (e.g., NAT gateway 638 in Figure 6). The control plane VCN 816 may include the service gateway 836 and the NAT gateway 838.

[0069] The data plane VCN818 may include a data plane application layer 846 (e.g., data plane application layer 646 in Figure 6), a data plane DMZ layer 848 (e.g., data plane DMZ layer 648 in Figure 6), and a data plane data layer 850 (e.g., data plane data layer 650 in Figure 6). The data plane DMZ layer 848 may include an LB subnet 822 that can be communicatively coupled to the trusted application subnet 860 and untrusted application subnet 862 of the data plane application layer 846, and to the internet gateway 834 included in the data plane VCN818. The trusted application subnet 860 may be communicatively coupled to the service gateway 836 included in the data plane VCN818, the NAT gateway 838 included in the data plane VCN818, and the DB subnet 830 included in the data plane data layer 850. An untrusted application subnet 862 may be communicatively coupled to a service gateway 836 included in the data plane VCN 818 and a DB subnet 830 included in the data plane data layer 850. The data plane data layer 850 may include a DB subnet 830 that can be communicatively coupled to a service gateway 836 included in the data plane VCN 818.

[0070] Untrusted application subnet 862 affects tenant virtual machines (VMs) 866(1)~ (N) may include one or more primary VNICs 864(1) to (N) which can be communicatively coupled. Each tenant VM 866(1) to (N) may be communicatively coupled to a corresponding application subnet 867(1) to (N) which may be included in a corresponding container output VCN 868(1) to (N) which may be included in a corresponding customer tenancy 870(1) to (N). A corresponding secondary VNIC 872(1) to (N) may facilitate communication between an untrusted application subnet 862 included in a data plane VCN 818 and an application subnet included in a container output VCN 868(1) to (N). Each container output VCN 868(1) to (N) may include a NAT gateway 838 which can be communicatively coupled to the public internet 854 (e.g., public internet 654 in Figure 6).

[0071] The Internet gateway 834, included in the control plane VCN816 and the data plane VCN818, can be communicatively coupled to a metadata management service 852 (e.g., the metadata management system 652 in Figure 6), which can be communicatively coupled to the public internet 854. The public internet 854 can be communicatively coupled to a NAT gateway 838, included in the control plane VCN816 and the data plane VCN818. The service gateway 836, included in the control plane VCN816 and the data plane VCN818, can be communicatively coupled to a cloud service 856.

[0072] In some embodiments, the data plane VCN818 may be integrated with the customer tenancy 870. This integration may be useful or desirable in several cases for the IaaS provider's customers, such as when they may want support during code execution. Customers may provide code to be executed, but that code may be destructive, communicate with other customer resources, or otherwise cause undesirable effects. In response to this, the IaaS provider can decide whether or not to execute the code that the customer has provided to the IaaS provider.

[0073] In some examples, an IaaS provider's customer may request temporary network access to the IaaS provider and a function to be attached to a dataplane tier application 846. The code to perform the function may run in VM866(1)~(N), and this code does not need to be configured to run anywhere else on the dataplane VCN818. Each VM866(1)~(N) may be connected to one customer tenancy 870. The corresponding containers 871(1)~(N) contained within VM866(1)~(N) may be configured to run the code. In this case, a double isolation may exist (for example, container 871(1)~(N) runs the code, and container 871(1)~(N) may be contained in an untrusted application subnet 862, or at least in VM866(1)~(N)), but this may help prevent malicious or otherwise undesirable code from damaging the IaaS provider's network or another customer's network. Containers 871(1)-(N) can be communicatively coupled to customer tenancy 870 and can be configured to send or receive data to or from customer tenancy 870. Containers 871(1)-(N) do not need to be configured to send or receive data to or from any other entity in the data plane VCN818. Once code execution is complete, the IaaS provider can destroy or otherwise dispose of containers 871(1)-(N).

[0074] In some embodiments, a trusted application subnet 860 can execute code that may be owned or operated by the IaaS provider. In this embodiment, the trusted application subnet 860 may be communicatively coupled to a DB subnet 830 and configured to perform CRUD operations in the DB subnet 830. The resubnet 862 may be communicatively joined to the DB subnet 830, but in this embodiment, the untrusted application subnet may be configured to perform read operations in the DB subnet 830. Containers 871(1)~(N) that can be included in each customer's VM866(1)~(N) and execute code from the customer do not need to be communicatively joined to the DB subnet 830.

[0075] In other embodiments, the control plane VCN816 and the data plane VCN818 do not need to be directly communicatively coupled. In this embodiment, direct communication may not exist between the control plane VCN816 and the data plane VCN818. However, communication can be indirectly achieved by at least one method. The LPG810 may be established by an IaaS provider that can facilitate communication between the control plane VCN816 and the data plane VCN818. In another example, the control plane VCN816 or the data plane VCN818 can make a call to the cloud service 856 via the service gateway 836. For example, a call from the control plane VCN816 to the cloud service 856 may include a request for a service that can communicate with the data plane VCN818.

[0076] Figure 9 is a block diagram 900 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service provider 902 (e.g., service provider 602 in Figure 6) may be communicatively coupled to a secure host tenancy 904 (e.g., secure host tenancy 604 in Figure 6), which may include a virtual cloud network (VCN) 906 (e.g., VCN606 in Figure 6) and a secure host subnet 908 (e.g., secure host subnet 608 in Figure 6). VCN906 may include an LPG910 (e.g., LPG610 in Figure 6), which may be communicatively coupled to an SSH VCN912 (e.g., SSH VCN612 in Figure 6) via an LPG910 included in the SSH VCN912. SSH VCN912 may include SSH subnet 914 (e.g., SSH subnet 614 in Figure 6), and SSH VCN912 may be communicably coupled to control plane VCN916 (e.g., control plane VCN616 in Figure 6) via LPG910 included in control plane VCN916, and to data plane VCN918 (e.g., data plane 618 in Figure 6) via LPG910 included in data plane VCN918. Control plane VCN916 and data plane VCN918 may be included in service tenancy 919 (e.g., service tenancy 619 in Figure 6).

[0077] The control plane VCN916 may include a control plane DMZ layer 920 (e.g., control plane DMZ layer 620 in Figure 6) which may include an LB subnet 922 (e.g., LB subnet 622 in Figure 6), a control plane application layer 924 (e.g., control plane application layer 624 in Figure 6) which may include an application subnet 926 (e.g., application subnet 626 in Figure 6), and a control plane data layer 928 (e.g., control plane data layer 628 in Figure 6) which may include a DB subnet 930 (e.g., DB subnet 830 in Figure 8). The LB subnet 922 included in the control plane DMZ layer 920 can be communicatively coupled to the application subnet 926 included in the control plane application layer 924, and to an internet gateway 934 (e.g., internet gateway 634 in Figure 6) which may be included in the control plane VCN 916. The application subnet 926 can be communicatively coupled to the DB subnet 930 included in the control plane data layer 928, and to a service gateway 936 (e.g., service gateway in Figure 6) and a network address translation (NAT) gateway 938 (e.g., NAT gateway 638 in Figure 6). The control plane VCN 916 may include the service gateway 936 and the NAT gateway 938.

[0078] The data plane VCN918 may include a data plane application layer 946 (e.g., data plane application layer 646 in Figure 6), a data plane DMZ layer 948 (e.g., data plane DMZ layer 648 in Figure 6), and a data plane data layer 950 (e.g., data plane data layer 650 in Figure 6). The data plane DMZ layer 948 may include an LB subnet 922 that can be communicatively coupled to the trusted application subnet 960 (e.g., trusted application subnet 860 in Figure 8) and untrusted application subnet 962 (e.g., untrusted application subnet 862 in Figure 8) of the data plane application layer 946, and to the internet gateway 934 included in the data plane VCN918. The trusted application subnet 960 may be communicatively coupled to the service gateway 936 included in the data plane VCN918, the NAT gateway 938 included in the data plane VCN918, and the DB subnet 930 included in the data plane data layer 950. An untrusted application subnet 962 may be communicatively coupled to a service gateway 936 included in the data plane VCN 918 and a DB subnet 930 included in the data plane data layer 950. The data plane data layer 950 may include a DB subnet 930 that can be communicatively coupled to a service gateway 936 included in the data plane VCN 918.

[0079] An untrusted application subnet 962 may include primary VNICs 964(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 966(1)-(N) residing within the untrusted application subnet 962. Each tenant VM 966(1)-(N) can execute code within its corresponding container 967(1)-(N) and may be communicatively coupled to an application subnet 926 that may be included in a dataplane application layer 946 that may be included in a container output VCN 968. Corresponding secondary VNICs 972(1)-(N) may facilitate communication between the untrusted application subnet 962 included in a dataplane VCN 918 and the application subnet included in a container output VCN 968. A container output VCN may include a NAT gateway 938 that can be communicatively coupled to the public internet 954 (e.g., public internet 654 in Figure 6).

[0080] The Internet gateway 934, included in the control plane VCN916 and the data plane VCN918, can be communicatively coupled to a metadata management service 952 (e.g., the metadata management system 652 in Figure 6), which can be communicatively coupled to the public internet 954. The public internet 954 can be communicatively coupled to a NAT gateway 938, included in the control plane VCN916 and the data plane VCN918. The service gateway 936, included in the control plane VCN916 and the data plane VCN918, can be communicatively coupled to a cloud service 956.

[0081] In some examples, the pattern shown by the architecture in block diagram 900 of Figure 9 may be considered an exception to the pattern shown by the architecture in block diagram 800 of Figure 8, which may be desirable for the IaaS provider's customers when the IaaS provider cannot communicate directly with the customers (e.g., in isolated areas). The corresponding containers 967(1)~(N) contained within each customer's VM966(1)~(N) are accessible to the customer in real time. The containers 967(1)~(N) may be configured to make calls to the corresponding secondary VNICs 972(1)~(N) contained within the application subnet 926 of the data plane application layer 946, which may be contained within the container output VCN968. The secondary VNICs 972(1)~(N) can send calls to a NAT gateway 938, which can send calls to the public internet 954. In this example, the containers 967(1)~(N) accessible to the customer in real time can be isolated from the control plane VCN916 and from other entities contained within the data plane VCN918. Containers 967(1)~(N) are isolated from resources from other customers. It is possible.

[0082] In another example, a customer can use containers 967(1)-(N) to invoke the cloud service 956. In this example, the customer may execute code in containers 967(1)-(N) that requests the service from the cloud service 956. Containers 967(1)-(N) can send this request to secondary VNICs 972(1)-(N), which can then send the request to a NAT gateway that can send the request to the public internet 954. The public internet 954 can send the request to the LB subnet 922, which is included in the control plane VCN 916, via the internet gateway 934. In response to determining that the request is valid, the LB subnet can send the request to the application subnet 926, which can then send the request to the cloud service 956 via the service gateway 936.

[0083] Please note that the IaaS architectures 600, 700, 800, and 900 depicted in the figures may have components other than those shown. Furthermore, the embodiments shown in the figures are only a few examples of cloud infrastructure systems that may incorporate embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than those shown in the figures, may combine two or more components, or may have components with different configurations or arrangements.

[0084] In some embodiments, the IaaS system described herein may include a suite of applications, middleware, and database service products delivered to the customer in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is Oracle Cloud Infrastructure (OCI) by the Assignee.

[0085] Figure 10 shows an exemplary computer system 1000 in which various embodiments can be implemented. System 1000 can be used to implement any of the computer systems described above. As shown in the figure, computer system 1000 includes a processing unit 1004 that communicates with a number of peripheral subsystems via a bus subsystem 1002. These peripheral subsystems may include a processing acceleration unit 1006, an I / O subsystem 1008, a storage subsystem 1018, and a communication subsystem 1024. The storage subsystem 1018 includes a tangible computer-readable storage medium 1022 and system memory 1010.

[0086] The bus subsystem 1002 provides a mechanism for various components and subsystems of the computer system 1000 to communicate with each other as intended. Although the bus subsystem 1002 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus 1002 can be any of several types of bus structures that use any of the various bus architectures, including a memory bus or memory controller, peripheral bus, and local bus. For example, such architectures include the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA (EISA) bus, and the Video Electronics Standards Association (VESA) local bus. Examples include the LUBAS and the Peripheral Component Interconnect (PCI) bus, which can be implemented as a mezzanine bus compliant with the IEEE P1386.1 standard. It is possible.

[0087] The processing unit 1004 can be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers) and controls the operation of the computer system 1000. The processing unit 1004 may include one or more processors. These processors may include single-core or multi-core processors. In one embodiment, the processing unit 1004 may be implemented as one or more independent processing units 1032 and / or 1034, each containing a single-core or multi-core processor. In another embodiment, the processing unit 1004 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors onto a single chip.

[0088] In various embodiments, the processing unit 1004 can execute various programs in response to program code and can maintain multiple concurrently running programs or processes. At any given time, some or all of the program code to be executed can be deployed to the processor 1004 and / or the storage subsystem 1018. With appropriate programming, the processor 1004 can achieve the various functionalities described above. The computer system 1000 may also include a processing acceleration unit 1006, which may include a digital signal processor (DSP), a special-purpose processor, and / or similar.

[0089] The I / O subsystem 1008 may include user interface input devices and user interface output devices. Examples of user interface input devices include pointing devices such as keyboards, mice, or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. Examples of user interface input devices include motion sensing and / or gesture recognition devices, such as the Microsoft Kinect® motion sensor, which allows the user to control and interact with input devices, such as the Microsoft Xbox® 360 game controller, through a natural user interface using gestures and voice commands. Other examples of user interface input devices include eye gesture recognition devices, such as the Google Glass® blink detector, which detects the user's eye activity (e.g., blinking during photo taking and / or menu selection) and translates the eye gesture into input to an input device (e.g., Google Glass®). In addition, as a user interface input device, we can mention a voice recognition sensing device that enables users to interact with a voice recognition system (for example, Siri® Navigator) using voice commands.

[0090] Other examples of user interface input devices, though not limited to them, include 3D mice, joysticks or pointing sticks, gamepads, and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. In addition, medical imaging input devices such as computed tomography scanners, magnetic resonance imaging scanners, positional radiography scanners, and medical ultrasound machines can also be considered as user interface input devices. Other examples of audio input devices such as MIDI keyboards and digital musical instruments can also be considered as user interface input devices.

[0091] User interface output devices include non-visual displays such as display subsystems, indicator lights, or audio output devices. Display subsystems may include flat panel devices using cathode ray tubes (CRTs), liquid crystal displays (LCDs), or plasma displays, projection devices, touchscreens, etc. Generally, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 1000 to a user or another computer. For example, user interface output devices could include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0092] The computer system 1000 may include a storage subsystem 1018 that provides a tangible, non-temporary, computer-readable storage medium for storing software and data structures that realize the functionality of the embodiments described in this disclosure. The software may include programs, code modules, instructions, scripts, etc., that, when executed by one or more cores or processors of the processing unit 1004, realize the functionality described above. The storage subsystem 1018 may also provide a repository for storing data used in accordance with this disclosure.

[0093] As illustrated in the example in Figure 10, the storage subsystem 1018 may include various components, including system memory 1010, a computer-readable storage medium 1022, and a computer-readable storage medium reader 1020. System memory 1010 can store program instructions that can be loaded and executed by the processing unit 1004. System memory 1010 can also store data used during the execution of instructions, and / or data generated during the execution of program instructions. Various different types of programs may be loaded into system memory 1010, including but not limited to client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), virtual machines, and containers.

[0094] System memory 1010 can also store operating systems 1016. Examples of operating systems 1016 include various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or iOS, Windows® Phone, Android® OS, BlackBerry® OS, etc. Examples include mobile operating systems such as Palm® OS operating system. In a particular implementation where the computer system 1000 runs one or more virtual machines, the virtual machines are loaded into system memory 1010 along with their guest operating systems (GOS) and may be executed by one or more processors or cores of the processing unit 1004.

[0095] The system memory 1010 can have different configurations depending on the type of computer system 1000. For example, the system memory 1010 may be volatile memory (e.g., random access memory (RAM)) and / or non-volatile memory (e.g., read-only memory (ROM), flash memory, etc.). Various types of RAM configurations can be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), and others. In some implementations, the system memory 1010 performs information transfer between elements within the computer system 1000, such as during startup. It may include a basic input / output system (BIOS), which includes basic routines for providing support.

[0096] The computer-readable storage medium 1022 may represent remote, local, fixed, and / or removable storage devices and storage media for temporarily and / or more permanently storing computer-readable information used by the computer system 1000, including instructions executable by the processing unit 1004 of the computer system 1000.

[0097] The computer-readable storage medium 1022 can be any suitable medium known or used in the art, including storage and communication media, such as volatile and non-volatile removable and non-removable media, which are implemented in any method or technique for storing and / or transmitting information. This may include tangible computer-readable storage media, such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory, or other memory technologies; CD-ROM, digital multipurpose disk (DVD), or other optical storage; magnetic cassette, magnetic tape, magnetic disk storage device, or other magnetic storage device; or other tangible computer-readable media.

[0098] Examples of computer-readable storage media 1022 include hard disk drives that read from or write to non-removable non-volatile magnetic media, magnetic disk drives that read from or write to removable non-volatile magnetic disks, and optical disk drives that read from or write to removable non-volatile optical disks or other optical media such as CD-ROMs, DVDs, and Blu-ray® discs. Examples of computer-readable storage media 1022 include, but are not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, and digital videotapes. Other examples of computer-readable storage media 1022 include solid-state drives (SSDs) that utilize non-volatile memory, such as flash memory-based SSDs, enterprise flash drives, and solid-state ROMs, as well as SSDs that utilize volatile memory, such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. Disk drives and associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program modules, and other data of computer system 1000.

[0099] Machine-readable instructions executable by one or more processors or cores of the processing unit 1004 can be stored in a non-temporary computer-readable storage medium. The non-temporary computer-readable storage medium may include physically tangible memory, or storage devices including volatile memory storage devices and / or non-volatile storage devices. Examples of non-temporary computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard drives, floppy disk drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0100] The communication subsystem 1024 provides an interface with other computer systems and networks. The communication subsystem 1024 functions as an interface for receiving data from computer system 1000 and transmitting data from computer system 1000 to other systems. For example, the communication subsystem 1024 provides an interface with other computer systems and networks. The computer system 1000 may be enabled to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1024 may include radio frequency (RF) transceiver components, Global Positioning System (GPS) receiver components, and / or other components for accessing wireless voice and / or data networks (e.g., using cellular technology, 3G, 4G, or advanced data network technologies such as EDGE (enhanced data rates for global evolution), WiFi (IEEE 802.11 family standards, or other mobile communication technologies, or any combination thereof). In some embodiments, the communication subsystem 1024 may provide wired network connectivity (e.g., Ethernet®) in addition to or instead of a wireless interface.

[0101] In some embodiments, the communication subsystem 1024 may also receive input communications, such as structured and / or unstructured data feeds 1026, event streams 1028, and event updates 1030, on behalf of one or more users who may be using the computer system 1000.

[0102] For example, the communication subsystem 1024 handles Twitter® feeds and Facebook (Registered Trademark) updates, Rich Site Summary (RSS) feeds and other web feeds, It may be configured to receive data feeds 1026 from users of social networks and / or other communication services in real time, such as real-time updates from one or more third-party sources.

[0103] In addition, the communication subsystem 1024 may also be configured to receive data in the form of a continuous data stream. This data may include an event stream 1028 and / or event updates 1030 of real-time events, which may be inherently continuous or boundaryless without explicit termination. Examples of applications that generate continuous data include, for example, sensor data applications, financial indicators, network performance measurement tools (e.g., network monitoring applications and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0104] The communication subsystem 1024 may also be configured to output structured and / or unstructured data feeds 1026, event streams 1028, event updates 1030, etc., to one or more databases that can communicate with one or more streaming data source computers coupled to the computer system 1000.

[0105] Computer system 1000 can be one of various types, including portable devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or any other data processing systems.

[0106] Given the constantly changing nature of computers and networks, the description of computer system 1000 depicted in the figure is intended only as a specific example. Many other configurations are possible, having more or fewer components than the system depicted in the figure. For example, customized hardware may be used, and / or certain elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Furthermore, connections to other computing devices, such as network input / output devices, may be employed. Based on the disclosures and teachings provided herein, those skilled in the art will be able to find other ways and / or methods for carrying out various embodiments. They will understand the law.

[0107] While specific embodiments have been described, various modifications, alterations, alternative structures, and equivalents are also included within the scope of this disclosure. The embodiments are not limited to operation within a particular data processing environment, but can freely operate within multiple data processing environments. Furthermore, while the embodiments have been described using a specific set of transactions and steps, it will be apparent to those skilled in the art that the scope of this disclosure is not limited to the described set of transactions and steps. The various features and aspects of the embodiments described above may be used individually or in combination.

[0108] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments may be implemented using hardware alone, software alone, or a combination thereof. The various processes described herein can be carried out on the same processor or in any combination of different processors. Thus, where a component or module is described as being configured to perform a certain operation, such configuration can be achieved, for example, by designing electronic circuitry to perform that operation, by programming programmable electronic circuitry (such as a microprocessor) to perform that operation, or by any combination thereof. Processes can communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0109] This specification and its drawings should therefore be considered in an illustrative rather than restrictive sense. However, it will be apparent that additions, subtractions, deletions, and other modifications and alterations may be made to this specification and its drawings without departing from the broader spirit and scope set forth in the claims. Thus, specific embodiments of the disclosure have been described, but these are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0110] In the context describing the embodiments disclosed, the terms “a,” “an,” and “the,” as well as similar references, should be interpreted to encompass both singular and plural unless otherwise specifically indicated herein or unless it is clearly inconsistent with the context. The terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (i.e., “including but not limited to”) unless otherwise specified. The term “connected” should be interpreted to mean that some or all of them are encompassed, attached, or linked together, even if intervenes are present. The enumeration of value ranges herein is intended to serve as a concise way of referring individually to each distinct value that falls within that range unless otherwise specifically indicated herein, and each distinct value is incorporated herein as if it were individually enumerated herein. All methods described herein may be performed in any appropriate order unless otherwise specifically indicated herein or unless it is clearly inconsistent with the context. Any use of any example or illustrative language provided herein (e.g., "such as") is intended solely to better illustrate embodiments and, unless otherwise asserted, does not constitute a limitation of the scope of this disclosure. Nothing in this specification should be construed as indicating that any unclaimed element is essential for the practice of this disclosure.

[0111] "at least one of X, Y, or Z" Disjunctive wording such as "Y, or Z)" is intended to be understood in context as generally indicating that an item, term, etc., can be any one of X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z), unless explicitly stated otherwise. Therefore, such disjunctive wording is not intended, nor does it suggest, that in any embodiment, at least one of X, at least one of Y, or at least one of Z must each be present.

[0112] This specification describes preferred embodiments of the Disclosure, including the best known mode for carrying out the Disclosure. Those skilled in the art will recognize variations of these preferred embodiments by reading the preceding description. Those skilled in the art can, of course, adopt such variations as appropriate, and the Disclosure may be carried out in ways other than those specifically described herein. Therefore, to the extent permitted by applicable law, the Disclosure includes all modifications and equivalents of the subject matter described in the claims attached to this specification. Furthermore, unless otherwise specifically indicated herein, any combination of the elements described above in any possible variation is incorporated into the Disclosure.

[0113] All references cited herein, including publications, patent applications, and patents, are incorporated herein by reference to the same extent as if each reference had been included in its entirety, as indicated individually and specifically for each reference.

[0114] While aspects of the present disclosure are described in the aforementioned specification with reference to specific embodiments, those skilled in the art will recognize that the present disclosure is not limited thereto. The various features and aspects of the above-described disclosure may be used individually or in combination. Furthermore, embodiments may be used in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of this specification. This specification and the drawings should therefore be considered illustrative rather than restrictive.

Claims

1. A method by which a computer performs an action. The computing device includes monitoring the primary load and the backup load of the data center, The main load is powered by one or more main generator blocks having main capacity, and the backup load is powered by one or more backup generator blocks having backup capacity. The aforementioned method, The computing device determines whether the main load exceeds the main capacity, In response to the determination that the main load exceeds the main capacity, A method further comprising a switch controllable by the computing device connecting the auxiliary generator block to at least one of the main generator block or the main load.

2. The method according to claim 1, wherein power is supplied to the auxiliary load by the one or more auxiliary generator blocks and the commercial power supply connection, and the commercial power supply connection provides power to at least half of the auxiliary load.

3. The method according to claim 1 or 2, wherein the backup load is powered by the commercial power supply connection after the backup load is disconnected.

4. The method according to any one of claims 1 to 3, further comprising detecting that the main load exceeds the main capacity by determining that the power supplied to the backup load is below a threshold.

5. The method according to any one of claims 1 to 4, wherein detecting that the main load exceeds the main capacity includes determining that a malfunction has occurred in one of the one or more main generator blocks.

6. The method according to any one of claims 1 to 5, wherein the computing device is an industrial control system.

7. The method according to any one of claims 1 to 6, further comprising the determination that the combined load including the main load and the backup load exceeds the combined capacity including the main capacity and the backup capacity, the switch controllable by the computing device disconnects the backup load from the backup generator block.

8. The method according to any one of claims 1 to 7, further comprising reconnecting the backup load to one or more backup generator blocks in response to a determination that the combined capacity exceeds the combined load.

9. A computing system, Memory that stores computer executable program instructions, The system comprises a processing device that is communicably coupled to the memory to execute the computer executable program instructions, wherein executing the computer executable program instructions configures the processing device to perform an operation, and the operation is The computing system includes monitoring the main load and the backup load, wherein the main load is powered by one or more main generator blocks having a main capacity, The backup load is powered by one or more backup generator blocks having backup capacity. The aforementioned operation is, The computing system determines whether the main load exceeds the main capacity, In response to the determination that the main load exceeds the main capacity, A computing system further comprising a switch controllable by the computing system connecting the auxiliary generator block to at least one of the main generator block and the main load.

10. The system according to claim 9, wherein power is supplied to the auxiliary load by one or more auxiliary generator blocks and a commercial power supply connection, and the commercial power supply connection provides power to at least half of the auxiliary load.

11. The system according to claim 9 or 10, wherein the backup load is powered by the commercial power supply connection after the backup load is disconnected.

12. The system according to any one of claims 9 to 11, further comprising detecting that the main load exceeds the main capacity by determining that the power supplied to the backup load is below a threshold.

13. The system according to any one of claims 9 to 12, wherein detecting that the main load exceeds the main capacity further includes determining that a malfunction has occurred in one of the one or more main generator blocks.

14. The system according to any one of claims 9 to 13, further comprising the determination that the combined load including the main load and the backup load exceeds the combined capacity including the main capacity and the backup capacity, a switch controllable by the computing system disconnects the backup load from the backup generator block.

15. The computing system is an industrial control system, according to any one of claims 9 to 14.

16. One or more non-temporary computer-readable media for storing computer executable program instructions, wherein, when the computer executable program instructions are executed by a computing system, the computing system causes the computing system to perform an action, and the action is The computing system includes monitoring the primary load and the backup load of the data center, The main load is powered by one or more main generator blocks having main capacity, and the backup load is powered by one or more backup generator blocks having backup capacity. The aforementioned operation is, The computing system determines whether the main load exceeds the main capacity, In response to determining that the main load exceeds the main capacity, One or more non-temporary computer-readable media, further comprising: a switch controllable by the computing system connecting the auxiliary generator block to at least one of the main generator block and the main load.

17. The one or more auxiliary generator blocks and the commercial power supply connection provide power to the auxiliary load. One or more non-temporary computer-readable media according to claim 16, wherein power is supplied and the commercial power connection provides power to at least half of the backup load.

18. The backup load is powered by the commercial power connection after the backup load is disconnected, one or more non-temporary computer-readable media according to claim 16 or 17.

19. One or more non-temporary computer-readable media according to any one of claims 16 to 18, wherein detecting that the main load exceeds the main capacity further includes determining that a malfunction has occurred in one of the one or more main generator blocks.

20. One or more non-temporary computer-readable media according to any one of claims 16 to 19, further comprising the calculation that a combined load including the main load and the backup load exceeds a combined capacity including the main capacity and the backup capacity, a switch controllable by the computing system disconnects the backup load from the backup generator block.

Images