Increasing resource utilization in cloud computing clusters

Abstract

A computer-implemented method for provisioning a cloud computing cluster includes receiving a request to create a cloud computing cluster, the cloud computing cluster having a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster, and initiating an initialization process for a single processing node and corresponding attached storage device in response to receiving the request to create the cloud computing cluster. The method may also include, prior to completion of the initialization process, requesting and receiving IP addresses of the single processing nodes and device IDs of the corresponding attached storage devices from one or more cloud infrastructure controllers, and configuring the clustered file system and corresponding WAN cache using the received IP addresses and the received device IDs. Systems and computer program products corresponding to the above methods are also disclosed herein. (Figure 2)

Description

[Background technology]

[0001] The subject matter disclosed herein relates generally to launching cloud-based applications and, more particularly, to provisioning cloud computing clusters and associated resources.

[0002] Currently, cloud-based applications, when deployed, experience significant delays in resource availability. For example, consider a cloud bursting scenario in which a customer intends to immediately start scaling an application to the cloud, and the customer may wait for each resource to be allocated and initialized. Such a scenario can result in a delay of an hour or more before the cloud-based application can begin processing. Following the start of processing, significant additional time is often required to load the large datasets associated with cloud-based applications. [Summary of the Disclosed Embodiments]

[0003] A computer-implemented method for provisioning a cloud computing cluster includes receiving a request to create a cloud computing cluster, the cloud computing cluster having a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster, and initiating an initialization process for a single processing node and corresponding attached storage device in response to receiving the request to create the cloud computing cluster. The method may also include, prior to completion of the initialization process, requesting and receiving IP addresses of the single processing nodes and device IDs of the corresponding attached storage devices from one or more cloud infrastructure controllers, and configuring the clustered file system and corresponding WAN cache using the received IP addresses and the received device IDs.

[0004] Systems and computer program products incorporating the above methods are also disclosed herein. The computer program product includes a computer-readable storage medium having program instructions embodied thereon, where the computer-readable storage medium is not itself a transitory signal, and the program instructions are executable by a processor to cause the processor to perform the above methods. The system includes one or more processors and a computer-readable storage medium similar to the computer-readable storage medium included in the computer program product. [Brief explanation of the drawings]

[0005] So that the advantages of the disclosed embodiments can be readily understood, a more particular description of the embodiments briefly described above will be made by reference to specific embodiments that are illustrated in the accompanying drawings. It being understood that these drawings illustrate only some embodiments and are therefore not to be considered limiting in scope, the embodiments will be described and explained with additional specificity and detail through the use of the following accompanying drawings.

[0006] [Figure 1] FIG. 1 is a block diagram illustrating various portions of a computing environment in accordance with at least one embodiment disclosed herein.

[0007] [Figure 2] 1 is a flowchart of an example method for provisioning a cloud computing cluster according to at least one embodiment disclosed herein.

[0008] [Figure 3] 1 is a flowchart of an example method for adding resources to a cloud computing cluster, according to at least one embodiment disclosed herein.

[0009] [Figure 4]1 is a flowchart of an example method for releasing a cloud computing cluster, according to at least one embodiment disclosed herein.

[0010] [Figure 5] FIG. 1 is a block diagram illustrating an example of a computing stack in accordance with at least one embodiment disclosed herein. DETAILED DESCRIPTION OF THE INVENTION

[0011] Those skilled in the art will understand that references throughout this specification to "one embodiment," "an embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. Thus, appearances of the phrases "in one embodiment," "in an embodiment," and similar language throughout this specification mean "one or more, but not all, embodiments," unless expressly specified otherwise, although they may, but do not necessarily, all refer to the same embodiment. The terms "including," "comprising," and "having," and variations thereof, mean "including, but not limited to," unless expressly specified otherwise. An enumerated list of items does not imply that any or all of the items are mutually exclusive and / or mutually inclusive, unless expressly specified otherwise. The terms "a," "an," and "the" also refer to "one or more," unless expressly specified otherwise.

[0012] The techniques and solutions disclosed herein reduce delays experienced by cloud computing clients when provisioning cloud computing clusters and associated resources for use in running cloud-based applications.

[0013] 1 is a block diagram illustrating various portions of a computing environment 100 in accordance with at least one embodiment disclosed herein. Computing environment 100 includes an example environment for execution of at least some of the computer code associated with performing an inventive method or process, such as code block 201 (corresponding to method 200 shown in FIG. 2). In some embodiments, portions of code block 201 reside within operating system 122. In addition to block 201, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes a set of processors 110 (including processing circuitry 120 and cache 121), a communications fabric 111, volatile memory 112, persistent storage 113 (including operating system 122 and block 201 shown above), a set of peripheral devices 114 (including a set of user interface (UI) devices 123, storage 124, and a set of Internet of Things (IoT) sensors 125), and a network module 115. Remote server 104 includes a remote database 130. Public cloud 105 includes a gateway 140, a cloud orchestration module 141, a set of host physical machines 142, a set of virtual machines 143, and a set of containers 144.

[0014] Computer 101 may take the form of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch or other wearable computer, a mainframe computer, a quantum computer, or any other form of computer or mobile device now known or later developed that is capable of executing programs, accessing a network, or querying a database, such as remote database 130. As is well understood in the field of computer technology, and depending on the technology, execution of a computer-implemented method may be distributed among multiple computers and / or among multiple locations. However, in this description of computing environment 100, for purposes of brevity, the detailed discussion focuses on a single computer, specifically computer 101. While computer 101 is not shown in FIG. 1 within a cloud, it may be located within a cloud. However, computer 101 is not required to reside within a cloud except to any extent that may be expressly indicated.

[0015] Processor set 110 includes one or more computer processors of any type now known or later developed. Processing circuitry 120 may be distributed across multiple packages, e.g., multiple linked integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by threads or cores executing on processor set 110. Cache memory is typically organized into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all caches for a processor set may be located “off-chip.” In some computing environments, processor set 110 may be designed to operate with qubits and perform quantum computing.

[0016] Computer-readable program instructions are typically loaded into computer 101 and cause a series of operational steps to be executed by processor set 110 of computer 101, thereby realizing a computer-implemented process, such that the instructions so executed instantiate the methods specified in the flowcharts and / or descriptions of the computer-implemented methods contained herein (collectively referred to as "invention methods"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by processor set 110 to control and direct the execution of the inventive methods. In computing environment 100, at least some of the instructions for executing the inventive methods may be stored in block 201 within persistent storage 113.

[0017] Communications fabric 111 is the signal-conducting pathway that allows various components of computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive pathways, such as those that make up buses, bridges, physical input / output ports, etc. Other types of signal communication pathways may be used, such as fiber optic and / or wireless communication pathways.

[0018] Volatile memory 112 may be any type of volatile memory now known or later developed. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 112 is characterized by random access, although this is not required unless expressly stated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 101.

[0019] Persistent storage 113 is any form of non-volatile storage for a computer, now known or later developed. The non-volatility of this storage means that stored data remains regardless of whether power is supplied to computer 101 and / or directly to persistent storage 113. While persistent storage 113 may be read-only memory (ROM), typically at least a portion of persistent storage allows data to be written, data to be deleted, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 122 may take several forms, such as various known proprietary operating systems employing a kernel or open-source Portable Operating System Interface-type operating systems. The code included in block 201 typically includes at least some of the computer code related to performing the methods of the invention, such as provisioning a cloud computing cluster and associated resources.

[0020] Peripheral device set 114 includes a set of peripheral devices of computer 101. Data communication connections between peripheral devices and other components of computer 101 may be implemented in various ways, such as Bluetooth® connections, near field communication (NFC) connections, connections made by cables (such as Universal Serial Bus (USB)-type cables), insertion-type connections (e.g., Secure Digital (SD) cards), connections made through local area communication networks, and even connections made through wide area networks such as the Internet. In various embodiments, UI device set 123 may include components such as display screens, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage 124 may be external storage, such as an external hard drive, or insertable storage, such as an SD card. Storage 124 may be persistent and / or volatile. In some embodiments, storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (e.g., where computer 101 stores and manages large databases locally), this storage may be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple, geographically distributed computers. IoT sensor set 125 consists of sensors that may be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0021] Network module 115 is a collection of computer software, hardware, and firmware that enables computer 101 to communicate with other computers over WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for implementing the methods of the present invention may be downloaded to computer 101 from an external computer or external storage device, typically through a network adapter card or network interface included in network module 115.

[0022] WAN 102 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology for communicating computer data, now known or later developed. In some embodiments, WAN 102 may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include copper transmission cables, optical fiber transmissions, wireless transmissions, and computer hardware such as routers, firewalls, switches, gateway computers, and edge servers.

[0023] End-user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise that operates computer 101) and may take any of the forms described above with respect to computer 101. EUD 103 typically receives useful and useful data from the operation of computer 101. For example, in the hypothetical case where computer 101 is designed to provide recommendations to the end user, the recommendations would typically be communicated from network module 115 of computer 101 over WAN 102 to EUD 103. In this manner, EUD 103 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 103 may be a client device such as a thin client, a heavy client, a mainframe computer, a desktop computer, and the like.

[0024] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 may be controlled and used by the same entity that operates computer 101. Remote server 104 represents a machine that collects and stores useful and useful data for use by other computers, such as computer 101. For example, in the hypothetical case where computer 101 is designed and programmed to provide recommendations based on historical data, then this historical data may be provided to computer 101 from remote database 130 of remote server 104.

[0025] A public cloud 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer functionality, particularly data storage (cloud storage) and computing power, without direct active management by users. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct active management of the computing resources of the public cloud 105 is performed by computer hardware and / or software in a cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments running on various computers that comprise a host physical machine set 142, which is the universe of physical computers within and / or available to the public cloud 105. The virtual computing environments (VCEs) typically take the form of virtual machines from a virtual machine set 143 and / or containers from a container set 144. It is understood that these VCEs may be stored as images and transferred among and between various hosts of physical machines either as images or after instantiation of the VCE. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages active instantiations of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that enables public cloud 105 to communicate over WAN 102.

[0026] Here, we provide some further explanation of virtual computing environments (VCEs). A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from the image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of an operating system in which the kernel allows the existence of multiple isolated user space instances called containers. These isolated user space instances typically behave as actual computers from the perspective of the programs running within them. A computer program running on a typical operating system can utilize all of the computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and the devices assigned to the container; this feature is known as containerization.

[0027] Private cloud 106 is similar to public cloud 105, except that the computing resources are available only for use by a single enterprise. While private cloud 106 is shown in communication with WAN 102, in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. While each of the multiple clouds remains a separate, discrete entity, the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the constituent clouds. In this embodiment, both public cloud 105 and private cloud 106 are part of a larger hybrid cloud.

[0028] 2 is a flowchart of an example method 200 for provisioning a cloud computing cluster, in accordance with at least one embodiment disclosed herein. As shown, the method includes receiving a request to create a cloud computing cluster (210), initiating an initialization process (220), requesting and receiving an IP address and device ID (230), configuring a clustered file system and corresponding WAN cache (240), determining whether the initialization process is complete (260), configuring one or more threads (270), retrieving data (280), and adding additional resources (290). The illustrated method reduces initialization delays experienced by cloud applications when provisioning a cloud computing cluster and associated resources.

[0029] Receiving 210 a request to create a cloud computing cluster may include receiving a request specifying a requested number of processing nodes and attached storage devices to be allocated by the cloud computing cluster. The request may specify a file system type, such as a clustered file system.

[0030] Initiating 220 the initialization process may include initiating the initialization process for a single processing node and corresponding attached storage device. The initiated initialization process may be limited to a single processing node and corresponding attached storage device to reduce initialization latency and imposed wait time before useful processing can occur. Furthermore, the initiating party may offload the initialization process and continue additional useful operations (e.g., operations 230 and 240) while the initialization process is completing.

[0031] Requesting and receiving 230 IP addresses and device IDs may include requesting IP addresses of processing nodes and device IDs of corresponding attached storage devices. The request may be submitted to one or more cloud infrastructure controllers, which may respond with the IP addresses and device IDs.

[0032] Configuring 240 the clustered file system and corresponding WAN cache may include using the received IP address and device ID to initialize various configuration settings and parameters in the clustered file system and corresponding WAN cache, which settings and parameters may be changed before the initialization process is complete.

[0033] Determining 260 whether the initialization process is complete may include testing a semaphore that indicates the status of the initialization process. Alternatively, a callback function may be invoked by the initialization process when initialization of the single processing node and corresponding attached storage device is complete. The callback function may reactivate method 200 in operation 270.

[0034] Configuring 270 one or more threads may include configuring threads associated with a single processing node. Obtaining 280 data may include obtaining data needed by threads associated with the single processing node. An effect of obtaining data may be placing the data in a WAN cache.

[0035] Adding additional resources (290) may include adding additional resources to the cloud computing cluster as required by the cloud computing cluster, such as additional processing nodes and corresponding attached storage devices.

[0036] Those skilled in the art will appreciate that the above-described method for provisioning a cloud computing cluster allows a cloud application to begin executing application threads before all resources associated with the cloud computing cluster are fully provisioned. As a result, cloud-based applications may begin processes sooner and delays associated with loading new cloud-based applications may be reduced.

[0037] 3 is a flowchart of an example method 300 for adding resources to a cloud computing cluster, according to at least one embodiment disclosed herein. As shown, method 300 includes initiating initialization of the additional resource (310), requesting and receiving an ID (320), and configuring the cloud computing cluster (330).

[0038] Initiating initialization of additional resources (310) may include dispatching an initialization process to the additional processing node or attached storage device. Requesting and receiving IDs (320) may be similar to requesting and receiving operation 240 and may include requesting IP addresses of the additional processing nodes and / or device IDs of corresponding attached storage devices. The request may be submitted to one or more cloud infrastructure controllers, which may respond with the IP addresses and / or device IDs.

[0039] Configuring 330 the cloud computing cluster may include using the received IDs to configure clustered file system data structures in such a way that they are populated with processing node details (that are still under-provisioned) and / or storage device details (such as labels that are still under-provisioned). Those skilled in the art will appreciate that the above approach to adding resources to a cloud computing cluster enables cloud-based applications to use the resources that have been provisioned and to prepare to progressively use additional resources associated with the cloud computing cluster once they are fully provisioned.

[0040] 4 is a flowchart of an example method 400 for releasing a cloud computing cluster, according to at least one embodiment disclosed herein. As shown, the method includes determining changes to data (410), committing the changes (420), identifying processing nodes and storage devices (430), and releasing the processing nodes and storage devices (440).

[0041] Determining 410 changes to the data may include accessing cache coherency data to determine data that has been updated in the cache but not written to secondary storage. Committing 420 changes may include writing the data to secondary storage. Identifying 430 processing nodes and storage devices may include communicating with a cloud infrastructure controller to determine allocated but not yet released processing nodes and storage devices.

[0042] Releasing 440 the processing nodes and storage devices may include sending a request to a cloud infrastructure controller to release the allocated, but not yet released, processing nodes and storage devices.

[0043] The methods disclosed herein may be embodied partially or completely within the cloud resource provisioning code block 201 shown in Figure 1. Those skilled in the art will understand that the methods disclosed herein may be adapted to the cloud computing environment in which they are deployed without changing the intent of the disclosed methods. Those skilled in the art will also understand the usefulness and effectiveness of the methods and solutions disclosed herein.

[0044] The disclosed solution essentially consists in: A provisioning framework is provided that helps optimize the overall time / dollars spent for WAN caching solution deployment (or preparing a solution, such as software, hardware, or middleware, for an application) in public / hybrid cloud environments by pre-populating clustered file system records with (not yet provisioned) instance / VM / node details and expanding (flattening) disk structures with (not yet provisioned) disk (volume / EBS) details to create expanded node files, allocation maps, and log files. Furthermore, data prefetching threads can be scaled (where the speed of scaling depends on the infrastructure provisioning speed), and floating-window-based restriping of storage devices can occur (where the window size depends on the disk size added to the pre-expanded file system). Consequently, WAN caching (populating the dataset required by the application) can begin during the cluster provisioning phase itself (as opposed to starting it after the cluster configuration phase).

[0045] The described framework offers the following features and flexibility over traditional clustered file system based WAN caching deployment solutions in hybrid / public cloud: When the cloud deployment manager receives a request to create a WAN caching clustered file system with a defined threshold of nodes, the disks it runs on (where the threshold may increase or decrease depending on the cloud infrastructure processing unit workload); A single processing node / VM will be provisioned with an attached storage device. The initial device type and size may depend on the network bandwidth used to ingest data as part of WAN caching vs. the overall provisioning time required to reach full file system capacity, and is usually a fraction of the file system capacity. A request is made to the cloud infrastructure processing unit (IPU) (also known as the cloud infrastructure controller) to share node IP details and storage device IDs prior to node and device provisioning. The cloud IPU reserves these details and uses them to provision resources at the configured infrastructure processing rate. Clustered filesystem and WAN caching relationships are initially set up using a single (processing) node cluster. Clustered filesystem data structures are configured in such a way that they are populated with processing node details (still under-provisioned) and disk details (still under-provisioned, labels, etc.). Processing threads are configured and data prefetching begins. The number of threads increases based on the number of nodes that are provisioned and joining the cluster. This sequencing helps scale WAN cache data prefetching linearly based on the number of new nodes joining the cluster. The deployment continues by spinning up new nodes, attaching NSDs, and adding storage devices to the file system. Data prefetching continues and the framework linearly scales disk stripping (using floating window based data restriping) so that data fragments stored on the initial disk start migrating to other newly provisioned disks. In case of erasure in an erasure-encoder-based clustered file system, data prefetching continues and the framework linearly scales erasure-coded data segment migration (which can be an "rsync" equivalent operation) so that data fragments stored on the initial disk start migrating to other newly provisioned disks. When the cloud deployment manager receives a request for teardown of a WAN caching clustered file system, a similar flow is used, where nodes and attached storage devices are selected, changes to the data are identified, and the changes are replayed to object storage or a remote site based on WAN caching, resulting in an incremental contraction of the file system data structures. The identities of the selected nodes and attached storage devices are sent to the cloud IPU for resource deallocation ("destruction").

[0046] The following may occur in conjunction with the above framework and methodology: User enters the required number of nodes / VMs and NSDs (disks) along with their sizes into the cloud deployment manager / IaC tool. The framework identifies the current workload on the cloud infrastructure processing (IPU) units, and if the set processing IPU speed is lower and takes longer to provision, it will cause applications to wait (speed and duration are user configurable, as cloud vendors publish or observe speed of infrastructure provisioning duration). Provision a single node+disk attached or minimally supported file system blueprint and configure WAN caching. During creation of the minimally supported blueprint, the framework extends data structures to pre-allocate remaining blocks, ranges, allocation maps, and log groups for future attached (or not yet provisioned) NSD disks. The application starts prefetching data and begins working with the data. As soon as other NSDs reach a situation where they are available, they are simply linked to these pre-provisioned data structures (or some special counter flags may be enabled). Prefetch thread (pthreads) configuration is dynamically changed (increased) based on the rate at which new nodes / NSDs are added. Configuration change for restriping (floating window pattern) does not do a full stripe, but rather just stripes based on the calculated NSD attachments that will occur in the future. Explanation: (Number of files = 8, NSD = 8) Iteration-1: File-1 was fetched by NSD-1 Iteration 2: File 2 is fetched from NSD-1 and NSD-2. Restriping is performed and only stripe-2 blocks of File 1 are moved to NSD-2. Iteration-3: File-3 is fetched from NSD-1, NSD-2, and NSD-3. Re-stripe is triggered, and only stripe-3 blocks from File-1 are moved to NSD-3, and stripe-3 blocks from File-2 are moved to NSD-2. For teardown, the cloud deployment manager receives a request to tear down the WAN caching clustered file system. A similar flow is used, where nodes, disks are selected, changes are identified, changes are replayed to object storage or a remote site based on WAN caching, and the file system data structures are collapsed. The selected nodes are sent to the cloud IPU for destruction.

[0047] As disclosed herein, a computer-implemented method for provisioning a cloud computing cluster includes: receiving a request to create a cloud computing cluster, the cloud computing cluster having a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; Initiating an initialization process for a single processing node and a corresponding attached storage device in response to receiving the request to create the cloud computing cluster. requesting and receiving IP addresses of the single processing nodes and device IDs of the corresponding attached storage devices from one or more cloud infrastructure controllers prior to completion of the initialization process. configuring the clustered file system and corresponding WAN cache using the received IP addresses and the received device IDs before completing the initialization process; may include:

[0048] Additional features of the above method are: configuring one or more threads for the single processing node in response to completion of the initialization process. obtaining data for the one or more threads in response to completion of the initialization process. wherein obtaining data for one or more threads includes adding the data to the WAN cache. In response to completion of the initialization process, adding additional resources to the cloud computing cluster until the requested number of processing nodes and attached storage devices is achieved. Adding additional resources to the cloud computing cluster comprises initiating initialization of the additional resources. requesting and receiving identification of the additional resource from the one or more cloud infrastructure controllers before completing initialization of the additional resource. using the ID of the additional resource to further configure the cloud computing cluster before completing initialization of the additional resource. wherein the additional resource is an additional processing node. configuring at least one thread for said additional processing node; obtaining data for the at least one thread performing a computing cluster termination process in response to receiving a request to terminate the cloud computing cluster. Wherein the cluster termination process: determining changes to the captured data in the WAN cache; Committing the changes to local storage or a remote site Identifying processing nodes and corresponding attached storage devices associated with the cloud computing cluster. Releasing the processing node and the corresponding attached storage device have may include:

[0049] Systems and computer program products corresponding to the above methods are also disclosed herein. Various aspects of the present disclosure are described through narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. For any flowchart, depending on the technology involved, operations may be performed in a different order than shown in a given flowchart. For example, again depending on the technology involved, two operations shown in successive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in an at least partially overlapping manner.

[0050] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any set of one or more storage media (also referred to as "media") collectively included in a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded device (such as pits / lands formed on the major surface of a punch card or disk), or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in this disclosure, is not to be construed as storage in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals communicated through wires, and / or other transmission media. As will be appreciated by those skilled in the art, data is typically moved at some infrequent time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but the foregoing does not qualify a storage device as transitory because the data is not transitory while it is stored.

[0051] FIG. 5 is a block diagram illustrating one example of a computing stack 570, in accordance with at least one embodiment disclosed herein. As illustrated, the computing stack 570 includes a number of computing layers 572 used to perform computing operations. In the illustrated embodiment, the layers include hardware layers and software layers. The various software layers include an operating system layer associated with executing one or more operating systems, a middleware layer associated with executing middleware that extends and / or improves the functionality of the hardware layer, and the operating system. The software layers may also include various application-specific layers. The application-specific layers may include application frameworks that further extend and / or improve the functionality of the hardware and operating system layers.

[0052] The memory layer may include volatile memory, non-volatile memory, persistent storage, and hardware associated with controlling such memory. The logic units may include CPUs, arithmetic units, graphics processing units, and hardware associated with controlling such units. The microcode layer may include executable instructions for controlling process flow associated with moving data between memory and logic units. The processor layer may include instruction fetch units, instruction decode units, and the like that enable execution of processing instructions and utilization of the underlying hardware layers.

[0053] Hardware drivers (also known as the hardware abstraction layer) may contain executable code that allows the operating system to access and control storage devices, DMA hardware, I / O buses, peripheral devices, and other hardware associated with a computing environment. The operating system kernel layer may receive I / O requests from higher layers and manage memory and other hardware resources through hardware drivers. The operating system kernel layer may also provide other functions such as inter-process communication and file management.

[0054] Operating system libraries and utilities may extend the functionality provided by the operating system kernel and provide interfaces for accessing those functions. Libraries are typically utilized by higher layers of software by linking library object code into higher-level software executables. In contrast, operating system utilities are typically standalone executables that can be invoked via an operating system shell that receives commands and / or script files from the user. Examples of operating system libraries include file I / O libraries, math libraries, memory management libraries, process control libraries, data access libraries, and the like. Examples of operating system utilities include antivirus managers, disk formatters, disk defragmenters, file compressors, data or file sorters, data archivers, memory testers, program installers, package managers, network utilities, system monitors, system profilers, and the like.

[0055] Services are often provided by running executables or processes that receive local or remote requests from other processes or devices, called clients. Computers running services are often referred to as servers. Examples of servers include database servers, file servers, mail servers, print servers, web servers, game servers, and application servers.

[0056] Application frameworks provide functionality commonly required by applications and include system infrastructure frameworks, middleware integration frameworks, enterprise application frameworks, graphical rendering frameworks, and gaming frameworks. An application framework may support application development for a specific environment or industry. In some cases, an application framework is available for multiple operating systems, providing developers with a common programming interface across multiple platforms.

[0057] General-purpose applications include applications needed by most users. Examples of general-purpose applications include email applications, calendaring and scheduling applications, and web browsers. Such applications may be automatically included with the operating system.

[0058] Those skilled in the art will understand that improvements to any of the illustrated layers, or similar layers not illustrated herein, will result in improvements to the computer itself, including computer 101 and / or end-user device 103. Those skilled in the art will also understand that the illustrated layers, given by way of example, are not representative of all computing devices. Nevertheless, the concept of improving the computer itself by improving one or more layers of functionality is general in nature.

[0059] The executable files and programs described herein are identified based on the application or software layer in which they are implemented in a particular embodiment of the invention. However, it should be understood that any specific program terminology herein is used for convenience only, and therefore the invention should not be limited to use only in any particular identified application or software layer.

[0060] The features, advantages, and characteristics of the embodiments described herein may be combined in any suitable manner. Those skilled in the relevant art will recognize that embodiments may be practiced without one or more of the specific features or advantages of a particular embodiment. In other cases, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments.

[0061] Some of the functional units described herein may be labeled as modules to more specifically emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in a programmable hardware device such as a field programmable gate array, programmable array logic, programmable logic device, or the like.

[0062] The modules may be implemented in software for execution by various types of processors. An identified module of program instructions may comprise, for example, one or more physical or logical blocks of computer instructions, which may be organized, for example, as an object, procedure, or function. Nevertheless, the executable files of the identified modules need not be physically located together, but may include different instructions stored in different locations such that, when logically linked together, they comprise the modules and achieve the purpose described above for the modules.

[0063] In the preceding description, numerous specific details are provided, such as examples of programs, software modules, user selections, networks, transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, etc., to provide a thorough understanding of the embodiments. However, one skilled in the relevant art will recognize that the embodiments can be practiced without one or more of the specific details, or with other methods, processes, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the embodiments.

[0064] The symbols of elements in each figure may refer to elements in other figures. Like symbols refer to like elements in all figures, including alternative embodiments of like elements. The embodiments may be embodied in other specific forms. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims, rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are intended to be embraced within their scope.

Claims

1. receiving a request to create a cloud computing cluster, the cloud computing cluster having a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; initiating an initialization process for a single processing node and a corresponding attached storage device in response to receiving the request to create the cloud computing cluster; Before completing the initialization process, requesting and receiving an IP address of the single processing node to generate a received IP address and a device ID of the corresponding attached storage device from one or more cloud infrastructure controllers to generate a received device ID; and configuring the clustered file system and corresponding WAN cache using the received IP address and the received device ID prior to completion of the initialization process; A computer-implemented method comprising:

2. 2. The method of claim 1, further comprising configuring one or more threads for the single processing node in response to completion of the initialization process.

3. 3. The method of claim 2, further comprising the step of obtaining data for the one or more threads in response to completion of the initialization process.

4. The method of claim 3 , wherein the step of obtaining data for one or more threads adds data to the WAN cache.

5. 10. The method of claim 1, further comprising, in response to completion of the initialization process, adding additional resources to the cloud computing cluster until the requested number of processing nodes and attached storage devices is achieved.

6. The method of claim 5 , wherein adding additional resources to the cloud computing cluster comprises initiating initialization of the additional resources.

7. The method of claim 6 , further comprising requesting and receiving an identity of the additional resource from the one or more cloud infrastructure controllers before completing initialization of the additional resource.

8. The method of claim 7 , further comprising using the ID of the additional resource to further configure the cloud computing cluster before completing initialization of the additional resource.

9. The method of claim 8 , wherein the additional resource is an additional processing node.

10. The method of claim 9 , further comprising configuring at least one thread for the additional processing node.

11. The method of claim 10 further comprising obtaining data for the at least one thread.

12. The method of claim 1 , further comprising: executing a computing cluster termination process in response to receiving a request to terminate the cloud computing cluster.

13. The computing cluster termination process: determining changes to data captured in the WAN cache; committing the changes to local storage or a remote site; identifying processing nodes and corresponding attached storage devices associated with the cloud computing cluster; and Releasing the processing node and corresponding attached storage device.

13. The method of claim 12, comprising:

14. 1. A computer program product comprising a computer-readable storage medium having program instructions embodied thereon, the computer-readable storage medium not itself being a transitory signal, wherein the program instructions include: receiving a request to create a cloud computing cluster, the cloud computing cluster having a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; initiating an initialization process for a single processing node and a corresponding attached storage device in response to receiving the request to create the cloud computing cluster; Before completion of the initialization process, requesting and receiving an IP address of the single processing node to generate a received IP address and a device ID of the corresponding attached storage device from one or more cloud infrastructure controllers to generate a received device IP; and configuring the clustered file system and corresponding WAN cache using the received IP address and the received device ID prior to completion of the initialization process; 10. A computer program product executable by a processor to cause the processor to perform a method comprising:

15. 15. The computer program product of claim 14, wherein the method further comprises configuring one or more threads for the single processing node in response to completion of the initialization process.

16. 16. The computer program product of claim 15, wherein the method further comprises obtaining data for the one or more threads in response to completion of the initialization process.

17. 15. The computer program product of claim 14, wherein the method further comprises, in response to completion of the initialization process, adding additional resources to the cloud computing cluster until the requested number of processing nodes and attached storage devices is achieved.

18. 20. The computer program product of claim 17, wherein adding additional resources to the cloud computing cluster comprises initiating initialization of the additional resources, and requesting and receiving identification of the additional resources from the one or more cloud infrastructure controllers before completing initialization of the additional resources.

19. 20. The computer program product of claim 18, wherein the method further comprises using the identity of the additional resource to further configure the cloud computing cluster before completing initialization of the additional resource.

20. one or more processors; and a computer-readable storage medium having program instructions embodied thereon; wherein the computer-readable storage medium is not itself a transitory signal, and the program instructions are: receiving a request to create a cloud computing cluster, the cloud computing cluster having a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; initiating an initialization process for a single processing node and a corresponding attached storage device in response to receiving the request to create the cloud computing cluster; Before completion of the initialization process, requesting and receiving an IP address of the single processing node to generate a received IP address and a device ID of the corresponding attached storage device from one or more cloud infrastructure controllers to generate a received device IP; and configuring the clustered file system and corresponding WAN cache using the received IP address and the received device ID prior to completion of the initialization process; 10. A system executable by the one or more processors to cause the one or more processors to perform a method comprising:

Images