Increasing resource utilization in cloud computing clusters

Abstract

A computer-implemented method for provisioning a cloud computing cluster, comprising: receiving a request to create a cloud computing cluster, the cloud computing cluster comprising a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; and starting an initialization process for the single processing node and the corresponding attached storage device in response to receiving a request for creating the cloud computing cluster. The method may also include requesting and receiving an IP address for the single processing node and a device ID for the corresponding attached storage device from the one or more cloud infrastructure controllers prior to completing the initialization process; and configuring the clustered file system and the corresponding WAN cache using the received IP address and the received device ID. Systems and computer program products corresponding to the above method are also disclosed herein.

Description

BACKGROUND OF THE INVENTION

[0001] The subject matter disclosed herein generally relates to launching cloud-based applications and, more particularly, to provisioning cloud computing clusters and associated resources.

[0002] Currently, cloud-based applications experience significant delays in resource availability when being deployed. For example, assume a cloud bursting scenario where a customer intends to immediately start scaling an application into the cloud. The customer may wait for each resource to be allocated and initialized. Such a scenario can result in delays of over one hour before the cloud-based application can start processing. After starting processing, a significant additional amount of time is required to load the large data sets typically associated with cloud-based applications. SUMMARY OF THE INVENTION

[0003] A computer-implemented method for provisioning a cloud computing cluster includes: receiving a request to create a cloud computing cluster, the cloud computing cluster including a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; and in response to receiving the request to create the cloud computing cluster, initiating an initialization process for a single processing node and a corresponding attached storage device. The method may further include: requesting and receiving, from one or more cloud infrastructure controllers, an IP address for the single processing node and a device ID for the corresponding attached storage device before completing the initialization process; and using the received IP address and the received device ID to configure the clustered file system and the corresponding WAN cache.

[0004] Also disclosed herein are systems and computer program products incorporating the above method. The computer program product includes a computer-readable storage medium having program instructions embodied therewith, where the computer-readable storage medium is not a transitory signal per se, and where the program instructions are executable by a processor to cause the processor to perform the above method. The system includes one or more processors and a computer-readable storage medium similar to the computer-readable storage medium included in the computer program product. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] To facilitate an understanding of the advantages of the disclosed embodiments, a more specific description of the embodiments briefly described above will be presented by reference to specific embodiments illustrated in the drawings. It should be understood that these drawings depict only some embodiments and should not be considered limiting of the scope. The embodiments will be described and explained with additional features and details by use of the drawings, in which:

[0006] Figure 1 is a block diagram showing various parts of a computing environment in accordance with at least one embodiment disclosed herein; and

[0007] Figure 2 is a flowchart of an example of a method for provisioning a cloud computing cluster according to at least one embodiment disclosed herein;

[0008] Figure 3 is a flowchart of an example of a method for adding resources to a cloud computing cluster according to at least one embodiment disclosed herein;

[0009] Figure 4 is a flowchart of an example of a method for releasing a cloud computing cluster according to at least one embodiment disclosed herein;

[0010] Figure 5 is a block diagram showing an example of a computing stack according to at least one embodiment disclosed herein. Detailed Description

[0011] Those of ordinary skill in the art will understand that references to "one embodiment", "an embodiment", or similar language throughout the specification mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the phrases "in one embodiment", "in an embodiment", and similar language that appear throughout the specification may, but do not necessarily, all refer to the same embodiment, but rather mean "one or more but not all embodiments" unless expressly stated otherwise. The terms "comprises", "comprising", "has", and variations thereof mean "including but not limited to" unless expressly stated otherwise. A list of items does not imply that any or all of the items are mutually exclusive and / or mutually inclusive unless expressly stated otherwise. The terms "a", "an", and "the" also refer to "one or more" unless expressly stated otherwise.

[0012] The techniques and solutions disclosed herein reduce latency caused by cloud computing clients when provisioning cloud computing clusters and associated resources for executing cloud-based applications.

[0013] Figure 1 is a block diagram showing various parts of a computing environment 100 according to at least one embodiment disclosed herein. The computing environment 100 includes means for executing a method or process of the present invention, such as code block 201 (corresponding to Figure 2An example of an environment for at least some of the computer code involved in the method 200 shown in [description]. In some embodiments, portions of the code block 201 reside within the operating system 122. In addition to block 201, the computing environment 100 also includes, for example, a computer 101, a wide area network (WAN) 102, an end-user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, the computer 101 includes a processor group 110 (including processing circuitry 120 and cache 121), a communication fabric 111, volatile memory 112, persistent storage 113 (including the operating system 122 and block 201, as described above), a peripheral group 114 (including a user interface (UI) device group 123, a storage device 124, and an Internet of Things (IoT) sensor group 125), and a network module 115. The remote server 104 includes a remote database 130. The public cloud 105 includes a gateway 140, a cloud orchestration module 141, a host physical machine group 142, a virtual machine group 143, and a container group 144.

[0014] The computer 101 can take the form of a desktop computer, a laptop computer, a tablet computer, a smart phone, a smart watch, or other wearable computer, a mainframe computer, a quantum computer, or any other form of computer or mobile device that is now known or will be developed in the future and is capable of running programs, accessing a network, or querying a database, such as the remote database 130. As understood in the computer technology field, according to this technology, the execution of a computer-implemented method can be distributed among multiple computers and / or multiple locations. On the other hand, in the presentation of this computing environment 100, the detailed discussion focuses on a single computer, specifically the computer 101, to keep the presentation as simple as possible. The computer 101 may be located in the cloud, although Figure 1 it is not shown in the cloud in [description]. On the other hand, the computer 101 does not need to be in the cloud unless it can be definitely stated to any extent.

[0015] The processor group 110 includes one or more computer processors of any type that are now known or will be developed in the future. The processing circuitry 120 can be distributed across multiple packages, such as multiple coordinated integrated circuit chips. The processing circuitry 120 can implement multiple processor threads and / or multiple processor cores. The cache 121 is memory located within the processor chip package and is typically used for data or code that the threads or cores running on the processor group 110 should be able to access quickly. Depending on the relative proximity to the processing circuitry, the cache memory is typically organized into multiple levels. Alternatively, some or all of the cache of the processor group may be located "off-chip". In certain computing environments, the processor group 110 can be designed to use qubits and perform quantum computing.

[0016] Computer-readable program instructions are typically loaded onto computer 101 so that a processor group 110 of the computer 101 executes a series of operational steps to implement a computer-implemented process, such that the instructions so executed will instantiate the flowchart of the computer-implemented method and / or the method specified in the descriptive description contained in this document (collectively referred to as "the method of the present invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor group 110 to control and directly execute the method of the present invention. In computing environment 100, at least some of the instructions for executing the method of the present invention may be stored in block 201 of persistent storage device 113.

[0017] Communication structure 111 is a signal conduction path that allows various components of computer 101 to communicate with each other. Typically, this structure consists of switches and conductive paths, such as switches and conductive paths that make up a bus, a bridge, a physical input / output port, etc. Other types of signal communication paths can be used, such as fiber optic communication paths and / or wireless communication paths.

[0018] Volatile memory 112 is any type of volatile memory known currently or developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, volatile memory 112 is characterized by random access, but this is not required unless explicitly indicated. In computer 101, volatile memory 112 is located in a single package and is inside the computer 101. However, optionally or additionally, volatile memory can be distributed across multiple packages and / or be located externally relative to computer 101.

[0019] Persistent storage device 113 is any form of non-volatile storage device of a computer known currently or to be developed in the future. The non-volatility of this storage device means that the stored data will be retained regardless of whether power is supplied to computer 101 and / or whether power is directly supplied to persistent storage device 113. Persistent storage device 113 can be read-only memory (ROM), but typically at least a portion of the persistent storage device allows writing, deleting, and rewriting of data. Some common forms of persistent storage devices include magnetic disks and solid-state storage devices. Operating system 122 can take various forms, such as various known proprietary operating systems or open-source portable operating system interface type operating systems that employ a kernel. The code included in block 201 typically includes at least some computer code involved in executing the method of the present invention, such as provisioning a cloud computing cluster and associated resources.

[0020] The peripheral device group 114 includes the peripheral device group of the computer 101. The data communication connection between the peripheral devices and other components of the computer 101 can be implemented in various ways, such as a Bluetooth connection, a near field communication (NFC) connection, a cable connection (such as a universal serial bus (USB) type cable), a plug-in connection (such as a secure digital (SD) card), a connection via a local communication network, or even a connection via a wide area network such as the Internet. In various embodiments, the UI device group 123 may include components such as a display screen, a speaker, a microphone, wearable devices (such as goggles and smartwatches), a keyboard, a mouse, a printer, a touchpad, a game controller, and a haptic device. The storage device 124 is an external memory (such as an external hard disk drive) or a pluggable storage device (such as an SD card). The storage device 124 can be persistent and / or volatile. In some embodiments, the storage device 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where the computer 101 needs to have a large amount of storage (for example, the computer 101 locally stores and manages a large database), the storage device may be provided by a peripheral storage device designed to store a very large amount of data, such as a storage area network (SAN) shared by multiple geographically dispersed computers. The IoT sensor group 125 includes sensors that can be used in Internet of Things applications. For example, one sensor can be a thermometer and another sensor can be a motion detector.

[0021] The network module 115 is a collection of computer software, hardware, and firmware that allows the computer 101 to communicate with other computers via the WAN 102. The network module 115 may include hardware, such as a modem or a Wi-Fi signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or network browser software for transmitting data over the Internet. In some embodiments, the network control function and the network forwarding function of the network module 115 are executed on the same physical hardware device. In other embodiments (for example, embodiments using software-defined networking (SDN)), the control function and the forwarding function of the network module 115 are executed on physically separate devices, such that the control function manages multiple different network hardware devices. The computer-readable program instructions for performing the methods of the present invention can generally be downloaded to the computer 101 from an external computer or an external storage device via a network adapter card or a network interface included in the network module 115.

[0022] The WAN 102 is any wide area network, such as the Internet, capable of transmitting computer data over non-local distances by any technology for transmitting computer data, whether now known or hereafter developed. In some embodiments, the WAN 102 may be replaced and / or supplemented by a local area network (LAN) designed to transmit data between devices located in a local area such as a Wi-Fi network. The WAN and / or LAN typically includes computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0023] The end user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating the computer 101) and may take any form discussed above in connection with the computer 101. The EUD 103 typically receives helpful and useful data from the operation of the computer 101. For example, in the hypothetical case where the computer 101 is designed to provide recommendations to an end user, the recommendation will typically be transmitted from the network module 115 of the computer 101 through the WAN 102 to the EUD 103. Thus, the EUD 103 can display or otherwise present the recommendation to the end user. In some embodiments, the EUD 103 can be a client device such as a thin client, thick client, mainframe, desktop, etc.

[0024] The remote server 104 is any computer system that provides at least some data and / or functionality to the computer 101. The remote server 104 may be controlled and used by the same entity that operates the computer 101. The remote server 104 represents a machine for collecting and storing useful data for use by other computers, such as the computer 101. For example, in the hypothetical case where the computer 101 is designed and programmed to provide recommendations based on historical data, the historical data can be provided to the computer 101 from the remote database 130 of the remote server 104.

[0025] A public cloud 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computing capabilities, particularly data storage (cloud storage) and computing power, without the need for direct active management by the user. Cloud computing typically exploits resource sharing to achieve consistency and economies of scale. The direct active management of the public cloud 105 computing resources is performed by the computer hardware and / or software of the cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments running on various computers that make up the host physical machine group 142, which are the various physical computers within and / or available for the public cloud 105. The virtual computing environment (VCE) typically takes the form of virtual machines from the virtual machine group 143 and / or containers from the container group 144. It can be understood that these VCEs can be stored as images and can be transferred between individual physical machine hosts either as images or after the VCE is instantiated. The cloud orchestration module 141 manages the transfer and storage of the images, deploys new instances of the VCE, and manages the active instances of the VCE deployment. The gateway 140 is a collection of computer software, hardware, and firmware that allows the public cloud 105 to communicate over the WAN 102.

[0026] Some further explanations of the virtualized computing environment (VCE) will now be provided. A VCE can be stored as an "image". New active instances of the VCE can be instantiated from the image. Two familiar types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows for the existence of multiple isolated user space instances (called containers). From the perspective of the programs running within them, these isolated user space instances typically appear as real computers. A computer program running on a normal operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU capabilities, and quantifiable hardware capabilities. However, a program running within a container can only use the contents of that container and the devices allocated to that container, a feature known as containerization.

[0027] A private cloud 106 is similar to the public cloud 105, except that the computing resources are only available for use by a single enterprise. Although the private cloud 106 is depicted as communicating with the WAN 102, in other embodiments, the private cloud can be completely disconnected from the Internet and only accessible through a local / private network. A hybrid cloud is a combination of multiple clouds of different types (such as private, community, or public cloud types), typically implemented separately by different providers. Each of the multiple clouds remains an independent discrete entity, but the larger hybrid cloud architecture is bound together through standardized or proprietary technologies to support orchestration, management, and / or data / application portability between the multiple constituent clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0028] Figure 2 is a flowchart of an example of a method 200 for provisioning a cloud computing cluster according to at least one embodiment disclosed herein. As depicted, the method includes: receiving (210) a request to create a cloud computing cluster, starting (220) an initialization process, requesting and receiving (230) an IP address and a device ID, configuring (240) a clustered file system and a corresponding WAN cache, determining (260) whether the initialization process is complete, configuring (270) one or more threads, retrieving (280) data, and adding (290) additional resources. The depicted method reduces the initialization latency experienced by cloud applications when provisioning a cloud computing cluster and associated resources.

[0029] Receiving (210) a request to create a cloud computing cluster can include receiving a request specifying a requested number of processing nodes and attached storage devices to be allocated to the cloud computing cluster. The request can specify a file system type, such as a clustered file system.

[0030] Starting (220) the initialization process can include starting an initialization process for a single processing node and a corresponding attached storage device. The started initialization process can be limited to a single processing node and a corresponding attached storage device in order to reduce the initialization latency and waiting time imposed before useful processing can occur. Additionally, the starting party can offload the initialization process and continue with additional useful operations (e.g., operations 230 and 240) when the initialization process is complete.

[0031] Requesting and receiving (230) an IP address and a device ID can include requesting an IP address of a processing node and a device ID of a corresponding attached storage device. The request can be submitted to one or more cloud infrastructure controllers, which can respond with the IP address and the device ID.

[0032] Configuring (240) a clustered file system and a corresponding WAN cache can include using the received IP address and device ID to initialize various configuration settings and parameters within the clustered file system and the corresponding WAN cache. The settings and parameters can be changed before the initialization process is complete.

[0033] Determining (260) whether the initialization process is complete can include testing a semaphore indicating the status of the initialization process. Alternatively, when the initialization of a single processing node and a corresponding attached storage device is complete, the initialization process can call a callback function. The callback function can react to method 200 at operation 270.

[0034] Configuring (270) one or more threads may include configuring threads associated with a single processing node. Retrieving (280) data may include retrieving data required for threads associated with a single processing node. The act of retrieving data may place the data in a WAN cache.

[0035] Adding (290) additional resources may include adding additional resources required for a cloud computing cluster to the cloud computing cluster, such as additional processing nodes and corresponding attached storage devices.

[0036] Those skilled in the art will understand that the above method for provisioning a cloud computing cluster enables a cloud application to begin execution of application threads before all resources associated with the cloud computing cluster have been fully provisioned. Thus, cloud-based applications can begin processing more quickly, and the latency associated with loading a new cloud-based application can be reduced.

[0037] Figure 3 is a flowchart of an example of a method 300 for adding resources to a cloud computing cluster according to at least one embodiment disclosed herein. As shown, method 300 includes: initiating (310) the initialization of additional resources, requesting and receiving (320) an ID, and configuring (330) the cloud computing cluster.

[0038] Initiating (310) the initialization of additional resources may include dispatching an initialization process for an additional processing node or an attached storage device. Requesting and receiving (320) an ID may be similar to requesting and receiving operation 240, and may include requesting the IP address of an additional processing node and / or the device ID of a corresponding attached storage device. The request may be submitted to one or more cloud infrastructure controllers, which may respond with the IP address and / or device ID.

[0039] Configuring (330) the cloud computing cluster may include using the received ID to configure a clustered file system data structure such that the clustered file system data structure is populated with processing node details (still in provisioning) and / or storage device details (such as labels still in provisioning, etc.). Those skilled in the art will understand that the above method for adding resources to a cloud computing cluster enables cloud-based applications to be prepared to use resources that are being provisioned and to incrementally use these additional resources when the additional resources associated with the cloud computing cluster are fully provisioned.

[0040] Figure 4 is a flowchart of an example of a method (400) for releasing a cloud computing cluster according to at least one embodiment disclosed herein. As depicted, the method includes: determining (410) changes to data, submitting (420) the changes, identifying (430) processing nodes and storage devices, and releasing (440) the processing nodes and storage devices.

[0041] Determining (410) changes to data may include accessing cache coherent data to determine data that has been updated in the cache but has not yet been written to secondary storage. Committing (420) the changes may include writing the data to secondary storage. Identifying (430) processing nodes and storage devices may include communicating with a cloud infrastructure controller to determine processing nodes and storage devices that have been allocated but not yet released.

[0042] Releasing (440) processing nodes and storage devices may include sending a request to the cloud infrastructure controller to release processing nodes and storage devices that have been allocated but not yet released.

[0043] The methods disclosed herein may be partially or fully incorporated within Figure 1 the cloud resource provisioning code block 201 shown. Those skilled in the art will understand that the methods disclosed herein can be adapted to the cloud computing environments to which they are deployed without changing the intent of the disclosed methods. Those skilled in the art will also understand the utility and effectiveness of the methods and solutions disclosed herein.

[0044] The disclosed solution substantially provides a provisioning framework that helps optimize the total time / dollars spent in deploying (or making the solution - software, hardware, middleware ready for the application) a WAN cache solution in a public / hybrid cloud environment by pre-populating clustered file system records with instance / virtual machine / node details (which have not yet been provisioned) and expanding (flattening) the disk structure with disk (volume / EBS) details (which have not yet been provisioned), creating extended node files, allocation maps, and log files. Additionally, data prefetch threads can be scaled (where the rate of scaling depends on the infrastructure provisioning rate), and floating window-based re-striping of storage devices can occur (where the window size depends on the disk size added to the pre-expanded file system). Thus, WAN caching (populating the dataset required for the application) can be initiated during the cluster provisioning phase itself (instead of after the cluster configuration phase).

[0045] The described framework provides the following features and flexibility to traditional clustered file system-based WAN cache deployment solutions on hybrid / public clouds.

[0046] When the cloud deployment manager receives a request to create a WAN cache clustered file system for nodes, disks (where the threshold can increase or decrease based on the cloud infrastructure processing unit workload) with defined thresholds, it will execute;

[0047] A single processing node / virtual machine is provisioned along with an attached storage device. The initial device type and size can depend on the network bandwidth used to pull data as part of the WAN cache relative to the total provisioning time required to reach full file system capacity – it is typically a small fraction of the file system capacity.

[0048] A request is made to the cloud infrastructure processing unit (IPU) [also known as the cloud infrastructure controller] to share the node IP details and storage device ID before provisioning the node and device. The cloud IPU retains these details and uses them to provision resources at the configured infrastructure processing rate.

[0049] The clustered file system and WAN cache relationship is initially established using a single (processing) node cluster. The clustered file system data structures are configured in such a way that they break down into processing node details (still being provisioned) and disk details (such as labels, still being provisioned).

[0050] Processing threads are configured and data prefetching is started. The number of threads increases based on the number of nodes that are provisioned and joined to the cluster. This ordering helps to linearly scale the WAN cache data prefetching based on the number of new nodes added to the cluster.

[0051] Deployment continues to start new nodes, attach NSDs, and add storage devices to the file system.

[0052] Data prefetching continues, and the framework linearly scales disk striping (using data re-striping based on a floating window) so that data segments stored on the initial disks start to move to other newly provisioned disks.

[0053] In the case of an erasure code-based clustered file system, data prefetching continues, and the framework linearly scales the movement of erasure code data segments (which can be an "rsync" equivalent operation) so that data segments stored on the initial disks start to move to other newly provisioned disks.

[0054] When the cloud deployment manager receives a request to disassemble the WAN cache clustered file system, a similar process is used, where nodes and attached storage devices are selected, changes to the data are identified, and the changes are replayed to the object storage or remote site based on the WAN cache, resulting in an incremental shrinkage of the file system data structure. The IDs of the selected nodes and attached storage devices are sent to the cloud IPU for resource deallocation ("destruction").

[0055] The following operations can be carried out in combination with the above framework and methods.

[0056] The user fills in the number and size of the requested nodes / virtual machines and NSDs (disks) in the cloud deployment manager / IaC tool.

[0057] The framework identifies the current workload on the cloud infrastructure processing unit (IPU). If the set processing IPU rate is low and it takes more time to provision, this causes the application to wait (the rate and duration are user-configurable, e.g., according to the rate published or observed by the cloud provider for infrastructure provisioning duration).

[0058] Provision a single node + attached disk of the file system or the minimally supported blueprint, and configure the WAN cache.

[0059] When creating the minimally supported blueprint, - the framework extends the data structure so that it pre-allocates block ranges, allocation maps, and log groups for the remaining NSD disks to be attached (or to be provisioned) in the future. –

[0060] The application starts data prefetching and begins to process the data. –

[0061] Once the other NSDs are created and reach the available state, they are linked (or a special counter flag can also be enabled) to these pre-provisioned data structures.

[0062] The configuration of the prefetch thread (pthread) changes (increases) dynamically based on the rate at which new nodes / NSDs are added.

[0063] Configuration changes related to re-striping (floating window mode), except that it is not a complete stripe but a stripe based on the calculated NSD attachments, which will occur in the future. For example: (number of files = 8, NSD = 8)

[0064] Iteration - 1: File - 1 retrieved using NSD - 1

[0065] Iteration - 2: File - 2 retrieved using NSD - 1, NSD - 2. Re-striping starts, only moving stripe - 2 blocks of File - 1 to NSD - 2

[0066] Iteration - 3: File - 3 has been retrieved using NSD - 1, NSD - 2, NSD - 3. Restriping is initiated, moving only stripe - 3 chunks of File - 1 to NSD - 3 and only stripe - 3 chunks of File - 2 to NSD - 2.

[0067] In the case of disassembly, the cloud deployment manager receives a request to disassemble the WAN - cached clustered file system. A similar process is used where nodes and disks are selected, changes are identified, and the changes are replayed to the object storage device or remote site based on the WAN cache and the file system data structure shrinks. The selected nodes are sent to the cloud IPU for destruction.

[0068] As disclosed herein, a computer - implemented method for provisioning a cloud computing cluster may include:

[0069] Receiving a request to create a cloud computing cluster, the cloud computing cluster including a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster

[0070] In response to receiving the request to create a cloud computing cluster, initiating an initialization process for a single processing node and a corresponding attached storage device

[0071] Before completing the initialization process, requesting and receiving an IP address for the single processing node and a device ID for the corresponding attached storage device from one or more cloud infrastructure controllers

[0072] Before completing the initialization process, configuring the clustered file system and the corresponding WAN cache using the received IP address and the received device ID

[0073] Additional features of the above - described method may include:

[0074] In response to completion of the initialization process, configuring one or more threads for the single processing node

[0075] In response to completion of the initialization process, retrieving data for one or more threads

[0076] Wherein retrieving data for one or more threads adds the data to the WAN cache

[0077] In response to completion of the initialization process, adding additional resources to the cloud computing cluster until the requested number of processing nodes and attached storage devices is reached

[0078] Among them, adding additional resources to the cloud computing cluster includes: initiating the initialization of the additional resources

[0079] Before completing the initialization of the additional resources, request and receive the ID of the additional resources from one or more cloud infrastructure controllers

[0080] Before completing the initialization of the additional resources, use the ID of the additional resources to further configure the cloud computing cluster

[0081] Among them, the additional resources are additional processing nodes

[0082] Configure at least one thread for the additional processing node

[0083] Retrieve data for at least one thread

[0084] Execute the computing cluster termination process in response to receiving a request to terminate the cloud computing cluster

[0085] Among them, the cluster termination process includes:

[0086] Determine changes to the data captured in the WAN cache

[0087] Commit the changes to a local storage device or a remote site

[0088] Identify the processing nodes associated with the cloud computing cluster and the corresponding attached storage devices

[0089] Release the processing nodes and the corresponding attached storage devices

[0090] The present disclosure also discloses a system and a computer program product corresponding to the above method. Aspects of the present disclosure are described by the explanatory text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of a computer program product (CPP). For any flowchart, depending on the technology involved, the operations may be performed in an order different from the order shown in the given flowchart. For example, also depending on the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

[0091] An example of a computer program product implementation ("CPP implementation" or "CPP") is a term used in this disclosure to describe any collection of one or more storage media (also referred to as "media"), which together are included in a collection of one or more storage devices, which together include machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" is any tangible device that can retain and store instructions for use by a computer processor. A computer-readable storage medium can be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media are: floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices (such as punched cards or pits / islands formed in the main surface of a disc), or any suitable combination of the foregoing. As a term used in this disclosure, a computer-readable storage medium is not construed as storage in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, optical pulses passing through an optical fiber, electrical signals transmitted through a wire, and / or other transmission media. As will be understood by those skilled in the art, data is typically moved at certain occasional points in time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but this does not make the storage device transitory because the data is not transitory when stored.

[0092] Figure 5 is a block diagram showing an example of a computing stack 570 according to at least one embodiment disclosed herein. As depicted, the computing stack 570 includes a plurality of computing layers 572 for performing computing operations. In the depicted embodiment, these layers include a hardware layer and a software layer. The various software layers include an operating system layer associated with executing one or more operating systems, a middleware layer associated with executing middleware that extends and / or improves the functionality of the hardware layer, and the operating system being executed. The software layer may also include various application-specific layers. The application-specific layers may include application frameworks that further extend and / or improve the functionality of the hardware layer and the operating system layer.

[0093] The memory layer may include volatile memory, non-volatile memory, permanent storage devices, and the hardware associated with controlling such memory. The logic units may include a CPU, arithmetic units, graphics processing units, and the hardware associated with controlling these units. The microcode layer may include executable instructions for controlling the processing flow associated with moving data between the memory and the logic units. The processor layer may include an instruction fetch unit, an instruction decode unit, etc., which enable the execution of processing instructions and the utilization of the underlying hardware layer.

[0094] Hardware drivers (also known as the hardware abstraction layer) may include executable code that enables an operating system to access and control storage devices, DMA hardware, I / O buses, peripheral devices, and other hardware associated with the computing environment. The operating system kernel layer may receive I / O requests from higher levels and manage memory and other hardware resources via the hardware drivers. The operating system kernel layer may also provide other functions such as inter-process communication and file management.

[0095] Operating system libraries and utilities may extend the functions provided by the operating system kernel and provide interfaces for accessing these functions. Higher-level software typically utilizes libraries by linking the library object code to the higher-level software executable file. In contrast, operating system utilities are typically stand-alone executable programs that can be invoked via an operating system shell that receives commands from users and / or script files. Examples of operating system libraries include file I / O libraries, math libraries, memory management libraries, process control libraries, data access libraries, etc. Examples of operating system utilities include antivirus managers, disk formatters, disk defragmenters, file compressors, data or file sorters, data archivers, memory testers, program installers, package managers, network utilities, system monitors, system analyzers, etc.

[0096] Services are typically provided by a running executable program or process that receives local or remote requests from other processes or devices called clients. The computer running the service is typically referred to as a server. Examples of servers include database servers, file servers, mail servers, print servers, web servers, game servers, and application servers.

[0097] Application frameworks provide the functions typically required by applications and include system infrastructure frameworks, middleware integration, frameworks, enterprise application frameworks, graphics rendering frameworks, and game frameworks. Application frameworks may support application development for specific environments or industries. In some cases, application frameworks may be used for multiple operating systems and provide a common programming interface to developers across multiple platforms.

[0098] General applications include those that are required by most users. Examples of general applications include mail applications, calendar and scheduling applications, and web browsers. Such applications can be automatically included in the operating system.

[0099] Those skilled in the art will understand that improvements to any of the depicted layers or similar layers not depicted herein result in improvements to the computer itself, including computer 101 and / or end-user device 103. Those skilled in the art will also understand that the depicted layers are given as examples and are not representative of all computing devices. However, the concept of improving the computer itself by improving one or more functional layers is generally applicable.

[0100] The executable programs and programs described herein are identified based on the application or software layer in which they are implemented in a particular embodiment of the present invention. However, it should be understood that any specific program terms herein are used for convenience only, and thus the present invention should not be limited to use only in any of the identified specific applications or software layers.

[0101] The features, advantages, and characteristics of the embodiments described herein can be combined in any suitable manner. Those skilled in the relevant art will recognize that embodiments can be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in a particular embodiment that may not be present in all embodiments.

[0102] Some of the functional units described in this specification may have been labeled as modules to more specifically emphasize their implementation independence. For example, a module can be implemented as a hardware circuit including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module can also be implemented in programmable hardware devices such as field-programmable gate arrays, programmable array logic, programmable logic devices.

[0103] A module can also be implemented in software executed by various types of processors. The identified modules of program instructions can include, for example, one or more physical or logical blocks of computer instructions, which can be organized, for example, as objects, procedures, or functions. However, the executable code of the identified modules need not physically be located together, but can include different instructions stored in different locations, which, when logically combined, include the module and implement the claimed purpose of the module.

[0104] In the foregoing description, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, etc., to provide a thorough understanding of the embodiments. However, those skilled in the relevant art will recognize that the embodiments may be practiced without one or more of the specific details, or with other methods, processes, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the embodiments.

[0105] The description of the elements in each figure may refer to the elements in the previous figures. In all the figures, the same reference numerals represent the same elements, including alternative embodiments of the same element. The embodiments may be practiced in other specific forms. The described embodiments are to be considered in all respects only illustrative and not restrictive. Thus, the scope of the present invention is indicated by the appended claims rather than by the previous description. All changes within the meaning and range of equivalence of the claims will be embraced within their scope.

Claims

1. A computer-implemented method, comprising: Receiving a request to create a cloud computing cluster, the cloud computing cluster including a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; In response to receiving the request to create the cloud computing cluster, initiating an initialization process for a single processing node and a corresponding attached storage device; Before completing the initialization process, requesting and receiving from one or more cloud infrastructure controllers an IP address for the single processing node to generate the received IP address and a device ID for the corresponding attached storage device to generate the received device ID; And Before completing the initialization process, using the received IP address and the received device ID to configure the clustered file system and the corresponding WAN cache.

2. The method according to claim 1 further comprises: In response to completion of the initialization process, configuring one or more threads for the single processing node.

3. The method according to claim 2 further comprises: In response to completion of the initialization process, retrieving data for the one or more threads.

4. The method according to claim 3, wherein Retrieving data for the one or more threads adds the data to the WAN cache.

5. The method according to claim 1 further comprises: In response to completion of the initialization process, adding additional resources to the cloud computing cluster until the requested number of processing nodes and attached storage devices is reached.

6. The method according to claim 5, wherein, Adding additional resources to the cloud computing cluster includes: initiating the initialization of the additional resources.

7. The method according to claim 6, further comprising: Before completing the initialization of the additional resources, requesting and receiving from the one or more cloud infrastructure controllers an ID for the additional resources.

8. The method according to claim 7 further comprises: Before completing the initialization of the additional resources, using the ID of the additional resources to further configure the cloud computing cluster.

9. The method according to claim 8, wherein, The additional resources are additional processing nodes.

10. The method according to claim 9 further comprises: Configuring at least one thread for the additional processing node.

11. The method according to claim 10, further comprising: Retrieving data for the at least one thread.

12. The method according to claim 1 further comprises: Performing a compute cluster termination process in response to receiving a request to terminate the cloud computing cluster.

13. The method according to claim 12, wherein, The compute cluster termination process includes: Determining changes to data captured in the WAN cache; Committing the changes to a local storage device or a remote site; Identifying processing nodes and corresponding attached storage devices associated with the cloud computing cluster; and Releasing the processing nodes and corresponding attached storage devices.

14. A computer program product comprising a computer-readable storage medium having program instructions embodied therewith, wherein, The computer-readable storage medium is not a transient signal itself, wherein the program instructions are executable by a processor to cause the processor to perform a method, the method including: Receiving a request to create a cloud computing cluster, the cloud computing cluster including a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; In response to receiving the request to create the cloud computing cluster, initiating an initialization process for a single processing node and a corresponding attached storage device; Before completing the initialization process, requesting and receiving from one or more cloud infrastructure controllers an IP address for the single processing node to generate the received IP address and a device ID for the corresponding attached storage device to generate the received device ID; and Before completing the initialization process, configure the clustered file system and the corresponding WAN cache using the received IP address and the received device ID.

15. The computer program product according to claim 14, wherein, The method further includes: in response to completion of the initialization process, configuring one or more threads for the single processing node.

16. The computer program product according to claim 15, wherein, The method further includes: in response to completion of the initialization process, retrieving data for the one or more threads.

17. The computer program product according to claim 14, wherein, The method further includes: in response to completion of the initialization process, adding additional resources to the cloud computing cluster until the requested number of processing nodes and attached storage devices is reached.

18. The computer program product according to claim 17, wherein, Adding additional resources to the cloud computing cluster includes: initiating initialization of the additional resources and, before completion of the initialization of the additional resources, requesting and receiving an ID for the additional resources from one or more cloud infrastructure controllers.

19. The computer program product according to claim 18, wherein, The method further includes: before completion of the initialization of the additional resources, using the ID of the additional resources to further configure the cloud computing cluster.

20. A system, comprising: one or more processors; and a computer-readable storage medium having program instructions embodied thereon, where the computer-readable storage medium is not a transitory signal per se, where the program instructions are executable by the one or more processors to cause the one or more processors to perform a method, the method including: receiving a request to create a cloud computing cluster, the cloud computing cluster including a clustered file system and a requested number of processing nodes and attached storage devices associated with the cloud computing cluster; in response to receiving the request to create the cloud computing cluster, initiating an initialization process for a single processing node and the corresponding attached storage device; before completing the initialization process, requesting and receiving an IP address for the single processing node from one or more cloud infrastructure controllers to produce the received IP address and a device ID for the corresponding attached storage device to produce the received device ID; and before completing the initialization process, configuring the clustered file system and the corresponding WAN cache using the received IP address and the received device ID.

Images